Haohe Liu 刘濠赫 (Leo)

Email: haohe.liu AT surrey dot ac dot uk

At the Pont de Bir-Hakeim, Paris

I’m Haohe Liu, a final year PhD student at the Centre for Vision Speech and Signal Processing (CVSSP), University of Surrey. I’m the first author of paper such as AudioLDM, AudioLDM 2, NaturalSpeech, VoiceFixer, MusicLDM, AudioSR, etc., with 50+ research publications and over 1800 citations. My open-source projects/checkpoints on GitHub have received over 8300 stars and have been downloaded more than 150000 times.

My research includes topics related to speech, music, and general audio. I am fortunate to be advised by Prof. Mark D. Plumbley, co-supervised by Prof. Wenwu Wang. And I’m lucky to be jointly funded by BBC R&D and the Doctoral College. I’m a team member of the EPSRC AI for Sound Project (EP/T019751/1). Most of my studies are open-sourced.

Research highlights

My research includes tasks related to the audio generative model, source separation, quality enhancement, and recognition, appeared in journals and conferences such as TPAMI, TASLP, ICML, AAAI, NeurIPS, INTERSPEECH, and ICASSP.

Highlighted research performed as the first author:

Text-to-audio generation model: AudioLDM and AudioLDM2.
Ultra-low bitrate audio codec: Semanticodec
Audio super-resolution model on any audio type and any sampling rate: AudioSR.
First text-to-speech model that achieves on par CMOS with human recording: NaturalSpeech.
Restore the quality of human speech signal regardless of how the signal is degraded: VoiceFixer.
The music source separation system that achieves leading performance on Music Demixing Challenge 2021: CWS-PResUNet.
Speech super-resolution model: NVSR.
A module that makes the temporal-resolution of the spectrogram differentiable for efficient audio classification: DiffRes.
Few-shot bioacoustic detection: The 2nd ranking system in the DCASE 2022 Challenge Task 5.

Please refer to my Google Scholar Page for the full publication list:

Education Experience

Centre for Vision, Speech and Signal Processing @ University of Surrey, UK, 01/2022 - 01/2025
– PhD in Vision, Speech and Signal Processing; Main advisor: Prof. Mark D. Plumbley
– With a studentship from the CVSSP and the EPSRC Grant EP/T019751/1 AI for Sound

School of Computer Science @ Northwestern Polytechnical University, China, 09/2016 - 07/2020
– Bachelor of Engineering, Outstanding graduate, Computer Science and Technology; Advisor: Prof. Lei Xie
– GPA: 3.8/4.0 (Top 5%)

Competitions

First place, DCASE Challenge Task 7:Foley Sound Synthesis, 2022
Yi Yuan, Haohe Liu, Xubo Liu, Xiyuan Kang, Mark D.Plumbley, Wenwu Wang
[Paper] [Demo] [Code] [leaderboard]

Second place, DCASE Challenge Task 5:Few-shot Bioacoustic Event Detection, 2022
Haohe Liu, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Wenwu Wang, Mark D. Plumbley
[Paper] [Code] [leaderboard]

Second place on vocal score and fifth place on overall score, ISMIR Music Demixing Challenge, 2021
Haohe Liu and Qiuqiang Kong and Jiafeng Liu
[Paper] [Code] [Challenge details] [leaderboard]

Second place, DCASE Challenge Task 6B: Language-Based Audio Retrieval, 2022
Xinhao Mei, Xubo Liu, Haohe Liu, Jianyuan Sun, Mark D. Plumbley, Wenwu Wang
[Report] [Challenge details]

Third place, DCASE Challenge Task 6A: Automated Audio Captioning, 2022
Xinhao Mei, Xubo Liu, Haohe Liu, Jianyuan Sun, Mark D. Plumbley, Wenwu Wang
[Report] [Challenge details]

Honors & Awards

Scholarships

Judges’ Award - DCASE 2023, Tampere, Finland
Doctoral College Scholarship, tuition waiver with stipend, University of Surrey
Outstanding Graduate Students, Northwestern Polytechnical University, 2020-2021
WuYaJun third class scholarship, 2019
National Scholarship, Northwestern Polytechnical University (top 1.5%), 2017 and 2018, two consecutive years
First-class Scholarship, Northwestern Polytechnical University (top 14.7%), 2016 to 2019, three consecutive years
Gratitude to Scientists of Modern Times National Scholarship (top 0.5%), 2018

Teaching

Guest Lecturer, EEEM068 Applied Machine Learning, University of Surrey, 2024
Demonstrator, EEE3008 Fundamentals of DSP, University of Surrey, 2022/23 Semester 1
Demonstrator, EEE1033 Computer and Digital Logic, University of Surrey, 2022/23 Semester 1
Demonstrator, EEEM068 Applied Machine Learning, University of Surrey, 2022/23 Semester 2

Tweets by LiuHaohe