Haohe Liu 刘濠赫

Email: haohe.liu AT surrey dot ac dot uk

At the Pont de Bir-Hakeim, Paris (August 2024)

I’m a final year PhD student at the Centre for Vision Speech and Signal Processing (CVSSP), University of Surrey. I am fortunate to be supervised by Prof. Mark D. Plumbley, co-supervised by Prof. Wenwu Wang. And I’m lucky to be jointly funded by BBC R&D and the Doctoral College. I’m a team member of the EPSRC AI for Sound Project (EP/T019751/1). I’m a UK Global Talent Visa holder, endorsed by the Royal Academy of Engineering.

Research highlights

My research includes tasks related to the audio generative model, source separation, quality enhancement, and recognition, appeared in journals and conferences such as TPAMI, TASLP, JSTSP, ICML, AAAI, NeurIPS, INTERSPEECH, and ICASSP.

I’m the first author of paper such as AudioLDM, AudioLDM 2, NaturalSpeech, VoiceFixer, SemantiCodec, MusicLDM, AudioSR, etc., with around 50 research publications and 2100 citations. Most of my research studies are open-sourced. My open-source projects/checkpoints on GitHub have received over 8800 stars and have been downloaded more than 200000 times.

Highlighted research performed as the first author:

Text-to-audio generation model: AudioLDM and AudioLDM2.
Ultra-low bitrate audio codec: Semanticodec
Audio super-resolution model on any audio type and any sampling rate: AudioSR.
First text-to-speech model that achieves on par CMOS with human recording: NaturalSpeech.
Restore the quality of human speech signal regardless of how the signal is degraded: VoiceFixer.
The music source separation system that achieves leading performance on Music Demixing Challenge 2021: CWS-PResUNet.
Speech super-resolution model: NVSR.
A module that makes the temporal-resolution of the spectrogram differentiable for efficient audio classification: DiffRes.
Few-shot bioacoustic detection: The 2nd ranking system in the DCASE 2022 Challenge Task 5.

Please refer to my Google Scholar Page for the full publication list:

Educations

Centre for Vision, Speech and Signal Processing @ University of Surrey, UK, 01/2022 - 06/2025
– PhD in Vision, Speech and Signal Processing; Main supervisor: Prof. Mark D. Plumbley
– With a studentship from the CVSSP and the EPSRC Grant EP/T019751/1 AI for Sound
– 2024 Postgraduate Researcher of the Year Award - University of Surrey, CSEE

School of Computer Science @ Northwestern Polytechnical University, China, 09/2016 - 07/2020
– Bachelor of Engineering, Outstanding graduate, Computer Science and Technology; Supervisor: Prof. Lei Xie
– GPA: 3.8/4.0 (Top 5%)

Work/Internship Experience

Research Scientist Intern, Meta, Paris, 06/2024 - 10/2024

Consultant, Riffusion, Remote, 10/2023 - 05/2024

Research Intern, Microsoft, Beijing, 10/2021 - 04/2022

Research Intern, ByteDance, Shanghai, 07/2020 - 09/2021

Community Services

Workshop/Challenge Organizations

Co-organizer of the special session: “Generative AI for Media Generation” at 2024 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), London, UK.
Co-organizer of the IEEE 2024 ICME Grand Challenge “Semi-supervised Acoustic Scene Classification under Domain Shift”

Talks

2024: MIT CSAIL Spoken Language Systems group (16 May), UK Acoustic Network (6 March), Spotify (20 June), Télécom Paris Listen Lab (17 Oct), Shanghai Jiao Tong University (9 Nov)
2023: NetEase (17 Feb), TikTok (22 Feb), ByteDance (24 Feb), Mila - Quebec AI Institute (26 Feb), Chinese Academy of Science (14 April), Remin University of China (4 Oct), University of Cambridge (11 May), Huawei Helsinki R&D (11 Oct), Meta FAIR (1 Nov)

Conference Oral Presentations

2024 Conference on Neural Information Processing Systems
2024 IEEE International Conference on Acoustics, Speech, and Signal Processing
2023 IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events
2021: A Tutorial on Music Source Separation, ISMIR Music Demixing Workshop, Alexandre Défossez, Woosung Choi, Haohe Liu (12 Nov)

Reviewer Services

I serve as a regular reviewer for the following journals:

IEEE/ACM Transactions on Audio Speech and Language Processing
IEEE Transactions on Multimedia
IEEE Transactions on Circuits and Systems for Video Technology
IEEE Transactions on Neural Networks and Learning Systems
International Journal on Information Fusion
Computer Speech & Language

I also serve as a reviewer for the following conferences:

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2024, 2025
INTERSPEECH 2023, 2024, 2025
International Conference on Learning Representations (ICLR) 2025
Conference on Neural Information Processing Systems (NeurIPS) 2024
ACM MultiMedia 2023, 2024
IEEE International Joint Conference on Neural Networks (IJCNN) 2025
IEEE International Conference on Multimedia & Expo (ICME) 2024, 2025
IEEE International Workshop on Machine Learning for Signal Processing (MLSP) 2024

Competitions

First place, DCASE Challenge Task 7:Foley Sound Synthesis, 2022
Yi Yuan, Haohe Liu, Xubo Liu, Xiyuan Kang, Mark D.Plumbley, Wenwu Wang
[Paper] [Demo] [Code] [leaderboard]

Second place, DCASE Challenge Task 5:Few-shot Bioacoustic Event Detection, 2022
Haohe Liu, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Wenwu Wang, Mark D. Plumbley
[Paper] [Code] [leaderboard]

Second place on vocal score and fifth place on overall score, ISMIR Music Demixing Challenge, 2021
Haohe Liu and Qiuqiang Kong and Jiafeng Liu
[Paper] [Code] [Challenge details] [leaderboard]

Second place, DCASE Challenge Task 6B: Language-Based Audio Retrieval, 2022
Xinhao Mei, Xubo Liu, Haohe Liu, Jianyuan Sun, Mark D. Plumbley, Wenwu Wang
[Report] [Challenge details]

Third place, DCASE Challenge Task 6A: Automated Audio Captioning, 2022
Xinhao Mei, Xubo Liu, Haohe Liu, Jianyuan Sun, Mark D. Plumbley, Wenwu Wang
[Report] [Challenge details]

Honors & Awards

Scholarships

2024 Vice-Chancellor’s Award for Teaching and Research Excellence - School Post Graduate Researcher of the Year - University of Surrey
Judges’ Award - DCASE 2023, Tampere, Finland
Doctoral College Scholarship, tuition waiver with stipend, University of Surrey
Outstanding Graduate Students, Northwestern Polytechnical University, 2020-2021
WuYaJun third class scholarship, 2019
National Scholarship, Northwestern Polytechnical University (top 1.5%), 2017 and 2018, two consecutive years
First-class Scholarship, Northwestern Polytechnical University (top 14.7%), 2016 to 2019, three consecutive years
Gratitude to Scientists of Modern Times National Scholarship (top 0.5%), 2018

Teaching

Guest Lecturer, EEEM068 Applied Machine Learning, University of Surrey, 2024
Demonstrator, EEE3008 Fundamentals of DSP, University of Surrey, 2022/23 Semester 1
Demonstrator, EEE1033 Computer and Digital Logic, University of Surrey, 2022/23 Semester 1
Demonstrator, EEEM068 Applied Machine Learning, University of Surrey, 2022/23 Semester 2

Tweets by LiuHaohe

–>