Biography

Dr. Liu received his Ph.D. degree in Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology (HKUST) in 2002. Dr. Liu Joined Tsinghua University as an Associate Professor in Nov.2006. He is now the administrative deputy director of Center of Speech and Language Technologies of Research Institute of Information Technology, Tsinghua University. Before that, he was in Department of Electronic and Computer Engineering, HKUST, Hong Kong, from Aug.2002, where he was Postdoctoral Research Associate, Visiting Assistant Professor and Research Fellow of the Human Language Technology Center. Dr. Liu has published over 60 refereed papers in international leading journals and key conferences in the areas of speech recognition, audio processing, multimedia, and data mining and resources. He is the inventor of about 10 pending patents. His current research interests are in the areas of speech recognition, intelligent media processing, speech and audio signal processing, data mining and processing, and Internet multimedia information retrieval. He also participated many activities in embedded systems and consumer electronics.

 

In 2000, Dr. Liu was the first Ph.D. student from the Great China to attend the 2000 Johns Hopkins University Workshop on Speech and Language Technologies, supported by the National Science Foundation (NSF) and Department of Defense of the US government. He was a core member in the “Pronunciation modeling of Mandarin causal speech” group. Dr. Liu has been leading and participating in many R&D projects since 2002. Starting from 2004, Dr. Liu’s group has cooperated with the Linguistic Data Consortium (LDC). His group was acknowledged by LDC as a sole partner in China. His group and LDC finished the first large-scale Chinese telephony conversational speech corpus in the world. The corpus now is widely used by universities, research institutes and National Institute of Standards and Technology (NIST) in the world. Dr. Liu’s publications in the area of “accented speech recognition” ranked top 1 most cited paper by Google Scholar, and 6 of the top 10 most cited papers for “Mandarin pronunciation modeling”.

 

Dr. Liu is the member of IEEE Signal Processing Society, and International Speech Communication Association.

 

 

Research

 

Research interests

·      Speech recognition

·      Intelligent media processing

·      Speech and audio signal processing

·      Data mining and processing

·      Internet multimedia information retrieval

·      Embedded systems and consumer electronics

 

Current projects

·      “Adaptive and Layered Pronunciation Modeling for Mixed Accented Speech Recognition”, National Natural Science Foundation of China（国家自然科学基金）, 2010-2012.

·      “Acoustic and Phonetic Pronunciation Variations for Speech Recognition”, New Teacher Grant of Ministry of Education of China（教育部新教师基金）, 2010-2011.

·      “Integrated Linguistic Resources for Language Exploitation Technologies”, US DARPA Grant, 2008-2010.

·      “Content Based Perceptual Evaluation of Speech Quality for Telephone and Mobile Networks”, 2009-2010.

·      “Robustness in Multimodal UI”, 2008-2010.

·      “Content Based Perceptual Evaluation of Speech Quality for Telephone and Mobile Networks”, 2009-2010.

·      “Digital Information Security and Content Service Platform for High-end Oriented Service”, 2008-2010.

 

Recent Projects

·      “Speaker independent short phrase recognition and understanding”, 2008-2009

·      “Large Scale Query by Humming System”, 2007-2009.

·      “Digital Audio and Video Coding/Decoding System on SoC”, Guangdong-Hong Kong Technology Cooperation Funding Scheme, Dongguan Special Grant, No.20061681, 2006-2008.

·      “Integrated Linguistic Resources for Language Exploitation Technologies - Phase II”, supported by the Defense Advanced Research Projects Agency (DARPA), Linguistic Data Consortium (LDC) of the US government. 2007 -2008.

·      “Integrated Linguistic Resources for Language Exploitation Technologies”, supported by the Defense Advanced Research Projects Agency (DARPA), Linguistic Data Consortium (LDC) of the US government, No.5-45663-A. 2005 - 2006.

·      “Speech Data Collection: Lecture in Chinese (Mandarin)”, supported by Carnegie Mellon University and University of Karlsruhe. 2005－2006.

·      “Embedded Speech Recognition”, supported by the Innovation and Technology Fund, S/P845/04B, Innovation and Technology Commission, HKSAR. 2005 - 2006.

·      “EARS Chinese Telephony Conversational Speech Recognition and Database Collection”, supported by the Defense Advanced Research Projects Agency (DARPA), Linguistic Data Consortium (LDC) of the US government, PENN001.03/04. 2004 - 2005.

·      “Programming Interfaces with Audio Transcription and Data Mining Technologies”, supported by the Innovation and Technology Fund, S/P584/03A, Innovation and Technology Commission, HKSAR. 2003 - 2004.

·      “Using Boosting and Phonological Rules to Improve Acoustic Models for Accented Speech Recognition”, Direct Allocation Grant, HKSAR, 2003 - 2004.

 

Teaching

·      Principle of Signal Processing (Tsinghua University)

·      Signal and Systems (Hong Kong University of Science and Technology)

·      Digital Signal Processing (Hong Kong University of Science and Technology)

 

Students

·      Wenxiao CAO (Master Student, 2007): Segmental Information Based Matching Algorithm for Query by Humming

·      Jue HOU (Master Student, 2008): Chinese Accent Identification Based on Multiple Features

·      Chao ZHANG (Master Student, 2009)

 

To Prospective Graduate Students and Postdoctoral Research Associates

I am currently recruiting well motivated and dedicated graduate students and Postdoctoral Research Associates in the speech and audio signal processing, speech recognition, intelligent media processing, internet multimedia information retrieval, and embedded systems areas.

Please contact with me if you have interest in those research areas.

 

Publications

[2011]

·      Chao ZHANG, Yi LIU, Chin-Hui Lee, “Detection-based Accented Speech Recognition Using Articulatory Features”, in Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Hawaii, USA, 2011.

·      Jue HOU, Yi LIU, Chao ZHANG, Shilei HUANG, “An in-car Chinese Noisedatabase for Speech Recogni-tion”, in Proceedings of the International Conference on Asian Language Processing (IALP), Penang, Malaysia, 2011.

·      Chao ZHANG, Yi LIU, Thomas Fang ZHENG, “Asymmetric Acoustic Model for Accented Speech Recogni-tion”, in Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Xi’an, China, 2011.

·      Chao ZHANG, Yi LIU, Thomas Fang ZHENG, "Acoustic Model Reconstruction for Multi-accent Chinese Speech Recognition ", Journal of Tsinghua University (Science and Technology), 2011 (in Chinese).

·      Ying LI, Pascale FUNG, Ping XU, Yi LIU, "Asymmetric Acoustic Modeling of Mixed Language Speech", in Proceedings of the IEEE International Conference on Acoustic, Speech, and Signal Processing (ICASSP), Prague, Czech Republic, 2011.

·      Chao ZHANG, Yi LIU, Yunqing XIA, Thomas Fang ZHENG, Jesper OLSEN, Jilei TIAN, "Reliable Accent Specific Unit Generation with Dynamic Gaussian Mixture Selection for Multi-accent Speech Recognition", in Proceedings of the IEEE International Conference on Multimedia & Expo (ICME), Barcelona, Spain, 2011.

 

[2010]

·      Weifeng Su, Jiying Wang, Fred Lochovsky and Yi Liu. "Combining Tag and Value Similarity for Data Extraction and Alignment". IEEE Transaction on Knowledge and Data Engineering (TKDE), pp. 319-322, 2010.

·    Jue HOU, Yi LIU, Thomas Fang ZHENG, Jesper OLSEN, Jilei TIAN, "Multi-layered Features with SVM for Chinese Accent Identification", in Proceedings of the International Conference on Audio, Language and Image Processing (ICALIP), 2010.

·      Jue HOU, Yi LIU, Thomas Fang ZHENG, Jesper OLSEN, Jilei TIAN, "Using Cepstral and Prosodic Features for Chinese Accent Identification", in Proceedings of the International Symposium on Chinese Spoken Language Processing (ISCSLP), 2010.

·    Qibo Liu, Yi LIU, et al, "Combining Sub-bands SNR on Cochlear Model for Voice Activity Detection", in Proceedings of the International Conference on Asian Language Processing (IALP), 2010.

·      Shilei, HUANG, Jing WANG, Yi LIU, "Improvement of PESQ based on UVS classification and syllable stability detection", in Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2010), 2010.

 

[2009]

·      Wenxiao CAO, Danning JIANG, Jue HOU, Yong QIN, Shilei HUANG, Yi LIU "A Phrase-level Piecewise Linear Scaling Algorithm for Melody match in Query-by-Humming Systems", in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 2009.

·      Jue Hou, Danning JIANG, Wenxiao CAO, Yong QIN, Shilei HUANG, Yi LIU "Effectiveness of N-Gram Fast Match for Query-by-Humming Systems", in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 2009.

·     Wenxiao CAO, Yi LIU, Thomas Fang ZHENG, Danning JIANG, Yong QIN, "Linear scailing based dynamic programming algorithm for accurate matching in QBH", Journal of Tsinghua University(Science and Technology)，Vol. 49, No. S1, pp.1402-1407, 2009 (in Chinese).

·      Jue HOU, Yi LIU, Thomas Fang ZHENG, Danning JIANG, Yong QIN, Shilei HUANG, Yong LIU, "VP-tree based multi-stage matching algorithm for query-by-humming systems", Journal of Tsinghua University (Science and Technology), Vol. 49, No. S1, pp.1419-1424, 2009 (in Chinese).

·     Yi LIU, Yongsheng YANG, Yunqing XIA, Shilei HUANG, Wenxiao CAO, "Pronunciation Modeling for Nonstandard Speech Using Phonetic Feature Distance and Optimal Gaussian Mixture Sharing ", 2009 National Conference on Man-Machine Speech Communication (NCMMSC 2009), 2009.

 

[2008]

·      Wenxiao CAO, Yi LIU, Thomas Fang ZHENG, "Local Mismatch Phone for Confidence Measure in Standard and Accented Chinese Speech Recognition", in Proceedings of the International Symposium on Chinese Spoken Language Processing (ISCSLP), Kunming, China, 2008.

 

[2007]

·    Y. Liu, F.Zheng, L.He and Y.Xia, "State-Dependent Mixture Tying with Variable Codebook Size for Accented Speech Recognition," in Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Kyoto, Japan, Dec.2007.

·     LIU Yi, HE Lei, ZHENG Fang,"Modeling Sound Changes in Mandarin Spontaneous Speech Using Deleted Interpolation of Mixture Component Weights ", 2007 National Conference on Man-Machine Speech Communication (NCMMSC 2007), 2007.

·    Yunqing Xia, Jianxin Wang, Fang Zheng and Yi Liu, "A Binarization Approach to Email Categorization using Binary Decision Tree", In Proceedings of ICMLC-2007. 19-22 Aug. 2007. Hong Kong, China.

 

[2006]

·    Y. Liu, P. Fung, Y.S. Yang, C. Cieri, S.D. Huang and D. Graff, "HKUST/MTS: A very large scale Mandarin telephone speech corpus". Lecture Notes in Computer Science, Springer, LNAI, pp.724-735, 2006.

·    P. Fung and Y. Liu, "Spontaneous Mandarin Speech Pronunciation Modeling". Advances in Chinese Spoken Language Processing, World Scientific Publisher, 2006.

·    Y. Liu and P. Fung, "Multi-Accent Chinese Speech Recognition", In Proceedings of the International Conference on Spoken Language Processing, Pittsburgh PA, USA, 2006.

 

[2005]

·    P. Fung and Y. Liu, "Effects and Modeling of Phonetic and Acoustic Confusions in Accented Speech Recognition". Journal of the Acoustical Society of America, Vol.118, Issue 5, pp.3279 - 3293, Nov. 2005

·    Y. Liu, P. Fung, Christopher Cieri, Yongsheng, Yang and Shudong Huang, "EARSCTS: A Chinese Telephony Conversational Corpus for Speech Processing", NCMMSC2005, Technical Acoustics, Vol. 24, pp.486 - 489, 2005.

·    Y. Liu and P. Fung, "Acoustic and Phonetic Confusions in Accented Speech Recognition", in Proceedings of the European Conference on Speech Communication and Technology, Lisbon, Portugal, 2005.

 

[2004]

·    Y. Liu and P. Fung, "State-Dependent Phonetic Tied Mixtures with Pronunciation Modeling for Spontaneous Speech Recognition" IEEE Transactions on Speech and Audio Processing, Vol.12, No.4, pp.351-364, July 2004

·    Y. Liu and P. Fung, "Pronunciation Modeling for Spontaneous Mandarin Speech Recognition," International Journal of Speech Technology, Vol. 7, No. 2-3, pp. 155-172, 2004

·    Y. Liu, P. Fung, et al., "Development of a Chinese Telephony Conversational Corpus for Speech Processing," in Proceedings of the International Symposium on Chinese Spoken Language Processing, Hong Kong, December 2004.

·    C. Xu, Y. Liu, et al., "A System for Mandarin Short Phase Recognition on Portable Devices," in Proceedings of the International Symposium on Chinese Spoken Language Processing, Hong Kong, December 2004.

·    P. Fung, Y. Liu, Y.S. Yang, Y.H. Shen and D.K. Wu, "A Grammar-based Chinese to English Speech Translation System for Portable Devices," In Proceedings of the International Conference on Spoken Language Processing, Jeju island, Korea 2004.

 

[2003]

·    Y. Liu and P. Fung, "Modeling Partial Pronunciation Variations for Spontaneous Mandarin Speech Recognition," Computer Speech and Language, Vol.17, No.4, pp. 357-379, October 2003. (Top 20 downloaded and cited paper.)

·    Y. Liu and P. Fung, "Partial Change Accent Models for Accented Mandarin Speech Recognition," in Proceedings of the IEEE Automatic Speech Recognition and Understanding, St. Thomas, U.S. Virgin Islands, December, 2003.

·    Y. Liu and P. Fung, "Automatic Phone Set Extension with Confidence Measure for Spontaneous Speech," in Proceedings of the European Conference on Speech Communication and Technology, Geneva, September 2003.

·    P. Fung and Y. Liu, "Triphone Model Reconstruction for Mandarin Variations," in Proceedings of the IEEE International Conference on Acoustic Speech and Signal Processing, Hong Kong, April 2003.

 

[2002]

·    Y. Liu and P. Fung, "Model Partial Pronunciation Variations for Spontaneous Mandarin Speech Recognition," In Proceedings of the International Conference on Spoken Language Processing, Denver, Colorado, September 2002.

·    Y. Liu and P. Fung, "Partial Change Phone Models for Pronunciation Variations in Spontaneous Mandarin Speech," in Proceedings of the International Symposium on Chinese Spoken Language Processing, Taipei, Taiwan, August 2002.

 

[2001]

·    Y. Liu and P. Fung, "Estimating Pronunciation Variations from Acoustic Likelihood Score for HMM Reconstruction", in Proceedings of the European Conference on Speech Communication and Technology, Aalborg, Denmark, September 2001.

·    Y. Liu, P. Fung, W. Byrne and U. Ruhi, "Pronunciation Modeling of Spontaneous Mandarin Speech Using Phonetic Feature Distance and Optimal Gaussian Mixture Sharing", in Proceedings of the IEEE International Conference on Acoustic Speech and Signal Processing, Salt Lake City, USA, April 2001. (Student Paper.)

·    W. Byrne, V. Venkataramani, T. Kamm, T.F. Zheng, Z. Song, P. Fung, Y. Liu, U. Ruhi, "Automatic Generation of Pronunciation Lexicons for Mandarin Spontaneous Speech", in Proceedings of the IEEE International Conference on Acoustic Speech and Signal Processing, Salt Lake City, USA, April 2001.

 

[2000]

·    Y. Liu and P. Fung, "Modeling Pronunciation Variations in Spontaneous Mandarin Speech", in Proceedings of the International Conference on Spoken Language Processing, Beijing, China, October 2000.

·    Y. Liu and P. Fung "Rule-based Word Pronunciation Networks Generation for Mandarin Speech Recognition", in Proceedings of the International Symposium on Chinese Spoken Language Processing, Beijing, China, October 2000.

·    P. Fung, W. Byrne, F. Zheng, T. Kamm, Y. Liu, Z. Song, V. Venkataramani, U. Ruhi, "Pronunciation Modeling of Mandarin Casual Speech", 2000 Summer Research Workshop Technical Reports, Johns Hopkins University, 2000.

 

[1999]

·    Y. Liu and P. Fung, "Decision Tree-based Triphones are Robust and Practical for Mandarin Speech Recognition", in Proceedings of the European Conference on Speech Communication and Technology, Budapest, Hungary, September 1999.

 

[1998]

·    Y. Liu, C.F. Wang B.Q. Dai and H. Li, "Chinese Speech Synthesis Based on Wavelet Transform" Journal of Mini-Micro Systems, Vol.19, No.3 March 1998.

·    Y. Liu, C.F. Wang "A Scheme for PSOLA Based on Wavelet Transform" Journal of China University of Science and Technology, Vol.28, No.4 August 1998.

·    C.F. Wang, B.Q Dai and Y. Liu, "A Scheme for High Quality Linear Prediction Analysis of Speech" Journal of China University of Science and Technology, Vol.19, No.3 March 1998.

 

Professional Activities

 

Paper Reviewers (Selected)

·      IEEE Transactions on Audio, Speech and Language Processing

·      IEEE Transactions on Intelligent Transportation Systems

·      Machine Learning Journal

·      IEICE Transactions on Communications

·      ACM Transactions on Asian Language Information Processing

·      IEEE International Conference on Acoustic Speech and Signal Processing

·      The International Conference on Spoken Language Processing

·      The European Conference on Speech Communication and Technology

·      The International Symposium on Chinese Spoken Language Processing

 

Honors and Awards (Selected)

·      Research Travel Grant, HKUST, 2003.

·      Eurospeech 2001 Awards, International Speech Communication Association (ISCA), 2001.

·      The US Department of Defense Fellowship, 2000.

·      Research Travel Grant, HKUST, 2000.

·      Research Travel Grant, HKUST. 1999.

·      Japanese Communication Network Scholarship, the University of Science and Technology of China, 1998.

·      Huawei Scholarship, the University of Science and Technology of China, 1997.

 

Last updated: Aug.22, 2011