Prof. Biing-Hwang Fred Juang

Title

Accretionary Learning with Deep Neural Networks

Abstract

One of the fundamental limitations of Deep Neural Networks (DNN) is their inability to acquire and accumulate new cognitive capabilities. When data from new object classes appear, a conventional DNN would not be able to recognize them due to its formulation of the recognition problem, which is predicated by the prescribed number of classes. The current solution is typically to re-design and re-learn the entire network, perhaps with a new configuration, from a newly expanded dataset to accommodate new knowledge. This process is quite different from that of a human learner. For example, when a child learns to recognize the 10 digits, the child is not foretold to prepare to learn exactly 10 digits; almost without exception, children learn one digit a time in an accretionary manner. In this talk, we describe a new learning method named Accretionary Learning (AL) to emulate human learning, in that the set of objects to be recognized may not be pre-specified. The corresponding learning structure is modularized, which can dynamically expand to register and use new knowledge. During accretionary learning, the learning process does not require the system to be totally re-designed and re-trained as the set of object classes grows. The proposed DNN structure does not forget previous knowledge when learning to recognize new data classes. We show that the new structure and the design methodology led to a system that can grow to cope with increased cognitive complexity while providing stable and superior overall performance.

Biography

Biing-Hwang (Fred) Juang is the Motorola Foundation Chair Professor and a Georgia Research Alliance Eminent Scholar at Georgia Institute of Technology. He is also enlisted as Honorary Chair Professor at several renowned universities. He received a Ph.D. degree from University of California, Santa Barbara. He conducted research work at Speech Communications Research Laboratory (SCRL) and Signal Technology, Inc. (STI) in late 1970s on several Government-sponsored research projects and at Bell Labs during the 80s and 90s until he joined Georgia Tech in 2002. Prof. Juang’s notable accomplishments include development of vector quantization for voice applications, voice coders at extremely low bit rates (800 bps and ~300 bps), robust vocoders for satellite communications, fundamental algorithms in signal modeling for automatic speech recognition, mixture hidden Markov models, discriminative methods in pattern recognition and machine learning, stereo- and multi-phonic teleconferencing, and a number of voice-enabled interactive communication services. He was Director of Acoustics and Speech Research at Bell Labs (1996-2001).

Prof. Juang is well known for his work in data and signal modeling for compression, recognition, verification, enhancement, physical and statistical analyses, secure communication, and synthesis. He is accredited with the original concept of signal modeling for discrimination which serves as an important guiding principle in deep learning.

Prof. Juang has published extensively, including the book “Fundamentals of Speech Recognition”, co-authored with L.R. Rabiner, and holds about two dozen patents. He received the Technical Achievement Award from the IEEE Signal Processing Society in 1998 for contributions to the field of speech processing and communications and the Third Millennium Medal from the IEEE in 2000. He also received two Best Senior Paper Awards, in 1993 and 1994 respectively, and a Best Paper Awards in 1994, from the IEEE Signal Processing Society. He served as the Editor-in-Chief of the IEEE Transactions on Speech and Audio Processing from 1996 to 2002. He was elected an IEEE Fellow (1991), a Bell Labs Fellow (1999), a member of the US National Academy of Engineering (2004), and an Academician of the Academia Sinica (2006). He was named recipient of the IEEE Field Award in Audio, Speech and Acoustics, the J.L. Flanagan Medal, and a Charter Fellow of the National Academy of Inventors (NAI), in 2014. He is also a co-recipient of the 2022 Fred W. Ellersick Prize of the IEEE Communications Society.