kCCA Based Audio-Visual Speech Recognition

In recent years significant amounts of research time has been expended examining the possibility of using facial visual information to improve the poor performance of automatic (audio only) speech recognition (ASR) in acoustically noisy environments [2]-[3]. The major direction of these efforts is to seek to classify speech articulator positions and from these to form sets of articulatory features. Articulatory feature sets are employed in collaboration with various forms of Bayesian Networks [2] and Hidden Markov Models [3] to accomadate for the asynchronisation between speech production and articulator movement.