An Introduction to Sine-Wave Speech

MRC Cognition and Brain Sciences Unit
Chaucer Road
Cambridge CB2 7EF.

1) Introduction:

Sine-wave speech is a form of artificially degraded speech first developed at Haskins Laboratory . Several seminal experiments on the perception of sine-wave speech are described here:

Remez, R.E., Rubin, P.E., Pisoni, D.B., Carrell, T.D. (1981) Speech perception without traditional speech cues. Science, 212, 947-9. PubMed

In this work, Remez and colleagues demonstrated a dramatic change in the way in which sine-wave speech sentences are perceived, depending on listener's specific prior knowledge. For instance, listen to this sound:

Most naive listeners hear this as a set of simultaneous whistles, or science fiction sounds. However, for listeners that have previously heard this sound:

Listening to the sine-wave speech sound again produces a very different percept of a fully intelligible spoken sentence. This dramatic change in perception is an example of "perceptual insight" or pop-out. We have argued that this form of pop-out is an example of a top-down perceptual process produced by higher-level knowledge and expectations concerning sounds that can potentially be heard as speech:

Davis, M.H., Johnsrude, I.S. (2007) "Hearing speech sounds: Top-down influences on the interface between audition and speech perception." Hearing Research, 229(1-2), 132-147.PDF.

There are four more example pairs of sine-wave and clear speech in the table below:

As you listen to these four examples, you may find that you get better at understanding the sine-wave speech first time around. This is an example of perceptual learning. Having heard several examples of sine-wave speech, your perceptual system has tuned into this form of distortion, so as to be able to perceive new sine-wave speech sentences more clearly.

To my knowledge, no one has done controlled experiments to demonstrate that pop-out helps with learning sine-wave speech. However, for another form of distortion (noise-vocoded speech), we have shown that pop-out enhances perceptual learning so that people more rapidly learn to understand new distorted sentences. These experiments withvocoded speech suggest that perceptual learning is also a top-down process:

Davis, M.H., Johnsrude, I.S., Hervais-Adelman, A., Taylor, K. & McGettigan, C.M. (2005) Lexical information drives perceptual learning of distorted speech: Evidence from the comprehension of noise-vocoded sentences. Journal of Experimental Psychology: General, 134 (2), 222-241.PDF.

2) Generating Sine-Wave Speech:

Sine-wave speech is generated by using a formant tracker to detect the formant frequencies found in an utterance, and then synthesising sine waves that track the centre of these formants. This is illustrated in the figures below:

A number of pieces of software exist for generating sine-wave versions of utterances. These sentences shown above were generated using Praat software and ascript written byChris Darwin. There's also Matlab code to generate sine-wave speech written by Dan Ellis .

3) Other forms of perceptual insight:

There are a number of examples of perceptual insight in the visual domain that have been documented. For instance, turning grey-scale images into high contrast, black/white images can produce a similar phenomenon to sine-wave speech. This manipulation was originally described by Craig Mooney (1957).

Click on the image to receive a visual hint about the content of this image. This form of visual perceptual insight is discussed in greater detail by Nava Rubin in this paper:

Rubin, N., Nakayama, K. and Shapley, R. (2002), The role of insight in perceptual learning: evidence from illusory contour perception. In: Perceptual Learning, Fahle, M. and Poggio, T. (Eds.), MIT Press.

Which is where the example image above comes from. I'd be keen to hear of forms of perceptual insight, in other sensory modalities.

4) Media reports of this work:

New Scientist , Mind Hacks , Boing Boing

This page was last updated on 24th November 2007. Comments and suggestions to matt.davis @ mrc-cbu.cam.ac.uk.