This is an introduction to EAI 820 — Speech and Language Technologies, an elective in the MSc programme in Embedded Artificial Intelligence. Its purpose is to set out what the subject is, why it is difficult, what vocabulary you will need, and how the topics named in the course outline fit together into a coherent whole.

The course outline lists its contents as a sequence of technologies: speech synthesis and the machinery behind it; speech recognition; language recognition; automatic translation, summarizers, spell checkers and grammar checkers; and the development of mobile speech and language applications. Read as a list, these look like eight or nine separate subjects. They are not. They rest on a small set of shared ideas, and a good deal of this chapter is spent identifying those ideas, so that the rest of the course can proceed by variation rather than by starting over each week.

What This Course Is About

Language is the interface humans were built for. We acquire it without instruction, use it without effort, and route almost all of our knowledge through it. It is therefore unsurprising that getting machines to handle language has been a goal of computing since before there were computers to speak of — and that it has turned out to be one of the hardest things we have ever asked machines to do.

The field has two historically distinct halves that have now largely merged. Natural language processing deals with language as text: segmenting it, analysing it, translating it, generating it. Speech processing deals with language as sound: recognising it, synthesising it, identifying who is speaking and in what language. For fifty years these were separate communities with separate conferences and separate mathematics. Today they share an architecture, a training strategy, and increasingly a single model.

For a student of embedded artificial intelligence, the convergence matters for a specific and practical reason. A single architecture that handles audio, text and their combination is an architecture you can compress, quantise and deploy once, rather than a collection of task-specific components each needing its own optimisation. The engineering question throughout this course is therefore not only “how does this work?” but “what does it cost, and can it run on the device in front of me?” That second question is what distinguishes EAI 820 from a general course in language technology, and it is the question the final weeks are built around.

Figure 1.1.  The human–machine language loop, with the week of EAI 820 in which each stage is treated. This course covers the four inner stages; the outer two are hardware.

Figure 1.1 shows the loop that organises the whole subject. Sound enters, becomes words, becomes meaning, becomes new words, becomes sound again. Almost every technology named in the course outline is a component of that loop, a shortcut across it, or a variation on it. A dictation application runs only the left half. A screen reader runs only the right half. A speech-to-speech translator runs the whole loop with the language changing in the middle. A wake-word detector is a tiny, cheap, always-listening fragment of the leftmost stage whose only job is to decide when to wake the rest of the system up.

Keep the loop in mind as a filing system. When a new technique appears in a later week, the first question worth asking is which stage it belongs to and what it consumes and produces.

Downloads

1. Lecture Note

2. Presentation

3. Main Text: Speech and Language Processing