Imagine you are sitting in a quiet room and someone says your name. Before you have even thought about it, you have turned your head. What happened in that half-second is, when you slow it down, almost miraculous. A puff of air left someone's mouth, wobbled the air between you, pressed gently on your eardrum, and your brain unscrambled that wobble into a word, a name, a meaning. Speech-to-text — STT, the thing that writes your voice notes for you — is our attempt to teach a machine that same trick. Let us walk through how it does it, slowly, the way you would explain a card trick to a curious child.
First, we have to catch the sound. Sound is not a thing; it is a motion. It is air being squeezed and stretched, squeezed and stretched, many hundreds of times a second. A microphone is just a tiny eardrum: a little membrane that gets pushed by those squeezes and turns the pushing into a wiggling electrical voltage. But a computer cannot hold a smooth, continuous wiggle. It can only hold numbers. So it does something called sampling — it measures the height of the wiggle very, very often, typically sixteen thousand times every second for speech, and writes down each measurement. That stream of numbers is your raw waveform. If you have ever seen that jagged blue mountain range in a voice-recording app, that is it. It is honest, but it is also a mess.
Why a mess? Because that waveform contains everything. Your voice, yes, but also the hum of the fridge, the hiss of the room, a car outside, the little click of your tongue. So the next stage is preprocessing — tidying up. We might gently remove very low rumbles and very high hisses that carry no speech. We often normalise the volume, so a whisper and a shout get treated more fairly. And then we do something that feels almost like cheating: we chop the audio into tiny overlapping slices, each about twenty-five milliseconds long, sliding along every ten milliseconds. We call these frames. The idea is that speech changes fast, but not infinitely fast — within one little twenty-five-millisecond window, the sound is roughly steady, like a single held note. So we agree to look at speech one short note at a time.
Now comes my favourite part, because it is where the physics of waves earns its keep: feature extraction. A raw waveform, even one tiny frame of it, is hard to read directly. So we ask a different question. Instead of "how is the air moving moment to moment," we ask "what mixture of pitches is in this little slice?" Every sound is secretly a blend of pure tones — a low one, some middle ones, some high ones — stacked together like ingredients in a soup. There is a beautiful mathematical sieve called the Fourier transform that takes a slice of sound and tells you exactly how much of each pitch is in it. Run it on every frame, line the results up side by side, and you get a spectrogram: a picture of sound, with time running left to right and pitch running bottom to top, brightness showing how much energy sits at each pitch. Vowels become broad glowing bands; an "s" becomes a bright high smear. You can almost read it like sheet music once you get the eye for it.
We usually go one step further and compute MFCCs — mel-frequency cepstral coefficients, which is a terrifying name for a sensible idea. "Mel" just means we rescale the pitches to match human hearing, because our ears care far more about the difference between low pitches than between very high ones. The rest of the recipe squeezes that mel-spectrogram down into a dozen or so tidy numbers per frame that capture the shape of the sound — the position of your tongue and lips, more or less — while throwing away things that do not matter, like the exact pitch of your particular voice. So now each frame is no longer thousands of raw samples; it is a small, meaningful fingerprint.
Here the machine has a list of fingerprints and needs to guess what was actually said. That is the job of the acoustic model. This is a neural network — a large pattern-matcher trained on enormous piles of recorded speech paired with their correct transcripts. Its job is humble but hard: for each frame, estimate which speech sound it probably is. The smallest units it works with are usually phonemes — the atoms of speech, like the "k," "a," and "t" in "cat." The acoustic model does not output letters; it outputs probabilities. "This frame is sixty percent likely to be an 'aaa,' thirty percent an 'eh,' ten percent silence." It is a confident guesser, but a guesser nonetheless, and it has no idea what words are.
That is why we bring in a second brain: the language model. The acoustic model hears sound; the language model knows language. It has read mountains of text and learned which words tend to follow which. So when the acoustic evidence is muddy — and it often is, because "recognise speech" and "wreck a nice beach" sound almost identical — the language model leans in and says, "given the words so far, one of those is far more plausible." It is the part that knows you probably did not say "I'll see you at the bored meeting."
Stitching these two voices together is decoding. Think of it as finding the single most likely path through an enormous branching maze of possibilities, where the acoustic model votes on sounds and the language model votes on words, and we search for the route that makes both happiest at once. Older systems used a careful graph-walking method; modern end-to-end systems often fold much of this into the neural network itself and use a clever scoring trick called CTC, or a sequence-to-sequence decoder, to handle the fact that we never know in advance exactly how many frames each letter will stretch across. The result, finally, is text — the words, hopefully with a little punctuation sprinkled on top.
Now let me be honest, because that is the only way I know how to teach. None of this truly understands you. It is pattern matching of breathtaking scale, not comprehension. It stumbles on accents it has not heard enough of, on names, on two people talking at once, on a noisy café. It will write "their" when you meant "there," because to the air, they are the same wiggle. Knowing this is not a disappointment; it is the beginning of using the tool wisely.
If you want to go further, I would start by recording yourself saying a single word and opening it in a free audio tool to stare at its spectrogram until the shapes start to mean something. That picture is the whole story, hiding in plain sight. From there, read about the Fourier transform and then about CTC — and you will find the machine's miracle is just our own, patiently rebuilt one frame at a time.