Minuted transcribes meetings on your own machine. No upload, no cloud transcription service, no bot in the call — the audio stays where it was recorded. That is the whole point of the product, and it means the speed of transcription is our problem rather than someone else’s data centre’s.
For months, ours was bad. A five-minute meeting took about five minutes to transcribe. Long meetings were worse than useless: you finished the call, pressed stop, and waited.
We assumed the model was simply heavy. That assumption was wrong in an embarrassing and instructive way.
The measurement
The first useful thing we did was stop guessing and run the engine directly, on a real recording, on a twenty-core desktop. whisper.cpp prints its own timings, which is more than we were doing.
| Threads | Time for 333s of audio | Versus realtime |
|---|---|---|
| 20 | 699.6s | 2.10× slower |
| 4 | 33.8s | 9.8× faster |
Same audio. Same model. Same beam search. Byte-identical transcripts — same words, same timestamps, no fallbacks in either run. The only difference was how many CPU threads we asked for, and asking for fewer made it twenty times faster.
The line that did it
Our code contained this, with a comment explaining itself:
whisper.cpp defaults to four threads. We read that as an oversight and corrected it. It is not an oversight; it is tuned, and the correction cost us a factor of twenty.
The per-operation timings say why. Whisper’s decoder is six layers of 512-wide matrices — genuinely small pieces of arithmetic. Split twenty ways, each thread’s share of the work is smaller than the cost of synchronising with the other nineteen, so they spend their time waiting at barriers instead of computing.
| Phase | 20 threads | 4 threads |
|---|---|---|
| Batched decode | 114.30 ms/run | 2.23 ms/run |
| Decode | 2122.48 ms/run | 5.87 ms/run |
| Encode | 2167.58 ms/run | 1178.54 ms/run |
Even the encoder — the half of the model that genuinely does parallelise, and the part that actually processes the audio — ran faster on four threads than on twenty.
Why it hid for so long
Because it looks exactly like its own opposite.
A powerful machine transcribing slowly reads as “this model is expensive”. It does not read as “we asked for too much”. Every instinct it produces is the wrong one: use a smaller model, accept worse accuracy, add GPU support, tell users their laptop is not up to it. Any of those would have shipped, would have helped a little, and would have built permanently around a one-line bug.
It also never errored. Nothing crashed, nothing logged a warning, no test failed. The transcripts were correct. The only symptom was a number nobody had written down, because we had never measured the thing we were about to make decisions about.
What it unblocked
We had removed live transcription from the Windows app because it “could not keep up”. That judgement was made against an engine running at 0.48× realtime. At 9×, it is not close.
So it is back — and better than it was, because the speed also paid for something we could not previously afford. The live transcript is now a preview: words appear while the meeting runs, and when you stop, the whole recording is transcribed again properly, with acoustic speaker separation that a chunk-at-a-time pass cannot produce. Before, a second full pass would have cost twice the length of the meeting. Now it costs a fraction of it.
The part worth taking away
The fix was four characters. Finding it took running the engine once, by hand, and reading what it printed.
We had a benchmark harness. We had a scorer. What we did not have was a single number for “how fast does this machine actually transcribe” — so the app now measures itself on every transcription and shows you the answer in Settings. Not because users need it, but because we did, and we would not have known to look.
If a default in a mature library looks conservative, it is worth finding out what it was measured against before improving on it.