Uhm: On-device Filler-Word Detection for Audio and Video
On-device filler-word detection that finds 'um', 'uh', 'hmm', and other fillers straight from the waveform, with one prediction every 20 milliseconds.
Uhm is a frame-precise classifier that finds fillers directly in audio, one prediction every 20 milliseconds. It reads the waveform, so there is no transcript to run first, which is the usual cost of finding disfluencies.
It was trained on English and transfers acoustically to Spanish, French, German, and Dutch without retraining. A transcript would not even help: models like Whisper leave fillers out of their output, so the ums never appear in the text to find or cut. Uhm analyzes the audio and detects them on device, with no upload and no per-minute cost.
Demo
Performance
Scans audio at 296x realtime, on device, with a prediction every 20 ms.
Realtime factor (audio duration over analyze time), fp16 Core ML, warm (internal)
| Device | Realtime factor |
|---|---|
| iPhone 17 Pro | ~296x |
| iPad Pro (M4) | ~279x |
| iPhone 15 Pro | ~169x |
Internal benchmarks. Trained on English; transfers acoustically to Spanish, French, German, and Dutch.
Use cases
Podcast and video editing
Locate every filler to cut or tighten, without running a transcription pass first.
Transcription cleanup
Flag disfluencies acoustically so downstream transcripts read cleanly.
No per-minute cost
Detect fillers on device instead of gluing WhisperX plus regex or paying per-minute cloud speech-to-text.
Speaking feedback
Measure filler frequency in recorded speech for practice and coaching tools.
What it does
- Frame-precise: one prediction every 20 ms
- Acoustic detection: works directly on the waveform, no transcript required
Biaspreset: precision, balanced, or recall- Trained on English, transfers to Spanish, French, German, and Dutch without retraining
Platforms and install
// Swift Package Manager .package(url: "https://github.com/Desert-Ant-Labs/uhm-swift", from: "0.1.0")
import Uhm
let result = try await Uhm().analyze(audioURL: url)
for f in result.fillers { print(f.start, f.end, f.type ?? .other) }
Specs
- Resolution
- One prediction every 20 ms
- Approach
- Acoustic, transcript-free
- Languages
- English, transfers to ES, FR, DE, NL
- Privacy
- Runs on device; audio never leaves the device