§1The problem
The earbuds' built-in live translation has flaws (it closes, it doesn't always speak into the earbuds). There was no app with the lowest possible latency that works with the other person without them installing anything.
§2What I built
- PlansProduct and technical plans (billing by minute packs, balance as a statement, login).
- PrototypeApproved screen prototype, with light and dark themes from my own design system.
- Test benchCompared translation engines with real audio, leading to the choice of primary and fallback engines.
- ServerCloudflare Worker + D1 database, still a draft, running on my home server until the app reaches the store.
- Guest pageA web page the other person joins by QR code.
- AppSwiftUI (iPhone/iPad/Watch in sync), built on the Mac; the app is ready.
§3The hard parts
5 entries- 01
Measuring latency and quality of different engines
Speech-to-speech versus listen + translate + speak, on a bench with the same input.
- 02
Playing only the right direction
A second service detects who is speaking and filters.
- 03
Capture through the earbuds' microphone
Only one microphone works at a time, yet the other person must be captured too.
- 04
Real-time sync across iPhone, iPad and Watch
And support for two phones, each in its own language.
- 05
Minute balance as a server-side statement
So subscriptions can come later without a rewrite.
§4Result
The engine comparison is done and the engine is chosen, with the results written down. The backend is still a draft and runs on my home server for now. The iPhone/iPad/Watch app is ready, but it isn't in the stores yet and has no users.
§5Stack
- Swift / SwiftUI
- Cloudflare Workers + Durable Objects + D1
- Real-time speech translation API
- Speech-to-text and text-to-speech services (fallback)
- Node
- pm2
- Tailscale (temporary)