Join the Conversation
A desktop app that listens along to whatever your computer is playing. Hold a key, ask about what was just said, and get an answer without leaving the video, podcast or call.
- When
- 2026
- Role
- Design and engineering, on my own
- Status
- In development, not released yet
- Built with
- Electron, FoldKit, Effect, whisper.cpp, llama.cpp, Cloudflare

Asking the people on the podcast
When someone on a podcast or in a lecture says something you don’t follow, the usual fix is to pause, open a browser and try to describe something you only half heard. I wanted to be able to ask the conversation itself.
Join the Conversation keeps a rolling two-minute transcript of whatever your computer plays: a video, a course, an audiobook, a podcast or a meeting. Hold a key, ask out loud and let go, and the answer streams into a small bar at the bottom of the screen. It answers from what was just said, so it feels a bit like putting your hand up mid-conversation. Then you carry on listening.
One key to ask
You hold a key, ask and let go. It’s fn on a Mac and Ctrl with the Windows key on a PC, and you can change both. Double-tap it and it keeps listening, so you can talk for a while without holding anything. Esc cancels, with an Undo in case you hit it by accident.
A press longer than 400 ms is a question, and letting go sends it. A shorter one is treated as the start of a double-tap, and if no second tap comes it’s dropped without a message. Brushing the key should never ask the model anything.
Quiet until you need it
At rest the bar is a 32 by 6 pixel handle. It grows into a pill while you talk, says “Thinking…” once you let go, and opens into the answer panel when the first word arrives. There’s only ever one thing on screen. The bar never takes focus, and clicks pass straight through it unless there’s something to press, so whatever you were doing keeps working.
I studied a voice dictation app I like for this part, and measured its interface to get the sizes, motion and timings right. The icons, sounds and words are my own.
The start sound plays when the microphone is actually sending audio, not when you press the key, and the waveform is drawn from your real mic levels. If the bar says it’s listening, it is.
Answers are text, and you can have them read aloud. Each answer says how much it heard, like “Last 45 s · on this computer”. If that wasn’t enough, “Use more context” asks again with the last two minutes, and holding the key while an answer is open asks a follow-up.
Learning it with the real thing
There are no videos of the product pretending to work. Setup tests your real microphone on a live meter, asks you to hold your shortcut and checks it lights up, then gives you a short clip to ask about. The answer arrives in the real bar.

After that, tips are rationed. You get at most two a day, each one only once, and never while you’re recording. The Home checklist only ticks things off when you’ve actually done them, like asking a follow-up or trying hands-free.
Being clear about what it heard
An app that’s always listening has to be obvious about what it keeps. The second screen of setup asks where your data goes, and the default is that everything stays on your computer. Raw audio is never written to disk. Transcripts are kept for a day and questions for 30 days, and you can cut either down to the current session.

The Live transcript page shows exactly what it can answer about right now, labelled as automatic and possibly wrong. You can also type a question there instead of speaking.

Dressed in wine
I wanted it to feel quiet and a little luxurious, so I took the palette from this autumn’s fashion: berry, burgundy and merlot, worn against cream and black. The interface is mostly neutral and the wines are the accessories. Each one has a single job.
- BurgundyAction. One coloured button per screen.
#7f1b3a / #ac3a5c - BerrySignal: live, hands-free, focus and the tour.
#ae3368 / #d9588c - MerlotDepth: shadows, gradient ends and the mark.
#4d1420 / #7a2b3d - Porcelain and Black CherryThe neutrals, pulled about 1% towards wine.
#f6efee / #120d0e
Type follows the same idea. Instrument Serif is for things you look at once, like page titles, headlines and big numbers. Inter is for everything you read or press. The serif never goes below 20 pixels and never goes on a control.


All of it lives in StyleX tokens. A script generates the forced light and dark themes from them, so a colour is only ever defined once, and every window shares the same buttons, toggles, fields and key caps.
Mostly outside the browser
It’s an Electron app, but most of the work happens in other processes. A small native host captures audio and watches the shortcut. Speech and answer models run as local servers that the app starts, health-checks and restarts. The main process runs a single Effect application that owns all the state, and the windows just draw it.
- Control messages in, audio frames outNative audio hostSwift on macOS, C++ on Windows. Captures system audio, opens the mic while the key is held and watches the shortcut.
- HTTP on loopbackInference serverswhisper.cpp or Parakeet for speech and llama.cpp for answers, started on demand and restarted if they crash.
- Electron utilityProcessUtility processesSilero voice detection and the Kokoro voice, kept off the main thread.
- Typed IPC: requests in, state outWindowsThe bar, main window, tip card and tour, each a sandboxed FoldKit program.
The runtime only touches the outside world through ports, so the whole thing also runs against a simulated native host that speaks the real protocol. That means the tests run without models, a network or any audio hardware.
Recording everything except itself
If the app reads an answer aloud, it mustn’t then transcribe its own voice. The native host captures system audio with the app’s own processes excluded. On Windows that brought an excluded player down to −77.7 dBFS while everything else stayed at −20. A second check drops any transcript text that matches what the app just said, in case something slips through.
On Windows, opening the mic in the “communications” role tells Windows a call has started, and its default is to turn every other sound down by 80%, which is exactly the audio the app is trying to hear. So it uses the normal console role instead. And when your shortcut includes the Windows key, the host sends a masking key first so the Start menu doesn’t open every time you ask something.
Two passes over the same audio
A fast speech model transcribes system audio all the time. When you press the key, the app freezes what it heard up to that moment, and while you’re still talking it runs the slower, more accurate model over that stretch. When you let go it waits up to 2.5 seconds for the better text, and uses the fast version if it isn’t ready.
Then it picks which lines to send. By default that’s the last 45 seconds, rounded out to whole lines.
RECENT AUDIO (last 45s) [00:00–00:12] But regional banks fund themselves differently. Their deposits reprice faster than their loans. [00:13–00:21] Savers move their money the moment a better rate shows up. [00:22–00:33] Meanwhile most of their loans are fixed for years, so the income barely moves. [00:34–00:42] That gap is the squeeze. [00:43–00:49] So the real question is how long that squeeze lasts. USER Why would that hurt regional banks?
The system prompt tells the model the transcript may contain recognition errors, to keep what the speakers said separate from anything it adds, and never to invent a quote. If the audio doesn’t answer the question, it should say so.
Fast enough to feel like a reply
I set two targets: under a second from letting go to seeing your question, and under 2.5 seconds to the first word of the answer. An end-to-end test plays a real clip through the simulated host, speaks a real question into the fake mic and checks the answer mentions the right thing.
| Speech and answer models | Range | Fastest to slowest |
|---|---|---|
| Parakeet and Qwen3.5 4B, both on the GPU | 0.30–0.39 s | |
| whisper base on the CPU, Qwen3.5 4B on the GPU | 0.60–0.69 s | |
| whisper base and Qwen3.5 2B, both on the CPU | 0.59–0.89 s | |
| whisper small on the CPU, Qwen3.5 4B on the GPU | 1.47–1.62 s | |
| whisper base and Qwen3.5 4B, both on the CPU | 0.93–2.04 s |
The answer target held in every setup I measured. The question target didn’t with whisper small on the CPU, which took about 1.35 seconds, and the automatic speed test still needs to learn not to pick it for questions on a machine like mine. The biggest single fix was loading the system prompt into the model’s cache at startup, which took the first answer on a CPU-only setup from 3.56 seconds to 2.04.
FoldKit for the windows
Every window is a FoldKit program, which is the Elm architecture on top of Effect: a model, a set of messages, a pure update function and a view. The main process broadcasts one app state, and the windows turn it into pixels and send back typed requests. The preload script exposes exactly two functions, and the main process checks every request against an Effect Schema and ignores anything that didn’t come from one of its own windows.
Because updates are pure, the bar’s behaviour is tested as stories: given this state and these messages, expect these commands. The waveform is the one place that skips the virtual DOM, and writes its styles directly on each animation frame.
A server it never needs
The app works on a plane with the Wi-Fi off. I checked that by running the real-model test with nothing but loopback networking. So the Cloudflare side is optional, and the app never imports from it. It’s the CAFE stack: Cloudflare and Alchemy here, FoldKit and Effect in the app.
Alchemy defines a Worker, a D1 database and an R2 bucket in TypeScript, and the Worker’s bindings are typed from that same file, so renaming one fails the type check. It serves a signed list of models, update details and feature flags, and has a store for syncing settings and history that only ever sees encrypted bytes. No audio, transcript or question ever reaches it.
- Sign offlineEd25519 over canonical JSON. The private key never goes near Cloudflare.
- PublishThe Worker checks the signature again, so a leaked admin token alone can’t ship one the app would accept.
- FetchThe app asks with a short timeout. Being offline is normal.
- Verify firstThe signature is checked before any decoding, because decoding can reorder keys.
- Fall backAny failure means the manifest built into the app. Files are still checked by SHA-256.
It’s tested against a real SQLite database with the same migrations, but I haven’t deployed it. There’s no real sign-in yet, and billing is a stub.
Where it’s at
The whole loop works on my PC. The real-model test passes, including with the network switched off, and I’ve run the Electron app through onboarding, scripted spoken questions, follow-ups and hands-free on a Linux dev build. The Windows audio host is built and tested on Windows 11 but hasn’t run inside the packaged app yet. The macOS host is written and unit tested, but I haven’t had a Mac to compile it on.
Nothing is signed or released yet. Next on the list are running it on a Mac and pausing whatever’s playing when you hold the key. For now you pause it yourself.






