ANE vs GPU for Voice Transcription on Mac: Why the Chip Choice Changes Everything

Most voice dictation apps on Mac run transcription on the GPU. It's the obvious choice — GPUs are well-documented, the PyTorch tooling is mature, and Whisper (the model most of them use) was built with CUDA in mind.

The problem is that your GPU is already busy.

If you're a developer running Cursor, Claude Code, a browser with open tabs, and a couple of background agents — your GPU is handling rendering, model inference, and whatever else the OS is throwing at it. Starting voice transcription on the same chip means your dictation is now competing with everything else for resources.

This is why Superwhisper shipped a crash fix three days after their last feature release. It's not a quality problem — it's a resource contention problem.

There's a different approach.

What the Apple Neural Engine Actually Is

Every Mac with Apple Silicon (M1 and later) ships with three compute units: CPU, GPU, and ANE. The ANE — Apple Neural Engine — is a dedicated chip built specifically for neural network inference. It's not a general-purpose compute unit. It doesn't render frames. It doesn't run your browser. It runs one class of workload: machine learning models.

Apple introduced the ANE in 2017 with the A11 Bionic. By the time M1 shipped in 2020, the ANE in a MacBook Pro could run 11 trillion operations per second. The M4 generation pushes that to 38 TOPS.

The reason most apps don't use it: the tooling. ANE requires CoreML — Apple's framework for on-device ML. PyTorch doesn't target ANE directly. Converting a model from its training format to a CoreML-compatible format that actually runs on ANE (and not just falls back to the GPU) requires work that most developers don't want to do.

So they use the GPU instead.

The Practical Difference for Developers

When you're running a vibe coding session — speaking prompts into Cursor, switching to Claude Code, dictating a PR description — your Mac is working hard. The GPU is rendering four different windows, running any UI-accelerated model inference in your IDE, and handling graphics for the browser you have open with the documentation you're referencing.

Add GPU-based voice transcription to that and one of two things happens: the dictation gets slower as the GPU context-switches, or something crashes.

ANE-based transcription sidesteps this entirely. The transcription workload lives on a separate chip. Your GPU never sees it. Your Cursor performance is unaffected. Your transcription runs at whatever latency the ANE supports — which, for a model like Parakeet, is effectively real-time.

The numbers matter here. ANE inference for a voice model runs with a latency profile that GPU-based approaches can't match under load. But more important than raw speed is the stability: a dedicated chip doesn't have bad days when your machine is busy.

The Cold-Start Trade-Off

ANE-based transcription has one known limitation worth being honest about: cold start.

The first time a CoreML model loads onto the ANE in a session, there's a one-time compilation and caching step. For Parakeet running on the ANE, this is roughly 60 seconds on first launch. Subsequent runs — including every restart after the first — are under 2 seconds because the compiled model is cached.

This is a real trade-off. If you're someone who starts and stops your dictation tool constantly throughout the day, that 60-second first-run latency is annoying. If you launch it once in the morning and leave it running — which is how most voice dictation workflows actually operate — you never notice it.

GPU-based tools don't have this cold-start behaviour because they're running in a more dynamic execution environment. If you prioritise instantaneous startup over stable throughput, GPU is the technically correct choice.

For everything else, ANE wins.

Why This Matters for Privacy

There's a second reason to care about ANE-based transcription that has nothing to do with performance: it makes fully local processing practical.

Cloud voice transcription isn't just slower on bad networks — it's a different architecture. Your audio leaves your device, gets processed somewhere else, and returns as text. The latency is a symptom of the architecture, not a performance problem to be solved.

On-device transcription with ANE means the audio never enters a network stack. Not because of a privacy setting. Because there's no network in the processing path. Activity Monitor shows zero outbound traffic during transcription. The architecture makes the privacy claim trivially verifiable.

For developers handling client code, unreleased product specs, or anything they'd rather not have processed on a third party's infrastructure — this matters.

The Model: Parakeet

Both VoizerFlow and Superwhisper use Parakeet — NVIDIA NeMo's English-language transcription model, widely regarded as the best open-weight model for English dictation at the moment.

The model is the same. What's different is which chip runs it.

Superwhisper runs Parakeet on the GPU. VoizerFlow runs Parakeet on the ANE via CoreML. Same input, same output, different execution path.

If you're evaluating voice dictation tools and comparing transcription quality, the model choice matters more than which app you use. But if you care about stability under load, resource isolation, or private-by-architecture processing — chip choice is where the products actually diverge.

Who Should Care About This

If you're a Mac developer who uses an AI-assisted workflow — Cursor, Claude Code, or similar — and you want voice dictation that doesn't fight with your existing tooling for compute, ANE-based transcription is worth understanding.

If you're a developer who handles sensitive information and can't route audio through a cloud service, on-device transcription isn't a nice-to-have. It's a requirement.

If you're neither of those things and you just want accurate dictation and don't mind cloud processing — there are good options. Wispr is polished. Superwhisper has a large feature set.

But if you're in the category of developer who builds in Cursor at 2am and needs the transcription to stay on the machine — the chip choice matters.

VoizerFlow is a macOS voice dictation app for Apple Silicon that runs Parakeet on the ANE. Audio never leaves your Mac. $49.99, one-time.