Self-host a transcription server your own devices can call
Self-hosted speech to text: three ways to run a transcription server on hardware you own, with auth, TLS, GPU sizing, throughput numbers and the honest running cost.
A self-hosted transcription server is one machine you control running a speech model behind an HTTP endpoint, so every laptop, phone and script on your network posts audio to the same place and gets text back. It is the middle ground between a commercial API and a model on each device: the audio still crosses a network, but it goes to your hardware. Below: three routes to a working endpoint, the throughput to expect, the security work, and the running cost.
Who this is for, and who it is not for
It fits when several devices would otherwise each keep a copy of a multi-gigabyte model, and some of them, phones and thin laptops, cannot run one at all. It fits when a home server or a NAS with a GPU already runs anyway, or when something automated needs a stable endpoint. Liking this sort of thing is a valid reason too.
It does not fit one person with one Mac. Running the model on that Mac is quicker to set up, faster for short clips, and cannot be broken by a router reboot: start with run Whisper locally on a Mac instead. It is also wrong if you need transcripts away from your network without a VPN.
The shape of the thing
Every route below is the same three steps: the client POSTs audio as multipart form data, the server holds a loaded model in memory and decodes it, the client gets back text, JSON or subtitles. The box becomes a single point of failure, down sometimes for a kernel upgrade or a container that did not come back after a power cut, and a single point of access, since anything reaching the port can spend your compute and read what it sends. Auth and a backup plan belong in the build, not a later pass.
Route A: the HTTP server that ships with whisper.cpp
whisper.cpp includes a server example: one binary, one model file, no Python and no container runtime. One catch: the Homebrew formula builds with the server target disabled (-DWHISPER_BUILD_SERVER=OFF as of formula version 1.9.2), so brew install whisper-cpp gives you whisper-cli, not whisper-server. Build from source:
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
sh ./models/download-ggml-model.sh large-v3-turbo
cmake -B build
cmake --build build -j --config Release
./build/bin/whisper-server \
-m models/ggml-large-v3-turbo.bin \
-t 8 -l auto \
--host 127.0.0.1 --port 8080 --convert
Documented defaults are host 127.0.0.1, port 8080, model models/ggml-base.en.bin, four threads and language en, so set each explicitly. The flags that matter: -m model, -t threads, -l language (auto detects), --convert to accept anything other than 16 kHz WAV by converting with ffmpeg on the server, -ng to disable the GPU, --vad to skip silence. The full list is in the server example README.
Two endpoints are documented: POST /inference and POST /load, which swaps the model at runtime. Inference takes the audio as the file field, plus temperature, temperature_inc, prompt, carry_initial_prompt and response_format:
curl 127.0.0.1:8080/inference \
-H "Content-Type: multipart/form-data" \
-F file="@meeting.wav" \
-F temperature="0.0" \
-F response_format="json"
--inference-path moves that route and --request-path prefixes every route, which helps behind a proxy. Note what the options do not include: any authentication.
Route B: a container several machines can point at
If the box runs Docker, a packaged service saves the build step and adds subtitle output and engine choices. The most widely used is whisper-asr-webservice, published as onerahmet/openai-whisper-asr-webservice with a CPU and a GPU tag.
services:
asr:
image: onerahmet/openai-whisper-asr-webservice:latest
restart: unless-stopped
ports:
- "127.0.0.1:9000:9000"
environment:
ASR_ENGINE: faster_whisper
ASR_MODEL: large-v3
MODEL_IDLE_TIMEOUT: "300"
volumes:
- ./cache:/root/.cache/
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
Documented variables: ASR_ENGINE (openai_whisper, faster_whisper, whisperx), ASR_MODEL (tiny through large-v3), ASR_MODEL_PATH, ASR_DEVICE (cuda or cpu) and MODEL_IDLE_TIMEOUT, which unloads the model after a quiet period. Mounting /root/.cache/ stops a restart re-downloading gigabytes of weights. It listens on port 9000 and serves API docs at the root URL. The route is POST /asr, audio in a field called audio_file, the rest as query parameters: output (text, json, vtt, srt, tsv), task, language, encode, word_timestamps and vad_filter on faster-whisper, diarize with min_speakers and max_speakers on WhisperX.
curl -X POST -H "content-type: multipart/form-data" \
-F "audio_file=@meeting.m4a" \
"http://127.0.0.1:9000/asr?output=srt&language=en"
docker run -d --gpus all -p 127.0.0.1:9000:9000 \
-e ASR_MODEL=large-v3 -e ASR_ENGINE=faster_whisper \
-v "$PWD/cache:/root/.cache/" --restart unless-stopped \
onerahmet/openai-whisper-asr-webservice:latest-gpu
The GPU image needs the NVIDIA driver and the NVIDIA Container Toolkit on the host, then sudo nvidia-ctk runtime configure --runtime=docker and a Docker restart, per NVIDIA's install guide; faster-whisper also wants cuBLAS and cuDNN 9 for CUDA 12. None of it applies on a Mac, where Docker Desktop documents GPU support only on Windows with the WSL2 backend, so the container runs on the CPU. Pin a version tag rather than latest.
Route C: an endpoint your existing clients already know
Several open-source servers implement the same transcription route as the widely used commercial speech API, POST /v1/audio/transcriptions, taking file and model as multipart fields. A client written for that API can then be aimed at your machine by changing one base URL.
Speaches is a faster-whisper server that documents itself as API-compatible and says tools and SDKs written for the commercial API should work with it. It ships CPU and CUDA images on port 8000. vLLM is the other well-documented option, aimed at a real GPU: pip install vllm[audio], then vllm serve openai/whisper-large-v3. It documents /v1/audio/transcriptions and /v1/audio/translations with file, model, language, prompt, response_format (json, text, verbose_json, diarized_json) and temperature.
docker run --rm --detach --name speaches \
--publish 127.0.0.1:8000:8000 \
--volume hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
ghcr.io/speaches-ai/speaches:latest-cpu
curl -s "http://127.0.0.1:8000/v1/audio/transcriptions" \
-F "file=@meeting.wav" -F "model=Systran/faster-whisper-small"
The client-side swap is one line:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="token-abc123")
with open("meeting.mp3", "rb") as f:
result = client.audio.transcriptions.create(
model="openai/whisper-large-v3", file=f, language="en",
)
print(result.text)
Compatible means the route, the multipart fields and the common response formats, which covers most client code. It does not mean a clone of a whole commercial product: model names are the server's own, and streaming, word timing, diarization and upload limits vary per project. vLLM caps uploads at 25 MB by default through VLLM_MAX_AUDIO_CLIP_FILESIZE_MB, calls beam search on encoder-decoder models currently inefficient, and does not populate no_speech_prob in verbose_json. Set any non-empty API key string, which the SDKs require even where the server ignores it, and test your real client before migrating. whisper.cpp's server is a separate case: field names overlap and --inference-path can move the route there, but the project documents no compatibility, so do not assume the response shape matches.
Hardware and throughput
Read speed as a multiple of real time: 12x means an hour of audio finishes in five minutes. These figures come from the faster-whisper project's benchmark table for 13 minutes of audio, and are estimates for your hardware, not promises.
| Machine and model | Reported time | Roughly | Memory |
|---|---|---|---|
| RTX 3070 Ti 8GB, large-v2, fp16 | 1m03s | ~12x real time | 4525 MB VRAM |
| Same, fp16, batch size 8 | 17s | ~46x real time | 6090 MB VRAM |
| Same, int8 | 59s | ~13x real time | 2926 MB VRAM |
| i7-12700K, 8 threads, small, int8, batch 8 | 51s | ~15x real time | 3608 MB RAM |
| Same CPU, small, fp32, no batching | 2m37s | ~5x real time | 2257 MB RAM |
An 8 GB consumer card runs a large model comfortably; a CPU box suits a small model and overnight batches and is hopeless for a large model at volume; batching is where most of the GPU speedup lives. Allow 3 to 6 GB of VRAM for a large model depending on quantization, more with batching. whisper.cpp's own table lists about 852 MB of RAM for small, 2.1 GB for medium and 3.9 GB for large; how much memory a Mac needs goes deeper. Apple Silicon sits in between: whisper.cpp uses the GPU through Metal natively, so a Mac mini hosts Route A well and the container routes badly.
Concurrency is where people go wrong. One loaded model decodes one request at a time, and extra workers load extra copies of the weights, so two large models on an 8 GB card fail rather than share. Queue instead: a slow queue is an inconvenience, running out of VRAM is an outage.
Making it reachable without making it public
- Bind to localhost. whisper.cpp's server already defaults to
127.0.0.1. In Docker, publish as127.0.0.1:9000:9000, not9000:9000, which listens on every interface. - Terminate TLS in a reverse proxy. Caddy obtains and renews certificates from Let's Encrypt or ZeroSSL automatically for a public domain with the right DNS records and ports 80 and 443 reachable, and it can carry the auth. It refuses plaintext passwords, so generate the hash with
caddy hash-password.
asr.example.com {
basic_auth {
bob $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG
}
reverse_proxy 127.0.0.1:9000
}
- Or check a bearer token. The check is a string comparison and a 401, since in nginx
$http_authorizationis that request header as a variable. Use a long random token, one per client.
location /asr {
if ($http_authorization != "Bearer replace-with-a-long-random-string") {
return 401;
}
proxy_pass http://127.0.0.1:9000;
}
- Prefer a private overlay network to an open port. The better answer to "how do I reach it from my phone" is an encrypted overlay putting your devices on one private address range wherever they are. WireGuard is the protocol most of this category builds on; Tailscale is a managed network on top of it that connects devices behind firewalls without inbound port forwarding; Nebula is a certificate-based overlay using discovery nodes and UDP hole punching. All are less work than hardening a public endpoint.
Plainly: an unauthenticated transcription endpoint on the open internet will be found and abused, because scanners sweep the address space continuously and free GPU time is worth taking. Set a request size limit and a rate limit at the proxy too.
Keeping it running
A systemd unit gives a bare binary restart on failure and start on boot: Restart=always restarts regardless of exit status, on-failure lets a clean exit stay stopped. For containers, --restart unless-stopped, or restart: unless-stopped in compose, survives crashes and reboots but respects a manual stop.
[Unit]
Description=Whisper transcription server
After=network-online.target
[Service]
User=asr
ExecStart=/opt/whisper.cpp/build/bin/whisper-server -m /opt/models/ggml-large-v3-turbo.bin -t 8 -l auto --host 127.0.0.1 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
Three details matter. The model cache must be a volume, or weights are re-downloaded on every recreate: mount /root/.cache/ for whisper-asr-webservice, the Hugging Face hub cache for Speaches. Cap the logs, because Docker's json-file driver defaults to no size limit and one file, so --log-opt max-size=10m --log-opt max-file=3 saves a full disk later. Monitor free disk, VRAM and RAM, GPU temperature, restart count, and end-to-end latency by posting a known file on a schedule: a model that quietly failed to load returns empty strings and still passes a port check.
Diarization and subtitles on the server
Both are cheaper where the model already lives. whisper-asr-webservice exposes output=srt and output=vtt, plus diarize=true with min_speakers and max_speakers on the WhisperX engine. whisper.cpp's server has -di for stereo-channel diarization and -tdrz for tinydiarize models, and vLLM documents a diarized_json format for models that support it. Quality varies: see speaker diarization on a Mac and SRT subtitles from audio. Once the server returns transcripts reliably, a meeting notes pipeline is the natural thing to put in front of it.
What this actually costs
| Setup | Cost shape |
|---|---|
| Existing always-on box, CPU only | Electricity only. 100 W continuous is about 73 kWh a month, roughly $15 at $0.20 per kWh |
| Desktop with a consumer GPU, left on | Higher idle and load draw, plus the card |
| Rented cloud GPU, running continuously | Billed hourly whether busy or idle. On Lambda's published on-demand pricing, one A10 is $1.29 per GPU hour and one Quadro RTX 6000 is $0.69, so about $940 and $500 a month if you never turn it off |
| Metered speech API | Per minute of audio, nothing when idle. See cloud transcription API costs |
The other costs are not money: a day to build it properly, hours a year for OS upgrades and versions that move underneath you, and being on call for something other people now depend on.
Under about 10 hours of audio a month a metered API is cheaper than anything you can build, and that is the honest recommendation absent a non-cost reason. From there up to heavy daily use, hardware you already own and leave running wins by a wide margin: the marginal cost is close to zero. Renting a GPU by the hour makes sense only for batches you start, run and destroy.
What self-hosting does and does not buy you for privacy
What changes is real: no third-party account, no retention policy you did not write, no subprocessor list, no default that quietly enables training on your recordings.
What does not change is that the audio leaves the device and crosses a network. The question moves rather than disappears, from what a vendor does with it to who can reach this machine: anyone with shell access, anyone on the LAN if you skipped TLS, whoever holds the backups, and the proxy logs. A rented instance swaps a transcription vendor for an infrastructure vendor, a different trade rather than a better one.
So decide with a threat model, not a slogan: write down who you are keeping the audio from, then check whether this design stops that person. If your obligations are professional rather than personal, confirm them with your own advisor. On-device versus cloud transcription covers the stricter end of that spectrum.
If you'd rather not maintain this
Frequently asked questions
Can I run a self-hosted transcription server on a Mac?
Yes. whisper.cpp runs natively on macOS and uses the GPU through Metal, so its server example makes a reasonable always-on endpoint on a Mac mini. The container routes are weaker there: Docker Desktop documents GPU support only on Windows with the WSL2 backend, so a container on a Mac transcribes on the CPU.
Will my existing client work if I point it at my own server?
Often, if the server documents compatibility with the common transcription route. Speaches and vLLM both implement POST /v1/audio/transcriptions with file and model fields, so most clients need only a new base URL and any non-empty API key. Model names, streaming, upload limits and diarization differ per project, so test first.
How much GPU memory do I need for a self-hosted Whisper server?
The faster-whisper project's benchmark shows a large model using about 4.5 GB of VRAM in fp16 and 2.9 GB in int8, rising to about 6 GB at batch size 8. An 8 GB card handles one large model comfortably; 12 GB or more leaves room for batching. Two instances double the requirement, so queue instead.
Is self-hosting cheaper than paying a per-minute API?
It depends on volume and on whether the hardware exists already. On a machine you already run, the extra cost is electricity, so heavy use is very cheap. Renting hardware for the purpose is not, because a GPU bills every hour whether it transcribes or idles. Under roughly ten hours of audio a month, a metered API usually wins on cost alone.
