AI developer tooling. 2026. Author. Python, PTY, VAD, STT and TTS.

J.A.R.V.I.S.

A hands-free developer interaction system: a PTY wrapper, a streaming lexer, real-time voice activity detection and interchangeable STT, TTS and LLM adapters.

Role: Author
Built in: 2026
Surface: CLI
Stack: Python

Source on GitHub Index

Fig. 01

$ jarvis attach -- zsh

[vad]  listening
[stt]  "run the integration suite"
[pty]  $ pytest tests/integration -q
[tts]  "forty one passed, one skipped"

session active

2400 × 880

Fig. 01. A voice session driving the CLI.

The problem

A terminal workflow assumes two hands and a screen. Speaking to it is the easy half. The hard half is knowing who is talking: the user, the model, or the shell still streaming output.

So the system wraps a real PTY rather than reimplementing a shell, lexes the stream as it arrives, and keeps every speech and model vendor behind an adapter. Nothing about the design assumes a particular provider.

PythonPTYVADSTT / TTSLLM adapters

A real shell, not a simulation

Wrapping a PTY means anything that runs in a terminal runs here, including full-screen programs and interactive prompts.

Vendors are configuration

STT, TTS and the model each sit behind an adapter, so swapping engines is a config change rather than a rewrite.

The stream is the interface

Output is lexed as it arrives instead of after the command exits, which is what makes speaking over a running process possible.

Decisions and the price I paid for them

Decoupled providers, everywhere

Everything runs through interchangeable adapters and real-time streams, so no single model or speech vendor is baked into the system.

Trade-off accepted

An adapter layer for every provider is more surface to maintain than calling one API directly, and it is the only way to swap engines by config.

Wrap a PTY instead of writing a shell

A pseudo-terminal gives real job control, real signal handling and real TTY behaviour for free, which a reimplementation would have to earn back one edge case at a time.

Trade-off accepted

Parsing a PTY stream means dealing with escape sequences and partial writes rather than clean structured output.

Voice activity detection before transcription

Running VAD in front of speech-to-text is what lets the user interrupt while the system is still talking, instead of waiting for a turn.

Trade-off accepted

A detection threshold that is wrong in either direction is felt immediately: it either clips the user or talks over them.

Inside the build

Fig. 02
900 × 700Lexer output, token stream view

Fig. 02
The streaming lexer classifying output while the command is still running.

Fig. 03

# jarvis.toml

[stt]
adapter = "whisper-local"

[tts]
adapter = "piper"

[llm]
adapter = "anthropic"

900 × 700

Fig. 03

Provider adapters: each engine is one config block.