App AI Assistant launches a 100% local voice assistant for PCs

A local voice assistant often looks good only in a diagram. In practice, developers face a tangle of speech recognition, a local model, voice synthesis, audio queues, and interruption logic. App AI Assistant attempts to turn that tangle into a working desktop product: the user speaks, the system recognizes the phrase, a local Gemma 3 generates a response, Silero begins voicing it in chunks, and playback stops when the user starts speaking again.

The project works without cloud APIs and is published under the MIT license. That makes it useful not only as a ready-made "Jarvis," but also as an engineering example: its individual parts can run independently and connect to applications written in Python, C#, Java, JavaScript, or Go.

How the voice pipeline works

The author divided the system into a graphical orchestrator and three services: srv_stt for recognition, srv_llm for response generation, and srv_tts for speech synthesis. They exchange data over HTTP and WebSocket. This approach makes it possible to replace the interface or use only the required component without rewriting the entire assistant.

Recognition is built around a combination of Vosk and Faster-Whisper. Vosk quickly detects the start of a new utterance and helps implement barge-in—the ability to interrupt the assistant by speaking. Faster-Whisper then produces a more accurate final transcription. Gemma 3 generates the response through llama-cpp-python, while Silero turns the text into speech. Before synthesis, a separate layer normalizes numbers and abbreviations, which voice engines often pronounce incorrectly.

In a simple example, the process works like this. The user asks the assistant to open a work folder. The recognition module passes the text to the orchestrator, the model selects a configured scenario, and the assistant launches the corresponding Python script. If the user immediately speaks another command, Vosk detects the speech, playback stops, and a new cycle begins. A short summary of previous conversations can be retained between sessions.

Why the services are separated

Each component is available as a standalone HTTP server. A developer working on a C# application can launch only srv_stt and srv_tts to add a voice interface. A Java application can call srv_llm as a local API. This is more practical than monolithic integration, where replacing the model or recognition engine affects the entire product.

The author has prepared a separate setup wizard. It downloads portable versions of Python and Git, verifies files using SHA-256, creates virtual environments, installs dependencies, and builds executable files with PyInstaller. Docker is deliberately not used: the target audience is owners of ordinary Windows computers, for whom WSL2, virtualization, and Docker Desktop would create an additional barrier.

The package also includes infrastructure2, a shared collection of modules for logging, HTTP clients, settings, process management, streaming data, and building executable files. It is not a required part of the voice assistant, but the repository demonstrates how the author reduces duplication across several Python projects.

Limitations and practical conclusions

For now, the project is designed only for Windows 10 and 11. Model support is tied to the Gemma template, and echo cancellation is unfinished. As a result, sound from the speakers may be picked up by the microphone again and interpreted as new speech; the author recommends using headphones. The infrastructure2 repository is also still poorly documented.

Developers should view App AI Assistant not as a universal replacement for cloud assistants, but as a ready-made test bed for a local voice interface. Its strength is a transparent pipeline built from accessible components and the ability to take only the STT, TTS, or LLM layer. Before deployment, it is important to test latency and memory consumption on the target computer, review the permitted voice commands, and isolate the execution of user scripts. Local operation prevents conversations from being sent to the cloud, but does not by itself make command execution secure.