Jitsi Meet: Why Transcription for Electronic Medical Records Requires Jigasi and Vosk
Transcription in Jitsi Meet turned out not to be a button but a separate stack: Jigasi joins the call as a participant, sends the audio to Vosk and saves the result after the session ends. For an electronic medical record that is not enough. It takes consistent configs across three levels, control over XMPP authorization and LLM post-processing to recover medications, dosages and the structure of the physician's note.
AI-processed from Habr AI; edited by Hamidun News
Jitsi Meet supports transcription, but in a real product it is not a toggle in the interface. It is a separate infrastructure layer. In a case study on automatically filling an EMR after video consultations, this layer turned out to be the most labor-intensive: Jigasi, XMPP/SIP, Vosk and LLM post-processing all had to be wired together.
Architecture under the hood
Jitsi's basic idea looks simple only from the outside. The video call itself is handled by the Jitsi Meet frontend, the Jitsi Videobridge media server, the Jicofo conference manager and the Prosody XMPP layer. But transcription does not live inside a single button in the interface. It is handled by Jigasi, a separate gateway that joins the room as an ordinary participant, receives audio from the other participants and sends the stream to an external speech recognition service.
This creates a false sense of simplicity at the start. As a result, the task quickly moves from "connect an API" to an infrastructure problem. It is not enough to enable an option in the UI: you have to align several services, network connections and a separate STT backend. In the case examined, that backend was Vosk, running over WebSocket. The approach suits asynchronous processing after the consultation: the system does not have to meet the strict latency demands of live mode, and the final text can be processed calmly once the call is over.
Where the setup breaks
The main problem is that transcription here has several independent points of failure. The Jigasi config, the Jitsi Meet frontend parameters and the availability of the STT service must all line up at the same time. If one layer is misconfigured, the system often does not crash with a clear error. It simply fails to deliver the expected result: the bot does not enter the room, the file is not saved, or the text is too weak for practical use. Without reading the logs, such failures are easy to mistake for chance.
"Jigasi is a SIP gateway with transcription, not the other way around."
- a separate XMPP account for Jigasi in Prosody: if it is misconfigured, the transcriber bot will not appear in the conference at all;
- permissions on the transcripts directory: interim subtitles may flow while the final file is never saved to disk;
- choice of STT model: base Vosk is fine for an MVP, but handles medical terms, drug names and dosages worse;
- detecting the end of the session: Jigasi writes the final text only when the room has actually emptied, and the downstream pipeline needs a reliable trigger to start processing.
One more nuance is the separation of the channels for saving and sending results. One set of parameters controls writing the final text to disk after the consultation ends; another controls forwarding interim fragments to participants via XMPP. For a product that fills the EMR after the fact, reliably getting the final file matters more than showing live subtitles. Otherwise there is nothing to start the next processing stage from, and the whole automation hangs.
From text to EMR
Even after Jitsi is configured successfully, the job is not done. Jigasi's output is a plain raw dialogue with timecodes: the physician asks questions, the patient answers, then come prescriptions and recommendations. For a medical record, such text is almost useless in its original form, because the system needs not utterances as such but structured entities: complaints, symptom history, medications, dosages, intake schedule and next steps. A large layer of transformations remains between speech recognition and the EMR.
That is why another layer was needed on top of STT: LLM processing. The model normalizes the text, corrects some recognition errors from context and sorts the result into fields compatible with FHIR structures. The data then goes to a frontend form, where the physician reviews and confirms the entry before it is finally saved to the EMR. This human-in-the-loop step is not overcaution but a hard requirement: in a clinical scenario, medications, doses and prescriptions cannot be written automatically to the record without review.
This is where the limitation of "cheap" STT becomes clear. If the base model recognizes domain vocabulary poorly, the rest of the chain spends its resources fixing errors. For a production setup, the obvious options are heavier Vosk models, a specialized engine such as Deepgram with a medical profile, or a combination of STT and LLM normalization in which the language model compensates for recognition misses. Otherwise the cost of errors is too high at the level of the medical record itself.
What this means
The Jitsi Meet story shows one simple thing: transcription for an applied AI product is a separate subsystem, not a cosmetic feature. An asynchronous setup with Jigasi and Vosk will do for an MVP, but production in medicine requires precise tuning of the whole stack, good logs, control over session completion and a normalization layer that turns conversation into EMR-ready data. The stricter the domain, the more expensive the illusion that a single checkbox in the interface solves everything.
- Nov 12, 2026Google opens preorders for Gemini glasses starting at $395 and blocks covert recording
- Nov 12, 2026Samsung Is Preparing a Paid Galaxy AI Subscription: From $4.99 a Month and Credits Instead of Unlimited Use
- Nov 12, 2026Adobe showcases four AI prototypes at Max: 3D from photos, a sound square, and copy-and-paste edits
- Nov 12, 2026Instagram launches prompt-based filter creation for users aged 18+ in the US
Key Questions Answered
What are the key details on Architecture under the hood?
Jitsi's basic idea looks simple only from the outside. The video call itself is handled by the Jitsi Meet frontend, the Jitsi Videobridge media server, the Jicofo conference manager and the Prosody XMPP layer. But transcription does not live inside a single button in the interface. It is handled by Jigasi, a separate gateway that joins the room as an ordinary participant, receives audio from the other participants and sends the stream to an external speech recognition service.
Where the setup breaks?
The main problem is that transcription here has several independent points of failure. The Jigasi config, the Jitsi Meet frontend parameters and the availability of the STT service must all line up at the same time. If one layer is misconfigured, the system often does not crash with a clear error. It simply fails to deliver the expected result: the bot does not enter the room, the file is not saved, or the text is too weak for practical use. Without reading the logs, such failures are easy to mistake for chance.
What are the key details on From text to EMR?
Even after Jitsi is configured successfully, the job is not done. Jigasi's output is a plain raw dialogue with timecodes: the physician asks questions, the patient answers, then come prescriptions and recommendations. For a medical record, such text is almost useless in its original form, because the system needs not utterances as such but structured entities: complaints, symptom history, medications, dosages, intake schedule and next steps.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.