AI research2026
LangVis
A real-time voice language tutor that corrects every sentence and takes learners from A2 to B2.
- Course
- 24 units · A2→B2
- Skills tracked
- ~40
- Teaching methods
- 18
- Audio
- 16 kHz in · 24 kHz out
01Overview
LangVis is a desktop speaking course. The learner talks; the tutor listens, corrects the sentence, asks for a repeat, and offers a stronger way to say it. It remembers mistakes and turns repeated ones into short drills.
It is deliberately not an assistant: it has one job - to make the learner speak better - and every part of the architecture serves that loop.
02The problem
Speaking practice needs instant, low-latency conversation, but good correction needs careful analysis of each sentence. Doing both in one model call either slows the conversation or makes feedback shallow.
03Architecture
Two model paths run side by side: a live audio session keeps the conversation fluid, while a small, fast model analyses each sentence in the background and updates the learner model that shapes the next prompt.
Interface
Syllabus
Stages, units, rules
Coaching
Corrections, tips, upgrades
Dictionary
Words one step above level
Tutor face
Drawn at runtime from audio level
Live session
Audio I/O
Mic & speaker streams
Gemini Live
Two-way real-time voice
Reconnect
Session resumption
Tutor domain
Language tutor
observe → measure → teach
Analysis
Per-sentence, off the hot path
Curriculum
Skills, units, methods
Progress
Level, skills, vocabulary
Core
Plugin loader
Tools, observers, prompt blocks
Wake word
Optional, fully offline
Self-log
Own output & errors
Local storage
level.json
Source of truth
progress.md
Readable, regenerated
memory/
Settings & API key
04Components
Live session
Conversation without waiting
- Streams 16 kHz microphone audio to the Gemini Live API and plays 24 kHz responses.
- Handles server GoAway messages and settings changes with a controlled reconnect that can keep or drop conversation context.
google-genai · sounddevice · asyncio TaskGroup
Sentence analysis
Measures every utterance
- A lightweight model returns structured JSON per sentence: errors, corrected form, skill tags, CEFR evidence.
- Runs in the background, so the live model never waits for it.
- Local word-level checks answer cheap questions (was this word used?) without a model call.
gemini-flash-lite · tolerant JSON parsing
Curriculum & progress
What to teach next
- 6 stages × 4 units with target forms, model sentences and named teaching methods.
- ~40 grammar skills, each with mastery, real mistakes and a review date.
- The three weakest skills become the focus that is injected into the next session prompt.
Plugin system
Extensible without touching the core
- Any file in plugins/ with a PLUGIN dict and run() becomes a tool the tutor can call.
- Optional hooks let a plugin observe every sentence, add standing prompt instructions or appear in the UI.
05How it works
- 1
Speak
Audio streams to the live model; the tutor answers with correction first, then continues.
- 2
Analyse
The transcript goes to the background analyser, which tags errors and skills.
- 3
Update
Level, skill mastery and vocabulary are written to level.json; progress.md is regenerated.
- 4
Adapt
The same mistake three times triggers a drill; weakest skills shape the next prompt.
- 5
Progress
When target forms are strong, the next unit starts without restarting the session.
06Technical deep dives
Two models, two speeds
The live model is optimised for latency, the analysis model for structure and cost. Splitting them means the conversation stays natural while every sentence still gets a full grammatical review.
Results flow back through the plugin hook format_for_prompt(), so the next turn of the tutor knows what to focus on.
Controlled reconnects
Long voice sessions end, devices change and the language can be switched mid-session. Instead of crashing, a reconnect signal is raised inside the session's TaskGroup. A keep_context flag decides whether the session-resumption handle survives - kept for an audio-device change, dropped for a new language.
class _ReconnectSignal(Exception):
"""Raised inside the session TaskGroup to force a clean reconnect."""
def request_reconnect(self, keep_context: bool = True, reason: str = ""):
self._reconnect_keep = keep_context # False → drop resumption handle
self._reconnect_reason = reason
self._reconnect_event.set()
# e.g. switching language starts a fresh context,
# changing the microphone keeps the conversation
self.request_reconnect(keep_context=False, reason="new language")
self.request_reconnect(keep_context=True, reason="audio device")Plugin contract
Plugins are discovered at start-up. The same interface powers the language tutor itself, which keeps the core small and makes adding a language a data change rather than a code change.
PLUGIN = {
"name": "my_plugin", # unique snake_case tool name
"description": "When the tutor should call this tool.",
"parameters": {"type": "OBJECT", "properties": {...}, "required": []},
}
def run(parameters: dict, player=None, session_memory=None) -> str:
# Return a short sentence - it is spoken back to the learner.
# Never raise: catch errors and return them as text.
...
# Optional hooks
def observe(text, player): ... # see every sentence
def format_for_prompt() -> str: ... # add standing instructions
def set_language(name: str): ... # react to the language select07Design decisions
Level is measured, never guessed
CEFR level is a rolling score over the learner's own recent sentences, not a placement quiz.
Local-first privacy
API key, settings and all progress files stay on the machine and are git-ignored; audio goes only to the model during a session.
No shipped images
The tutor's face, icons and progress bars are drawn at runtime, so the interface stays sharp at any display scale.
Languages as data
Adding a language means writing its skills and stages in the curriculum and a word-level detector - the interface already supports it.
08Tech stack
- AI
- Gemini Live APIGemini Flash / Flash-LiteopenWakeWord (optional)
- App
- PythonPyQt6asynciosounddeviceNumPy
- Storage
- JSON (level.json)Markdown (progress.md)
09What's next
- Slovak course (curriculum and detector already planned).
- Pronunciation scoring from the audio stream.
Next case study
DevCode Academy LMS
A role-based learning management system for programming schools.