All projects

AI research2026

LangVis

A real-time voice language tutor that corrects every sentence and takes learners from A2 to B2.

Course
24 units · A2→B2
Skills tracked
~40
Teaching methods
18
Audio
16 kHz in · 24 kHz out

01Overview

LangVis is a desktop speaking course. The learner talks; the tutor listens, corrects the sentence, asks for a repeat, and offers a stronger way to say it. It remembers mistakes and turns repeated ones into short drills.

It is deliberately not an assistant: it has one job - to make the learner speak better - and every part of the architecture serves that loop.

02The problem

Speaking practice needs instant, low-latency conversation, but good correction needs careful analysis of each sentence. Doing both in one model call either slows the conversation or makes feedback shallow.

03Architecture

Two model paths run side by side: a live audio session keeps the conversation fluid, while a small, fast model analyses each sentence in the background and updates the learner model that shapes the next prompt.

Interface

Syllabus

Stages, units, rules

Coaching

Corrections, tips, upgrades

Dictionary

Words one step above level

Tutor face

Drawn at runtime from audio level

Live session

Audio I/O

Mic & speaker streams

Gemini Live

Two-way real-time voice

Reconnect

Session resumption

Tutor domain

Language tutor

observe → measure → teach

Analysis

Per-sentence, off the hot path

Curriculum

Skills, units, methods

Progress

Level, skills, vocabulary

Core

Plugin loader

Tools, observers, prompt blocks

Wake word

Optional, fully offline

Self-log

Own output & errors

Local storage

level.json

Source of truth

progress.md

Readable, regenerated

memory/

Settings & API key

04Components

Live session

Conversation without waiting

  • Streams 16 kHz microphone audio to the Gemini Live API and plays 24 kHz responses.
  • Handles server GoAway messages and settings changes with a controlled reconnect that can keep or drop conversation context.

google-genai · sounddevice · asyncio TaskGroup

Sentence analysis

Measures every utterance

  • A lightweight model returns structured JSON per sentence: errors, corrected form, skill tags, CEFR evidence.
  • Runs in the background, so the live model never waits for it.
  • Local word-level checks answer cheap questions (was this word used?) without a model call.

gemini-flash-lite · tolerant JSON parsing

Curriculum & progress

What to teach next

  • 6 stages × 4 units with target forms, model sentences and named teaching methods.
  • ~40 grammar skills, each with mastery, real mistakes and a review date.
  • The three weakest skills become the focus that is injected into the next session prompt.

Plugin system

Extensible without touching the core

  • Any file in plugins/ with a PLUGIN dict and run() becomes a tool the tutor can call.
  • Optional hooks let a plugin observe every sentence, add standing prompt instructions or appear in the UI.

05How it works

  1. 1

    Speak

    Audio streams to the live model; the tutor answers with correction first, then continues.

  2. 2

    Analyse

    The transcript goes to the background analyser, which tags errors and skills.

  3. 3

    Update

    Level, skill mastery and vocabulary are written to level.json; progress.md is regenerated.

  4. 4

    Adapt

    The same mistake three times triggers a drill; weakest skills shape the next prompt.

  5. 5

    Progress

    When target forms are strong, the next unit starts without restarting the session.

06Technical deep dives

Two models, two speeds

The live model is optimised for latency, the analysis model for structure and cost. Splitting them means the conversation stays natural while every sentence still gets a full grammatical review.

Results flow back through the plugin hook format_for_prompt(), so the next turn of the tutor knows what to focus on.

Controlled reconnects

Long voice sessions end, devices change and the language can be switched mid-session. Instead of crashing, a reconnect signal is raised inside the session's TaskGroup. A keep_context flag decides whether the session-resumption handle survives - kept for an audio-device change, dropped for a new language.

main.py (simplified)python
class _ReconnectSignal(Exception):
    """Raised inside the session TaskGroup to force a clean reconnect."""

def request_reconnect(self, keep_context: bool = True, reason: str = ""):
    self._reconnect_keep = keep_context   # False → drop resumption handle
    self._reconnect_reason = reason
    self._reconnect_event.set()

# e.g. switching language starts a fresh context,
# changing the microphone keeps the conversation
self.request_reconnect(keep_context=False, reason="new language")
self.request_reconnect(keep_context=True, reason="audio device")

Plugin contract

Plugins are discovered at start-up. The same interface powers the language tutor itself, which keeps the core small and makes adding a language a data change rather than a code change.

plugins/_template.py (condensed)python
PLUGIN = {
    "name": "my_plugin",          # unique snake_case tool name
    "description": "When the tutor should call this tool.",
    "parameters": {"type": "OBJECT", "properties": {...}, "required": []},
}

def run(parameters: dict, player=None, session_memory=None) -> str:
    # Return a short sentence - it is spoken back to the learner.
    # Never raise: catch errors and return them as text.
    ...

# Optional hooks
def observe(text, player): ...             # see every sentence
def format_for_prompt() -> str: ...        # add standing instructions
def set_language(name: str): ...           # react to the language select

07Design decisions

Level is measured, never guessed

CEFR level is a rolling score over the learner's own recent sentences, not a placement quiz.

Local-first privacy

API key, settings and all progress files stay on the machine and are git-ignored; audio goes only to the model during a session.

No shipped images

The tutor's face, icons and progress bars are drawn at runtime, so the interface stays sharp at any display scale.

Languages as data

Adding a language means writing its skills and stages in the curriculum and a word-level detector - the interface already supports it.

08Tech stack

AI
Gemini Live APIGemini Flash / Flash-LiteopenWakeWord (optional)
App
PythonPyQt6asynciosounddeviceNumPy
Storage
JSON (level.json)Markdown (progress.md)

09What's next

  • Slovak course (curriculum and detector already planned).
  • Pronunciation scoring from the audio stream.

Next case study

DevCode Academy LMS

A role-based learning management system for programming schools.