Hey Nabi 🦋

Real-time lecture translation for international students. Listen in Korean, Chinese, or Japanese — read in your language, instantly.

---

Features

• Real-time transcription — Stream lecture audio directly from your browser, no uploads required
• Multilingual translation — Listen in Korean, Chinese, or Japanese; read in Vietnamese or English
• Smart sentence buffering — Sentence boundary detection prevents mid-thought translations
• Korean spacing correction — Auto-fixes Deepgram's missing whitespace in Korean output
• Session history — Save, search, and export past lectures (Supabase + RLS)
• Multiple STT engines — Switch between Deepgram, Soniox, or browser-native Web Speech API
• Responsive UI — Mobile-first layout with side-by-side or interleaved transcript view
• Auth + RLS — Supabase Auth with row-level security per user

---

How It Works

🎤 Browser Mic
 → AudioWorklet (PCM 16kHz, 100ms chunks)
 → WebSocket → Deepgram Nova-3 (Speech-to-Text)
 → SentenceBoundaryDetector (smart buffering + commit)
 → Zustand Store
 → Gemini Flash (AI translation + spacing correction)
 → Interleaved UI (original ↔ translation)

Tech Stack

Layer	Technology
Framework	TanStack Start (React 19 + SSR + file-based routing)
Styling	Tailwind CSS v4 + Shadcn UI + Framer Motion
State	Zustand
Database	Supabase (PostgreSQL + Auth + RLS)
AI Translation	Google Gemini Flash via Vercel AI SDK
STT	Deepgram Nova-3 (primary) · Soniox · Web Speech API
Validation	Zod
Build	Vite
Testing	Vitest

Supported Languages

Language	Code	UI	STT Source	Translation Target
Tiếng Việt	vi	✅	—	✅
English	en	✅	—	✅
한국어	ko	✅	✅	✅
中文	zh	✅	✅	✅
日本語	ja	✅	✅	✅

Project Structure

src/
├── features/
│   ├── stt/              # Speech-to-text pipeline
│   │   ├── hooks/        # use-deepgram, use-soniox, use-web-speech, use-stt
│   │   └── lib/          # Audio pipeline, WebSocket, shared types
│   └── translation/
│       ├── hooks/        # useTranslation hook
│       └── prompts.ts    # Gemini translation prompts
├── components/
│   ├── auth/             # Login, user menu
│   ├── landing/          # Landing page
│   ├── layout/           # App shell (header, sidebar, mobile nav)
│   ├── session/          # Session controls, panels, language selector
│   ├── shared/           # Reusable UI (empty state, loading, search)
│   └── ui/               # Shadcn primitives
├── server/               # TanStack server functions (auth, translate, sessions, export, STT tokens)
├── stores/               # Zustand stores
├── lib/                  # Utilities (i18n, Supabase client, sentence boundary, Korean spacing)
├── routes/               # File-based routing (/{lang}/app/...)
└── types/                # TypeScript type definitions

Getting Started

Prerequisites

• Node.js 20+
• npm

Setup

# 1. Install dependencies
npm install

# 2. Configure environment
cp .env.example .env

Fill in .env:

# Client-side (exposed to browser)
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your-anon-key

# Server-side only
GEMINI_API_KEY=your-gemini-key
SONIOX_API_KEY=your-soniox-key
DEEPGRAM_API_KEY=your-deepgram-key
DEEPGRAM_PROJECT_ID=your-deepgram-project-id

# 3. Apply database migrations
npx supabase db push

# 4. Start dev server
npm run dev

App runs at http://localhost:3000

Scripts

Command	Description
npm run dev	Start development server
npm run build	Production build
npm run preview	Preview production build
npm run test	Run tests (Vitest)
npm run lint	ESLint check
npm run format	Prettier check
npm run check	Auto-fix (Prettier + ESLint)

STT Engines

Engine	Quality	Cost	Notes
Deepgram Nova-3	High	$200 free credit	Default, multilingual
Soniox	Highest	Paid	Best Korean accuracy
Web Speech API	Basic	Free	Chrome/Edge only, fallback

Contributing

Contributions are welcome! Please:

1. Fork the repo and create a feature branch
2. Run npm run check before committing
3. Make sure npm run test and npm run build pass
4. Open a PR with a clear description of changes

License

ISC