active Neural Computer Interface or activeNCI is an AI system for interpreting a real time signal from a human (currently a hand tracker, meant to be an EEG) and classifying its current state. the innovation here is that the classification doesnt NEED labels, instead relying on online K means.
Each frame, a MediaPipe hand tracker produces a bone-offset representation of the hand. A small autoencoder compresses this into a latent vector, which an online k-means classifier assigns to a gesture cluster. Holding a label key associates the current cluster with that key. The whole pipeline trains continuously in the background.
- Python 3.12+
- CUDA-capable GPU (recommended)
pip install -r requirements.txtDownload the MediaPipe hand landmark model from the MediaPipe model card and place it in the project root as hand_landmarker.task.
Open activeNCI.ipynb and run the cells top to bottom. Adjust the settings cell for your hardware (camera index, device, label keys). In the main loop, hold a label key while performing a gesture to train the classifier. Press q to stop.
core/ signal-agnostic ML pipeline
autoencoder.py encoder/decoder for arbitrary signal shapes
online_k_means.py online k-means++ with dead centroid reinit
live_dataset.py per-class rolling queue storage
ae_trainer.py AdamW training loop
centroid_labeler.py centroid-to-label mapping with majority vote
label_getter.py keyboard-based ground truth capture
hand_tracker_sig/ MediaPipe hand tracking
hand_tracker.py landmark detection + bone offset conversion
camera_wrapper.py OpenCV camera input
hand_tracker_demo.py minimal visualisation demo
utils/ visualizers
hand_visualizer.py bone skeleton renderer with EMA scale
stats_visualizer.py auto-sizing text overlay window
activeNCI.ipynb main demo notebook