From camera to cortex: everyday markers, anchored in the brain
Picture a frail patient at home, a phone propped on the table, and measures of gaze, pupil and gait as good as a lab's. Eye trackers, motion capture and EEG rarely leave the lab, and many patients can rarely come to them. This challenge brings lab-grade measures to ordinary cameras, checks them against lab references, and anchors them in brain activity, with EEG kept clean while people talk and walk and pipelines that hold across sites. Markerless tracking, EEG foundation models and open EEG and imaging data are the starting kit for a first prototype and a study plan. The open question at its heart: what neurophysiology/imaging adds beyond the camera.
Resources
- Brain signals and imaging
- Wearables and phone sensing
- Sensitive data handling
- LEMON and OpenNeuro ↗
- MPIIFaceGaze — webcam gaze dataset (research use) ↗
- AddBiomechanics — motion-capture reference data ↗
- Simultaneous EEG–fMRI, neurofeedback (OpenNeuro ds002336, CC0) ↗
- EEG while walking — Audiocue (OpenNeuro ds001971) ↗
- EEGEyeNet — simultaneous EEG and eye tracking (OSF) ↗
- Healthy Brain Network — EEG, eye tracking, MRI (data use terms) ↗
- Dreem Open Datasets — headband vs lab sleep EEG ↗
- TUH EEG Corpus — clinical EEG (registration required) ↗
- NEMAR — open EEG/MEG repository ↗
- Speech models
- Language-model endpoints
- Data platform and governance
- Video / vision models
- Lab equipment
- OptiTrack and eye trackers ↗
- OpenCap — motion analysis from phone video, validated against motion capture (Apache-2.0) ↗
- Pose2Sim — 3D markerless pose from several cameras (BSD-3) ↗
- MMPose / RTMPose — pose estimation (Apache-2.0) ↗
- MediaPipe Pose Landmarker ↗
- OpenFace — gaze, head pose, facial action units (non-commercial academic) ↗
- WebGazer.js — webcam eye tracking in the browser (check licence) ↗
- Lab Streaming Layer — synchronise EEG, gaze, audio and motion ↗
- MNE-Python — EEG/MEG analysis (BSD-3) ↗
- pyprep — EEG preprocessing (PREP pipeline) ↗
- autoreject — automated EEG artefact rejection ↗
- Braindecode — deep learning on EEG (BSD-3) ↗
- LaBraM — EEG foundation model (MIT) ↗
- fMRIPrep — fMRI preprocessing (Apache-2.0) ↗
- FastSurfer — fast MRI segmentation (Apache-2.0) ↗
- SynthSeg — segmentation for any T1 scan (FreeSurfer) ↗
- Lead-DBS — deep brain stimulation electrode localisation ↗
- BIDS — standard layout for brain data ↗
