What speech reveals: memory and mood in clinical audio
A recorded memory, a therapy session, a parent playing with a child: each carries pauses, prosody and word choices that say something about memory and mood, and no clinician can score them at scale. How much of that signal can be captured from French clinical audio, compared over time, and shared without exposing the speaker? The team will build the pieces: who spoke when, voice and language markers, and anonymisation that protects identity while keeping the signal. Speech foundation models, language models and on-device processing work on open corpora, and the two days end with a plan to bring them to the clinic. Still wide open: markers that survive a phone call.
Resources
- Sensitive data handling
- Brain signals and imaging
- Wearables and phone sensing
- Common Voice FR — French read speech (Mozilla, open) ↗
- Multilingual LibriSpeech French — audiobook speech (OpenSLR 94, CC BY 4.0) ↗
- VoxPopuli — French parliamentary speech (Meta, CC0) ↗
- VoxConverse — who-spoke-when benchmark (CC BY 4.0, research) ↗
- AMI Meeting Corpus — multi-speaker, overlapping speech ↗
- CaFE — Canadian French emotional speech (CC BY-NC-SA 4.0) ↗
- RAVDESS — emotional speech and song (CC BY-NC-SA 4.0) ↗
- CHILDES French — child–parent speech (TalkBank, ground rules apply) ↗
- DementiaBank — speech in dementia (TalkBank, membership required) ↗
- EATD-Corpus — depression speech, Mandarin ↗
- DAIC-WOZ — depression interviews, English (EULA required) ↗
- Speech models
- Language-model endpoints
- Video / vision models
- Lab equipment
- Data platform and governance
- faster-whisper — fast Whisper transcription (MIT) ↗
- WhisperX — word timings and diarisation on Whisper (BSD-2) ↗
- NVIDIA Canary-1B-v2 — multilingual ASR incl. French (CC BY 4.0) ↗
- NVIDIA Parakeet-TDT-0.6B-v3 — multilingual ASR (CC BY 4.0) ↗
- pyannote.audio — who spoke when (MIT; accept model terms on Hugging Face) ↗
- openSMILE — acoustic features (audEERING research licence, non-commercial) ↗
- VoicePrivacy Challenge 2024 — voice anonymisation baselines ↗
- SA-toolkit — speaker anonymisation in Python (Apache-2.0) ↗
- Presidio — text de-identification (MIT) ↗
- Vosk — on-device speech recognition incl. French ↗
- CamemBERT — French language model ↗
