SzPredict

Quickstart

github.com/hyperreal-ai/SzPredict — Five minutes from git clone to a Protocol 3 baseline result on CHB-MIT.

git clone https://github.com/hyperreal-ai/SzPredict.git
cd SzPredict
./install.sh

install.sh prepares the environment (venv), runs a mock-pipeline smoke test, downloads CHB-MIT (or accepts a path to an existing local copy), prepares the data per spec, trains the reference 1D-CNN, evaluates it, and produces a Protocol 3 submission JSON. Fast, streamlined setup - swap the CNN body for your own architecture and submit results via PR to results/.

Summary

Seizure prediction research on CHB-MIT has a comparison problem. Across 19 recent papers, reported sensitivities range from 58% to nearly 100% — not because models differ that much, but because papers use incompatible task conventions, preictal windows, patient cohorts, and post-processing rules. Numbers look impressive in isolation; most aren't directly comparable.

SzPredict pins every axis. Four evaluation protocols (patient-specific [protocol 1], leave-one-patient-out [protocol 2], cross-patient fixed split [protocol 3], and Protocol 4 — clinical transition timing). A single metric suite applied to every submission. Fixed dataset splits. Reference baselines runnable in five minutes. Anyone with a spare weekend and a single GPU can train their own model and submit results that slot directly into the same comparison table as ours.

Protocol 4 is the metric that actually matters. Not 'classification accuracy on balanced test segments' but 'how many minutes before seizure onset does the model reliably warn?' The clinical target is specific: a device that knows a patient's interictal baseline and says "Caution — Preictal detected" or, at full strength, "Seizure Predicted — onset in approximately 13 minutes." Benchmarking against real-time transition timing, not sliced-up windows.

Our own model — NeuroWave Gen 6 with a 10-minute patient calibration step — was evaluated under SzPredict's spec using our internal pipeline (Protocol 3 + Protocol 4 metrics, identical labelling rules). It reaches a 27.5 minute median preictal lead time on held-out patients (15 seizures, 5 patients the model never saw, 73% detected in advance). Adapting NeuroWave to consume SzPredict's standard window interface — so the result is independently reproducible from the public repo — is in progress for v0.2. Trained across progressively constrained hardware (Dell C4130 with 4×P100 → Surface Pro 5 CPU-only → Ryzen 5900X + A100 32GB) — under \$2,000 total, no cloud compute. The constraints forced architectural choices toward efficient inference, which happens to be the same constraint wearable seizure-warning hardware requires.

The Rosetta Stone

Across 19 recent CHB-MIT seizure prediction papers, reported sensitivity ranges from 58% to 99.96%. The spread isn't a measure of model quality — it's a measure of protocol heterogeneity. The same dataset, evaluated under different task definitions, preictal windows, splits, and post-processing, produces numbers that look vastly different but are solving vastly different problems. Below, 10 representative papers alongside the axes that actually determine what their numbers mean.

None of these numbers are wrong. Most aren't directly comparable. A paper reporting 99.96% sensitivity isn't necessarily better than one reporting 58.55% — they're often solving different problems. Rukhsar merges preictal+ictal into one positive class. Dissanayake's 58.55% is the LOPO result hidden behind their patient-independent 91.54% headline. Xu 2021's 98.8% is on 7 hand-picked patients with random CV.

SzPredict pins every axis. Your model's Protocol 3 result becomes a single number any other Protocol 3 result can be measured against directly. That's the rosetta stone.

For the full 19-paper translation table with all 10+ heterogeneity axes, see the lit review summary on GitHub.

A field consensus hiding in plain sight

In our lit review, six independent groups reinvented the same post-processing idea: 'require sustained positive predictions before firing an alarm.' Each called it something different:

• Dissanayake 2020 — Hanning smoothing
• Ben Messaoud 2020 — Sustainability rule (T* threshold for n* consecutive epochs)
• Liu & Richardson 2021 — Weighted Majority Voting
• Bhattacharya 2022 — k-of-n (k=8, n=10 over 300s)
• Xu 2023 — Accumulative probability
• Shu 2024 — k-of-n + 30-min refractory

Quantified effect (Liu 2021): temporal smoothing reduces detection FPR by 4.4× and prediction FPR by 3.3×. It's mandatory for clinical deployment — but every paper implements it differently, and published 'model' results usually don't disclose which smoothing was applied.

SzPredict logs post-processing as a first-class rosetta axis. Submissions declare their smoothing explicitly. You get to see the model performance separately from the wrapper.

Why this matters beyond seizure prediction

Honest evaluation is a safety problem for any ML system deployed into decisions with real consequences. Seizure prediction is one domain; the pattern is general. When evaluation methodology inflates reported performance — through cherry-picked cohorts, leaky cross-validation, missing specificity numbers, or silent post-processing — the model fails in deployment and the field learns nothing from the failure.

SzPredict's design choices (cross-patient primary evaluation, clinical-utility metric in Protocol 4, methodology rubric on each submission, transparent baselines including degenerate ones) are general-purpose answers to the evaluation-integrity problem, translated here to EEG seizure prediction. The same discipline applies to any ML domain where deployment consequences are asymmetric and the cost of being fooled by your own metric is real.

Dataset

CHB-MIT Scalp EEG Database — 22 paediatric subjects, 844+ hours of continuous EEG recorded at Children's Hospital Boston. The most widely used public dataset for seizure prediction research.

Protocol 3 splits: 12 patients for training, 5 patients for evaluation. 33 GB total prepared data (21 GB train, 12 GB validation).