Skip to content

Add PTB-XL dataset support (ECG) - #1186

Closed
AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:feature/ptbxl-dataset
Closed

Add PTB-XL dataset support (ECG)#1186
AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:feature/ptbxl-dataset

Conversation

@AxelNoun

@AxelNoun AxelNoun commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Contributor: AxelNoun (GitHub: @AxelNoun) — external contributor, no NetID
Contribution Type: New Dataset + Task
Description: Adds PTB-XL (12-lead ECG, PhysioNet open access) as a
PyHealth dataset, with a 5-superclass multi-label diagnosis task and a
split helper using the official stratified folds. Motivated by the CPBench
effort: it gives conformal prediction a multi-label ECG benchmark that needs
no credentialing, with metadata suited to covariate/label-shift splits.
Based on Wagner et al., Scientific Data 2020
(https://www.nature.com/articles/s41597-020-0495-6).

Files to Review:

  • pyhealth/datasets/ptbxl.py — dataset implementation
  • pyhealth/datasets/configs/ptbxl.yaml — table config (template)
  • pyhealth/tasks/ptbxl.py — 5-superclass multi-label task
  • pyhealth/datasets/splitter.pysplit_by_strat_fold helper
  • tests/core/test_ptbxl.py — tests with synthetic fixtures
  • docs/api/datasets/pyhealth.datasets.ptbxl.rst, docs/api/tasks/pyhealth.tasks.ptbxl.rst
  • examples/ecg/ptbxl/ptbxl_superclass_quickstart.py

Implementation notes:

  • wfdb is an optional extra (pip install pyhealth[ptbxl]), lazily imported.
    Happy to make it a hard dependency if you'd prefer.
  • Derived metadata CSV goes to MODULE_CACHE_PATH, not root. Filenames include
    sampling rate and a short hash of the resolved data root so different roots
    never share a cache; reuse also validates signal_file ⊆ data_root.
  • Resolved cache YAML (new vs EEGBCI): BaseDataset only accepts
    config_path, so PTB-XL writes a root+rate-keyed YAML next to the derived CSV
    (ptbxl-config-{rate}hz-{root_hash}.yaml) before super().__init__(), with
    file_path already pointing at the derived metadata CSV. EEGBCI instead
    patches self.config.tables[...].file_path after super(). Open to aligning
    with maintainers' preferred pattern.
  • Unrelated pre-existing # noqa: E402 on the EEGBCI import line in
    pyhealth/datasets/__init__.py — left untouched (out of diff).
  • Signals returned as (n_leads, n_samples), matching EEGBCI's convention.
  • Age ≥90 is encoded as 300 per HIPAA; kept raw with age_is_censored,
    distinct from age_is_missing (written as pd.NA).
  • Empty label sets (407/21799) dropped by default via drop_empty_labels.
  • Fixture under test-resources/ptbxl/ is synthetic WFDB (.hea/.dat)
    plus tiny CSVs covering censored age, missing age, multilabel, and empty superclass.

Testing / CI:

  • Local: pytest tests/core/test_ptbxl.py17 passed (with wfdb installed).
  • Without wfdb / pyhealth[ptbxl], the 2 waveform I/O tests skip; metadata /
    task / e2e event-read tests still run.
  • pixi test feature pins wfdb so make testall under pixi covers waveform I/O;
    CI paths that do not install that extra will not exercise load_ptbxl_record.

Follow-up: 71-SCP-code classification and age regression are structured
for but not implemented here — planned as a separate PR.

Test plan

  • pytest tests/core/test_ptbxl.py — 17 passed (with wfdb)
  • python tools/check_pr_rules.py --base upstream/master --head HEAD
  • CI make testall / contribution-rules on the PR

@AxelNoun
AxelNoun force-pushed the feature/ptbxl-dataset branch from f371eea to 23febb4 Compare August 12, 2026 19:00
@AxelNoun

Copy link
Copy Markdown
Contributor Author

Note for reviewers: there is an unrelated pre-existing # noqa: E402 on the EEGBCI import line in pyhealth/datasets/__init__.py; left untouched because it is out of this PR's diff.

@AxelNoun
AxelNoun force-pushed the feature/ptbxl-dataset branch from 23febb4 to 55b29e4 Compare August 12, 2026 19:21
AxelNoun and others added 4 commits August 12, 2026 21:23
Introduce PTBXLDataset (BaseDataset + YAML) with optional wfdb extra,
root-keyed metadata cache, and a resolved cache YAML so file_path is
correct before BaseDataset init.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add PTBXLSuperclassClassification and split_by_strat_fold (folds 1-8 /
9 / 10) in separate modules so they can move to benchmarks later.

Co-authored-by: Cursor <cursoragent@cursor.com>
Cover helpers, metadata cache keyed by root, event reads via
PTBXLDataset, waveform IO (wfdb), and the 5-superclass task path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document the dataset/task Overview RST pages and a minimal ECG
superclass example for CPBench users.

Co-authored-by: Cursor <cursoragent@cursor.com>
@AxelNoun
AxelNoun force-pushed the feature/ptbxl-dataset branch from 55b29e4 to 47aa3a4 Compare August 12, 2026 19:29
@AxelNoun AxelNoun closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant