Skip to content

feat(pose): SLEAP and DeepLabCut become installable plugins - #148

Merged
gbradham merged 1 commit into
mainfrom
feat/pose-converter-plugins
Aug 25, 2026
Merged

feat(pose): SLEAP and DeepLabCut become installable plugins#148
gbradham merged 1 commit into
mainfrom
feat/pose-converter-plugins

Conversation

@gbradham

Copy link
Copy Markdown
Member

Four installable plugins, and pose conversion becomes an extension point.

Core runs ONNX. A plugin turns one vendor's folder into it. That division is
the change; everything else follows from it. It is what lets each vendor's
framework stay out of GLIDER's dependency tree — TensorFlow is a large thing to
hand a lab that tracks with YOLO and has never opened SLEAP.

Plugin Provides Dependencies
glider-harp driver, device harp-protocol
glider-maimu device, node bleak
glider-sleap pose converter tensorflow-cpu, tf2onnx
glider-dlc pose converter none

The extension point

A new glider.pose entry-point group and a POSE_CONVERTERS registry, shaped
exactly like the driver, device and node registries already there.

A converter answers three questions about a folder — is it mine, is what I made
from it still current, please convert it — and answers the first two from
paths alone
. That constraint is the load-bearing one: claims() is asked on
every model selection, including for folders belonging to other vendors, and
importing TensorFlow to decide would cost seconds on the common path where the
answer is no. There is a test for it in each plugin.

Core needed no decoder changes. The sidecar already parameterises both
decoders, and YOLO stays in core.

glider-sleap

The existing core converter, moved out, unchanged in behaviour: point at the
folder SLEAP wrote and it still converts on first selection and runs.
pip install glider-sleap replaces the glider[sleap] extra, which is gone.

It still does not depend on sleap, and does not need to — a SLEAP model is an
ordinary Keras checkpoint.

The dependency floors moved with it and are not cosmetic:

  • tensorflow-cpu>=2.20 — earlier releases publish no cp313 wheels, so a lower
    floor would silently cap this plugin below GLIDER's own Python range. (I had
    it capped at <3.13 first, on a stale belief that TF has no 3.13 wheels at
    all. It does, from 2.20.)
  • tf2onnx>=1.17 — 1.16 cannot convert a Keras 3 model, which is what any
    TensorFlow new enough to be worth installing loads a checkpoint into.

Verified end to end on tensorflow-cpu 2.21 + tf2onnx 1.17.

glider-dlc

New, and carries no dependencies at all. DeepLabCut is needed to read a
snapshot — a checkpoint is a bare PyTorch state_dict, so rebuilding the
network needs DLC's own model classes — but it brings about 1.3 GB with it.

So instead of carrying that, the first conversion builds a private Python 3.12
environment under ~/.glider/envs/deeplabcut with uv and converts in there.
GLIDER's own environment is untouched. GLIDER_DLC_ENV points at an existing
DeepLabCut venv and skips the download entirely.

A new optional preflight hook puts the size in the confirmation dialog before
anything downloads. It is deliberately not on the converter Protocol, so
adding it cannot make an existing converter stop satisfying the check.

Two things in the converter worth a reviewer's attention:

The output stride is measured from the network, not read from its config.
It is the backbone's stride divided by whatever the head's deconvolutions undo,
and a ResNet-50 and an HRNet-w32 trained on the same data differ by a factor of
four. It is measured from two input sizes, because every DLC head adds one
cell past the edge (out = in/stride + 1) — a single ratio reads a stride-8
ResNet as 7.53 at 128 px and 7.88 at 512 px. The difference cancels the
constant and lands on 8 exactly.

The snapshot is looked for beside its own config, not anywhere under the
selected folder. A project with two training runs has two of each, and pairing
one run's config with another's weights is not an error that surfaces:
load_state_dict accepts it whenever the shapes happen to match, and the model
then answers with a network nobody trained for this. The search is also
depth-bounded — a DLC project holds labeled-data with thousands of frames in
it, and this runs on every model selection.

Nothing in either sidecar is defaulted. A wrong stride or normalisation does not
fail; it shifts every keypoint by a constant and still draws a plausible
skeleton, which is the failure worth refusing outright.

Verified against DeepLabCut 3.0.1

torch 2.13, Python 3.12, both backbone families:

stride ONNX vs. DeepLabCut dynamic input staleness
ResNet-50 8 2.5e-06 relative ✅ non-square
HRNet-w32 2 2.1e-06 relative ✅ non-square

Relative, not absolute: an untrained HRNet emits values around 1e7, where an
absolute tolerance says nothing about whether the graph matches.

PyPI

glider-maimu, glider-sleap and glider-dlc each get a release workflow
using Trusted Publishing, exactly as glider-harp already does — no API token
exists anywhere. All three names are available on PyPI; all three build and pass
twine check locally.

Each release asserts the built wheel's entry-point group and name. That is
the one failure that is otherwise invisible: a plugin registered under the wrong
name loads without error and does nothing, which is how glider-maimu shipped
once. The assertion was tested against a deliberately-corrupted wheel and
catches exactly that shape.

Three near-identical workflow files on purpose. PyPI matches its trusted
publisher against the workflow that requests the token and does not support
reusable workflows
(warehouse#11096),
so factoring the shared body into a workflow_call would stop publishing
working at all.

Before the first release, three pending publishers are needed

At https://pypi.org/manage/account/publishing/Add a new pending
publisher
, once per plugin:

PyPI project name Owner Repository Workflow name Environment
glider-maimu LaingLab glider release-glider-maimu.yml pypi
glider-sleap LaingLab glider release-glider-sleap.yml pypi
glider-dlc LaingLab glider release-glider-dlc.yml pypi

Then, after merging:

git tag glider-maimu-v0.1.0 && git push origin glider-maimu-v0.1.0
git tag glider-sleap-v0.1.0 && git push origin glider-sleap-v0.1.0
git tag glider-dlc-v0.1.0   && git push origin glider-dlc-v0.1.0

Each workflow also has a workflow_dispatch → TestPyPI option for a dry run
first. A tag that disagrees with the declared version fails before the build,
because PyPI never lets a version be re-uploaded.

Also

glider-maimu added to the bundled plugin catalogue. It was installable but
absent from it, so it never appeared in the Plugins window at all — which is
what you hit earlier.

Scope

Single animal, as agreed. Multi-animal SLEAP models (top-down, bottom-up) and
multi-animal DLC heads (paf, identity) are reported by name rather than
converted wrongly. DeepLabCut 2.x is TensorFlow rather than PyTorch and is not
converted automatically; a 2.x folder is recognised and says so rather than
reporting a missing pytorch_config.yaml.

Testing

4438 passed, 4 skipped, 0 failures across core and all four plugins. Ruff
and black clean.

The plugin suites run without the frameworks they convert for — that is the
point of the plugins, so it is also the point of the tests. glider-dlc fakes
every uv call and checks the decision-making around them: when to build, when
not to, what to do with a half-built environment, and what a person is told when
it fails. The one test that performs a real DLC conversion is marked slow and
skips wherever DeepLabCut is not importable, including CI.

CI installs all four plugins. glider-sleap goes in with --no-deps — its one
TensorFlow test skips without it, and downloading TF on nine matrix jobs to skip
one test is not a trade worth making.

One PR, not two

I offered to split this into extension-point-plus-SLEAP and then DLC. I have
kept it as one: the DLC half is what motivates preflight, and reconstructing
a green intermediate state I never actually ran the suite against would be
inventing a commit rather than splitting one. Happy to split if you would
rather review it in halves.

Core runs ONNX; a plugin turns one vendor's folder into it. That division
is the point of this change, and it is what lets each vendor's framework
stay out of GLIDER's dependency tree -- TensorFlow is a large thing to
hand a lab that tracks with YOLO and has never opened SLEAP.

A new `glider.pose` entry-point group and a POSE_CONVERTERS registry,
shaped exactly like the driver, device and node registries already there.
A converter answers three questions about a folder from *paths alone* --
is it mine, is what I made from it still current, please convert it --
because it is asked on every model selection, including for folders
belonging to other vendors, and importing TensorFlow to decide that would
cost seconds on the common path where the answer is no.

glider-sleap is the existing core converter moved out, unchanged in
behaviour: point at the folder SLEAP wrote and it still converts on first
selection. `pip install glider-sleap` replaces the `glider[sleap]` extra,
which is gone. It still does not depend on sleap. The dependency floors
moved with it and are not cosmetic -- tensorflow-cpu>=2.20 because earlier
releases publish no 3.13 wheels and would have capped the plugin below
GLIDER's own Python range, and tf2onnx>=1.17 because 1.16 cannot convert
the Keras 3 model any usable TensorFlow loads a checkpoint into.

glider-dlc is new, and carries no dependencies at all. DeepLabCut is
needed to read a snapshot -- a checkpoint is a bare PyTorch state_dict, so
rebuilding the network needs DLC's own model classes -- but it brings
1.3 GB with it. So the first conversion builds a private Python 3.12
environment under ~/.glider/envs with uv and converts in there. A new
optional `preflight` hook puts that size in the dialog before anything is
downloaded; it is deliberately not on the converter Protocol, so adding it
cannot make an existing converter stop satisfying the check.
GLIDER_DLC_ENV points at an existing DeepLabCut venv and skips it all.

Two things in the DLC converter are worth knowing about:

The output stride is measured from the network, not read from its config.
It is the backbone's stride divided by whatever the head's deconvolutions
undo, and a ResNet-50 and an HRNet-w32 trained on the same data differ by
a factor of four. Measured from *two* input sizes, because every DLC head
adds one cell past the edge (out = in/stride + 1) -- a single ratio reads
a stride-8 ResNet as 7.53 at 128 px and 7.88 at 512 px.

The snapshot is looked for beside its own config rather than anywhere
under the selected folder. A project with two training runs has two of
each, and pairing one run's config with another's weights is not an error
that surfaces: load_state_dict accepts it whenever the shapes match, and
the model then answers with a network nobody trained for this. The search
is also depth-bounded, because a DLC project holds labeled-data with
thousands of frames and this runs on every model selection.

Nothing in either sidecar is defaulted. A wrong stride or normalisation
does not fail -- it shifts every keypoint by a constant and still draws a
plausible skeleton, which is the failure worth refusing outright.

Verified against DeepLabCut 3.0.1 (torch 2.13, py3.12) for both backbone
families: ResNet-50 at stride 8 and HRNet-w32 at stride 2, each matching
DLC's own inference to within 3.1e-06 relative, exporting with dynamic
height and width, and correctly reading a retrained snapshot as stale.
SLEAP conversion was verified earlier against SLEAP's own
minimal_robot.UNet.single_instance fixture to 7e-7.

Also here:

- Release workflows for glider-maimu, glider-sleap and glider-dlc, using
  Trusted Publishing like glider-harp already does. Each asserts the built
  wheel's entry-point group and name, which is the one failure that is
  otherwise invisible: a plugin registered under the wrong name loads
  without error and does nothing, which is how glider-maimu shipped once.
  Three near-identical files on purpose -- PyPI matches its publisher
  against the workflow requesting the token and does not support reusable
  workflows (warehouse#11096).
- glider-maimu added to the bundled plugin catalogue. It was installable
  but absent from it, so it never appeared in the Plugins window at all.
@gbradham
gbradham merged commit bbd7e2f into main Aug 25, 2026
4 checks passed
@gbradham
gbradham deleted the feat/pose-converter-plugins branch August 25, 2026 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant