Upload exactly the server-validated package - #1806
Open
PaulHax wants to merge 1 commit into
Open
Conversation
Classify each selected file into a semantic upload role on the server, then upload exactly what that classification accepts and report the rest, instead of re-deriving the set in the browser.
This was referenced Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of a four-PR stack. Foundation only: no attachment semantics or frame metadata yet.
Summary
Make server validation authoritative for the browser upload package.
Upload validated one interpretation of the selected files, then rebuilt the package in the browser
with separate rules. The two could disagree: an accepted file could be dropped, an ignored file
could still upload, and ordinary and multicamera imports kept different side files.
validate_filesnow returns semanticroles(media, annotations, dataset configuration) plusignoredwith a per-file reason. The browser uploads the selection minus the ignored names — onederivation, no second classifier, same contract for ordinary and multicamera camera-folder uploads.
Files that upload today still upload; a skipped file is now listed with its reason instead of
vanishing.
The per-role slots stay; their contents and meaning change.
processImportis deleted andsuggestUploadSlotsreplaces it as a convenience split that is never trusted — slots arere-validated at Start upload, and the server decides media type, folder name, fan-out, and what
uploads. A rejected selection now yields a row in an error state rather than a discarded pick,
since the slot editor is the only place to fix it without reopening the OS picker.
Four intentional behavior changes ride along:
annotation extensions alone, so
.yml,.yaml,.json, and.csvwere selectable but.txtwas not — the browser silently withheld a file the server would have classified, which is the
drift this PR exists to remove. The filter is now the union of the annotation and metadata
attachment extensions.
media/annotationslists are replaced byroles+ignored. Bothin-tree consumers (browser upload, girder-worker zip path) are updated here; external callers of
this endpoint must adopt the new shape.
mainaccepted and then mishandled viasilent last-write-wins or an ambiguous config pick are now rejected explicitly: two annotation
JSONs, CSV + JSON, YAML + JSON, more than one YAML, two config JSONs. Everything else validates as
before.
UploadGirderemitted a row index whileUploadexpected the row object, sosplice(-1, 1)retired the last queued row instead of the finished one. A pre-existingmainbug, fixed here because per-row error state makes retiring the wrong row destructive.
The zip path is unchanged: validation gates the whole archive and every extracted file uploads. Only
interactive upload drops
ignoredfiles — the same filesmainalready dropped, silently.Reviewer focus
pre-empts it by withholding a selectable side file.
main; the five new rejections are the intended change.side files
main's media-only path dropped.the displaced folder copy is reported as ignored. Re-picking the folder's own file displaces
nothing.
import menu opened until the server confirms it), never uploads into an unnamed folder.
Manual test
Uses
okeanos-upload-authority-ignorefromokeanos-frame-metadata-fixtures.zip, attached to thisPR — CC0 NOAA Okeanos Explorer media with synthetic metadata and annotations. Select all files in
the directory.
okeanos-frame-metadata-fixtures.zip
notes.txtis listed under "Ignored (not uploaded)", below the pickers, with theunsupported-side-file reason.
notes.txtuploaded as neither media, annotation,nor dataset config.