Skip to content

Suggestion: add pydantic versioned schema migration + base DLC schemas #48

Description

@C-Achard

This is a reminder/centralized post regarding where to host DLC config/project schemas and related tools.

Several of our libraries in the software stack could benefit from having access to:

  • DLC project layout/file structure schemas
  • DLC config schemas (available in main DLC)
  • Versioned schema migration code (available in main DLC)
  • DLC keypoint/data spec schema(s)

and potentially other pieces of code.

Criteria for inclusion in DLCLibrary (requires discussion, this is an outline):

  • Lighweight dependencies
    • No core DLC
  • Useful to at least two DLC repos in a concrete way
    • e.g. repo-specific migration logic should be hosted at the lowest stage where needed only
    • Adding code "for later" should be strictly documented and motivated, and ideally have an associated WIP PR in involved repos
  • Should have one central domain-correct behavior
    • Centralizing prevents incompatible interpretations across repositories
    • DLCLibrary should be a source of truth, not a copy
    • Existing implementations should migrate to importing it, adding here should have corresponding migration logic ready downstream ideally
    • Must not become a drawer of loosely useful helpers that may be repo specific/general-purpose code
    • In general adding code here should be heavily scrutinized, only the bare required minimum should be accepted
  • Testing can also be done in this repo on a different level:
    • Public schemas and migration behavior can be versioned and tested
    • Breaking changes require an explicit release and migration path
    • Round-trip and compatibility tests
    • Unknown or newer schema versions must fail clearly if broken/problematic

Metadata

Metadata

Labels

enhancementNew feature or request

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions