Problem
sil-lift currently generates no timestamps of its own. dateCreated/dateModified are read from the source and written back verbatim; the only other root attributes are version (fixed at 0.13) and producer (caller-supplied). Serialization is therefore fully deterministic today — the same input plus the same in-memory edits always produce the same bytes, which is what makes the byte-exact fidelity contract testable.
Any feature that generates a timestamp from the wall clock breaks that. A dateModified stamped with datetime.now(...) means two runs of the same script over the same input produce different bytes, which:
- makes byte-exact round-trip tests of the stamping feature itself unwritable without monkeypatching the clock;
- defeats diff-based CI gates that assert an export is unchanged;
- makes container/Action runs non-reproducible, which matters because sil-lift ships both a
Dockerfile and a GitHub Action (action.yml) explicitly aimed at pipeline use.
Proposed solution
Honor SOURCE_DATE_EPOCH as the default clock source anywhere the library generates a timestamp, with explicit arguments always taking precedence.
Precedence: an explicit when= argument > SOURCE_DATE_EPOCH > wall clock.
def _default_now() -> datetime:
"""The clock for generated timestamps: SOURCE_DATE_EPOCH if usable, else now."""
raw = os.environ.get("SOURCE_DATE_EPOCH")
if raw is not None:
try:
return datetime.fromtimestamp(int(raw.strip()), tz=timezone.utc)
except (ValueError, OverflowError, OSError):
pass # a malformed value is not honored; fall through to the wall clock
return datetime.now(timezone.utc).replace(microsecond=0)
Two properties make this a clean fit:
SOURCE_DATE_EPOCH is integer POSIX seconds, so the result is naturally seconds-precision — no truncation needed, and no fractional-second output to worry about.
_fmt_date in src/sil_lift/_writer.py already rewrites +00:00 to Z, so an aware UTC datetime serializes as YYYY-MM-DDTHH:MM:SSZ. That is the shape real FieldWorks exports use: across the seven FLEx 8.3–9.0 exports in The Combine's Backend.Tests/Assets, all 70,636 dateCreated/dateModified literals are exactly that 20-character form, with no bare dates, numeric offsets, or fractional seconds.
The variable is already set by Debian, Nix, Buildah, setuptools, and Sphinx among others, so pipelines that set it for other tools get sil-lift determinism for free rather than needing sil-lift-specific configuration.
Dependency
This is only observable once something in sil-lift generates a timestamp; nothing does today. It is a prerequisite or companion to any timestamp-stamping API rather than an independently visible change — and arguably a prerequisite rather than a follow-up, since it is what makes byte-exact tests of such a feature possible without clock monkeypatching.
Open questions
- Malformed values: the spec permits a consumer to either error or ignore. The sketch above ignores and falls through, which keeps a library from raising on an environment variable the caller may not control — but it means a typo silently reintroduces nondeterminism. An alternative is to ignore it but surface a
Problem-style warning, which fits the existing validation vocabulary.
- Scope of "generated timestamp":
dateModified only, or dateCreated too? Both are generated in a from-scratch export, so both presumably.
- Forcing the wall clock: with the env var set, is passing
when=datetime.now(timezone.utc) explicitly the sanctioned escape hatch, or is a dedicated override wanted?
- CLI surface: whether the
validate/export commands should also accept an explicit --now value, which would document the behavior in --help where an environment variable is invisible.
- Documentation home: the reproducibility guarantee belongs next to the fidelity contract in
docs/en/fidelity.md, since that is where readers go for byte-exactness claims.
Alternatives considered
- A sil-lift-specific variable (e.g.
SIL_LIFT_NOW). Rejected: it duplicates an established cross-ecosystem standard, and a pipeline already pinning SOURCE_DATE_EPOCH would have to learn a second name for the same concept.
- An injectable clock parameter and nothing else. A
when= parameter should exist regardless, but it does not help when the timestamp is generated inside a call the pipeline does not control — the CLI subcommands and the Docker/Action entrypoint being the concrete cases here.
- Making generated timestamps opt-in only, and declaring reproducibility the caller's problem. Viable, but it pushes every pipeline that wants both stamping and determinism into monkeypatching or post-processing, and it leaves the library's own test suite without a supported way to exercise stamping byte-exactly.
Problem
sil-lift currently generates no timestamps of its own.
dateCreated/dateModifiedare read from the source and written back verbatim; the only other root attributes areversion(fixed at0.13) andproducer(caller-supplied). Serialization is therefore fully deterministic today — the same input plus the same in-memory edits always produce the same bytes, which is what makes the byte-exact fidelity contract testable.Any feature that generates a timestamp from the wall clock breaks that. A
dateModifiedstamped withdatetime.now(...)means two runs of the same script over the same input produce different bytes, which:Dockerfileand a GitHub Action (action.yml) explicitly aimed at pipeline use.Proposed solution
Honor
SOURCE_DATE_EPOCHas the default clock source anywhere the library generates a timestamp, with explicit arguments always taking precedence.Precedence: an explicit
when=argument >SOURCE_DATE_EPOCH> wall clock.Two properties make this a clean fit:
SOURCE_DATE_EPOCHis integer POSIX seconds, so the result is naturally seconds-precision — no truncation needed, and no fractional-second output to worry about._fmt_dateinsrc/sil_lift/_writer.pyalready rewrites+00:00toZ, so an aware UTC datetime serializes asYYYY-MM-DDTHH:MM:SSZ. That is the shape real FieldWorks exports use: across the seven FLEx 8.3–9.0 exports in The Combine'sBackend.Tests/Assets, all 70,636dateCreated/dateModifiedliterals are exactly that 20-character form, with no bare dates, numeric offsets, or fractional seconds.The variable is already set by Debian, Nix, Buildah, setuptools, and Sphinx among others, so pipelines that set it for other tools get sil-lift determinism for free rather than needing sil-lift-specific configuration.
Dependency
This is only observable once something in sil-lift generates a timestamp; nothing does today. It is a prerequisite or companion to any timestamp-stamping API rather than an independently visible change — and arguably a prerequisite rather than a follow-up, since it is what makes byte-exact tests of such a feature possible without clock monkeypatching.
Open questions
Problem-style warning, which fits the existing validation vocabulary.dateModifiedonly, ordateCreatedtoo? Both are generated in a from-scratch export, so both presumably.when=datetime.now(timezone.utc)explicitly the sanctioned escape hatch, or is a dedicated override wanted?validate/exportcommands should also accept an explicit--nowvalue, which would document the behavior in--helpwhere an environment variable is invisible.docs/en/fidelity.md, since that is where readers go for byte-exactness claims.Alternatives considered
SIL_LIFT_NOW). Rejected: it duplicates an established cross-ecosystem standard, and a pipeline already pinningSOURCE_DATE_EPOCHwould have to learn a second name for the same concept.when=parameter should exist regardless, but it does not help when the timestamp is generated inside a call the pipeline does not control — the CLI subcommands and the Docker/Action entrypoint being the concrete cases here.