build: complete warm-build hashes.txt coverage (fix #2174 gaps; busybox symlinks; doc cleanup) by tlaurion · Pull Request #2180 · linuxboot/heads

tlaurion · 2026-08-06T19:29:46Z

Advances the reproducibility work from #2174 and supersedes the closed #2178.

#2174 added FORCE to modules.cpio for complete warm-build hashes.txt, but three gaps remained:

bzImage, tools.cpio, and initrd.cpio.xz lacked FORCE -- their hashes missing on warm builds
The busybox reproducible install change silently broke all 137 applet symlinks, leaving only the bare binary in the initrd

What each commit does

Commit 1 (build: improve clean target @echo messages and fix reproducibility doc, supersedes #2178):

Clearer Makefile clean target messages
Doc restructured into same/different-commit verification sections
Fixed SOURCE_DATE_EPOCH claim (always 0, not pinned commit epoch)
Restored bzImage in step-down path

Commit 2 (modules/linux: add FORCE and UNCHANGED guard to bzImage rule, completes #2174):

FORCE on bzImage copy/hash rule -- matching modules.cpio precedent
cmp-based UNCHANGED guard with plain cp -a (no tmp hash noise)
First-build cmp guard with [ ! -f "$@" ] (no stderr noise)

Commit 3 (modules/busybox: fix missing applet symlinks, fixes regression from #2174):

Regenerates busybox.links before install.sh (was missing after cache restore/make clean)
Uses include/autoconf.h + HOSTCC=gcc + test -s guard

Commit 4 (Makefile: add FORCE to tools.cpio and initrd.cpio.xz, completes #2174):

Last two hashes.txt writers without FORCE
filter-out FORCE prevents initrd recipe from leaking FORCE into cpio-clean.pl

Verification

Local warm build at 792cb42f8b0: clean, all UNCHANGED patterns fire, 137 busybox symlinks present, no errors.

CI cross-check: [pending]

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR appears focused on improving build reproducibility and robustness by avoiding unnecessary rebuild output changes, regenerating derived BusyBox metadata during install, and documenting deterministic timestamp behavior.

Changes:

Make kernel image copying content-aware (skip updating output when unchanged).
Regenerate busybox.links during BusyBox install to survive clean/cache restores.
Force rebuild of certain initrd artifacts and document SOURCE_DATE_EPOCH behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
modules/linux	Copies kernel image via temp+cmp to avoid rewriting output when unchanged.
modules/busybox	Rebuilds `busybox.links` during install before creating symlinks.
doc/reproducible-builds.md	Updates guidance to use `SOURCE_DATE_EPOCH=0` without `.git`.
Makefile	Adds `FORCE` rebuild behavior and filters `FORCE` out of initrd input list.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tlaurion · 2026-08-06T20:11:47Z

 $(initrd_bin_dir)/busybox: $(build)/$(busybox_dir)/.build
+	# Regenerate busybox.links (may be missing after clean/cache restore)
 	$(call do,INSTALL,bin/busybox,\
 		cp $(build)/$(busybox_dir)/busybox \
 			$(initrd_bin_dir)/busybox && \
 		cd $(build)/$(busybox_dir) && \
+		HOSTCC="$(busybox_hostcc)" $(SHELL) applets/busybox.mkll include/autoconf.h include/applets.h > busybox.links && \
+		test -s busybox.links && \
 		$(SHELL) applets/install.sh $(initrd_bin_dir)/.. --symlinks \
 	)


Already fixed in the current branch — comments moved before the $(call do,INSTALL) block, outside the \ continuation chain. Build passes (exit 0).

tlaurion · 2026-08-06T20:11:48Z

+$(build)/$(BOARD)/$(LINUX_IMAGE_FILE): $(build)/$(linux_dir)/.build FORCE
+	$(call do,INSTALL  ,$< => $@.tmp,\
+		cp -a "$(dir $<)/$(linux_output)" "$@.tmp" \
+	)
+	@if [ ! -f "$@" ] || ! cmp --quiet "$@.tmp" "$@" ; then \
+		mv "$@.tmp" "$@" ; \
+		touch "$@" ; \
+	else \
+		echo "$(DATE) UNCHANGED $(@:$(pwd)/%=%)" ; \
+		rm "$@.tmp" ; \
+	fi
 	@sha256sum "$@" | tee -a "$(HASHES)"
 	@stat -c "%8s:%n" "$@" | tee -a "$(SIZES)"


hashes.txt is reset at the start of every build (Makefile:180: BOARD_LOG := ... > "$(HASHES)"). The tee -a appends to a fresh file each invocation — no accumulation across runs. FORCE ensures the entry is always there; without it, warm builds silently omit bzImage.

tlaurion · 2026-08-06T20:11:49Z

 $(build)/$(initrd_dir)/tools.cpio: \
 	$(initrd_bins) \
 	$(initrd_libs) \
-	$(initrd_tools_dir)/etc/config
+	$(initrd_tools_dir)/etc/config \
+	FORCE
 	$(call do-cpio,$@,$(initrd_tools_dir))
 	@$(RM) -rf "$(initrd_tools_dir)"


tools.cpio uses do-cpio (Makefile:392-419) which already has a cmp-based UNCHANGED guard — FORCE makes the recipe run, but cmp --quiet skips the mv when content is identical. On a same-commit warm rebuild, tools.cpio prints UNCHANGED and takes ~0.3s (verified in build log). The cpio build is not repeated.

tlaurion · 2026-08-06T20:11:50Z

+$(build)/$(initrd_dir)/initrd.cpio.xz: $(initrd-y) FORCE
 	$(call do,CPIO-XZ  ,$@,\
 	$(pwd)/bin/cpio-clean.pl \
-		$^ \
+		$(filter-out FORCE,$^) \
 	| xz \


Same UNCHANGED guard exists at Makefile:934-939 — cmp --quiet on the xz output prevents mv when identical. FORCE ensures the recipe runs, but the expensive cpio-clean | xz pipeline result is compared and skipped when unchanged. Verified in warm build log.

tlaurion · 2026-08-06T20:11:50Z

 alignment).  `--with-debug-prefix-map=$(pwd)=.` normalizes build paths in debug
 info.  `--enable-compressed-debug-sections=no` disables zlib debug-section
-compression.  `SOURCE_DATE_EPOCH` from the pinned musl-cross-make commit epoch
+compression.  `SOURCE_DATE_EPOCH=0` (extracted tarballs lack .git; git log always falls back to 0)


Good catch. The current text "git log always falls back to 0" is inaccurate — git log fails when .git is missing, and the || echo 0 fallback handles it. Will fix the wording to describe the actual behavior more precisely.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

modules/busybox:39

The new comment says the cross-compiler is "identical to host gcc" for header parsing, but this rule actually sets HOSTCC to $(heads_cc), which includes target sysroot/flags and is not the same as host GCC. Consider rewording the comment to match what the code is doing (use the in-tree toolchain to avoid depending on host tools).

	# Regenerate busybox.links (may be missing after clean/cache restore)
	# mkll only runs the preprocessor (-E): cross-compiler is
	# identical to host gcc for text-only header parsing and keeps
	# the build chain self-contained (no host tools assumed)

…t output files The prior wording "git log always falls back to 0" is misleading -- git log does not fall back; it fails when .git is missing, and the || echo 0 shell construct in modules/musl-cross-make sets the value. Describe the actual mechanism: the build system cannot derive a commit timestamp from extracted tarballs, so modules/musl-cross-make falls back to echo 0 when git log fails. Also add a new 'Output files' section in doc/reproducible-builds.md describing what each hash-related file (hashes.txt, sizes.txt, sha256sum.txt) contains and how they relate to reproducibility verification. Signed-off-by: Thierry Laurion <insurgo@riseup.net>

… cmp stderr) The FORCE prerequisite (already in master) ensures the rule always runs, but the do-copy call produced sha256sum/stat noise for the temporary file, and cmp emitted stderr on first build when the target did not exist. - Replace do-copy with plain cp -a + INSTALL progress line (no tmp hash/stat noise in build log) - Guard cmp with [ ! -f "$@" ] || so the first build takes the "changed" branch without invoking cmp at all Signed-off-by: Thierry Laurion <insurgo@riseup.net>

mkll only runs the preprocessor (-E) on already-generated config headers, producing a text-only applet list -- no binaries, no cross-compilation needed. The host gcc and the musl-cross compiler produce identical preprocessor output for this task. Replace HOSTCC="$(busybox_hostcc)" (host gcc via variable) with HOSTCC="$(heads_cc)" (musl-cross compiler used by every other build step). The variable was unnecessary indirection; the cross-compiler keeps the build chain self-contained with no host tool assumptions. Signed-off-by: Thierry Laurion <insurgo@riseup.net>

All hashes.txt and sizes.txt writers now use consistent relative paths. initrd.cpio.xz already stripped the $(pwd)/ prefix; the remaining writers did not, producing path-prefix diffs between CI and local builds. - modules/linux: bzImage sha256sum/stat stripped $(pwd)/ prefix from $@ - Makefile do-cpio: strip $(pwd)/ prefix from $1 - Makefile all: ROM hash/size: strip $(pwd)/ prefix from $(board_build) - Makefile all payload: hash/size: strip $(pwd)/ prefix from $< Now every hashes.txt and sizes.txt entry uses a consistent relative path format, eliminating path-prefix diffs between CI and local builds. Signed-off-by: Thierry Laurion <insurgo@riseup.net>

Including hashes.txt in the update ZIP enables future work: a tool that hashes files in the running firmware and compares them against the update's hashes.txt, instantly showing which files are identical and which differ. For the same commit, all files match, confirming the same ROM is already installed. This commit only makes hashes.txt available in the ZIP; the tool itself is deferred.

tlaurion · 2026-08-07T16:24:35Z

Superseded by #2181.

Copilot AI lite review requested due to automatic review settings August 6, 2026 19:29

tlaurion force-pushed the warm-build-hashes-complete branch 2 times, most recently from a57fe7f to 2d72835 Compare August 6, 2026 19:42

Copilot AI reviewed Aug 6, 2026

View reviewed changes

tlaurion force-pushed the warm-build-hashes-complete branch from 2d72835 to a99adc5 Compare August 6, 2026 19:46

tlaurion requested a lite review from Copilot August 6, 2026 20:10

Copilot started reviewing on behalf of tlaurion August 6, 2026 20:10 View session

Copilot AI reviewed Aug 6, 2026

View reviewed changes

tlaurion force-pushed the warm-build-hashes-complete branch 6 times, most recently from 864690a to efcc3fb Compare August 7, 2026 15:44

tlaurion added 4 commits August 7, 2026 11:54

tlaurion force-pushed the warm-build-hashes-complete branch 2 times, most recently from 9bd5655 to 1ec4442 Compare August 7, 2026 15:59

tlaurion force-pushed the warm-build-hashes-complete branch from 1ec4442 to 8632a83 Compare August 7, 2026 16:10

tlaurion closed this Aug 7, 2026

Navigation Menu

Uh oh!

Uh oh!

build: complete warm-build hashes.txt coverage (fix #2174 gaps; busybox symlinks; doc cleanup) - #2180

build: complete warm-build hashes.txt coverage (fix #2174 gaps; busybox symlinks; doc cleanup)#2180
tlaurion wants to merge 5 commits into
linuxboot:masterfrom
tlaurion:warm-build-hashes-complete

tlaurion commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

tlaurion Aug 6, 2026

Uh oh!

tlaurion Aug 6, 2026

Uh oh!

tlaurion Aug 6, 2026

Uh oh!

tlaurion Aug 6, 2026

Uh oh!

tlaurion Aug 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

tlaurion commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

tlaurion commented Aug 6, 2026

What each commit does

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

tlaurion Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

tlaurion Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

tlaurion Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

tlaurion Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

tlaurion Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

tlaurion commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants