cluster: migrate registry from host container to in-cluster deployment by lkingland · Pull Request #3856 · knative/func

lkingland · 2026-05-26T07:53:53Z

Summary

Migrates func cluster create from running a standalone func-registry container on the host to deploying the registry as in-cluster Kubernetes resources, matching the approach already taken in hack/cluster.sh (PR #3718).

What changed:

Registry is now a Deployment + ClusterIP Service + Contour Ingress inside the Kind cluster (was a host-side docker run container with ExternalName Service)
registryAddr is registry.localtest.me (was localhost:50000)
Containerd mirrors point at http://localhost:5000 via hostPort (was http://func-registry:5000 via Docker network DNS)
Each cluster owns its own registry, destroyed automatically by kind delete cluster
Delete flow simplified: no shared container teardown needed
Removed: ensureRegistry, registryStatus, teardownRegistry, setupPodmanMacOSForwarding, and all host-container lifecycle constants

What's preserved:

Host trust config (Docker daemon.json / Podman registries.conf) — now uses registry.localtest.me
--skip-registry-config flag behavior unchanged
Last-cluster check for host trust revert
Kubeconfig stat guard in Delete() to suppress noisy warnings on orphan-cleanup path

knative-prow · 2026-05-26T07:54:00Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lkingland

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [lkingland]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2026-05-26T08:09:51Z

Codecov Report

❌ Patch coverage is 0% with 86 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.48%. Comparing base (da6327e) to head (05b13f2).
⚠️ Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/cluster/registry.go	0.00%	71 Missing ⚠️
pkg/cluster/delete.go	0.00%	14 Missing ⚠️
pkg/cluster/kubernetes.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3856      +/-   ##
==========================================
+ Coverage   53.43%   53.48%   +0.05%     
==========================================
  Files         200      200              
  Lines       23405    23381      -24     
==========================================
- Hits        12506    12505       -1     
+ Misses       9649     9626      -23     
  Partials     1250     1250

Flag	Coverage Δ
e2e	`33.73% <0.00%> (+0.03%)`	⬆️
e2e go	`29.61% <0.00%> (+<0.01%)`	⬆️
e2e node	`25.90% <0.00%> (+<0.01%)`	⬆️
e2e python	`29.94% <0.00%> (+<0.01%)`	⬆️
e2e quarkus	`26.00% <0.00%> (+0.02%)`	⬆️
e2e rust	`25.47% <0.00%> (+<0.01%)`	⬆️
e2e springboot	`24.14% <0.00%> (+<0.01%)`	⬆️
e2e typescript	`25.99% <0.00%> (+<0.01%)`	⬆️
e2e-config-ci	`26.84% <0.00%> (+<0.01%)`	⬆️
integration	`15.78% <0.00%> (-0.01%)`	⬇️
unit macos-14	`42.25% <0.00%> (+<0.01%)`	⬆️
unit macos-latest	`42.25% <0.00%> (+<0.01%)`	⬆️
unit ubuntu-24.04-arm	`42.58% <0.00%> (+0.04%)`	⬆️
unit ubuntu-latest	`43.12% <0.00%> (+<0.01%)`	⬆️
unit windows-latest	`42.32% <0.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

matejvasek · 2026-05-26T12:08:14Z


-	if err := ensureRegistry(ctx, cfg, out); err != nil {
-		return err
+	registryManifest := `apiVersion: apps/v1


possible improvement: make this Go struct instead of string literal.

🤖 Done in 47c768b — Deployment/Service/Ingress/ConfigMap are typed k8s objects marshaled to YAML; same apply path.

matejvasek · 2026-05-26T12:15:28Z

-		return fmt.Errorf("applying registry service: %w", err)
-	}
-
 	success(out, "Registry", time.Since(start))


potential improvement: we could wait until http://registry.localtest.me/v2/ returns 200.

🤖 Done in ed43102 — poll until /v2/ returns 200. Uses http://127.0.0.1:5000 (hostPort) rather than registry.localtest.me because Contour installs in parallel with the registry goroutine; hostPort is ready with the Deployment and matches the containerd mirror endpoint.

matejvasek · 2026-05-26T12:17:16Z

+// installRegistry deploys the container registry as in-cluster Kubernetes
+// resources (Deployment + ClusterIP Service + Contour Ingress), configures
+// host-side trust, and applies the local-registry-hosting ConfigMap.
 func installRegistry(ctx context.Context, cfg ClusterConfig, out io.Writer) error {


Not sure when/how installRegistry() is called but it preferably should be called after Contour installation (or in parallel to it). But it's not necessary.

But it's not necessary.

Because we use generic k8s ingress not some specific Contour resource.

🤖 Agreed — leaving order as-is (generic Ingress; Contour still parallel). No change.

matejvasek · 2026-05-26T12:19:35Z

unrelated to this PR: why we use exec() instead of using Go code, kind/kubectl is Go code. Also why call docker via CLI exec()? Why not API? We already extensively use k8s and docker API already.

lkingland · 2026-05-28T03:59:47Z

unrelated to this PR: why we use exec() instead of using Go code, kind/kubectl is Go code. Also why call docker via CLI exec()? Why not API? We already extensively use k8s and docker API already.

Exactly right! This is simply a stepping-stone. Translating script->Go code with this structure is much less error-prone. Next step is to replace the exec's with code wherever possible! Same way we did with the builders: wrap the cli, get it working, replace with the stdlib 👍🏻

lkingland · 2026-05-31T04:07:16Z

/retest

gauron99

note from AI: stale comment incmd/cluster.go -> Deletes a local development cluster and its associated registry container 😁

Replace the shared host-side func-registry container with Kubernetes-native resources deployed inside each Kind cluster: - Deployment (registry:2 with hostPort 5000 + emptyDir volume) - ClusterIP Service (port 5000) - Contour Ingress at registry.localtest.me Key changes: - registryAddr is now "registry.localtest.me" (was "localhost:50000") - containerd mirrors point at http://localhost:5000 via hostPort (was http://func-registry:5000 via Docker network DNS) - Each cluster owns its own registry, destroyed with kind delete cluster - Delete flow simplified: no shared container teardown, just host trust revert on last-cluster removal - Removed: ensureRegistry, registryStatus, teardownRegistry, setupPodmanMacOSForwarding, and all host-container lifecycle code

Help still said "registry container" (host-side lifecycle). Match the in-cluster Deployment model: Kind removes the registry with the cluster; host trust reverts only on last func-managed cluster. Functions#43 / knative#3856

lkingland · 2026-07-25T10:23:00Z

🤖 @gauron99 re stale delete help ("registry container"): fixed in f5a260a — docs now match in-cluster registry + last-cluster host-trust revert.

Replace YAML string literals for the in-cluster registry Deployment, Service, Ingress, and local-registry-hosting ConfigMap with typed k8s API objects marshaled via sigs.k8s.io/yaml. Same resources; apply path unchanged (kubectl apply -f -). Addresses matejvasek review on knative#3856. Functions#43

Poll http://127.0.0.1:5000/v2/ until 200 (hostPort path) before create returns, so the first push is less likely to race an unready registry. Uses hostPort rather than registry.localtest.me because Contour installs in parallel with the registry goroutine. Addresses matejvasek review on knative#3856. Functions#43

lkingland · 2026-07-25T10:36:14Z

🤖 @matejvasek re your note on exec vs Go APIs: agreed this PR stays on the script→Go stepping-stone. Follow-up for kind/kubectl/docker library calls: #3964

Used by pkg/cluster for typed registry manifest marshaling. Satisfies hack/update-codegen / verify-deps which require direct deps in go.mod. Functions#43 / knative#3856

knative-prow Bot requested review from dsimansk and jrangelramos May 26, 2026 07:53

knative-prow Bot added approved 🤖 PR has been approved by an approver from all required OWNERS files. size/L 🤖 PR changes 100-499 lines, ignoring generated files. labels May 26, 2026

lkingland force-pushed the push-qtztqtnuqqqy branch from f96a48b to 0470fa1 Compare May 26, 2026 08:00

matejvasek reviewed May 26, 2026

View reviewed changes

Comment thread CLAUDE.local.md Outdated

matejvasek reviewed May 26, 2026

View reviewed changes

lkingland force-pushed the push-qtztqtnuqqqy branch from 0470fa1 to 05b13f2 Compare May 26, 2026 12:45

gauron99 reviewed Jun 25, 2026

View reviewed changes

Comment thread pkg/cluster/delete.go

Comment thread pkg/cluster/registry.go

lkingland added 2 commits July 25, 2026 19:13

cmd: fix cluster delete help after in-cluster registry

f5a260a

Help still said "registry container" (host-side lifecycle). Match the in-cluster Deployment model: Kind removes the registry with the cluster; host trust reverts only on last func-managed cluster. Functions#43 / knative#3856

lkingland force-pushed the push-qtztqtnuqqqy branch from 05b13f2 to f5a260a Compare July 25, 2026 10:22

lkingland added 2 commits July 25, 2026 19:30

lkingland mentioned this pull request Jul 25, 2026

cluster: replace exec with API calls #3964

Open

deps: promote sigs.k8s.io/yaml to a direct require

472da09

Used by pkg/cluster for typed registry manifest marshaling. Satisfies hack/update-codegen / verify-deps which require direct deps in go.mod. Functions#43 / knative#3856

Uh oh!

Conversation

lkingland commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

knative-prow Bot commented May 26, 2026

Uh oh!

codecov Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

matejvasek May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkingland Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

matejvasek May 26, 2026

Choose a reason for hiding this comment

Uh oh!

lkingland Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

matejvasek May 26, 2026

Choose a reason for hiding this comment

Uh oh!

matejvasek May 26, 2026

Choose a reason for hiding this comment

Uh oh!

lkingland Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

matejvasek commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lkingland commented May 28, 2026

Uh oh!

lkingland commented May 31, 2026

Uh oh!

gauron99 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lkingland commented Jul 25, 2026

Uh oh!

lkingland commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lkingland commented May 26, 2026 •

edited

Loading

codecov Bot commented May 26, 2026 •

edited

Loading

matejvasek May 26, 2026 •

edited

Loading

matejvasek commented May 26, 2026 •

edited

Loading