From 446dd18b6e43a34c4465bcd804631b29ee7ff5e5 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Tue, 25 Aug 2026 15:06:37 +0200 Subject: [PATCH 1/8] docs: add the Blockstor storage backend and its migration guide Cozystack is gaining a second storage control plane, so the platform install page now presents the choice as tabs rather than assuming LINSTOR. LINSTOR stays the default and is the first tab; Blockstor carries an experimental warning. Adds a migration guide for moving an existing cluster across. It is written from a run against a live cluster, so it leads with the parts that are easy to get wrong: the adoption happens after the switch, not before, because the resources have nowhere to live until the CRDs exist; the controller has to be stopped while the converted resources are applied, or it auto-places replicas on freshly allocated ports and the real ones are then refused; the live DRBD ports have to be read from the kernel because LINSTOR does not keep them in its custom resources; and a cluster coming from LINSTOR hits a Service ownership collision a fresh install never sees. The guide also states the version requirement plainly: a Blockstor build that cannot read a pool name from StorDriver/StorPoolName adopts nothing, which is every release up to and including v0.1.17. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../docs/next/install/cozystack/platform.md | 43 +++++- content/en/docs/next/storage/_index.md | 4 +- .../docs/next/storage/blockstor-migration.md | 143 ++++++++++++++++++ 3 files changed, 187 insertions(+), 3 deletions(-) create mode 100644 content/en/docs/next/storage/blockstor-migration.md diff --git a/content/en/docs/next/install/cozystack/platform.md b/content/en/docs/next/install/cozystack/platform.md index 981033f8..597334f5 100644 --- a/content/en/docs/next/install/cozystack/platform.md +++ b/content/en/docs/next/install/cozystack/platform.md @@ -109,7 +109,46 @@ enable a provider-specific load balancer, or use a different network setup. Check out the [provider-specific installation]({{% ref "/docs/next/install/providers" %}}) section. It may include a complete guide for your provider that you can use to deploy a production-ready cluster. -### 2.3. Define Network Configuration +### 2.3. Choose a Storage Backend + +Cozystack runs one of two storage control planes. Both are driven by the same CSI driver and expose the same StorageClasses, so the choice does not change how workloads request volumes — only what manages them underneath. + +{{< tabs name="storage_backend" >}} +{{% tab name="LINSTOR (default)" %}} +LINSTOR is the default and the backend every existing cluster runs. Nothing needs to be set: omitting `storage` selects it. + +```yaml +spec: + components: + platform: + values: + storage: + backend: linstor +``` +{{% /tab %}} +{{% tab name="Blockstor (experimental)" %}} +Blockstor is a LINSTOR-API-compatible control plane that stores its state in Kubernetes custom resources instead of an in-cluster database. + +```yaml +spec: + components: + platform: + values: + storage: + backend: blockstor +``` + +{{% alert color="warning" %}} +Blockstor is **experimental**. Use it on clusters where you can tolerate storage-layer problems, and prefer LINSTOR for production until this notice is removed. +{{% /alert %}} +{{% /tab %}} +{{< /tabs >}} + +An unrecognised value fails the render rather than falling back to a default, so a typo cannot quietly deploy the storage control plane you did not ask for. + +To move an existing cluster from LINSTOR to Blockstor, see [Migrating from LINSTOR to Blockstor]({{% ref "/docs/next/storage/blockstor-migration" %}}). Do not simply flip this value on a cluster that already holds data — the switch installs a new control plane that knows nothing about the existing volumes until they are adopted. + +### 2.4. Define Network Configuration Replace `example.org` in `publishing.host` and `publishing.apiServerEndpoint` with a routable fully-qualified domain name (FQDN) that you control. If you only have a public IP, but no routable FQDN, use [nip.io](https://nip.io/) with dash notation. @@ -130,7 +169,7 @@ networking: Cozystack gathers anonymous usage statistics by default. Learn more about what data is collected and how to opt out in the [Telemetry Documentation]({{% ref "/docs/next/operations/configuration/telemetry" %}}). {{% /alert %}} -### 2.4. Apply Platform Package +### 2.5. Apply Platform Package Once the configuration file is ready, apply it: diff --git a/content/en/docs/next/storage/_index.md b/content/en/docs/next/storage/_index.md index ec5849d2..5f0bf9da 100644 --- a/content/en/docs/next/storage/_index.md +++ b/content/en/docs/next/storage/_index.md @@ -7,4 +7,6 @@ aliases: - /docs/next/operations/storage --- -These guides will show you how to perform typical tasks related to the LINSTOR storage system in Cozystack. +These guides will show you how to perform typical tasks related to the storage subsystem in Cozystack. + +Cozystack runs one of two storage control planes, selected at install time: LINSTOR, the default, or Blockstor, which is experimental. See [Choose a Storage Backend]({{% ref "/docs/next/install/cozystack/platform#23-choose-a-storage-backend" %}}) to pick one, and [Migrating from LINSTOR to Blockstor]({{% ref "/docs/next/storage/blockstor-migration" %}}) to move an existing cluster across. Unless a guide says otherwise, the pages in this section describe LINSTOR. diff --git a/content/en/docs/next/storage/blockstor-migration.md b/content/en/docs/next/storage/blockstor-migration.md new file mode 100644 index 00000000..0dc2dcf6 --- /dev/null +++ b/content/en/docs/next/storage/blockstor-migration.md @@ -0,0 +1,143 @@ +--- +title: "Migrating from LINSTOR to Blockstor" +linkTitle: "Migrating to Blockstor" +description: "How to move an existing cluster's storage control plane from LINSTOR to Blockstor without recreating volumes" +weight: 7 +--- + +Blockstor is a LINSTOR-API-compatible storage control plane that keeps its state in Kubernetes custom resources. Because it serves the same CSI wire shape through the same CSI driver, an existing cluster can move across without recreating volumes or re-provisioning workloads: the StorageClasses, the PersistentVolumes bound to them, and the DRBD devices on the nodes all stay exactly as they are. What changes is which control plane manages them. + +{{% alert color="warning" %}} +Blockstor is **experimental**. This migration rewrites which component owns your storage metadata. Read the whole page before starting, take the backup in step 1, and rehearse on a cluster you can afford to lose. +{{% /alert %}} + +## How the migration works + +Your data is not copied or moved. DRBD runs in the kernel and is host-scoped: it keeps serving volumes while the control plane is replaced above it. The migration converts LINSTOR's metadata — nodes, storage pools, resource definitions, replicas — into Blockstor custom resources, and Blockstor then *adopts* the running devices rather than creating new ones. + +The order matters, and it is the opposite of what feels natural. You switch the backend **first**, which installs Blockstor's CRDs and starts its control plane against an empty database, and you adopt the existing volumes **second**. Adoption cannot come first because the resources it creates have nowhere to live until the CRDs exist. + +## Before you start + +You need: + +- `linstor-migrate`, the converter shipped with Blockstor. Build it from the Blockstor repository with `make build` or take it from a release. +- A Blockstor release that can register your storage pools. This is not optional and it is the first thing to check — see the box below. +- Enough of a maintenance window that CSI cannot attach or detach volumes for its duration. Running workloads keep their volumes; new pods that need an attach will wait. + +{{% alert color="warning" %}} +**Check the Blockstor version before anything else.** Registering a storage pool that LINSTOR created requires reading the pool name from `StorDriver/StorPoolName`, which is where LINSTOR stores it. A Blockstor build without that support logs `unknown storage pool ""` on every reconcile and adopts nothing. The failure is safe — Blockstor refuses before touching the data plane — but the migration cannot proceed. Support landed after `v0.1.17`; confirm your build has it before switching anything. +{{% /alert %}} + +## 1. Back up LINSTOR's metadata + +Everything the migration reads lives in LINSTOR's custom resources. Save them, and the CRD definitions themselves, before you touch the cluster: + +```bash +kubectl get crds | grep -o ".*.internal.linstor.linbit.com" | \ + xargs kubectl get crds -ojson > crds.json + +kubectl get crds | grep -o ".*.internal.linstor.linbit.com" | \ + xargs -I{} sh -xc "kubectl get {} -ojson > {}.json" + +tar czvf backup-$(date +%d.%m.%Y).tgz *.json +``` + +Switching the backend does not delete these resources, so this backup is what lets you go back. + +## 2. Dump the LINSTOR tables + +The converter reads a directory of per-table JSON dumps: + +```bash +mkdir -p linstor-dump +for crd in $(kubectl get crd -o name | grep 'internal.linstor.linbit.com' \ + | sed 's|customresourcedefinition.apiextensions.k8s.io/||'); do + kubectl get "$crd" -o json > "linstor-dump/${crd}.json" +done +``` + +This works even when the LINSTOR controller itself is unhealthy: the converter reads the custom resources directly and never talks to the LINSTOR API. + +## 3. Capture the live DRBD ports + +Adopted replicas must keep the TCP port their running DRBD connection already uses. If the port is not supplied, Blockstor allocates a fresh one and the mesh reconnects — a brief interruption on every replicated volume. + +The port is not stored in LINSTOR's custom resources, so read it from the running kernel. It appears inside each connection's `path` block: + +```bash +for node in $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'); do + pod=$(kubectl get pods -n cozy-linstor -o name | grep "linstor-satellite.${node}-" | head -1) + kubectl exec -n cozy-linstor "$pod" -c linstor-satellite -- drbdsetup show 2>/dev/null \ + | awk '/^resource / { res = $2; gsub(/"/, "", res); next } + /_this_host[ \t]+ipv4/ { addr = $3; gsub(/;/, "", addr) + n = split(addr, p, ":") + if (res != "" && p[n] ~ /^[0-9]+$/) print res, p[n] }' +done | sort -u > drbd-ports.txt +``` + +A single-replica volume has no peer and therefore no port to preserve; it will not appear in this file, and that is correct. + +## 4. Convert + +```bash +linstor-migrate -in linstor-dump -drbd-ports drbd-ports.txt -out blockstor-resources.yaml +``` + +Read the warnings. Resources that LINSTOR has marked for deletion are skipped and named. Flags the converter does not recognise are reported and dropped rather than guessed at. + +## 5. Switch the backend + +Set the storage backend on the platform Package, as described in [Choose a Storage Backend]({{% ref "/docs/next/install/cozystack/platform#23-choose-a-storage-backend" %}}): + +```yaml +spec: + components: + platform: + values: + storage: + backend: blockstor +``` + +Apply it and wait for the Blockstor control plane to come up. The LINSTOR controller and the piraeus-managed satellites go away; piraeus-operator stays on in external mode to keep driving the CSI driver. + +{{% alert color="info" %}} +On a cluster that already ran LINSTOR, the compatibility Service that Blockstor provides for the scheduler and GUI collides with the one piraeus-operator owns. Helm refuses to import it. Take ownership of the existing Service so the release can proceed; a fresh install never hits this. +{{% /alert %}} + +Your volumes keep serving throughout this step. Blockstor does not yet know about them, so CSI cannot attach or detach until adoption finishes. + +## 6. Stop the controller, then adopt + +Scale the Blockstor controller to zero before applying the converted resources: + +```bash +kubectl -n cozy-linstor scale deploy/blockstor-controller --replicas=0 +kubectl apply -f blockstor-resources.yaml +kubectl -n cozy-linstor scale deploy/blockstor-controller --replicas=1 +``` + +The scale-down is not a nicety. Applying the file in one pass makes the resource definitions visible before the replicas further down the file exist, and a running controller reacts by auto-placing replicas from the resource group's placement policy — giving them freshly allocated ports. The real replicas are then rejected, because a replica's DRBD port can be set once and not changed. You are left with replicas whose ports disagree with the live mesh, which is worse than it sounds: reconciling that state reconfigures a running mesh onto ports its peers do not share. + +With the controller stopped, the whole file lands before anything reacts to it. + +## 7. Verify + +Confirm every replica was adopted rather than recreated, and that the ports match what the kernel is using: + +```bash +kubectl get resources.blockstor.cozystack.io -o custom-columns=\ +'RD:.spec.resourceDefinitionName,NODE:.spec.nodeName,PORT:.spec.drbdPort' +``` + +Compare against `drbd-ports.txt`. Then check the data plane is untouched — every peer should still be `UpToDate` and no resource should be syncing: + +```bash +kubectl exec -n cozy-linstor ds/blockstor-satellite -- drbdsetup status +``` + +A resource that starts a full resync after adoption means it was treated as new rather than adopted. Stop and investigate before letting it run. + +## Rolling back + +Switching back is the same operation in reverse: set `storage.backend` to `linstor` and apply. LINSTOR's custom resources are not deleted by the migration, so its controller finds its state where it left it. The Blockstor custom resources can be removed afterwards. From 14d99b1d2f59151b19b5e283ac968399f43ed274 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Tue, 25 Aug 2026 16:16:33 +0200 Subject: [PATCH 2/8] docs(storage): stop the CSI provisioner during a Blockstor migration linstor-csi keeps reconciling while the control plane is being replaced, and a volume it cannot find is a volume it re-provisions: it creates a fresh resource definition through the LINSTOR-compatible API, carrying newly allocated DRBD minors and ports and a different node ID, for a volume that already exists and holds data. Those definitions also lack spec.initialized, so the satellite treats them as new. Observed on a live migration, where two already-migrated volumes were re-provisioned this way during the switch window. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../en/docs/next/storage/blockstor-migration.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/content/en/docs/next/storage/blockstor-migration.md b/content/en/docs/next/storage/blockstor-migration.md index 0dc2dcf6..d7731a3c 100644 --- a/content/en/docs/next/storage/blockstor-migration.md +++ b/content/en/docs/next/storage/blockstor-migration.md @@ -88,6 +88,14 @@ Read the warnings. Resources that LINSTOR has marked for deletion are skipped an ## 5. Switch the backend +Stop the CSI provisioner first: + +```bash +kubectl -n cozy-linstor scale deploy/linstor-csi-controller --replicas=0 +``` + +It keeps reconciling while the control plane is being replaced, and a volume it cannot find is a volume it re-provisions. It creates a fresh resource definition through the LINSTOR-compatible API — with newly allocated DRBD minors and ports and a different node ID — for a volume that already exists and still holds data. Those definitions also lack `spec.initialized`, so the satellite treats them as new and queues them for `create-md`. Leaving the provisioner running is how a migration quietly acquires duplicate definitions that disagree with the live mesh. + Set the storage backend on the platform Package, as described in [Choose a Storage Backend]({{% ref "/docs/next/install/cozystack/platform#23-choose-a-storage-backend" %}}): ```yaml @@ -138,6 +146,12 @@ kubectl exec -n cozy-linstor ds/blockstor-satellite -- drbdsetup status A resource that starts a full resync after adoption means it was treated as new rather than adopted. Stop and investigate before letting it run. +Once the replicas check out, bring the CSI provisioner back: + +```bash +kubectl -n cozy-linstor scale deploy/linstor-csi-controller --replicas=1 +``` + ## Rolling back Switching back is the same operation in reverse: set `storage.backend` to `linstor` and apply. LINSTOR's custom resources are not deleted by the migration, so its controller finds its state where it left it. The Blockstor custom resources can be removed afterwards. From a4b6e3f8784d10f99776d3db95c6c84e4d08a4e3 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Tue, 25 Aug 2026 17:06:55 +0200 Subject: [PATCH 3/8] docs(storage): warn about pool capacity before a Blockstor migration Two things claim space during adoption, and both surfaced on a live migration only after the switch had started. Blockstor's thick provider reserves the full size of every volume it adopts. LINSTOR allows a pool declared thick to hold sparse volumes via StorDriver/ZfscreateOptions, so a comfortably oversubscribed pool can fill up mid-migration and leave the last volumes unadoptable. Blockstor also reconciles replica count against the resource group's placeCount continuously, where LINSTOR only places on request, so every under-replicated volume is topped up as soon as the controller starts. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- content/en/docs/next/storage/blockstor-migration.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/content/en/docs/next/storage/blockstor-migration.md b/content/en/docs/next/storage/blockstor-migration.md index d7731a3c..5e8fa368 100644 --- a/content/en/docs/next/storage/blockstor-migration.md +++ b/content/en/docs/next/storage/blockstor-migration.md @@ -24,11 +24,16 @@ You need: - `linstor-migrate`, the converter shipped with Blockstor. Build it from the Blockstor repository with `make build` or take it from a release. - A Blockstor release that can register your storage pools. This is not optional and it is the first thing to check — see the box below. - Enough of a maintenance window that CSI cannot attach or detach volumes for its duration. Running workloads keep their volumes; new pods that need an attach will wait. +- Room in the storage pool for what adoption will provision. Two separate things claim space: Blockstor brings replica counts up to each resource group's `placeCount`, and on a thick pool it reserves the full size of every volume it adopts. Both are covered below — check them against your free space before you switch, not after. {{% alert color="warning" %}} **Check the Blockstor version before anything else.** Registering a storage pool that LINSTOR created requires reading the pool name from `StorDriver/StorPoolName`, which is where LINSTOR stores it. A Blockstor build without that support logs `unknown storage pool ""` on every reconcile and adopts nothing. The failure is safe — Blockstor refuses before touching the data plane — but the migration cannot proceed. Support landed after `v0.1.17`; confirm your build has it before switching anything. {{% /alert %}} +{{% alert color="warning" %}} +**A thick pool holding sparse volumes will not survive adoption unchanged.** LINSTOR lets a pool be declared thick (`driver=ZFS`) while `StorDriver/ZfscreateOptions: -s` makes every volume sparse. Blockstor has no equivalent state: its thick provider reserves the full size of each volume it adopts, so a pool that was comfortably oversubscribed under LINSTOR can fill up during the migration and leave the last volumes unadoptable. Nothing is lost — a reservation is reversible — but check `zfs list -o name,used,avail` against the sum of your volume sizes first. +{{% /alert %}} + ## 1. Back up LINSTOR's metadata Everything the migration reads lives in LINSTOR's custom resources. Save them, and the CRD definitions themselves, before you touch the cluster: @@ -146,6 +151,8 @@ kubectl exec -n cozy-linstor ds/blockstor-satellite -- drbdsetup status A resource that starts a full resync after adoption means it was treated as new rather than adopted. Stop and investigate before letting it run. +Expect the controller to create replicas for any volume LINSTOR left under-replicated. Blockstor reconciles replica count against the resource group's `placeCount` continuously, where LINSTOR only places on request, so a volume sitting at one replica under a three-replica storage class gets two more — each a full sync. This is correct behaviour, not a migration fault, but it is worth knowing before it happens on a pool with no room for it. + Once the replicas check out, bring the CSI provisioner back: ```bash From 43657a83a076bbc1fed5e23fdc620c0fb7f80db9 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 26 Aug 2026 13:25:42 +0200 Subject: [PATCH 4/8] docs(storage): freeze the LINSTOR writers before the backup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The runbook stopped only the CSI provisioner, and only at the switch. Two writers were left running through the backup and the dump. piraeus-operator owns the linstor-controller Deployment and the satellite DaemonSets, so scaling the controller down while the operator runs achieves nothing — it puts it straight back. And the controller itself keeps writing to its database, so a backup taken underneath it does not match the dump taken a moment later, and the converted manifest then describes a cluster that no longer exists. The stop moves to its own first step, in dependency order, ahead of the backup. The CSI scale-down moves there with it; step 6 now explains the window rather than repeating the command. Two prerequisites corrected while here. The Blockstor version is not the operator's to choose — it arrives pinned inside the Cozystack release, so the check is a Cozystack version check and the remedy is an upgrade, not an image swap. And the thick-pool capacity warning is gone: the converter now carries a sparse ZFS pool over as thin, so adoption no longer reserves the full size of volumes LINSTOR created sparse. What remains of that prerequisite is the placeCount top-up, which is real. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../docs/next/storage/blockstor-migration.md | 50 +++++++++++-------- 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/content/en/docs/next/storage/blockstor-migration.md b/content/en/docs/next/storage/blockstor-migration.md index 5e8fa368..35059a98 100644 --- a/content/en/docs/next/storage/blockstor-migration.md +++ b/content/en/docs/next/storage/blockstor-migration.md @@ -22,19 +22,35 @@ The order matters, and it is the opposite of what feels natural. You switch the You need: - `linstor-migrate`, the converter shipped with Blockstor. Build it from the Blockstor repository with `make build` or take it from a release. -- A Blockstor release that can register your storage pools. This is not optional and it is the first thing to check — see the box below. +- A Cozystack release whose pinned Blockstor can register your storage pools. This is not optional and it is the first thing to check — see the box below. - Enough of a maintenance window that CSI cannot attach or detach volumes for its duration. Running workloads keep their volumes; new pods that need an attach will wait. -- Room in the storage pool for what adoption will provision. Two separate things claim space: Blockstor brings replica counts up to each resource group's `placeCount`, and on a thick pool it reserves the full size of every volume it adopts. Both are covered below — check them against your free space before you switch, not after. +- Room in the storage pool for the replicas adoption will add. Blockstor brings replica counts up to each resource group's `placeCount`, and a volume LINSTOR left under-replicated is topped up as soon as the controller starts — a new replica and a full sync each. Check that against your free space before you switch, not after. {{% alert color="warning" %}} -**Check the Blockstor version before anything else.** Registering a storage pool that LINSTOR created requires reading the pool name from `StorDriver/StorPoolName`, which is where LINSTOR stores it. A Blockstor build without that support logs `unknown storage pool ""` on every reconcile and adopts nothing. The failure is safe — Blockstor refuses before touching the data plane — but the migration cannot proceed. Support landed after `v0.1.17`; confirm your build has it before switching anything. +**Check your Cozystack version before anything else.** You do not pick a Blockstor version: it arrives pinned inside the Cozystack release, so what decides whether the migration can work is which Cozystack you are on. Registering a storage pool that LINSTOR created requires reading the pool name from `StorDriver/StorPoolName`, which is where LINSTOR stores it; a pinned Blockstor without that support logs `unknown storage pool ""` on every reconcile and adopts nothing. The failure is safe — Blockstor refuses before touching the data plane — but the migration cannot proceed, and the fix is to upgrade Cozystack rather than to swap an image. {{% /alert %}} -{{% alert color="warning" %}} -**A thick pool holding sparse volumes will not survive adoption unchanged.** LINSTOR lets a pool be declared thick (`driver=ZFS`) while `StorDriver/ZfscreateOptions: -s` makes every volume sparse. Blockstor has no equivalent state: its thick provider reserves the full size of each volume it adopts, so a pool that was comfortably oversubscribed under LINSTOR can fill up during the migration and leave the last volumes unadoptable. Nothing is lost — a reservation is reversible — but check `zfs list -o name,used,avail` against the sum of your volume sizes first. -{{% /alert %}} +## 1. Stop everything that writes + +Freeze the writers first, and only then take the backup. LINSTOR's controller keeps writing to its database for as long as it runs, so a backup taken underneath it does not match the dump you take a moment later, and the converted manifest would describe a cluster that no longer exists. + +Order matters. piraeus-operator owns the controller Deployment and the satellite DaemonSets, so it goes first: scale the controller down while the operator is still running and the operator simply puts it back. + +```bash +kubectl -n cozy-system scale deploy/piraeus-operator-controller-manager --replicas=0 +kubectl -n cozy-linstor scale deploy/linstor-controller --replicas=0 +kubectl -n cozy-linstor scale deploy/linstor-csi-controller --replicas=0 +``` + +The CSI provisioner is on that list for a reason of its own, covered in step 6: a volume it cannot find is a volume it re-provisions. -## 1. Back up LINSTOR's metadata +Confirm nothing is left writing before you continue: + +```bash +kubectl -n cozy-linstor get deploy +``` + +## 2. Back up LINSTOR's metadata Everything the migration reads lives in LINSTOR's custom resources. Save them, and the CRD definitions themselves, before you touch the cluster: @@ -50,7 +66,7 @@ tar czvf backup-$(date +%d.%m.%Y).tgz *.json Switching the backend does not delete these resources, so this backup is what lets you go back. -## 2. Dump the LINSTOR tables +## 3. Dump the LINSTOR tables The converter reads a directory of per-table JSON dumps: @@ -64,7 +80,7 @@ done This works even when the LINSTOR controller itself is unhealthy: the converter reads the custom resources directly and never talks to the LINSTOR API. -## 3. Capture the live DRBD ports +## 4. Capture the live DRBD ports Adopted replicas must keep the TCP port their running DRBD connection already uses. If the port is not supplied, Blockstor allocates a fresh one and the mesh reconnects — a brief interruption on every replicated volume. @@ -83,7 +99,7 @@ done | sort -u > drbd-ports.txt A single-replica volume has no peer and therefore no port to preserve; it will not appear in this file, and that is correct. -## 4. Convert +## 5. Convert ```bash linstor-migrate -in linstor-dump -drbd-ports drbd-ports.txt -out blockstor-resources.yaml @@ -91,15 +107,9 @@ linstor-migrate -in linstor-dump -drbd-ports drbd-ports.txt -out blockstor-resou Read the warnings. Resources that LINSTOR has marked for deletion are skipped and named. Flags the converter does not recognise are reported and dropped rather than guessed at. -## 5. Switch the backend - -Stop the CSI provisioner first: - -```bash -kubectl -n cozy-linstor scale deploy/linstor-csi-controller --replicas=0 -``` +## 6. Switch the backend -It keeps reconciling while the control plane is being replaced, and a volume it cannot find is a volume it re-provisions. It creates a fresh resource definition through the LINSTOR-compatible API — with newly allocated DRBD minors and ports and a different node ID — for a volume that already exists and still holds data. Those definitions also lack `spec.initialized`, so the satellite treats them as new and queues them for `create-md`. Leaving the provisioner running is how a migration quietly acquires duplicate definitions that disagree with the live mesh. +The CSI provisioner was stopped in step 1, and this is the window it was stopped for. It keeps reconciling while the control plane is being replaced, and a volume it cannot find is a volume it re-provisions: it creates a fresh resource definition through the LINSTOR-compatible API — with newly allocated DRBD minors and ports and a different node ID — for a volume that already exists and still holds data. Those definitions also lack `spec.initialized`, so the satellite treats them as new and queues them for `create-md`. Leaving the provisioner running is how a migration quietly acquires duplicate definitions that disagree with the live mesh. Set the storage backend on the platform Package, as described in [Choose a Storage Backend]({{% ref "/docs/next/install/cozystack/platform#23-choose-a-storage-backend" %}}): @@ -120,7 +130,7 @@ On a cluster that already ran LINSTOR, the compatibility Service that Blockstor Your volumes keep serving throughout this step. Blockstor does not yet know about them, so CSI cannot attach or detach until adoption finishes. -## 6. Stop the controller, then adopt +## 7. Stop the controller, then adopt Scale the Blockstor controller to zero before applying the converted resources: @@ -134,7 +144,7 @@ The scale-down is not a nicety. Applying the file in one pass makes the resource With the controller stopped, the whole file lands before anything reacts to it. -## 7. Verify +## 8. Verify Confirm every replica was adopted rather than recreated, and that the ports match what the kernel is using: From 11ba594912553457c921e48f7023f5552c67c8b0 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 26 Aug 2026 13:46:09 +0200 Subject: [PATCH 5/8] docs(install): un-nest the experimental notice so it renders A {{% %}} shortcode renders its body as Markdown, so an alert nested inside a tab runs its body through twice: the inner shortcode receives the HTML the outer already produced, and escapes it. The notice came out as a literal inside a code block. It lives outside the tabs now, which also means the reader sees it whichever tab is open. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- content/en/docs/next/install/cozystack/platform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/next/install/cozystack/platform.md b/content/en/docs/next/install/cozystack/platform.md index 597334f5..9ca44bb9 100644 --- a/content/en/docs/next/install/cozystack/platform.md +++ b/content/en/docs/next/install/cozystack/platform.md @@ -137,12 +137,12 @@ spec: storage: backend: blockstor ``` +{{% /tab %}} +{{< /tabs >}} {{% alert color="warning" %}} Blockstor is **experimental**. Use it on clusters where you can tolerate storage-layer problems, and prefer LINSTOR for production until this notice is removed. {{% /alert %}} -{{% /tab %}} -{{< /tabs >}} An unrecognised value fails the render rather than falling back to a default, so a typo cannot quietly deploy the storage control plane you did not ask for. From 64af4c7bc3c9a3e55ce7993fcf10b5ad1843ae56 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 26 Aug 2026 13:50:19 +0200 Subject: [PATCH 6/8] docs(install): give the storage section a Blockstor path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 2.3 lets the operator choose Blockstor, and then step 3 walked them into commands that cannot work there. Every command in 3.1 and 3.2 runs through `kubectl exec deploy/linstor-controller`, and on backend: blockstor the control plane runs in external mode — that Deployment is never created, so each one fails with NotFound. 3.1 and 3.2 now say which backend they are for, and 3.4 covers the same ground for Blockstor: list the nodes, create the ZFS pool through the satellite (which carries the tooling), register it as a StoragePool custom resource, check the result. It is kubectl throughout, because Blockstor keeps its state in custom resources rather than behind a controller pod — there is nothing to exec into. Storage classes are backend-independent, so 3.3 is shared and both paths lead into it. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../docs/next/install/cozystack/platform.md | 62 ++++++++++++++++++- 1 file changed, 60 insertions(+), 2 deletions(-) diff --git a/content/en/docs/next/install/cozystack/platform.md b/content/en/docs/next/install/cozystack/platform.md index 9ca44bb9..670ba884 100644 --- a/content/en/docs/next/install/cozystack/platform.md +++ b/content/en/docs/next/install/cozystack/platform.md @@ -237,9 +237,13 @@ kubectl taint nodes --all node-role.kubernetes.io/control-plane- ## 3. Configure Storage Kubernetes needs a storage subsystem to provide persistent volumes to applications, but it doesn't include one of its own. -Cozystack provides [LINSTOR](https://github.com/LINBIT/linstor-server) as a storage subsystem. +Cozystack provides [LINSTOR](https://github.com/LINBIT/linstor-server) as a storage subsystem, and [Blockstor](https://github.com/cozystack/blockstor) as an experimental alternative. -In the following steps, we'll access LINSTOR interface, create storage pools, and define storage classes. +In the following steps, we'll access the storage interface, create storage pools, and define storage classes. + +{{% alert color="warning" %}} +Steps 3.1 and 3.2 below are written for the **LINSTOR** backend. They run through `linstor-controller`, which exists only there — on `backend: blockstor` the control plane runs in external mode and that Deployment is never created, so every `linstor` command in them fails with `NotFound`. If you chose Blockstor in [step 2.3](#23-choose-a-storage-backend), follow [3.4](#34-configure-storage-on-the-blockstor-backend) instead. Step 3.3 is the same for both. +{{% /alert %}} ### 3.1. Check Storage Devices @@ -422,6 +426,60 @@ Create storage classes, one of which should be the default class. +### 3.4. Configure Storage on the Blockstor Backend + +Skip this step on the LINSTOR backend — 3.1 and 3.2 cover it there. + +Blockstor keeps its state in Kubernetes custom resources rather than in a database behind a controller pod, so its storage pools are configured with `kubectl` rather than through a CLI inside the cluster. There is no `linstor-controller` to exec into. + +1. List the nodes Blockstor knows about: + + ```bash + kubectl get nodes.blockstor.cozystack.io + ``` + +1. Create the ZFS pool on the backing device of each storage node. The satellite image carries the ZFS tooling, so run it there: + + ```bash + kubectl exec -n cozy-linstor ds/blockstor-satellite -- \ + zpool create -o failmode=continue data /dev/sdb + ``` + + Repeat per node, addressing the satellite pod on that node. `failmode=continue` is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) for the same reason as on LINSTOR: it lets DRBD handle a disk failure rather than ZFS. + +1. Register each pool with Blockstor. The object name is `.` in lower case — a validation rule on the custom resource pins that, so a mismatch is refused rather than stored: + + ```yaml + apiVersion: blockstor.cozystack.io/v1alpha1 + kind: StoragePool + metadata: + name: data.srv1 + spec: + nodeName: srv1 + poolName: data + providerKind: ZFS_THIN + props: + StorDriver/StorPoolName: data + ``` + + Use `ZFS` instead of `ZFS_THIN` for a thick pool, where every volume reserves its full size on creation. + +1. Check the result: + + ```bash + kubectl get storagepools.blockstor.cozystack.io + ``` + + ```console + NAME AGE + data.srv1 1m + data.srv2 1m + data.srv3 1m + dfltdisklessstorpool.srv1 1m + ``` + +Then continue with [3.3](#33-create-storage-classes) — storage classes are the same on both backends. + ## 4. Configure Networking Next, we will configure how the Cozystack cluster can be accessed. From 8fdbdcfc156b26506c3815976bec2a265cfac501 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 26 Aug 2026 14:12:00 +0200 Subject: [PATCH 7/8] docs(install): drive the Blockstor storage steps from its own CLI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first version of 3.4 configured storage by applying custom resources by hand, on the reasoning that Blockstor keeps its state in CRDs. That missed the point: Blockstor ships a client that speaks the same command grammar as LINSTOR, short forms included, so the steps mirror 3.1 and 3.2 one for one and an operator moving between backends types almost the same thing. What actually differs is the client. `blockstor` is a standalone binary that talks to the Kubernetes API directly — no controller pod to exec into, no alias to set, it reads the kubeconfig the way kubectl does. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../docs/next/install/cozystack/platform.md | 65 +++++++++++-------- 1 file changed, 39 insertions(+), 26 deletions(-) diff --git a/content/en/docs/next/install/cozystack/platform.md b/content/en/docs/next/install/cozystack/platform.md index 670ba884..c0f81071 100644 --- a/content/en/docs/next/install/cozystack/platform.md +++ b/content/en/docs/next/install/cozystack/platform.md @@ -430,52 +430,65 @@ Create storage classes, one of which should be the default class. Skip this step on the LINSTOR backend — 3.1 and 3.2 cover it there. -Blockstor keeps its state in Kubernetes custom resources rather than in a database behind a controller pod, so its storage pools are configured with `kubectl` rather than through a CLI inside the cluster. There is no `linstor-controller` to exec into. +Blockstor speaks the same command grammar as LINSTOR, short forms included, so the steps below mirror 3.1 and 3.2 one for one. What differs is the client: `blockstor` is a standalone binary that talks to the Kubernetes API directly. There is no controller pod to exec into and no alias to set — it reads your kubeconfig the way `kubectl` does. Build it from the [Blockstor repository](https://github.com/cozystack/blockstor) with `make build`, or take it from a release. -1. List the nodes Blockstor knows about: +1. Check the client reaches the cluster: ```bash - kubectl get nodes.blockstor.cozystack.io + blockstor controller version ``` -1. Create the ZFS pool on the backing device of each storage node. The satellite image carries the ZFS tooling, so run it there: +1. List your nodes and check their readiness: ```bash - kubectl exec -n cozy-linstor ds/blockstor-satellite -- \ - zpool create -o failmode=continue data /dev/sdb + blockstor node list ``` - Repeat per node, addressing the satellite pod on that node. `failmode=continue` is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) for the same reason as on LINSTOR: it lets DRBD handle a disk failure rather than ZFS. + ```console + +-------+-----------+--------------+--------+ + | Node | NodeType | Addresses | State | + +=======+===========+==============+========+ + | srv1 | SATELLITE | 10.20.0.11 | ONLINE | + | srv2 | SATELLITE | 10.20.0.12 | ONLINE | + | srv3 | SATELLITE | 10.20.0.13 | ONLINE | + +-------+-----------+--------------+--------+ + ``` -1. Register each pool with Blockstor. The object name is `.` in lower case — a validation rule on the custom resource pins that, so a mismatch is refused rather than stored: +1. List available empty devices: - ```yaml - apiVersion: blockstor.cozystack.io/v1alpha1 - kind: StoragePool - metadata: - name: data.srv1 - spec: - nodeName: srv1 - poolName: data - providerKind: ZFS_THIN - props: - StorDriver/StorPoolName: data + ```bash + blockstor physical-storage list + ``` + +1. Create the storage pools. The verb and its flags are the same as on LINSTOR: + + ```bash + blockstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data + blockstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data + blockstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data + ``` + + It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) to set `failmode=continue` on ZFS storage pools, so DRBD handles a disk failure rather than ZFS: + + ```bash + kubectl exec -ti -n cozy-linstor blockstor-satellite- -- zpool set failmode=continue data ``` - Use `ZFS` instead of `ZFS_THIN` for a thick pool, where every volume reserves its full size on creation. + Use `lvm` in place of `zfs` for an LVM pool. 1. Check the result: ```bash - kubectl get storagepools.blockstor.cozystack.io + blockstor sp l ``` ```console - NAME AGE - data.srv1 1m - data.srv2 1m - data.srv3 1m - dfltdisklessstorpool.srv1 1m + +----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ + | StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | + +======================+=======+==========+==========+==============+===============+==============+=======+ + | DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | + | data | srv1 | ZFS_THIN | data | 237.80 GiB | 254 GiB | True | Ok | + +----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ ``` Then continue with [3.3](#33-create-storage-classes) — storage classes are the same on both backends. From 49cecf1dc56a0542df2455283405bfbb3d741ab6 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 26 Aug 2026 14:57:02 +0200 Subject: [PATCH 8/8] docs(install): one storage-backend switch for the whole page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The backend choice was a tab set in 2.3, and the steps that act on it were a separate section further down. An operator picking Blockstor had to remember that and navigate to a different heading. The `tabs` shortcode gains an opt-in `sync` group. Tab sets sharing a group switch together and the choice is remembered, so picking a backend anywhere on the page switches every command that follows. Tab sets without the parameter behave exactly as before — nothing else on the site changes. Storage now reads as one flow: 3.1 and 3.2 are backend tabs in that group, and the separate Blockstor section is gone. ZFS/LVM inside 3.2 became plain headings rather than an inner tab set: the shortcode collects tabs through a page Scratch, so a nested set absorbs content from beyond its own boundary — the inner tabs came out carrying labels lifted from later paragraphs. Assisted-By: Claude Signed-off-by: Andrei Kvapil --- .../docs/next/install/cozystack/platform.md | 269 +++++++++--------- layouts/partials/hooks/body-end.html | 51 ++++ layouts/shortcodes/tabs.html | 6 +- 3 files changed, 187 insertions(+), 139 deletions(-) diff --git a/content/en/docs/next/install/cozystack/platform.md b/content/en/docs/next/install/cozystack/platform.md index c0f81071..7f03288f 100644 --- a/content/en/docs/next/install/cozystack/platform.md +++ b/content/en/docs/next/install/cozystack/platform.md @@ -113,9 +113,9 @@ It may include a complete guide for your provider that you can use to deploy a p Cozystack runs one of two storage control planes. Both are driven by the same CSI driver and expose the same StorageClasses, so the choice does not change how workloads request volumes — only what manages them underneath. -{{< tabs name="storage_backend" >}} -{{% tab name="LINSTOR (default)" %}} -LINSTOR is the default and the backend every existing cluster runs. Nothing needs to be set: omitting `storage` selects it. +{{< tabs name="storage_backend" sync="storage-backend" >}} +{{% tab name="LINSTOR" %}} +The default, and the backend every existing cluster runs. Nothing needs to be set: omitting `storage` selects it. ```yaml spec: @@ -126,8 +126,8 @@ spec: backend: linstor ``` {{% /tab %}} -{{% tab name="Blockstor (experimental)" %}} -Blockstor is a LINSTOR-API-compatible control plane that stores its state in Kubernetes custom resources instead of an in-cluster database. +{{% tab name="Blockstor" %}} +Experimental. A LINSTOR-API-compatible control plane that stores its state in Kubernetes custom resources instead of an in-cluster database. ```yaml spec: @@ -241,65 +241,91 @@ Cozystack provides [LINSTOR](https://github.com/LINBIT/linstor-server) as a stor In the following steps, we'll access the storage interface, create storage pools, and define storage classes. -{{% alert color="warning" %}} -Steps 3.1 and 3.2 below are written for the **LINSTOR** backend. They run through `linstor-controller`, which exists only there — on `backend: blockstor` the control plane runs in external mode and that Deployment is never created, so every `linstor` command in them fails with `NotFound`. If you chose Blockstor in [step 2.3](#23-choose-a-storage-backend), follow [3.4](#34-configure-storage-on-the-blockstor-backend) instead. Step 3.3 is the same for both. -{{% /alert %}} +Both backends speak the same command grammar, so the steps below differ only in which client you run. The backend tabs on this page are linked: pick one in [step 2.3](#23-choose-a-storage-backend) or in any step below, and the rest switch with it. ### 3.1. Check Storage Devices -1. Set up an alias to access LINSTOR: +{{< tabs name="devices_client" sync="storage-backend" >}} +{{% tab name="LINSTOR" %}} +The client lives inside the controller pod, so reach it with an alias: - ```bash - alias linstor='kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor' - ``` +```bash +alias linstor='kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor' +``` -1. List your nodes and check their readiness: +List your nodes and check their readiness: - ```bash - linstor node list - ``` +```bash +linstor node list +``` - Example output shows node names and state: +```console ++-------------------------------------------------------+ +| Node | NodeType | Addresses | State | +|=======================================================| +| srv1 | SATELLITE | 192.168.100.11:3367 (SSL) | Online | +| srv2 | SATELLITE | 192.168.100.12:3367 (SSL) | Online | +| srv3 | SATELLITE | 192.168.100.13:3367 (SSL) | Online | ++-------------------------------------------------------+ +``` - ```console - +-------------------------------------------------------+ - | Node | NodeType | Addresses | State | - |=======================================================| - | srv1 | SATELLITE | 192.168.100.11:3367 (SSL) | Online | - | srv2 | SATELLITE | 192.168.100.12:3367 (SSL) | Online | - | srv3 | SATELLITE | 192.168.100.13:3367 (SSL) | Online | - +-------------------------------------------------------+ - ``` +List available empty devices: -1. List available empty devices: +```bash +linstor physical-storage list +``` - ```bash - linstor physical-storage list - ``` +```console ++--------------------------------------------+ +| Size | Rotational | Nodes | +|============================================| +| 107374182400 | True | srv3[/dev/sdb] | +| | | srv1[/dev/sdb] | +| | | srv2[/dev/sdb] | ++--------------------------------------------+ +``` +{{% /tab %}} +{{% tab name="Blockstor" %}} +`blockstor` is a standalone client that talks to the Kubernetes API directly, so there is no controller pod to exec into and no alias to set — it reads your kubeconfig the way `kubectl` does. Build it from the [Blockstor repository](https://github.com/cozystack/blockstor) with `make build`, or take it from a release. The command grammar is LINSTOR's, short forms included. - Example output shows the same node names: +Check the client reaches the cluster: - ```console - +--------------------------------------------+ - | Size | Rotational | Nodes | - |============================================| - | 107374182400 | True | srv3[/dev/sdb] | - | | | srv1[/dev/sdb] | - | | | srv2[/dev/sdb] | - +--------------------------------------------+ - ``` +```bash +blockstor controller version +``` +List your nodes and check their readiness: +```bash +blockstor node list +``` -### 3.2. Create Storage Pools +```console ++-------+-----------+--------------+--------+ +| Node | NodeType | Addresses | State | ++=======+===========+==============+========+ +| srv1 | SATELLITE | 10.20.0.11 | ONLINE | +| srv2 | SATELLITE | 10.20.0.12 | ONLINE | +| srv3 | SATELLITE | 10.20.0.13 | ONLINE | ++-------+-----------+--------------+--------+ +``` -1. Create storage pools using ZFS or LVM. +List available empty devices: - You can also restore previously created storage pools after a node reset. +```bash +blockstor physical-storage list +``` +{{% /tab %}} +{{< /tabs >}} - {{< tabs name="create_storage_pools" >}} - {{% tab name="ZFS" %}} +### 3.2. Create Storage Pools + +Create storage pools using ZFS or LVM. The same commands restore a pool after a node reset. + +{{< tabs name="pools_client" sync="storage-backend" >}} +{{% tab name="LINSTOR" %}} +**ZFS:** ```bash linstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data @@ -307,8 +333,7 @@ linstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data linstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data ``` -It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) -to set `failmode=continue` on ZFS storage pools to allow DRBD to handle disk failures instead of ZFS. +It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) to set `failmode=continue` on ZFS storage pools, so DRBD handles a disk failure rather than ZFS: ```bash kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv1 -- zpool set failmode=continue data @@ -316,8 +341,7 @@ kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv2 -- zpool set failmod kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv3 -- zpool set failmode=continue data ``` - {{% /tab %}} - {{% tab name="LVM" %}} +**LVM:** ```bash linstor ps cdp lvm srv1 /dev/sdb --pool-name data --storage-pool data @@ -325,40 +349,76 @@ linstor ps cdp lvm srv2 /dev/sdb --pool-name data --storage-pool data linstor ps cdp lvm srv3 /dev/sdb --pool-name data --storage-pool data ``` - {{% /tab %}} - {{% tab name="Restore ZFS/LVM storage-pool on nodes after reset" %}} +**Restore after a node reset:** ```bash for node in $(kubectl get nodes --no-headers -o custom-columns=":metadata.name"); do echo "linstor storage-pool create zfs $node data data" done -# linstor storage-pool create zfs data data ``` - {{% /tab %}} - {{< /tabs >}} +Check the result: -1. Check the results by listing the storage pools: +```bash +linstor sp l +``` - ```bash - linstor sp l - ``` +```console ++-------------------------------------------------------------------------------------------------------+ +| StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | +|=======================================================================================================| +| DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | +| data | srv1 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | ++-------------------------------------------------------------------------------------------------------+ +``` +{{% /tab %}} +{{% tab name="Blockstor" %}} +**ZFS:** - Example output: +```bash +blockstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data +blockstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data +blockstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data +``` - ```console - +-------------------------------------------------------------------------------------------------------------------------------------+ - | StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | SharedName | - |=====================================================================================================================================| - | DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | srv1;DfltDisklessStorPool | - | DfltDisklessStorPool | srv2 | DISKLESS | | | | False | Ok | srv2;DfltDisklessStorPool | - | DfltDisklessStorPool | srv3 | DISKLESS | | | | False | Ok | srv3;DfltDisklessStorPool | - | data | srv1 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv1;data | - | data | srv2 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv2;data | - | data | srv3 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv3;data | - +-------------------------------------------------------------------------------------------------------------------------------------+ - ``` +Set `failmode=continue` for the same reason as on LINSTOR. The satellite carries the ZFS tooling: + +```bash +kubectl exec -ti -n cozy-linstor -- zpool set failmode=continue data +``` +**LVM:** + +```bash +blockstor ps cdp lvm srv1 /dev/sdb --pool-name data --storage-pool data +blockstor ps cdp lvm srv2 /dev/sdb --pool-name data --storage-pool data +blockstor ps cdp lvm srv3 /dev/sdb --pool-name data --storage-pool data +``` + +**Restore after a node reset:** + +```bash +for node in $(kubectl get nodes --no-headers -o custom-columns=":metadata.name"); do + echo "blockstor storage-pool create zfs $node data data" +done +``` + +Check the result: + +```bash +blockstor sp l +``` + +```console ++----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ +| StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | ++======================+=======+==========+==========+==============+===============+==============+=======+ +| DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | +| data | srv1 | ZFS_THIN | data | 237.80 GiB | 254 GiB | True | Ok | ++----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ +``` +{{% /tab %}} +{{< /tabs >}} ### 3.3. Create Storage Classes @@ -426,73 +486,6 @@ Create storage classes, one of which should be the default class. -### 3.4. Configure Storage on the Blockstor Backend - -Skip this step on the LINSTOR backend — 3.1 and 3.2 cover it there. - -Blockstor speaks the same command grammar as LINSTOR, short forms included, so the steps below mirror 3.1 and 3.2 one for one. What differs is the client: `blockstor` is a standalone binary that talks to the Kubernetes API directly. There is no controller pod to exec into and no alias to set — it reads your kubeconfig the way `kubectl` does. Build it from the [Blockstor repository](https://github.com/cozystack/blockstor) with `make build`, or take it from a release. - -1. Check the client reaches the cluster: - - ```bash - blockstor controller version - ``` - -1. List your nodes and check their readiness: - - ```bash - blockstor node list - ``` - - ```console - +-------+-----------+--------------+--------+ - | Node | NodeType | Addresses | State | - +=======+===========+==============+========+ - | srv1 | SATELLITE | 10.20.0.11 | ONLINE | - | srv2 | SATELLITE | 10.20.0.12 | ONLINE | - | srv3 | SATELLITE | 10.20.0.13 | ONLINE | - +-------+-----------+--------------+--------+ - ``` - -1. List available empty devices: - - ```bash - blockstor physical-storage list - ``` - -1. Create the storage pools. The verb and its flags are the same as on LINSTOR: - - ```bash - blockstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data - blockstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data - blockstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data - ``` - - It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) to set `failmode=continue` on ZFS storage pools, so DRBD handles a disk failure rather than ZFS: - - ```bash - kubectl exec -ti -n cozy-linstor blockstor-satellite- -- zpool set failmode=continue data - ``` - - Use `lvm` in place of `zfs` for an LVM pool. - -1. Check the result: - - ```bash - blockstor sp l - ``` - - ```console - +----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ - | StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | - +======================+=======+==========+==========+==============+===============+==============+=======+ - | DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | - | data | srv1 | ZFS_THIN | data | 237.80 GiB | 254 GiB | True | Ok | - +----------------------+-------+----------+----------+--------------+---------------+--------------+-------+ - ``` - -Then continue with [3.3](#33-create-storage-classes) — storage classes are the same on both backends. - ## 4. Configure Networking Next, we will configure how the Cozystack cluster can be accessed. diff --git a/layouts/partials/hooks/body-end.html b/layouts/partials/hooks/body-end.html index 3cf8ee44..d995b891 100644 --- a/layouts/partials/hooks/body-end.html +++ b/layouts/partials/hooks/body-end.html @@ -19,3 +19,54 @@ } {{ end }} + +{{- /* Page-wide tab switching for tab sets that opt in with `sync`. + Clicking "Blockstor" in one set selects it in every other set on + the page, and the choice survives navigation — so an operator + picks their storage backend once and the whole runbook follows. */ -}} + diff --git a/layouts/shortcodes/tabs.html b/layouts/shortcodes/tabs.html index 0108a4aa..279c663e 100644 --- a/layouts/shortcodes/tabs.html +++ b/layouts/shortcodes/tabs.html @@ -2,7 +2,11 @@ {{- $tab_set_id := .Get "name" | default (printf "tabset-%s-%d" (.Page.RelPermalink) (.Page.Scratch.Get "tabset-counter") ) | anchorize -}} {{- $tabs := .Scratch.Get "tabs" -}} {{- if .Inner -}}{{- /* We don't use the inner content, but Hugo will complain if we don't reference it. */ -}}{{- end -}} -