Skip to content
237 changes: 170 additions & 67 deletions content/en/docs/next/install/cozystack/platform.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,46 @@ enable a provider-specific load balancer, or use a different network setup.
Check out the [provider-specific installation]({{% ref "/docs/next/install/providers" %}}) section.
It may include a complete guide for your provider that you can use to deploy a production-ready cluster.

### 2.3. Define Network Configuration
### 2.3. Choose a Storage Backend

Cozystack runs one of two storage control planes. Both are driven by the same CSI driver and expose the same StorageClasses, so the choice does not change how workloads request volumes — only what manages them underneath.

{{< tabs name="storage_backend" sync="storage-backend" >}}
{{% tab name="LINSTOR" %}}
The default, and the backend every existing cluster runs. Nothing needs to be set: omitting `storage` selects it.

```yaml
spec:
components:
platform:
values:
storage:
backend: linstor
```
{{% /tab %}}
{{% tab name="Blockstor" %}}
Experimental. A LINSTOR-API-compatible control plane that stores its state in Kubernetes custom resources instead of an in-cluster database.

```yaml
spec:
components:
platform:
values:
storage:
backend: blockstor
```
{{% /tab %}}
{{< /tabs >}}

{{% alert color="warning" %}}
Blockstor is **experimental**. Use it on clusters where you can tolerate storage-layer problems, and prefer LINSTOR for production until this notice is removed.
{{% /alert %}}

An unrecognised value fails the render rather than falling back to a default, so a typo cannot quietly deploy the storage control plane you did not ask for.

To move an existing cluster from LINSTOR to Blockstor, see [Migrating from LINSTOR to Blockstor]({{% ref "/docs/next/storage/blockstor-migration" %}}). Do not simply flip this value on a cluster that already holds data — the switch installs a new control plane that knows nothing about the existing volumes until they are adopted.

### 2.4. Define Network Configuration

Replace `example.org` in `publishing.host` and `publishing.apiServerEndpoint` with a routable fully-qualified domain name (FQDN) that you control.
If you only have a public IP, but no routable FQDN, use [nip.io](https://nip.io/) with dash notation.
Expand All @@ -130,7 +169,7 @@ networking:
Cozystack gathers anonymous usage statistics by default. Learn more about what data is collected and how to opt out in the [Telemetry Documentation]({{% ref "/docs/next/operations/configuration/telemetry" %}}).
{{% /alert %}}

### 2.4. Apply Platform Package
### 2.5. Apply Platform Package

Once the configuration file is ready, apply it:

Expand Down Expand Up @@ -198,124 +237,188 @@ kubectl taint nodes --all node-role.kubernetes.io/control-plane-
## 3. Configure Storage

Kubernetes needs a storage subsystem to provide persistent volumes to applications, but it doesn't include one of its own.
Cozystack provides [LINSTOR](https://github.com/LINBIT/linstor-server) as a storage subsystem.
Cozystack provides [LINSTOR](https://github.com/LINBIT/linstor-server) as a storage subsystem, and [Blockstor](https://github.com/cozystack/blockstor) as an experimental alternative.

In the following steps, we'll access LINSTOR interface, create storage pools, and define storage classes.
In the following steps, we'll access the storage interface, create storage pools, and define storage classes.

Both backends speak the same command grammar, so the steps below differ only in which client you run. The backend tabs on this page are linked: pick one in [step 2.3](#23-choose-a-storage-backend) or in any step below, and the rest switch with it.


### 3.1. Check Storage Devices

1. Set up an alias to access LINSTOR:
{{< tabs name="devices_client" sync="storage-backend" >}}
{{% tab name="LINSTOR" %}}
The client lives inside the controller pod, so reach it with an alias:

```bash
alias linstor='kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor'
```
```bash
alias linstor='kubectl exec -n cozy-linstor deploy/linstor-controller -- linstor'
```

1. List your nodes and check their readiness:
List your nodes and check their readiness:

```bash
linstor node list
```
```bash
linstor node list
```

Example output shows node names and state:
```console
+-------------------------------------------------------+
| Node | NodeType | Addresses | State |
|=======================================================|
| srv1 | SATELLITE | 192.168.100.11:3367 (SSL) | Online |
| srv2 | SATELLITE | 192.168.100.12:3367 (SSL) | Online |
| srv3 | SATELLITE | 192.168.100.13:3367 (SSL) | Online |
+-------------------------------------------------------+
```

```console
+-------------------------------------------------------+
| Node | NodeType | Addresses | State |
|=======================================================|
| srv1 | SATELLITE | 192.168.100.11:3367 (SSL) | Online |
| srv2 | SATELLITE | 192.168.100.12:3367 (SSL) | Online |
| srv3 | SATELLITE | 192.168.100.13:3367 (SSL) | Online |
+-------------------------------------------------------+
```
List available empty devices:

1. List available empty devices:
```bash
linstor physical-storage list
```

```bash
linstor physical-storage list
```
```console
+--------------------------------------------+
| Size | Rotational | Nodes |
|============================================|
| 107374182400 | True | srv3[/dev/sdb] |
| | | srv1[/dev/sdb] |
| | | srv2[/dev/sdb] |
+--------------------------------------------+
```
{{% /tab %}}
{{% tab name="Blockstor" %}}
`blockstor` is a standalone client that talks to the Kubernetes API directly, so there is no controller pod to exec into and no alias to set — it reads your kubeconfig the way `kubectl` does. Build it from the [Blockstor repository](https://github.com/cozystack/blockstor) with `make build`, or take it from a release. The command grammar is LINSTOR's, short forms included.

Example output shows the same node names:
Check the client reaches the cluster:

```console
+--------------------------------------------+
| Size | Rotational | Nodes |
|============================================|
| 107374182400 | True | srv3[/dev/sdb] |
| | | srv1[/dev/sdb] |
| | | srv2[/dev/sdb] |
+--------------------------------------------+
```
```bash
blockstor controller version
```

List your nodes and check their readiness:

```bash
blockstor node list
```

### 3.2. Create Storage Pools
```console
+-------+-----------+--------------+--------+
| Node | NodeType | Addresses | State |
+=======+===========+==============+========+
| srv1 | SATELLITE | 10.20.0.11 | ONLINE |
| srv2 | SATELLITE | 10.20.0.12 | ONLINE |
| srv3 | SATELLITE | 10.20.0.13 | ONLINE |
+-------+-----------+--------------+--------+
```

1. Create storage pools using ZFS or LVM.
List available empty devices:

You can also restore previously created storage pools after a node reset.
```bash
blockstor physical-storage list
```
{{% /tab %}}
{{< /tabs >}}

### 3.2. Create Storage Pools

{{< tabs name="create_storage_pools" >}}
{{% tab name="ZFS" %}}
Create storage pools using ZFS or LVM. The same commands restore a pool after a node reset.

{{< tabs name="pools_client" sync="storage-backend" >}}
{{% tab name="LINSTOR" %}}
**ZFS:**

```bash
linstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data
linstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data
linstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data
```

It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020)
to set `failmode=continue` on ZFS storage pools to allow DRBD to handle disk failures instead of ZFS.
It is [recommended](https://github.com/LINBIT/linstor-server/issues/463#issuecomment-3401472020) to set `failmode=continue` on ZFS storage pools, so DRBD handles a disk failure rather than ZFS:

```bash
kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv1 -- zpool set failmode=continue data
kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv2 -- zpool set failmode=continue data
kubectl exec -ti -n cozy-linstor pod/linstor-satellite.srv3 -- zpool set failmode=continue data
```

{{% /tab %}}
{{% tab name="LVM" %}}
**LVM:**

```bash
linstor ps cdp lvm srv1 /dev/sdb --pool-name data --storage-pool data
linstor ps cdp lvm srv2 /dev/sdb --pool-name data --storage-pool data
linstor ps cdp lvm srv3 /dev/sdb --pool-name data --storage-pool data
```

{{% /tab %}}
{{% tab name="Restore ZFS/LVM storage-pool on nodes after reset" %}}
**Restore after a node reset:**

```bash
for node in $(kubectl get nodes --no-headers -o custom-columns=":metadata.name"); do
echo "linstor storage-pool create zfs $node data data"
done
# linstor storage-pool create zfs <node> data data
```

{{% /tab %}}
{{< /tabs >}}
Check the result:

1. Check the results by listing the storage pools:
```bash
linstor sp l
```

```bash
linstor sp l
```
```console
+-------------------------------------------------------------------------------------------------------+
| StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State |
|=======================================================================================================|
| DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok |
| data | srv1 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok |
+-------------------------------------------------------------------------------------------------------+
```
{{% /tab %}}
{{% tab name="Blockstor" %}}
**ZFS:**

Example output:
```bash
blockstor ps cdp zfs srv1 /dev/sdb --pool-name data --storage-pool data
blockstor ps cdp zfs srv2 /dev/sdb --pool-name data --storage-pool data
blockstor ps cdp zfs srv3 /dev/sdb --pool-name data --storage-pool data
```

```console
+-------------------------------------------------------------------------------------------------------------------------------------+
| StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State | SharedName |
|=====================================================================================================================================|
| DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok | srv1;DfltDisklessStorPool |
| DfltDisklessStorPool | srv2 | DISKLESS | | | | False | Ok | srv2;DfltDisklessStorPool |
| DfltDisklessStorPool | srv3 | DISKLESS | | | | False | Ok | srv3;DfltDisklessStorPool |
| data | srv1 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv1;data |
| data | srv2 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv2;data |
| data | srv3 | ZFS | data | 96.41 GiB | 99.50 GiB | True | Ok | srv3;data |
+-------------------------------------------------------------------------------------------------------------------------------------+
```
Set `failmode=continue` for the same reason as on LINSTOR. The satellite carries the ZFS tooling:

```bash
kubectl exec -ti -n cozy-linstor <blockstor-satellite-pod> -- zpool set failmode=continue data
```

**LVM:**

```bash
blockstor ps cdp lvm srv1 /dev/sdb --pool-name data --storage-pool data
blockstor ps cdp lvm srv2 /dev/sdb --pool-name data --storage-pool data
blockstor ps cdp lvm srv3 /dev/sdb --pool-name data --storage-pool data
```

**Restore after a node reset:**

```bash
for node in $(kubectl get nodes --no-headers -o custom-columns=":metadata.name"); do
echo "blockstor storage-pool create zfs $node data data"
done
```

Check the result:

```bash
blockstor sp l
```

```console
+----------------------+-------+----------+----------+--------------+---------------+--------------+-------+
| StoragePool | Node | Driver | PoolName | FreeCapacity | TotalCapacity | CanSnapshots | State |
+======================+=======+==========+==========+==============+===============+==============+=======+
| DfltDisklessStorPool | srv1 | DISKLESS | | | | False | Ok |
| data | srv1 | ZFS_THIN | data | 237.80 GiB | 254 GiB | True | Ok |
+----------------------+-------+----------+----------+--------------+---------------+--------------+-------+
```
{{% /tab %}}
{{< /tabs >}}

### 3.3. Create Storage Classes

Expand Down
4 changes: 3 additions & 1 deletion content/en/docs/next/storage/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,6 @@ aliases:
- /docs/next/operations/storage
---

These guides will show you how to perform typical tasks related to the LINSTOR storage system in Cozystack.
These guides will show you how to perform typical tasks related to the storage subsystem in Cozystack.

Cozystack runs one of two storage control planes, selected at install time: LINSTOR, the default, or Blockstor, which is experimental. See [Choose a Storage Backend]({{% ref "/docs/next/install/cozystack/platform#23-choose-a-storage-backend" %}}) to pick one, and [Migrating from LINSTOR to Blockstor]({{% ref "/docs/next/storage/blockstor-migration" %}}) to move an existing cluster across. Unless a guide says otherwise, the pages in this section describe LINSTOR.
Loading
Loading