Skip to content

Commit 7c6e3d5

Browse files
apedrizaapedriza
authored
Merge pull request #1340 from apedriza/add-ignition-docs
Add example about using Ignition provisioner
2 parents 89d3a37 + 181004b commit 7c6e3d5

File tree

2 files changed

+245
-0
lines changed

2 files changed

+245
-0
lines changed

‎docs/ignition-support.md‎

Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
# Ignition Support
2+
3+
k0smotron generates bootstrap data in *cloud-init* format by default. Some OS distributions used for workload clusters nodes rely on **Ignition** for early-boot provisioning instead of cloud-init, requiring bootstrap data to be provided in Ignition format. To support these environments, k0smotron can alternatively generate bootstrap data in Ignition format for distributions such [Fedora CoreOS](https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/) or [Flatcar Container Linux](https://www.flatcar.org/docs/latest/provisioning/ignition/).
4+
5+
This guide explains how to deploy an AWS workload cluster using Ignition. More documentation about Ignition support in AWS infrastructure provider can be found [here](https://cluster-api-aws.sigs.k8s.io/topics/ignition-support).
6+
7+
### Initialize the management cluster
8+
9+
Before workload clusters can be deployed, Cluster API components must be deployed to the management cluster.
10+
11+
Initialize the management cluster:
12+
13+
```sh
14+
export AWS_REGION=eu-west-1
15+
export AWS_ACCESS_KEY_ID=<your-access-key>
16+
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
17+
export AWS_SESSION_TOKEN=<your-session-token>
18+
19+
# Workload clusters need to call the AWS API as part of their normal operation.
20+
# The following command creates a CloudFormation stack which provisions the
21+
# necessary IAM resources to be used by workload clusters.
22+
clusterawsadm bootstrap iam create-cloudformation-stack
23+
24+
# The management cluster needs to call the AWS API in order to manage cloud
25+
# resources for workload clusters. The following command tells clusterctl to
26+
# store the AWS credentials provided before in a Kubernetes secret where they
27+
# can be retrieved by the AWS provider running on the management cluster.
28+
export AWS_B64ENCODED_CREDENTIALS=$(clusterawsadm bootstrap credentials encode-as-profile)
29+
30+
# Enable the feature gates controlling Ignition bootstrap.
31+
export EXP_BOOTSTRAP_FORMAT_IGNITION=true # Used by the AWS provider
32+
33+
# Initialize the management cluster.
34+
clusterctl init --infrastructure aws --control-plane k0sproject-k0smotron --bootstrap k0sproject-k0smotron
35+
```
36+
37+
!!! warning "Set `EXP_BOOTSTRAP_FORMAT_IGNITION` environment variable"
38+
Before deploying CAPA using `clusterctl`, make sure you set `EXP_KUBEADM_BOOTSTRAP_FORMAT_IGNITION=true` environment variables to enable experimental Ignition bootstrap support.
39+
40+
### Create a workload cluster
41+
42+
After deploying the bootstrap, control plane, and infrastructure provider controllers, we can create a workload cluster configured to use Ignition as bootstraping engine when using OS distributions supporting this engine. In our example, we will use Flatcar Container Linux as AMI for the cluster nodes.
43+
44+
Configure access to the machines by using your *SSH Key Name* and *SSH Public Key* in the following following manifest and apply it:
45+
46+
```yaml
47+
apiVersion: cluster.x-k8s.io/v1beta1
48+
kind: Cluster
49+
metadata:
50+
name: ignition-test-cluster
51+
namespace: ignition-test
52+
spec:
53+
clusterNetwork:
54+
pods:
55+
cidrBlocks:
56+
- 192.168.0.0/16
57+
serviceDomain: cluster.local
58+
services:
59+
cidrBlocks:
60+
- 10.128.0.0/12
61+
controlPlaneRef:
62+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
63+
kind: K0sControlPlane
64+
name: ignition-test-cluster-aws-test
65+
infrastructureRef:
66+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
67+
kind: AWSCluster
68+
name: ignition-test-cluster
69+
---
70+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
71+
kind: AWSMachineTemplate
72+
metadata:
73+
name: ignition-test-cluster-aws-test-mt
74+
namespace: ignition-test
75+
spec:
76+
template:
77+
spec:
78+
ignition:
79+
storageType: ClusterObjectStore
80+
version: "3.4"
81+
ami:
82+
id: ami-00d12617b68dbc62f # Flatcar Container Linux stable 3975.2.1 (HVM) in eu-west-1
83+
instanceType: t3.large
84+
publicIP: true
85+
iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
86+
uncompressedUserData: false
87+
sshKeyName: <your-ssh-key-name> # Your SSH key here
88+
---
89+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
90+
kind: K0sControlPlane
91+
metadata:
92+
name: ignition-test-cluster-aws-test
93+
namespace: ignition-test
94+
spec:
95+
replicas: 3
96+
version: v1.30.2+k0s.0
97+
updateStrategy: Recreate
98+
k0sConfigSpec:
99+
# Flatcar, as inmutable OS, needs k0s in /opt/bin. It cannot write k0s binary in the default /usr/local/bin.
100+
k0sInstallDir: /opt/bin
101+
ignition:
102+
variant: flatcar
103+
version: 1.1.0
104+
additionalConfig: |
105+
variant: flatcar
106+
version: 1.1.0
107+
passwd:
108+
users:
109+
- name: core
110+
ssh_authorized_keys:
111+
- ssh-rsa <your-ssh-public-key>
112+
args:
113+
- --enable-worker
114+
- --debug
115+
k0s:
116+
apiVersion: k0s.k0sproject.io/v1beta1
117+
kind: ClusterConfig
118+
metadata:
119+
name: k0s
120+
spec:
121+
api:
122+
extraArgs:
123+
anonymous-auth: "true"
124+
telemetry:
125+
enabled: false
126+
machineTemplate:
127+
infrastructureRef:
128+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
129+
kind: AWSMachineTemplate
130+
name: ignition-test-cluster-aws-test-mt
131+
namespace: ignition-test
132+
---
133+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
134+
kind: AWSCluster
135+
metadata:
136+
name: ignition-test-cluster
137+
namespace: ignition-test
138+
spec:
139+
# Ignition bootstrap data needs to be stored in an S3 bucket so that nodes can
140+
# read them at boot time. Store Ignition bootstrap data in the following bucket.
141+
s3Bucket:
142+
controlPlaneIAMInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
143+
# For simplicity and following AWS documentation: by default clusterawsadm creates IAM roles
144+
# to only allow interacting with buckets with cluster-api-provider-aws- prefix to reduce the
145+
# permissions of CAPA controller, so all bucket names should use this prefix.
146+
name: cluster-api-provider-aws-ignition-test
147+
nodesIAMInstanceProfiles:
148+
- nodes.cluster-api-provider-aws.sigs.k8s.io
149+
region: eu-west-1
150+
sshKeyName: <your-ssh-key-name> # Your SSH key here
151+
controlPlaneLoadBalancer:
152+
healthCheckProtocol: TCP
153+
network:
154+
additionalControlPlaneIngressRules:
155+
- description: "k0s controller join API"
156+
protocol: tcp
157+
fromPort: 9443
158+
toPort: 9443
159+
---
160+
apiVersion: cluster.x-k8s.io/v1beta1
161+
kind: MachineDeployment
162+
metadata:
163+
name: ignition-test-cluster-aws-test-md
164+
namespace: ignition-test
165+
spec:
166+
clusterName: ignition-test-cluster
167+
replicas: 1
168+
template:
169+
spec:
170+
clusterName: ignition-test-cluster
171+
bootstrap:
172+
configRef: # This triggers our controller to create cloud-init secret
173+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
174+
kind: K0sWorkerConfigTemplate
175+
name: ignition-test-cluster-machine-config
176+
namespace: ignition-test
177+
infrastructureRef:
178+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
179+
kind: AWSMachineTemplate
180+
name: ignition-test-cluster}-aws-test-mt
181+
namespace: ignition-test
182+
---
183+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
184+
kind: K0sWorkerConfigTemplate
185+
metadata:
186+
name: ignition-test-cluster-machine-config
187+
namespace: ignition-test
188+
spec:
189+
template:
190+
spec:
191+
version: v1.30.2+k0s.0
192+
# Flatcar, as inmutable OS, needs k0s in /opt/bin. It cannot write k0s binary in the default /usr/local/bin.
193+
k0sInstallDir: /opt/bin
194+
ignition:
195+
variant: flatcar
196+
version: 1.1.0
197+
additionalConfig: |
198+
variant: flatcar
199+
version: 1.1.0
200+
passwd:
201+
users:
202+
- name: core
203+
ssh_authorized_keys:
204+
- ssh-rsa <your-ssh-public-key>
205+
```
206+
207+
Ignition bootstrap is enabled implicitly by configuring an `ignition` section for `K0sControlPlane.spec.k0sConfigSpec` and `K0sWorkerConfigTemplate.spec.template.spec` resources. When an ignition section with a supported variant and version is present (for example flatcar), k0smotron generates Ignition-formatted bootstrap data instead of the default cloud-init format. Supported variants and their versions can be found [here](https://coreos.github.io/butane/specs/#butane-specifications-and-ignition-specifications).
208+
209+
k0smotron allows extending the generated Ignition bootstrap data using the `additionalConfig` field. In this example, `additionalConfig` is used to configure SSH access for the default `core` user by injecting an SSH public key, allowing access to the machine after provisioning.
210+
211+
Additionally, because Flatcar is an immutable operating system, it does not allow writing the k0s binary to the default `/usr/local/bin` location. For this reason, an alternative installation directory is specified using `k0sInstallDir`.
212+
213+
We can check the state of our recently created cluster by running:
214+
215+
```shell
216+
$ clusterctl describe cluster ignition-test-cluster -nignition-test
217+
NAME REPLICAS AVAILABLE READY UP TO DATE STATUS REASON SINCE MESSAGE
218+
Cluster/ignition-test-cluster 4/4 4 4 4 True Available 7m19s
219+
├─ClusterInfrastructure - AWSCluster/ignition-test-cluster True NoReasonReported 9m29s
220+
├─ControlPlane - K0sControlPlane/ignition-test-cluster-aws-test 3/3 3 3
221+
│ └─3 Machines... 3 3 0 True Ready 7m25s See ignition-test-cluster-aws-test-86hdh, ignition-test-cluster-aws-test-8qtnr, ...
222+
└─Workers
223+
└─MachineDeployment/ignition-test-cluster-aws-test-md 1/1 1 1 1 True Available 7m19s
224+
└─Machine/ignition-test-cluster-aws-test-md-9fbps-t77jh 1 1 1 1 True Ready 7m19s
225+
```
226+
227+
Once cluster is created, we can access to it and check everything is working as expected.
228+
229+
```shell
230+
# Retrieve workload cluster kubeconfig for accesing it
231+
$ clusterctl get kubeconfig ignition-test-cluster -nignition-test > ignition-test-cluster.conf
232+
```
233+
234+
```shell
235+
# List nodes in the workload cluster
236+
$ kubectl --kubeconfig ignition-test-cluster.conf get nodes
237+
NAME STATUS ROLES AGE VERSION
238+
ignition-test-cluster-aws-test-86hdh Ready control-plane 8m15s v1.30.2+k0s
239+
ignition-test-cluster-aws-test-8qtnr Ready control-plane 10m v1.30.2+k0s
240+
ignition-test-cluster-aws-test-bw4b5 Ready control-plane 9m21s v1.30.2+k0s
241+
ignition-test-cluster-aws-test-md-9fbps-t77jh Ready <none> 10m v1.30.2+k0s
242+
```
243+
244+

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ nav:
3030
- Worker Node Bootstrap: capi-bootstrap.md
3131
- Remote Machine Provider: capi-remote.md
3232
- Ingress support: ingress-support.md
33+
- Ignition support: ignition-support.md
3334
- ClusterClass: capi-clusterclass.md
3435
- Health Checks: capi-machine-health-checks.md
3536
- Examples:

0 commit comments

Comments
 (0)