From 1a78d7dc836b542883c212f6dd85f2af7e5a8995 Mon Sep 17 00:00:00 2001 From: Chris Tauchen Date: Wed, 2 Sep 2026 13:06:11 +0100 Subject: [PATCH] Correct the Forklift install pages after the RKE2 test run A test run of the v2.12.5-v3.24.0-2.0 release build on a five-node RKE2 cluster found five problems in the install pages. Each one was true on a uniform, well-provisioned cluster and wrong or unusable on a real one. The seccomp and AppArmor profile fields belong to the one ForkliftController, so they apply to every conversion pod in the cluster. The page said to set only the seccomp pair "on nodes that do not enforce AppArmor", which assumes every node is the same. A cluster whose conversion-capable nodes are mixed has no valid setting: naming the AppArmor profile makes the kubelet refuse the pod where AppArmor is absent, and omitting it lets the pod start where AppArmor is enforced, and the remount then fails. The page now gives the two valid configurations, and points to convertorNodeSelector on the Plan for mixed clusters. The storage prerequisite asked only for CDI and a default StorageClass. Migrations also fail before conversion when the volume cannot honour the file locking qemu-img takes, which NFS-backed storage needs mount options for. The address-preservation check named only the interface-scoped annotations. A primary NIC mapped by a type: pod item carries the unscoped keys instead, so a reader following the page concluded that preservation had failed when it had worked. Both forms are now shown, one per case. The command for reading the profile installer's log aggregated every pod into unlabelled lines, losing the node attribution the following step depends on. It now reads one pod at a time. The calico struct was shown as a fragment, which left its place in the YAML ambiguous. One complete map, with a type: pod item and a type: multus item, now shows it as a property of destination. Both the 3.24-2 and the unversioned copies are updated, since nothing compares them. --- .../kubevirt/install-forklift-kubernetes.mdx | 85 ++++++++++++++----- .../kubevirt/install-forklift-openshift.mdx | 48 +++++++---- .../kubevirt/install-forklift-kubernetes.mdx | 85 ++++++++++++++----- .../kubevirt/install-forklift-openshift.mdx | 48 +++++++---- 4 files changed, 196 insertions(+), 70 deletions(-) diff --git a/calico-enterprise/networking/kubevirt/install-forklift-kubernetes.mdx b/calico-enterprise/networking/kubevirt/install-forklift-kubernetes.mdx index 1ef8a0b119..7fa50bece2 100644 --- a/calico-enterprise/networking/kubevirt/install-forklift-kubernetes.mdx +++ b/calico-enterprise/networking/kubevirt/install-forklift-kubernetes.mdx @@ -43,6 +43,9 @@ Upstream Forklift's guest conversion fails on clusters whose nodes carry the run ``` - CDI and a default StorageClass, for migrations that import disks. Neither is needed to install Forklift. + The StorageClass backing the migrated disks must support the file locking `qemu-img` takes on an image it opens. + A volume that cannot honor those locks fails the migration before conversion starts. + NFS-backed storage needs mount options that permit locking, such as `nfsvers=3,nolock` or NFSv4, set on the StorageClass rather than only on the export. ## How to @@ -96,8 +99,7 @@ OpenShift hits neither: its nodes permit `unshare` and confine containers with S Put the profiles on every node that can run a conversion. Use the Security Profiles Operator if you already run it; otherwise you may apply the ConfigMap and DaemonSet patch below, after inspection. -**Note** This is a privileged DaemonSet. It writes both profiles under the kubelet's seccomp directory, and loads the AppArmor one on nodes that have `apparmor_parser`. Its log says which it did on each node: - +**Note** This is a privileged DaemonSet. It writes both profiles under the kubelet's seccomp directory, and loads the AppArmor one on nodes that have `apparmor_parser`. ```bash kubectl apply -f $[filesUrl]/forklift/forklift-virt-v2v-profiles.yaml @@ -105,11 +107,23 @@ kubectl apply -f $[filesUrl]/forklift/forklift-virt-v2v-profiles.yaml kubectl -n konveyor-forklift rollout status ds/forklift-virt-v2v-profile-installer ``` +Each pod reports what it did on its own node, and a single `kubectl logs` across the DaemonSet loses that attribution, so read the logs one pod at a time: + ```bash -kubectl -n konveyor-forklift logs -l service=forklift-virt-v2v-profile-installer -c install +for pod in $(kubectl -n konveyor-forklift get pods \ + -l service=forklift-virt-v2v-profile-installer -o name); do + echo "== $(kubectl -n konveyor-forklift get $pod -o jsonpath='{.spec.nodeName}')" + kubectl -n konveyor-forklift logs $pod -c install +done ``` -Then name the profiles on the `ForkliftController`: +A node that enforces AppArmor reports `loaded AppArmor profile`. +One that does not reports `no apparmor_parser on this node, skipping AppArmor`. +What you see decides which fields to set next. + +Then name the profiles on the `ForkliftController`. +The patch below sets all four fields, which is what a cluster of AppArmor-enforcing nodes needs. +Drop the two `apparmor` lines if no node enforces it: ```bash kubectl -n konveyor-forklift patch forkliftcontroller forklift-controller \ @@ -120,7 +134,22 @@ kubectl -n konveyor-forklift patch forkliftcontroller forklift-controller \ "virt_v2v_apparmor_profile_path": "forklift-virt-v2v-unshare"}}' ``` -All four fields are unset by default, which leaves every cluster on its runtime default: naming a profile that a node does not carry makes the kubelet refuse the pod outright. A path given without its type is rejected as the CR is saved. On nodes that do not enforce AppArmor, set only the seccomp pair. +All four fields are unset by default, which leaves every cluster on its runtime default. +A path given without its type is rejected as the CR is saved. + +The fields belong to the one `ForkliftController`, so they apply to every conversion pod in the cluster. +There is no per-node and no per-`Plan` override, which makes the AppArmor pair a property of the cluster rather than of a node: + +| Nodes that can run a conversion | Set | +| --- | --- | +| All enforce AppArmor | All four fields | +| None enforce AppArmor | The seccomp pair only | + +A cluster whose conversion-capable nodes are mixed has no valid setting. +Name the AppArmor profile and the kubelet refuses the conversion pod outright wherever AppArmor is absent, reporting `Cannot enforce AppArmor: AppArmor is not enabled on the host`. +Omit it and the pod is admitted on an AppArmor node, where the remount then fails. + +If your nodes are mixed, confine conversions to a uniform set of them with `spec.convertorNodeSelector` on the `Plan`, and set these fields to match the set you chose. ### 5. Map a VM's networks to $[prodname] @@ -137,10 +166,24 @@ can be included to flag that the primary network should also receive address-pre To additionally force the primary NIC to get attached to a pre-configured Calico `Network`, the struct has fields `network` and `vlan` which target the Calico `Network` to use. +`calico` is a property of a map item's `destination`, alongside `type`, and is valid only on an item of `type: pod`. +At most one item in a `NetworkMap` may carry it. + ```yaml + map: + - source: + id: + destination: + type: pod calico: network: mock-l2-net # cluster-scoped projectcalico.org/v3 Network CR vlan: 100 # must match a vlan.id entry in that Network + - source: + id: + destination: + type: multus + name: my-network-vlan10 + namespace: vms ``` | Field | Effect | @@ -152,28 +195,30 @@ fields `network` and `vlan` which target the Calico `Network` to use. Note: The MAC is always carried over; the addresses follow when the `Plan` sets `preserveStaticIPs`. Definitions that use another CNI are left untouched. The values are set before the VM starts, so each interface is created already holding them. -```yaml - map: - - source: - id: - destination: - type: multus - name: my-network-vlan10 - namespace: vms -``` - - ### Verify address preservation worked Inspect the `virt-launcher` pod, or the VM template created post-migration. -It should contain annotations for Calico address preservation, for each NIC -in the NetworkMap: +Each NIC in the `NetworkMap` carries a pair of annotations, and which keys to look for depends on the NIC. + +The primary NIC, mapped by an item of `type: pod` with a `calico` block, uses keys with no interface name: ```yaml -cni.projectcalico.org/.hwAddr: ... -cni.projectcalico.org/.ipAddrs: ... +cni.projectcalico.org/hwAddr: '52:54:00:3a:1c:04' +cni.projectcalico.org/ipAddrs: '["10.10.0.42"]' ``` +A secondary NIC, mapped by an item of `type: multus`, scopes each key to the interface it applies to: + +```yaml +cni.projectcalico.org/.hwAddr: '52:54:00:3a:1c:05' +cni.projectcalico.org/.ipAddrs: '["10.10.10.42"]' +``` + +For the name to use in place of ``, see [Which interface name to use](../l2-bridge/vm-identity.mdx#which-interface-name-to-use). + +The MAC is always stamped; `ipAddrs` appears only when the `Plan` sets `preserveStaticIPs`. +$[prodname] adds `cni.projectcalico.org/podIP` and `podIPs` once the interface is up, which is what confirms the requested address was assigned. + See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx) for the `Network`, IP pool and attachment definition, and [Set a VM's IP and MAC address](../l2-bridge/vm-identity.mdx) for the annotations themselves. diff --git a/calico-enterprise/networking/kubevirt/install-forklift-openshift.mdx b/calico-enterprise/networking/kubevirt/install-forklift-openshift.mdx index 8e259efa8c..ba059aaa25 100644 --- a/calico-enterprise/networking/kubevirt/install-forklift-openshift.mdx +++ b/calico-enterprise/networking/kubevirt/install-forklift-openshift.mdx @@ -27,6 +27,8 @@ Install the Tigera fork of Forklift on an OpenShift cluster and connect it to a - The BPF data plane enabled: `bpfEnabled: true` in the default `FelixConfiguration`. See [Enable eBPF on an existing cluster](../../operations/ebpf/enabling-ebpf.mdx). - OpenShift Virtualization (KubeVirt and CDI) installed, with `HyperConverged` Available. - A StorageClass for the migrated VM disks. + It must support the file locking `qemu-img` takes on an image it opens: a volume that cannot honor those locks fails the migration before conversion starts. + NFS-backed storage needs mount options that permit locking, such as `nfsvers=3,nolock` or NFSv4, set on the StorageClass rather than only on the export. - IP pools covering the VM addresses. The pod network uses an ordinary pool; each L2 VLAN needs its own pool with `allowedUses: [L2Workload]` and `disableBGPExport: true`. A VM keeps its address only if that address falls inside a pool. See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx). ## How to @@ -122,10 +124,24 @@ can be included to flag that the primary network should also receive address-pre To additionally force the primary NIC to get attached to a pre-configured Calico `Network`, the struct has fields `network` and `vlan` which target the Calico `Network` to use. +`calico` is a property of a map item's `destination`, alongside `type`, and is valid only on an item of `type: pod`. +At most one item in a `NetworkMap` may carry it. + ```yaml + map: + - source: + id: + destination: + type: pod calico: network: mock-l2-net # cluster-scoped projectcalico.org/v3 Network CR vlan: 100 # must match a vlan.id entry in that Network + - source: + id: + destination: + type: multus + name: my-network-vlan10 + namespace: vms ``` | Field | Effect | @@ -137,28 +153,30 @@ fields `network` and `vlan` which target the Calico `Network` to use. Note: The MAC is always carried over; the addresses follow when the `Plan` sets `preserveStaticIPs`. Definitions that use another CNI are left untouched. The values are set before the VM starts, so each interface is created already holding them. -```yaml - map: - - source: - id: - destination: - type: multus - name: my-network-vlan10 - namespace: vms -``` - - ### Verify address preservation worked Inspect the `virt-launcher` pod, or the VM template created post-migration. -It should contain annotations for Calico address preservation, for each NIC -in the NetworkMap: +Each NIC in the `NetworkMap` carries a pair of annotations, and which keys to look for depends on the NIC. + +The primary NIC, mapped by an item of `type: pod` with a `calico` block, uses keys with no interface name: ```yaml -cni.projectcalico.org/.hwAddr: ... -cni.projectcalico.org/.ipAddrs: ... +cni.projectcalico.org/hwAddr: '52:54:00:3a:1c:04' +cni.projectcalico.org/ipAddrs: '["10.10.0.42"]' ``` +A secondary NIC, mapped by an item of `type: multus`, scopes each key to the interface it applies to: + +```yaml +cni.projectcalico.org/.hwAddr: '52:54:00:3a:1c:05' +cni.projectcalico.org/.ipAddrs: '["10.10.10.42"]' +``` + +For the name to use in place of ``, see [Which interface name to use](../l2-bridge/vm-identity.mdx#which-interface-name-to-use). + +The MAC is always stamped; `ipAddrs` appears only when the `Plan` sets `preserveStaticIPs`. +$[prodname] adds `cni.projectcalico.org/podIP` and `podIPs` once the interface is up, which is what confirms the requested address was assigned. + See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx) for the `Network`, IP pool and attachment definition, and [Set a VM's IP and MAC address](../l2-bridge/vm-identity.mdx) for the annotations themselves. diff --git a/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-kubernetes.mdx b/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-kubernetes.mdx index 1ef8a0b119..7fa50bece2 100644 --- a/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-kubernetes.mdx +++ b/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-kubernetes.mdx @@ -43,6 +43,9 @@ Upstream Forklift's guest conversion fails on clusters whose nodes carry the run ``` - CDI and a default StorageClass, for migrations that import disks. Neither is needed to install Forklift. + The StorageClass backing the migrated disks must support the file locking `qemu-img` takes on an image it opens. + A volume that cannot honor those locks fails the migration before conversion starts. + NFS-backed storage needs mount options that permit locking, such as `nfsvers=3,nolock` or NFSv4, set on the StorageClass rather than only on the export. ## How to @@ -96,8 +99,7 @@ OpenShift hits neither: its nodes permit `unshare` and confine containers with S Put the profiles on every node that can run a conversion. Use the Security Profiles Operator if you already run it; otherwise you may apply the ConfigMap and DaemonSet patch below, after inspection. -**Note** This is a privileged DaemonSet. It writes both profiles under the kubelet's seccomp directory, and loads the AppArmor one on nodes that have `apparmor_parser`. Its log says which it did on each node: - +**Note** This is a privileged DaemonSet. It writes both profiles under the kubelet's seccomp directory, and loads the AppArmor one on nodes that have `apparmor_parser`. ```bash kubectl apply -f $[filesUrl]/forklift/forklift-virt-v2v-profiles.yaml @@ -105,11 +107,23 @@ kubectl apply -f $[filesUrl]/forklift/forklift-virt-v2v-profiles.yaml kubectl -n konveyor-forklift rollout status ds/forklift-virt-v2v-profile-installer ``` +Each pod reports what it did on its own node, and a single `kubectl logs` across the DaemonSet loses that attribution, so read the logs one pod at a time: + ```bash -kubectl -n konveyor-forklift logs -l service=forklift-virt-v2v-profile-installer -c install +for pod in $(kubectl -n konveyor-forklift get pods \ + -l service=forklift-virt-v2v-profile-installer -o name); do + echo "== $(kubectl -n konveyor-forklift get $pod -o jsonpath='{.spec.nodeName}')" + kubectl -n konveyor-forklift logs $pod -c install +done ``` -Then name the profiles on the `ForkliftController`: +A node that enforces AppArmor reports `loaded AppArmor profile`. +One that does not reports `no apparmor_parser on this node, skipping AppArmor`. +What you see decides which fields to set next. + +Then name the profiles on the `ForkliftController`. +The patch below sets all four fields, which is what a cluster of AppArmor-enforcing nodes needs. +Drop the two `apparmor` lines if no node enforces it: ```bash kubectl -n konveyor-forklift patch forkliftcontroller forklift-controller \ @@ -120,7 +134,22 @@ kubectl -n konveyor-forklift patch forkliftcontroller forklift-controller \ "virt_v2v_apparmor_profile_path": "forklift-virt-v2v-unshare"}}' ``` -All four fields are unset by default, which leaves every cluster on its runtime default: naming a profile that a node does not carry makes the kubelet refuse the pod outright. A path given without its type is rejected as the CR is saved. On nodes that do not enforce AppArmor, set only the seccomp pair. +All four fields are unset by default, which leaves every cluster on its runtime default. +A path given without its type is rejected as the CR is saved. + +The fields belong to the one `ForkliftController`, so they apply to every conversion pod in the cluster. +There is no per-node and no per-`Plan` override, which makes the AppArmor pair a property of the cluster rather than of a node: + +| Nodes that can run a conversion | Set | +| --- | --- | +| All enforce AppArmor | All four fields | +| None enforce AppArmor | The seccomp pair only | + +A cluster whose conversion-capable nodes are mixed has no valid setting. +Name the AppArmor profile and the kubelet refuses the conversion pod outright wherever AppArmor is absent, reporting `Cannot enforce AppArmor: AppArmor is not enabled on the host`. +Omit it and the pod is admitted on an AppArmor node, where the remount then fails. + +If your nodes are mixed, confine conversions to a uniform set of them with `spec.convertorNodeSelector` on the `Plan`, and set these fields to match the set you chose. ### 5. Map a VM's networks to $[prodname] @@ -137,10 +166,24 @@ can be included to flag that the primary network should also receive address-pre To additionally force the primary NIC to get attached to a pre-configured Calico `Network`, the struct has fields `network` and `vlan` which target the Calico `Network` to use. +`calico` is a property of a map item's `destination`, alongside `type`, and is valid only on an item of `type: pod`. +At most one item in a `NetworkMap` may carry it. + ```yaml + map: + - source: + id: + destination: + type: pod calico: network: mock-l2-net # cluster-scoped projectcalico.org/v3 Network CR vlan: 100 # must match a vlan.id entry in that Network + - source: + id: + destination: + type: multus + name: my-network-vlan10 + namespace: vms ``` | Field | Effect | @@ -152,28 +195,30 @@ fields `network` and `vlan` which target the Calico `Network` to use. Note: The MAC is always carried over; the addresses follow when the `Plan` sets `preserveStaticIPs`. Definitions that use another CNI are left untouched. The values are set before the VM starts, so each interface is created already holding them. -```yaml - map: - - source: - id: - destination: - type: multus - name: my-network-vlan10 - namespace: vms -``` - - ### Verify address preservation worked Inspect the `virt-launcher` pod, or the VM template created post-migration. -It should contain annotations for Calico address preservation, for each NIC -in the NetworkMap: +Each NIC in the `NetworkMap` carries a pair of annotations, and which keys to look for depends on the NIC. + +The primary NIC, mapped by an item of `type: pod` with a `calico` block, uses keys with no interface name: ```yaml -cni.projectcalico.org/.hwAddr: ... -cni.projectcalico.org/.ipAddrs: ... +cni.projectcalico.org/hwAddr: '52:54:00:3a:1c:04' +cni.projectcalico.org/ipAddrs: '["10.10.0.42"]' ``` +A secondary NIC, mapped by an item of `type: multus`, scopes each key to the interface it applies to: + +```yaml +cni.projectcalico.org/.hwAddr: '52:54:00:3a:1c:05' +cni.projectcalico.org/.ipAddrs: '["10.10.10.42"]' +``` + +For the name to use in place of ``, see [Which interface name to use](../l2-bridge/vm-identity.mdx#which-interface-name-to-use). + +The MAC is always stamped; `ipAddrs` appears only when the `Plan` sets `preserveStaticIPs`. +$[prodname] adds `cni.projectcalico.org/podIP` and `podIPs` once the interface is up, which is what confirms the requested address was assigned. + See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx) for the `Network`, IP pool and attachment definition, and [Set a VM's IP and MAC address](../l2-bridge/vm-identity.mdx) for the annotations themselves. diff --git a/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-openshift.mdx b/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-openshift.mdx index 8e259efa8c..ba059aaa25 100644 --- a/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-openshift.mdx +++ b/calico-enterprise_versioned_docs/version-3.24-2/networking/kubevirt/install-forklift-openshift.mdx @@ -27,6 +27,8 @@ Install the Tigera fork of Forklift on an OpenShift cluster and connect it to a - The BPF data plane enabled: `bpfEnabled: true` in the default `FelixConfiguration`. See [Enable eBPF on an existing cluster](../../operations/ebpf/enabling-ebpf.mdx). - OpenShift Virtualization (KubeVirt and CDI) installed, with `HyperConverged` Available. - A StorageClass for the migrated VM disks. + It must support the file locking `qemu-img` takes on an image it opens: a volume that cannot honor those locks fails the migration before conversion starts. + NFS-backed storage needs mount options that permit locking, such as `nfsvers=3,nolock` or NFSv4, set on the StorageClass rather than only on the export. - IP pools covering the VM addresses. The pod network uses an ordinary pool; each L2 VLAN needs its own pool with `allowedUses: [L2Workload]` and `disableBGPExport: true`. A VM keeps its address only if that address falls inside a pool. See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx). ## How to @@ -122,10 +124,24 @@ can be included to flag that the primary network should also receive address-pre To additionally force the primary NIC to get attached to a pre-configured Calico `Network`, the struct has fields `network` and `vlan` which target the Calico `Network` to use. +`calico` is a property of a map item's `destination`, alongside `type`, and is valid only on an item of `type: pod`. +At most one item in a `NetworkMap` may carry it. + ```yaml + map: + - source: + id: + destination: + type: pod calico: network: mock-l2-net # cluster-scoped projectcalico.org/v3 Network CR vlan: 100 # must match a vlan.id entry in that Network + - source: + id: + destination: + type: multus + name: my-network-vlan10 + namespace: vms ``` | Field | Effect | @@ -137,28 +153,30 @@ fields `network` and `vlan` which target the Calico `Network` to use. Note: The MAC is always carried over; the addresses follow when the `Plan` sets `preserveStaticIPs`. Definitions that use another CNI are left untouched. The values are set before the VM starts, so each interface is created already holding them. -```yaml - map: - - source: - id: - destination: - type: multus - name: my-network-vlan10 - namespace: vms -``` - - ### Verify address preservation worked Inspect the `virt-launcher` pod, or the VM template created post-migration. -It should contain annotations for Calico address preservation, for each NIC -in the NetworkMap: +Each NIC in the `NetworkMap` carries a pair of annotations, and which keys to look for depends on the NIC. + +The primary NIC, mapped by an item of `type: pod` with a `calico` block, uses keys with no interface name: ```yaml -cni.projectcalico.org/.hwAddr: ... -cni.projectcalico.org/.ipAddrs: ... +cni.projectcalico.org/hwAddr: '52:54:00:3a:1c:04' +cni.projectcalico.org/ipAddrs: '["10.10.0.42"]' ``` +A secondary NIC, mapped by an item of `type: multus`, scopes each key to the interface it applies to: + +```yaml +cni.projectcalico.org/.hwAddr: '52:54:00:3a:1c:05' +cni.projectcalico.org/.ipAddrs: '["10.10.10.42"]' +``` + +For the name to use in place of ``, see [Which interface name to use](../l2-bridge/vm-identity.mdx#which-interface-name-to-use). + +The MAC is always stamped; `ipAddrs` appears only when the `Plan` sets `preserveStaticIPs`. +$[prodname] adds `cni.projectcalico.org/podIP` and `podIPs` once the interface is up, which is what confirms the requested address was assigned. + See [Connect workloads to an existing VLAN](../l2-bridge/connect-vlan.mdx) for the `Network`, IP pool and attachment definition, and [Set a VM's IP and MAC address](../l2-bridge/vm-identity.mdx) for the annotations themselves.