KubeVirt: Running Virtual Machines on Kubernetes

Legacy workloads do not disappear just because the platform team standardised on Kubernetes. KubeVirt takes the pragmatic position that if you can express a VM as a Kubernetes resource, you get one control plane, one set of RBAC rules, one GitOps pipeline and one place to look for logs — for containers and virtual machines alike. VMs run as pods, scheduled by the same scheduler, with the same network and storage primitives. This guide covers installation, a working VirtualMachine manifest, console access, and the operational realities of live migration and storage.

Installation and requirements

export RELEASE=$(curl -s https://storage.googleapis.com/kubevirt-prow/release/kubevirt/kubevirt/stable.txt)

kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-operator.yaml
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-cr.yaml
kubectl -n kubevirt wait kv kubevirt --for condition=Available

kubectl -n kubevirt get pods
kubectl get crd | grep kubevirt

Requirements worth checking first: the apiserver must allow privileged pods because KubeVirt runs a privileged DaemonSet for the virt-handler, and the nodes need hardware virtualisation available. Without it, KubeVirt can run in software emulation by setting spec.configuration.developerConfiguration.useEmulation: true on the KubeVirt CR — useful on a laptop, unusably slow for anything real. A default install lands on the order of seven pods, three services and a DaemonSet in the kubevirt namespace; verify all pods are running before creating a VM, since a half-installed KubeVirt produces confusing failures at VM start time.

Define a virtual machine

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: legacy-app-01
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/domain: legacy-app-01
    spec:
      domain:
        cpu:
          cores: 2
        resources:
          requests:
            memory: 4Gi
        devices:
          disks:
            - name: rootdisk
              disk: {bus: virtio}
            - name: cloudinitdisk
              disk: {bus: virtio}
          interfaces:
            - name: default
              masquerade: {}
      networks:
        - name: default
          pod: {}
      volumes:
        - name: rootdisk
          containerDisk:
            image: quay.io/containerdisks/centos-stream:9
        - name: cloudinitdisk
          cloudInitNoCloud:
            userData: |
              #cloud-config
              user: ops
              ssh_authorized_keys:
                - ssh-ed25519 AAAA... ops@example.com
              packages: [qemu-guest-agent]
              runcmd:
                - [systemctl, enable, --now, qemu-guest-agent]

kubectl apply -f legacy-app-01.yaml
kubectl get vm,vmi
kubectl describe vmi legacy-app-01 | tail -30

The pattern to internalise: a VirtualMachine is a controller object (it restarts its VMI and holds desired state), while a VirtualMachineInstance is the running pod. Use a DataVolume or PVC instead of a containerDisk for anything with persistent state — containerDisk is a backing-store image, not a disk, and writing to it loses data when the pod is replaced.

Console access and exposure

# virtctl comes with the KubeVirt release and talks to the API
virtctl console legacy-app-01
virtctl vnc legacy-app-01
virtctl start legacy-app-01
virtctl stop legacy-app-01
virtctl restart legacy-app-01

# expose a service to reach the VM from inside the cluster
kubectl expose vmi legacy-app-01 --name legacy-app-svc --port 8080 --target-port 8080

With the default masquerade binding the VM shares the pod network and NATs outbound traffic, which is the simplest option and the right default for most migrations. Multus-based bridge or SR-IOV bindings give the VM a real interface on a VLAN — needed when the workload must be in the same broadcast domain as physical hosts, and where throughput matters.

Live migration and day-two operations

virtctl migrate legacy-app-01
kubectl get vmi -w

# make a VM migratable: storage must be shared (RWX PVC or Ceph/Longhorn RBD), not local-path
kubectl get pvc legacy-app-data -o jsonpath='{.spec.accessModes}'

Live migration is the feature that justifies KubeVirt in a Kubernetes platform: nodes can be drained for maintenance without a maintenance window for the guests. The prerequisite is storage: a VM on a node-local path can never migrate, so plan for a shared storage class from the start rather than retrofitting it. Enable the LiveMigratable condition check in your automation — kubectl get vmi legacy-app-01 -o jsonpath='{.status.conditions[?(@.type=="LiveMigratable")].status}' — before you start a drain, otherwise the drain blocks and the maintenance window is spent debugging.

Storage: the decision that determines everything else

KubeVirt gives VMs the same storage primitives as pods, which means the storage class you choose determines their operational properties. A VM on a node-local path cannot be live-migrated; a VM on a shared, read-write-many class can. This is the choice to get right on day one, because changing it later means moving the guest's data.

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: legacy-app-01-root
spec:
  source:
    registry:
      url: "docker://quay.io/containerdisks/centos-stream:9"
  pvc:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 40Gi
    storageClassName: ceph-rbd
---
# reference the DataVolume from the VM instead of a containerDisk
      volumes:
        - name: rootdisk
          dataVolume:
            name: legacy-app-01-root

CDI (the Containerized Data Importer) is what makes this work: it imports a cloud image into a PVC and can also clone volumes, which is the mechanism for golden templates. Once you have a prepared DataVolume, cloning it to create new VMs is faster and more consistent than importing the image again for each guest.

Networking: masquerade, bridge and SR-IOV

The interface binding determines what the guest sees and what your policies can do to it. With masquerade, the VM shares the pod network, gets NAT for outbound traffic, and is reachable through a Service — the simplest model and the right default for migrations. With a bridge binding on a Multus secondary network, the guest has a real interface on a VLAN and participates in that broadcast domain like a physical host. With SR-IOV, the guest gets a virtual function for near-line-rate performance and loses most of the platform's ability to inspect its traffic.

# multus attachment for a VLAN-backed VM interface
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: vlan-100
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "bridge",
    "bridge": "br100",
    "vlan": 100,
    "ipam": {}
  }'
---
      interfaces:
        - name: default
          masquerade: {}
        - name: vlan100
          bridge: {}
      networks:
        - name: default
          pod: {}
        - name: vlan100
          multus:
            networkName: vlan-100

Choose masquerade unless a specific requirement forces otherwise. Every step away from the pod network (bridge, SR-IOV) buys performance or broadcast-domain integration and costs you network policy, observability and ease of migration — and those costs are paid by whoever is on call at 03:00, not by whoever made the design decision.

Day-two operations: snapshots, exports and capacity

# snapshots are storage-class dependent; verify support before promising them
kubectl get volumesnapshotclass
virtctl snapshot create legacy-app-01-snap --vm legacy-app-01

# console, restart and migration
virtctl console legacy-app-01
virtctl restart legacy-app-01
virtctl migrate legacy-app-01

# capacity questions: VMIs are pods, so nodes and quotas apply
kubectl get vmi -o wide
kubectl describe node | grep -A5 "Allocated resources"
kubectl get resourcequota -A

Three operational realities. First, a VMI is a pod, so it competes with other pods for CPU and memory on the node; if the VMs are the critical workload, give them Guaranteed QoS by setting requests equal to limits, and use taints or a dedicated node pool to keep bursting workloads away. Second, live migration needs headroom on the destination node — a cluster running at 85% memory cannot evacuate a node, and the drain that was supposed to be routine blocks. Third, snapshots of a running VM depend on the storage provider's support; test a restore, because a snapshot you have never restored is a hypothesis.

FAQ

Is KubeVirt production ready? It is widely deployed in production for legacy VM workloads alongside containers. The honest framing is that you are adopting a virtualisation platform operated through Kubernetes APIs, so your operational maturity with Kubernetes is the limiting factor.

Can I run Windows guests? Yes, with a suitable image, virtio drivers and a machine type that matches the guest's expectations. Licensing is the harder part; plan it with whoever owns the Microsoft agreement.

How do VMs and pods share a node safely? Set explicit requests and limits, use dedicated node pools for latency-sensitive guests, and monitor node-level overcommit. The scheduler is happy to place a VM where it fits numerically, not where it fits operationally.

What about licence and support implications of moving a VM into Kubernetes? The hypervisor changed; the guest OS licence position usually has not, but confirm with your vendor. Where an application vendor only supports VMware or KVM directly, KubeVirt's KVM backend is normally acceptable — get it in writing before the migration, not after.

Quick reference: the objects and what they do

Read a VM as three layers that can each be wrong independently: the controller (is running: true and does the VMI exist), the pod (is it scheduled, is the image or DataVolume ready), and the guest OS (did it boot, is the agent running). Diagnosing failures in that order saves a great deal of time, because a guest that failed to boot and a pod that never scheduled produce equally unhelpful console output.

Where KubeVirt fits and where it does not

The workloads that justify it are legacy applications with no container path, appliances distributed as virtual disks, and Windows guests that cannot be containerised. The workloads that do not are stateful services that already run well as pods — putting them in a VM adds a nesting layer, a guest OS to patch and a slower path to the same outcome. A cluster hosting both is normal and healthy; a cluster where VMs are the default because "it is a virtual machine platform now" is the pattern that produces operational debt.

Decide the ratio deliberately and document it: which workloads are expected to remain VMs, which are scheduled to be containerised, and what the exit path looks like for each. VMs on Kubernetes make sense as a transitional and coexistence platform; the migration plan is what keeps them from becoming the permanent reason your cluster is complicated.

Related Reading

Related reading: KVM and libvirt: VM Management with virsh, K3s Lightweight Kubernetes: Single-Node and HA Install, Helm Charts: Packaging Kubernetes Applications

原文链接：https://kubevirt.io/user-guide/cluster_admin/installation/