文章大纲
COntainerd Data Importer(CDI) 是能够将多种来源的数据导入到 Kubernetes 中的 PVC,随后用做 VM 的磁盘。
所以 CDI 会创建相关的 PVC,并且使用 PV,建议在 Kubernetes 中部署 Rook-Ceph,提供持久的块存储,并且后续还需通过 CSI 实现对 VM 的快照和克隆。
Rook-Ceph 的部署和使用可参考:
安装 CDI
使用以下命令安装最新版本的 CDI 通过 Operator:
vagrant@master01:~$ export VERSION=$(basename $(curl -s -w %{redirect_url} https://github.com/kubevirt/containerized-data-importer/releases/latest))
vagrant@master01:~$ echo $VERSION
v1.59.0
vagrant@master01:~$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
namespace/cdi created
customresourcedefinition.apiextensions.k8s.io/cdis.cdi.kubevirt.io created
clusterrole.rbac.authorization.k8s.io/cdi-operator-cluster created
clusterrolebinding.rbac.authorization.k8s.io/cdi-operator created
serviceaccount/cdi-operator created
role.rbac.authorization.k8s.io/cdi-operator created
rolebinding.rbac.authorization.k8s.io/cdi-operator created
deployment.apps/cdi-operator created
vagrant@master01:~$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml
cdi.cdi.kubevirt.io/cdi created
等待 Operator 部署完成,然后检查状态:
vagrant@master01:~$ kubectl get cdi cdi -n cdi
NAME AGE PHASE
cdi 52s Deployed
vagrant@master01:~$ kubectl get pod -n cdi
NAME READY STATUS RESTARTS AGE
cdi-apiserver-75bb86c8c4-84m5j 1/1 Running 0 51s
cdi-deployment-645dc8bc99-vj4fc 1/1 Running 0 51s
cdi-operator-67d655447d-zhpb7 1/1 Running 0 67s
cdi-uploadproxy-67f586465b-fjntg 1/1 Running 0 51s
部署完成后,会引进 DataVolumes
,通过创建该资源,可以抽象的创建 PVC。
使用 CDI 导入磁盘映像
通过声明 DadaVolume 资源定义,来创建 PVC,并将一个 Fedora 的映像导入到该 PVC 中:
cat <<EOF > dv_fedora.yml
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: "fedora"
spec:
storage:
resources:
requests:
storage: 5Gi
source:
http:
url: "https://download.fedoraproject.org/pub/fedora/linux/releases/37/Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.raw.xz"
EOF
kubectl -n cdi-demo create -f dv_fedora.yml
这里的 source
用的是 http
,除此之外还可以使用 pvc
、registry
、s3
等,具体可以通过 kubectl explain DataVolume.spec.source
来进行查看。
然后,它就报错了。
vagrant@master01:~$ kubectl get pod -n cdi-demo
NAME READY STATUS RESTARTS AGE
importer-prime-4d87fc83-0832-42df-bd2c-f970098b75ac 0/1 CrashLoopBackOff 4 (45s ago) 2m35s
vagrant@master01:~$ kubectl logs importer-prime-4d87fc83-0832-42df-bd2c-f970098b75ac -n cdi-demo
I0428 04:53:21.137080 1 importer.go:103] Starting importer
E0428 04:53:21.138511 1 importer.go:133] exit status 1, blockdev: cannot open /dev/cdi-block-volume: Permission denied
kubevirt.io/containerized-data-importer/pkg/util.GetAvailableSpaceBlock
pkg/util/util.go:130
kubevirt.io/containerized-data-importer/pkg/util.GetAvailableSpaceByVolumeMode
pkg/util/util.go:100
main.main
cmd/cdi-importer/importer.go:131
runtime.main
GOROOT/src/runtime/proc.go:267
查看 PVC 和 DataVolume 的状态:
vagrant@master01:~$ kubectl get pvc -n cdi-test
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
fedora Pending rook-ceph-block <unset> 3m27s
prime-ac1aca3c-15cc-4c8b-ab77-3ef2cb044846 Bound pvc-e995b1ec-c7eb-41ed-9fae-7f6ce7a5ebdd 5Gi RWX rook-ceph-block <unset>
vagrant@master01:~$ kubectl get datavolume -n cdi-test
NAME PHASE PROGRESS RESTARTS AGE
fedora ImportInProgress N/A 7 16m
经过查找资料,解决方法参考 Non-root Containers And Devices
如果使用的 Containerd ,修改配置中的:
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
device_ownership_from_security_context = true
如果使用的是 CRI-O,修改配置中的:
[crio.runtime]
device_ownership_from_security_context = true
根据使用的 Container Runtime 修改不同的配置文件,然后重启,就正常了。
查看状态:
vagrant@master01:~$ kubectl get datavolume -n cdi-test
NAME PHASE PROGRESS RESTARTS AGE
fedora ImportInProgress 86.27% 11 35m
等待导入成功后,查看资源状态:
vagrant@master01:~$ kubectl get DataVolume,pvc,pod -n cdi-test
NAME PHASE PROGRESS RESTARTS AGE
datavolume.cdi.kubevirt.io/fedora Succeeded 100.0% 11 46m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
persistentvolumeclaim/fedora Bound pvc-e995b1ec-c7eb-41ed-9fae-7f6ce7a5ebdd 5Gi RWX rook-ceph-block <unset> 46m
执行导入的 Pod 已经被移除了。
使用导入的 PVC 创建 VM
使用以下的 VM 定义创建 fedora-vm,注意将 ssh 公钥换成自己的:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
labels:
kubevirt.io/os: linux
name: fedora-vm
spec:
running: true
template:
metadata:
labels:
kubevirt.io/domain: fedora-vm1
spec:
domain:
cpu:
cores: 1
devices:
disks:
- disk:
bus: virtio
name: disk0
- cdrom:
bus: sata
readonly: true
name: cloudinitdisk
machine:
type: q35
resources:
requests:
memory: 512M
volumes:
- name: disk0
persistentVolumeClaim:
claimName: fedora
- cloudInitNoCloud:
userData: |
#cloud-config
hostname: fedora-vm1
ssh_pwauth: True
disable_root: false
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxl86q6k6zEKHLmnNtMKZcgCTweRAm7ChsXTi0s7+40L4rCrRqB6Sq/D9e3tXc7vV3Mcqf0G6oNhh2Y0ZlYHu6ADCfj8Dw8fQscsflypDj0B9EXIb3j2Eetp0jI73H/tOe8X7dQGgmTG6bjJ2NmnQL/uAJpiGnGzAe06E4CrK5Q9plOD00E3gKbMhJbxZBOKY6rHfEPhVEhFC8jXjCwTshtRSpF8EA947oPzeAqun9WO5ffwewvBo1/FfyU9OOvDYXj0qvAYM0pUVlHtSts3vsvO3HaxkFSpyQxxT9StnIsD9dtY9UanaRQu0b3W7j+7xv+TqHHVTvAi8D25a/or3w3bpXwor488Qok+IhxE36LJiARoJ++4v7G5VYZSCkuPsmOPNUoMXJgksy9TAtQkU1PL7dF0v3s/4+BJ6BQ9s2BQtLhRXatIBiXU/Hxfhp7OewWt3vmlsy0qKKN6oEo6VUxMYjPTmlCaPX9YXdKwgAQv9/+Fmgadvpxha+R4qB4fk= vagrant@master01
name: cloudinitdisk
创建 vm:
vagrant@master01:~$ kubectl -n cdi-test apply -f fedora-vm1_pvc.yml
virtualmachine.kubevirt.io/fedora-vm created
查看状态:
vagrant@master01:~$ kubectl get vmi -n cdi-test
NAME AGE PHASE IP NODENAME READY
fedora-vm 28s Running 10.244.241.115 master01 True
连接到虚拟机:
vagrant@master01:~$ virtctl ssh -i .ssh/id_rsa fedora@fedora-vm.cdi-test
The authenticity of host 'vmi/fedora-vm.cdi-test:22 (192.168.121.1:7890)' can't be established.
ECDSA key fingerprint is SHA256:qmiiDH3244ZW1cfwYav30En+1CoLOR/dSY3ixx5Ul0Y.
Are you sure you want to continue connecting (yes/no)? yes
[fedora@fedora-vm ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1M 0 rom
zram0 251:0 0 435M 0 disk [SWAP]
vda 252:0 0 5G 0 disk
├─vda1 252:1 0 1M 0 part
├─vda2 252:2 0 1000M 0 part /boot
├─vda3 252:3 0 100M 0 part /boot/efi
├─vda4 252:4 0 4M 0 part
└─vda5 252:5 0 3.9G 0 part /home
/
VM 可以正常使用 CDI 导入的 PVC 用作根。