MacOS 11.5
MicroK8s 1.20.9
MinIO
Velero 1.6.3
Info:
- MinIO is a High Performance Object Storage released under GNU. It is API compatible with Amazon S3 cloud storage service. It can handle unstructured data such as photos, videos, log files, backups, and container images with (currently) the maximum supported object size of 5TB.
- Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.
- MicroK8s is the simplest production-grade upstream Kubernetes. Lightweight and focused. Single command install on Linux, Windows and macOS. Made for devops, great for edge, appliances and IoT. Full high availability Kubernetes with autonomous clusters.
Note:
- All of the following instructions assume that you have already installed Kubernetes and can control your cluster with the kubectl command. You can follow the instructions in this link to install MicroK8s on MacOS or Ubuntu.
Install:
- MinIO will store the kubernetes backup files.
- Run standalone MinIO server on Docker:
- Create a local folder to store the minio data:
- mkdir /Users/marcus/minio/data
- docker container create -p 9000:9000 -p 9001:9001 --name minio -v /Users/marcus/minio/data:/data -e "MINIO_ROOT_USER=myminioaccesskey" -e "MINIO_ROOT_PASSWORD=myminiosecretkey" minio/minio:latest server /data --console-address ":9001"
- OR, using the minio default user / password (minioadmin / minioadmin):
- docker container create -p 9000:9000 -p 9001:9001 --name minio -v /Users/marcus/minio/data:/data minio/minio:RELEASE.2021-09-09T21-37-07Z server /data --console-address ":9001"
- docker container start minio
- Velero consists of:
- A server that runs on the kubernetes cluster
- A command-line client that runs locally
- Install Velero server:
- ssh into the machine where the kubernetes cluster is running.
- cd ~/Downloads
- wget https://github.com/vmware-tanzu/velero/releases/download/v1.6.3/velero-v1.6.3-linux-amd64.tar.gz
- tar -xvzf velero-v1.6.3-linux-amd64.tar.gz
- cd velero-v1.6.3-linux-amd64/
- sudo mv velero /usr/local/bin/
- export KUBECONFIG=/var/snap/microk8s/current/credentials/client.config
- nano cred-velero
- [default]
- export BUCKET=velero
- export REGION=minio
- aws_access_key_id=myminioaccesskey
- aws_secret_access_key=myminiosecretkey
- velero install --default-volumes-to-restic --use-restic --provider aws --bucket velero --plugins velero/velero-plugin-for-aws:v1.0.0 --secret-file ./cred-velero --snapshot-location-config region=minio --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://<host-ip>:9000
- Attention: Do NOT use 127.0.0.1 or localhost in the s3Url parameter, because the velero and minio are not running on the same host.
- Wait until the velero pod is running:
- kubectl get ns
- velero version
- Patch the `hostPath` to be compatible with microk8s (issue 4035):
- kubectl -n velero patch daemonset restic -p '{"spec":{"template":{"spec":{"volumes":[{"name":"host-pods","hostPath":{"path":"/var/snap/microk8s/common/var/lib/kubelet/pods"}}]}}}}'
Testing:
- MinIO console:
- http://127.0.0.1:9001
- user: myminioaccesskey
- password: myminiosecretkey
- Create a new bucket named `velero` to store backup files:
- Buckets -> Create Bucket
- Velero Backup:
- velero backup create bkp-test --include-namespaces=test
- velero get backup
- velero backup describe bkp-test [--details]
- velero backup logs bkp-test
- Simulate an error deleting the entire namespace:
- kubectl delete ns test
- kubectl get ns
- Velero Restore:
- velero restore create --from-backup bkp-test
- velero get restore
- kubectl get ns
- kubectl -n test get pod
Troubleshooting:
- Check the Velero deployment log:
- kubectl logs deployment/velero -n velero
- Check the contents of the Velero secret:
- kubectl get secrets -n velero cloud-credentials -o jsonpath="{.data.cloud}" | base64 -d
Uninstalling Velero:
- kubectl delete namespace/velero clusterrolebinding/velero
- kubectl delete crds -l component=velero
Uninstalling Minio:
- docker stop minio
- docker rm minio
References:
- MinIO - Multi-Cloud Object Storage
- MinIO Docker Quickstart Guide
- Python MinIO Client API Reference
- Velero Basic Install
- Velero Git Repository
- Restic Integration - Default Pod Volume backup to restic
- Kubernetes Backup & Restore of Namespaces with Persistent Volumes
- Backup and Restore of Kubernetes Namespaces
- Backup and Restore of Kubernetes Applications using Heptio’s Velero with Restic and Rook-Ceph as the storage provider
