kubeadm에 대해 알아보고 설치해보자.
1. kubeadm이란?
kubeadm은 Kubernetes Cluster 생성을 위한 kubeadm init과 kubeadm join을 위한 툴이다.
- kubeadm init : Control-plane 노드의 부트스트랩
- kubeadm join : Worker 노드를 부트스트랩하고 Cluster에 조인
- kubeadm upgrade : Kubernetes Cluster를 신규 버전으로 업그레이드
- kubeadm config : kubeadm 1.7 이하 버전 사용 시 Cluster를 초기화할 때 kubeadm upgrade를 위한 Cluster 설정
- kubeadm token : kubeadm join을 위한 토큰 관리
- kubeadm reset : kubeadm 혹은 kubeadm join에 의해 호스트에 발생한 변경 사항을 원복
- kubeadm version : 버전 출력
- kubeadm alpha : 커뮤니티로부터 피드백을 수집하기 위해 준비된 기능들을 미리 확인
2. 환경
우분투 리눅스 기준
3. Docker 설치
도커를 설치한다.
# apt-get install docker.io Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package docker.io E: Couldn't find any package by glob 'docker.io' E: Couldn't find any package by regex 'docker.io'
만약 위와 같이 에러가 나면 apt update를 한 후에 다시 한다.
# apt-get update Hit:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease Get:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB] Get:3 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB] Get:4 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/main Sources [868 kB] Get:5 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted Sources [4,808 B] Get:6 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Sources [7,728 kB] Get:7 http://security.ubuntu.com/ubuntu xenial-security InRelease [107 kB] Get:8 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Sources [179 kB] Get:9 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,532 kB] Get:10 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,354 kB] Get:11 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [144 kB] Get:12 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [106 kB] Get:13 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Sources [318 kB] Get:14 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2,528 B] Get:15 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [217 kB] Get:16 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [8,408 B] Get:17 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [824 kB] Get:18 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [339 kB] Get:19 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [7,556 B] Get:20 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [677 kB] Get:21 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [273 kB] Get:22 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.4 kB] Get:23 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,344 B] Get:24 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Sources [4,488 B] Get:25 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Sources [6,736 B] Get:26 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [6,756 B] Get:27 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,180 B] Get:28 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [7,420 B] Get:29 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [3,996 B] Get:30 http://security.ubuntu.com/ubuntu xenial-security/main Sources [131 kB] Get:31 http://security.ubuntu.com/ubuntu xenial-security/restricted Sources [2,116 B] Get:32 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [69.5 kB] Get:33 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [2,088 B] Get:34 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [533 kB] Get:35 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [228 kB] Get:36 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [363 kB] Get:37 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [136 kB] Get:38 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,456 B] Get:39 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [1,744 B] Fetched 25.4 MB in 4s (5,356 kB/s) Reading package lists... Done
다시 도커를 설치한다.
# apt-get install docker.io Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: bridge-utils cgroupfs-mount ubuntu-fan Suggested packages: mountall aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils The following NEW packages will be installed: bridge-utils cgroupfs-mount docker.io ubuntu-fan 0 upgraded, 4 newly installed, 0 to remove and 46 not upgraded. Need to get 17.1 MB of archives. After this operation, 90.5 MB of additional disk space will be used. Do you want to continue? [Y/n]
잘 설치되었다.
Done. Setting up ubuntu-fan (0.12.8~16.04.2) ... Processing triggers for systemd (229-4ubuntu21.2) ... Processing triggers for ureadahead (0.100.0-19) ...
4. kubeadm 설치 사전 작업
우선 apt-transport-https 를 설치한다.
# apt-get install apt-transport-https Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be upgraded: apt-transport-https 1 upgraded, 0 newly installed, 0 to remove and 45 not upgraded. Need to get 26.1 kB of archives. After this operation, 0 B of additional disk space will be used. Get:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.27 [26.1 kB] Fetched 26.1 kB in 0s (1,617 kB/s) (Reading database ... 51422 files and directories currently installed.) Preparing to unpack .../apt-transport-https_1.2.27_amd64.deb ... Unpacking apt-transport-https (1.2.27) over (1.2.26) ... Setting up apt-transport-https (1.2.27) ...
아래 과정을 계속 진행한다. 당연하지만 curl 명령어가 있어야 함.
# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - OK
계속 수행..
# cat <<EOF >/etc/apt/sources.list.d/kubernetes.list > deb http://apt.kubernetes.io/ kubernetes-xenial main > EOF
파일을 확인한다.
# ls -l /etc/apt/sources.list.d/kubernetes.list -rw-r--r-- 1 root root 53 Aug 7 00:15 /etc/apt/sources.list.d/kubernetes.list
5. kubeadm 설치
# apt-get install -y kubelet kubeadm kubectl Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: cri-tools ebtables kubernetes-cni socat The following NEW packages will be installed: cri-tools ebtables kubeadm kubectl kubelet kubernetes-cni socat 0 upgraded, 7 newly installed, 0 to remove and 45 not upgraded. Need to get 53.7 MB of archives. After this operation, 351 MB of additional disk space will be used. Get:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ebtables amd64 2.0.10.4-3.4ubuntu2.16.04.2 [79.9 kB] Get:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 socat amd64 1.7.3.1-1 [321 kB] Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.11.0-00 [5,309 kB] Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 0.6.0-00 [5,910 kB] Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.11.1-00 [23.3 MB] Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.11.1-00 [9,388 kB] Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.11.1-00 [9,418 kB] Fetched 53.7 MB in 7s (7,298 kB/s) Selecting previously unselected package cri-tools. (Reading database ... 51422 files and directories currently installed.) Preparing to unpack .../cri-tools_1.11.0-00_amd64.deb ... Unpacking cri-tools (1.11.0-00) ... Selecting previously unselected package ebtables. Preparing to unpack .../ebtables_2.0.10.4-3.4ubuntu2.16.04.2_amd64.deb ... Unpacking ebtables (2.0.10.4-3.4ubuntu2.16.04.2) ... Selecting previously unselected package kubernetes-cni. Preparing to unpack .../kubernetes-cni_0.6.0-00_amd64.deb ... Unpacking kubernetes-cni (0.6.0-00) ... Selecting previously unselected package socat. Preparing to unpack .../socat_1.7.3.1-1_amd64.deb ... Unpacking socat (1.7.3.1-1) ... Selecting previously unselected package kubelet. Preparing to unpack .../kubelet_1.11.1-00_amd64.deb ... Unpacking kubelet (1.11.1-00) ... Selecting previously unselected package kubectl. Preparing to unpack .../kubectl_1.11.1-00_amd64.deb ... Unpacking kubectl (1.11.1-00) ... Selecting previously unselected package kubeadm. Preparing to unpack .../kubeadm_1.11.1-00_amd64.deb ... Unpacking kubeadm (1.11.1-00) ... Processing triggers for man-db (2.7.5-1) ... Processing triggers for systemd (229-4ubuntu21.2) ... Processing triggers for ureadahead (0.100.0-19) ... Setting up cri-tools (1.11.0-00) ... Setting up ebtables (2.0.10.4-3.4ubuntu2.16.04.2) ... update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults Setting up kubernetes-cni (0.6.0-00) ... Setting up socat (1.7.3.1-1) ... Setting up kubelet (1.11.1-00) ... Setting up kubectl (1.11.1-00) ... Setting up kubeadm (1.11.1-00) ... Processing triggers for systemd (229-4ubuntu21.2) ... Processing triggers for ureadahead (0.100.0-19) ...
계속 수행.
# apt-mark hold kubelet kubeadm kubectl kubelet set on hold. kubeadm set on hold. kubectl set on hold.
kubelet 재기동.
# systemctl daemon-reload # systemctl restart kubelet
6. init
시간이 많이 걸리니 기달린다.
# kubeadm init [init] using Kubernetes version: v1.11.1 [preflight] running pre-flight checks I0807 02:18:03.224218 10115 kernel_validator.go:81] Validating kernel version I0807 02:18:03.224446 10115 kernel_validator.go:96] Validating kernel config [preflight/images] Pulling images required for setting up a Kubernetes cluster [preflight/images] This might take a minute or two, depending on the speed of your internet connection [preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [preflight] Activating the kubelet service [certificates] Generated ca certificate and key. [certificates] Generated apiserver certificate and key. [certificates] apiserver serving cert is signed for DNS names [ip-10-0-10-220 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.0.10.220] [certificates] Generated apiserver-kubelet-client certificate and key. [certificates] Generated sa key and public key. [certificates] Generated front-proxy-ca certificate and key. [certificates] Generated front-proxy-client certificate and key. [certificates] Generated etcd/ca certificate and key. [certificates] Generated etcd/server certificate and key. [certificates] etcd/server serving cert is signed for DNS names [ip-10-0-10-220 localhost] and IPs [127.0.0.1 ::1] [certificates] Generated etcd/peer certificate and key. [certificates] etcd/peer serving cert is signed for DNS names [ip-10-0-10-220 localhost] and IPs [10.0.10.220 127.0.0.1 ::1] [certificates] Generated etcd/healthcheck-client certificate and key. [certificates] Generated apiserver-etcd-client certificate and key. [certificates] valid certificates and keys now exist in "/etc/kubernetes/pki" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf" [controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml" [controlplane] wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml" [controlplane] wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml" [etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml" [init] waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests" [init] this might take a minute or longer if the control plane images have to be pulled [apiclient] All control plane components are healthy after 46.505755 seconds [uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.11" in namespace kube-system with the configuration for the kubelets in the cluster [markmaster] Marking the node ip-10-0-10-220 as master by adding the label "node-role.kubernetes.io/master=''" [markmaster] Marking the node ip-10-0-10-220 as master by adding the taints [node-role.kubernetes.io/master:NoSchedule] [patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ip-10-0-10-220" as an annotation [bootstraptoken] using token: 09si41.rjipcw83obk3vna4 [bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 10.0.10.220:6443 --token 09si41.rjipcw83obk3vna4 --discovery-token-ca-cert-hash sha256:fc629584400772a5f0b61f4579a317399b1b430793e28129206ed02ea1882134
7. 클러스터 사용을 위한 작업
일반 사용자 계정에서 수행.
# mkdir -p $HOME/.kube # sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config # sudo chown $(id -u):$(id -g) $HOME/.kube/config