Print
카테고리: [ Cloud Computing & MSA ]
조회수: 2697

kubeadm에 대해 알아보고 설치해보자.


1. kubeadm이란?

kubeadm은 Kubernetes Cluster 생성을 위한 kubeadm init과 kubeadm join을 위한 툴이다. 


2. 환경

우분투 리눅스 기준


3. Docker 설치

도커를 설치한다.

# apt-get install docker.io
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package docker.io
E: Couldn't find any package by glob 'docker.io'
E: Couldn't find any package by regex 'docker.io'

만약 위와 같이 에러가 나면 apt update를 한 후에 다시 한다.

# apt-get update
Hit:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:3 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:4 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/main Sources [868 kB]
Get:5 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted Sources [4,808 B]
Get:6 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Sources [7,728 kB]
Get:7 http://security.ubuntu.com/ubuntu xenial-security InRelease [107 kB]
Get:8 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Sources [179 kB]
Get:9 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,532 kB]
Get:10 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,354 kB]
Get:11 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [144 kB]
Get:12 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [106 kB]
Get:13 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Sources [318 kB]
Get:14 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2,528 B]
Get:15 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [217 kB]
Get:16 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [8,408 B]
Get:17 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [824 kB]
Get:18 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [339 kB]
Get:19 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [7,556 B]
Get:20 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [677 kB]
Get:21 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [273 kB]
Get:22 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.4 kB]
Get:23 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,344 B]
Get:24 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Sources [4,488 B]
Get:25 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Sources [6,736 B]
Get:26 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [6,756 B]
Get:27 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,180 B]
Get:28 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [7,420 B]
Get:29 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [3,996 B]
Get:30 http://security.ubuntu.com/ubuntu xenial-security/main Sources [131 kB]
Get:31 http://security.ubuntu.com/ubuntu xenial-security/restricted Sources [2,116 B]
Get:32 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [69.5 kB]
Get:33 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [2,088 B]
Get:34 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [533 kB]
Get:35 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [228 kB]
Get:36 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [363 kB]
Get:37 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [136 kB]
Get:38 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,456 B]
Get:39 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [1,744 B]
Fetched 25.4 MB in 4s (5,356 kB/s)
Reading package lists... Done

다시 도커를 설치한다.

# apt-get install docker.io
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  bridge-utils cgroupfs-mount ubuntu-fan
Suggested packages:
  mountall aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils
The following NEW packages will be installed:
  bridge-utils cgroupfs-mount docker.io ubuntu-fan
0 upgraded, 4 newly installed, 0 to remove and 46 not upgraded.
Need to get 17.1 MB of archives.
After this operation, 90.5 MB of additional disk space will be used.
Do you want to continue? [Y/n]

잘 설치되었다.

Done.
Setting up ubuntu-fan (0.12.8~16.04.2) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...

4. kubeadm 설치 사전 작업

우선 apt-transport-https 를 설치한다.

# apt-get install apt-transport-https
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  apt-transport-https
1 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.
Need to get 26.1 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.27 [26.1 kB]
Fetched 26.1 kB in 0s (1,617 kB/s)
(Reading database ... 51422 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_1.2.27_amd64.deb ...
Unpacking apt-transport-https (1.2.27) over (1.2.26) ...
Setting up apt-transport-https (1.2.27) ...

아래 과정을 계속 진행한다. 당연하지만 curl 명령어가 있어야 함.

# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
OK

계속 수행..

# cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
> deb http://apt.kubernetes.io/ kubernetes-xenial main
> EOF

파일을 확인한다.

# ls -l /etc/apt/sources.list.d/kubernetes.list
-rw-r--r-- 1 root root 53 Aug  7 00:15 /etc/apt/sources.list.d/kubernetes.list

5. kubeadm 설치

# apt-get install -y kubelet kubeadm kubectl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  cri-tools ebtables kubernetes-cni socat
The following NEW packages will be installed:
  cri-tools ebtables kubeadm kubectl kubelet kubernetes-cni socat
0 upgraded, 7 newly installed, 0 to remove and 45 not upgraded.
Need to get 53.7 MB of archives.
After this operation, 351 MB of additional disk space will be used.
Get:1 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ebtables amd64 2.0.10.4-3.4ubuntu2.16.04.2 [79.9 kB]
Get:2 http://ap-northeast-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 socat amd64 1.7.3.1-1 [321 kB]
Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.11.0-00 [5,309 kB]
Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 0.6.0-00 [5,910 kB]
Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.11.1-00 [23.3 MB]
Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.11.1-00 [9,388 kB]
Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.11.1-00 [9,418 kB]
Fetched 53.7 MB in 7s (7,298 kB/s)
Selecting previously unselected package cri-tools.
(Reading database ... 51422 files and directories currently installed.)
Preparing to unpack .../cri-tools_1.11.0-00_amd64.deb ...
Unpacking cri-tools (1.11.0-00) ...
Selecting previously unselected package ebtables.
Preparing to unpack .../ebtables_2.0.10.4-3.4ubuntu2.16.04.2_amd64.deb ...
Unpacking ebtables (2.0.10.4-3.4ubuntu2.16.04.2) ...
Selecting previously unselected package kubernetes-cni.
Preparing to unpack .../kubernetes-cni_0.6.0-00_amd64.deb ...
Unpacking kubernetes-cni (0.6.0-00) ...
Selecting previously unselected package socat.
Preparing to unpack .../socat_1.7.3.1-1_amd64.deb ...
Unpacking socat (1.7.3.1-1) ...
Selecting previously unselected package kubelet.
Preparing to unpack .../kubelet_1.11.1-00_amd64.deb ...
Unpacking kubelet (1.11.1-00) ...
Selecting previously unselected package kubectl.
Preparing to unpack .../kubectl_1.11.1-00_amd64.deb ...
Unpacking kubectl (1.11.1-00) ...
Selecting previously unselected package kubeadm.
Preparing to unpack .../kubeadm_1.11.1-00_amd64.deb ...
Unpacking kubeadm (1.11.1-00) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up cri-tools (1.11.0-00) ...
Setting up ebtables (2.0.10.4-3.4ubuntu2.16.04.2) ...
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
Setting up kubernetes-cni (0.6.0-00) ...
Setting up socat (1.7.3.1-1) ...
Setting up kubelet (1.11.1-00) ...
Setting up kubectl (1.11.1-00) ...
Setting up kubeadm (1.11.1-00) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...

계속 수행.

# apt-mark hold kubelet kubeadm kubectl
kubelet set on hold.
kubeadm set on hold.
kubectl set on hold.

kubelet 재기동.

# systemctl daemon-reload
# systemctl restart kubelet

6. init

시간이 많이 걸리니 기달린다.

# kubeadm init
[init] using Kubernetes version: v1.11.1
[preflight] running pre-flight checks
I0807 02:18:03.224218   10115 kernel_validator.go:81] Validating kernel version
I0807 02:18:03.224446   10115 kernel_validator.go:96] Validating kernel config
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of your internet connection
[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[preflight] Activating the kubelet service
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [ip-10-0-10-220 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.0.10.220]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Generated etcd/ca certificate and key.
[certificates] Generated etcd/server certificate and key.
[certificates] etcd/server serving cert is signed for DNS names [ip-10-0-10-220 localhost] and IPs [127.0.0.1 ::1]
[certificates] Generated etcd/peer certificate and key.
[certificates] etcd/peer serving cert is signed for DNS names [ip-10-0-10-220 localhost] and IPs [10.0.10.220 127.0.0.1 ::1]
[certificates] Generated etcd/healthcheck-client certificate and key.
[certificates] Generated apiserver-etcd-client certificate and key.
[certificates] valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] this might take a minute or longer if the control plane images have to be pulled
[apiclient] All control plane components are healthy after 46.505755 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.11" in namespace kube-system with the configuration for the kubelets in the cluster
[markmaster] Marking the node ip-10-0-10-220 as master by adding the label "node-role.kubernetes.io/master=''"
[markmaster] Marking the node ip-10-0-10-220 as master by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ip-10-0-10-220" as an annotation
[bootstraptoken] using token: 09si41.rjipcw83obk3vna4
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.0.10.220:6443 --token 09si41.rjipcw83obk3vna4 --discovery-token-ca-cert-hash sha256:fc629584400772a5f0b61f4579a317399b1b430793e28129206ed02ea1882134

7. 클러스터 사용을 위한 작업

일반 사용자 계정에서 수행.

# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config