1. 사전 준비사항
Amazon Linux 또는 Ubuntu 머신(EC2 가상 서버)은 준비되어 있다고 가정합니다. 이 글은 EC2 인스턴스에 k3s를 설치하면서 Amazon Linux에서는 실패했고, Ubuntu에서는 성공한 과정을 정리한 기록입니다.
보안 그룹에서는 SSH 접속용 포트와, 워커 노드를 추가할 경우 k3s API 서버 포트인 6443에 접근할 수 있도록 설정해 둡니다. 운영 환경이라면 전체 공개가 아니라 필요한 IP 대역으로 제한하는 것이 좋습니다.
2. 서버 접속
SSH를 통해 서버에 접속합니다. 이후 명령어는 기본적으로 접속한 서버의 터미널에서 실행합니다.
3. Amazon Linux 설치 (실패)
먼저 Amazon Linux 환경에서 설치를 시도했습니다. 결과적으로 이 환경에서는 원하는 상태까지 진행하지 못했지만, 중간에 발생한 SELinux 관련 오류와 처리 과정을 기록해 둡니다.
3.1. 설치 커맨드
curl -sfL https://get.k3s.io | sh -
처음에는 구글링 후 아래 명령어로 설치를 시도했는데 잘 되지 않아, 위의 기본 설치 명령어로 다시 실행했습니다.
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="\
--disable traefik \
--disable metrics-server \
--node-name master --docker" \
INSTALL_K3S_VERSION="v1.18.6+k3s1" sh -s -
3.2. 오류 발생 시
다음과 같은 오류가 발생하면 SELinux 관련 필수 패키지를 설치한 뒤 다시 진행합니다.
[INFO] Using v1.18.6+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.18.6+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.18.6+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[ERROR] Failed to find the k3s-selinux policy, please install:
dnf install -y container-selinux
dnf install -y https://rpm.rancher.io/k3s/stable/common/centos/8/noarch/k3s-selinux-1.2-2.el8.noarch.rpm
오류 메시지에 안내된 대로 아래 dnf install 명령을 실행합니다.
$ sudo dnf install -y container-selinux Last metadata expiration check: 0:31:48 ago on Fri Apr 7 01:49:33 2023. Dependencies resolved. ========================================================================================================================================================================= Package Architecture Version Repository Size ========================================================================================================================================================================= Installing: container-selinux noarch 2:2.189.0-289.amzn2023.0.2 amazonlinux 47 k Transaction Summary ========================================================================================================================================================================= Install 1 Package Total download size: 47 k Installed size: 57 k Downloading Packages: container-selinux-2.189.0-289.amzn2023.0.2.noarch.rpm 576 kB/s | 47 kB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 298 kB/s | 47 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Running scriptlet: container-selinux-2:2.189.0-289.amzn2023.0.2.noarch 1/1 Installing : container-selinux-2:2.189.0-289.amzn2023.0.2.noarch 1/1 Running scriptlet: container-selinux-2:2.189.0-289.amzn2023.0.2.noarch 1/1 Verifying : container-selinux-2:2.189.0-289.amzn2023.0.2.noarch 1/1 Installed: container-selinux-2:2.189.0-289.amzn2023.0.2.noarch Complete!
$ sudo dnf install -y https://rpm.rancher.io/k3s/stable/common/centos/8/noarch/k3s-selinux-1.2-2.el8.noarch.rpm Last metadata expiration check: 0:37:12 ago on Fri Apr 7 01:49:33 2023. k3s-selinux-1.2-2.el8.noarch.rpm 34 kB/s | 20 kB 00:00 Dependencies resolved. ========================================================================================================================================================================= Package Architecture Version Repository Size ========================================================================================================================================================================= Installing: k3s-selinux noarch 1.2-2.el8 @commandline 20 k Transaction Summary ========================================================================================================================================================================= Install 1 Package Total size: 20 k Installed size: 94 k Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Running scriptlet: k3s-selinux-1.2-2.el8.noarch 1/1 Installing : k3s-selinux-1.2-2.el8.noarch 1/1 Running scriptlet: k3s-selinux-1.2-2.el8.noarch 1/1 Verifying : k3s-selinux-1.2-2.el8.noarch 1/1 Installed: k3s-selinux-1.2-2.el8.noarch Complete!
3.3. 설치 완료
필수 패키지 설치 후 다시 실행하면 k3s 서비스 파일과 관련 심볼릭 링크가 생성되고, systemd를 통해 k3s가 시작됩니다.
$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="\
--disable traefik \
--disable metrics-server \
--node-name master --docker" INSTALL_K3S_VERSION="v1.18.6+k3s1" sh -s -
[INFO] Using v1.18.6+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.18.6+k3s1/sha256sum-amd64.txt
[INFO] Skipping binary downloaded, installed k3s matches hash
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
3.4. Kubernetes 마스터 설정
kubectl을 일반 사용자로 실행하기 위해 k3s가 생성한 kubeconfig 파일을 사용자 홈 디렉터리로 복사합니다.
sudo update-ca-certificates mkdir ~/.kube sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config sudo chown -R $(id -u):$(id -g) ~/.kube echo "export KUBECONFIG=~/.kube/config" >> ~/.bashrc source ~/.bashrc
4. Ubuntu 설치 (성공)
Amazon Linux에서의 시도와 달리 Ubuntu 환경에서는 기본 설치 명령어만으로 정상 설치되었습니다.
4.1. 파일 설치
$ curl -sfL https://get.k3s.io | sh - [INFO] Finding release for channel stable [INFO] Using v1.26.3+k3s1 as release [INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.26.3+k3s1/sha256sum-amd64.txt [INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.26.3+k3s1/k3s [INFO] Verifying binary download [INFO] Installing k3s to /usr/local/bin/k3s [INFO] Skipping installation of SELinux RPM [INFO] Creating /usr/local/bin/kubectl symlink to k3s [INFO] Creating /usr/local/bin/crictl symlink to k3s [INFO] Creating /usr/local/bin/ctr symlink to k3s [INFO] Creating killall script /usr/local/bin/k3s-killall.sh [INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh [INFO] env: Creating environment file /etc/systemd/system/k3s.service.env [INFO] systemd: Creating service file /etc/systemd/system/k3s.service [INFO] systemd: Enabling k3s unit Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service. [INFO] systemd: Starting k3s
4.2. 상태 확인
설치가 끝나면 k3s 서비스가 정상적으로 실행 중인지 확인합니다.
$ service k3s status
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2023-04-07 05:08:58 UTC; 43min ago
Docs: https://k3s.io
Process: 1698 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
Process: 1700 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 1702 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 1704 (k3s-server)
Tasks: 92
Memory: 1.1G
CPU: 2min 26.327s
CGroup: /system.slice/k3s.service
├─1704 "/usr/local/bin/k3s server"
├─1724 containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/l>
├─2285 /var/lib/rancher/k3s/data/c26e7571d760c5f199d18efd197114f1ca4ab1e6ffe494f96feb65c87fcb8cf0/bin/containerd-shim-runc-v2 -namespace k8s.io -id f6dbfc1>
├─2446 /var/lib/rancher/k3s/data/c26e7571d760c5f199d18efd197114f1ca4ab1e6ffe494f96feb65c87fcb8cf0/bin/containerd-shim-runc-v2 -namespace k8s.io -id 741c3aa>
├─2469 /var/lib/rancher/k3s/data/c26e7571d760c5f199d18efd197114f1ca4ab1e6ffe494f96feb65c87fcb8cf0/bin/containerd-shim-runc-v2 -namespace k8s.io -id 35068a7>
├─3564 /var/lib/rancher/k3s/data/c26e7571d760c5f199d18efd197114f1ca4ab1e6ffe494f96feb65c87fcb8cf0/bin/containerd-shim-runc-v2 -namespace k8s.io -id 987dab2>
└─3634 /var/lib/rancher/k3s/data/c26e7571d760c5f199d18efd197114f1ca4ab1e6ffe494f96feb65c87fcb8cf0/bin/containerd-shim-runc-v2 -namespace k8s.io -id c856c88>
Apr 07 05:23:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:23:54Z" level=info msg="COMPACT revision 0 has already been compacted"
Apr 07 05:28:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:28:54Z" level=info msg="COMPACT revision 0 has already been compacted"
Apr 07 05:33:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:33:54Z" level=info msg="COMPACT compactRev=0 targetCompactRev=60 currentRev=1060"
Apr 07 05:33:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:33:54Z" level=info msg="COMPACT deleted 8 rows from 60 revisions in 709.916µs - compacted to 60/1060"
Apr 07 05:38:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:38:54Z" level=info msg="COMPACT compactRev=60 targetCompactRev=150 currentRev=1150"
Apr 07 05:38:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:38:54Z" level=info msg="COMPACT deleted 6 rows from 90 revisions in 650.919µs - compacted to 150/1150"
Apr 07 05:43:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:43:54Z" level=info msg="COMPACT compactRev=150 targetCompactRev=240 currentRev=1240"
Apr 07 05:43:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:43:54Z" level=info msg="COMPACT deleted 15 rows from 90 revisions in 1.144503ms - compacted to 240/1240"
Apr 07 05:48:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:48:54Z" level=info msg="COMPACT compactRev=240 targetCompactRev=330 currentRev=1330"
Apr 07 05:48:54 ip-10-21-110-130 k3s[1704]: time="2023-04-07T05:48:54Z" level=info msg="COMPACT deleted 17 rows from 90 revisions in 1.088716ms - compacted to 330/1330"
4.3. kubectl 명령어
kubectl get node 명령으로 노드가 Ready 상태인지 확인합니다.
$ sudo kubectl get node -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ip-10-21-110-130 Ready control-plane,master 44m v1.26.3+k3s1 10.21.110.130 Ubuntu 22.04.2 LTS 5.15.0-1031-aws containerd://1.6.19-k3s1
추가로 시스템 파드가 정상적으로 올라왔는지도 확인해 두면 좋습니다.
sudo kubectl get pods -A
4.4. 워커 노드 추가
워커 노드를 추가하려면, 추가할 워커 노드에서 토큰값과 마스터 URL을 환경변수로 설정한 뒤 k3s 설치를 실행합니다.
마스터 서버에서 토큰값을 확인하는 명령어는 다음과 같습니다.
cat /var/lib/rancher/k3s/server/node-token
워커 노드에서는 다음과 같이 환경변수를 설정합니다. K3S_URL에는 마스터 노드의 실제 접근 가능한 IP 주소와 포트 6443을 입력합니다.
# export K3S_TOKEN=.... # export K3S_URL=https://192.168.10.101:6443
그다음 각 워커 노드에서 아래 명령어를 실행하여 설치합니다.
curl -sfL https://get.k3s.io | sh -
워커 노드 설치가 끝난 뒤에는 마스터 노드에서 다시 kubectl get node -o wide를 실행해 새 워커 노드가 목록에 표시되고 Ready 상태가 되었는지 확인합니다.