陕西中洋建设工程有限公司网站友情链接适用网站
文章目录
- 安装部署过程
- 1.修改基本配置
- 2.安装docker
- 3.安装k8s
- 4.kubeadm建立集群
- 5.安装网络插件
- 6.部署dashboard
节点安排：
| name | IP |
|---|---|
| master | 172.16.10.21 |
| node1 | 172.16.10.22 |
| node2 | 172.16.10.23 |
如果接下来的步骤中没有特殊指明是哪台机器要做的话，就都要执行
安装部署过程
1.修改基本配置
1.1.首先安装ubuntu的虚拟机，配置静态IP地址，使其能够正常上网，更换为国内镜像源，并且能够被xshell正常连接上（如果这步出现问题可以查看我的之前博客：Ubuntu系统配置静态IP地址、更换国内源以及连接xshell
1.2.修改主机名，配置hosts文件，禁用防火墙跟selinux以及swap交换分区(ubuntu默认没有selinux这个功能模块，可以忽略）
例如master节点上就是：
hostnamectl set-hostname master
vim /etc/hosts
cat /etc/hosts
172.16.10.21 master
172.16.10.22 node1
172.16.10.23 node2
ufw disable
swapoff -a && sed -i '/swap/d' /etc/fstab
为什么需要关闭交换分区：在集群中，我们通常是希望如果出现OOM（内存溢出）的情况，就直接终止这个进程，然后kubernetes进行故障转移，把这个进程在其他节点上重启起来。而不是，出现OOM的时候，通过交换分区来延长使用，看似没有问题也不会有报错提示给我们，但是会导致节点hang住（没有响应，卡死）。更可怕的是有一些集群的swap位于机械硬盘阵列上，大量动用swap基本可以等同于死机，你甚至连root都登录不上，不用提杀掉问题进程了，往往结局就是硬盘重启
1.3.修改配置
# Enable kernel modules
sudo modprobe overlay && \
sudo modprobe br_netfilter# Add some settings to sysctl
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF# Reload sysctl
sudo sysctl --system
2.安装docker
2.1.安装docker
sudo apt update && \
sudo apt install apt-transport-https ca-certificates curl software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - && \
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" && \
apt-cache policy docker-ce && \
sudo apt install -y containerd.io docker-ce docker-ce-cli && \
sudo systemctl status docker
2.2配置docker
# Create required directories
sudo mkdir -p /etc/systemd/system/docker.service.d# Create daemon json config file
mkdir /etc/docker
sudo tee /etc/docker/daemon.json <<EOF
{"exec-opts": ["native.cgroupdriver=systemd"], "registry-mirrors": ["https://8i185852.mirror.aliyuncs.com"], "log-driver": "json-file","log-opts": {"max-size": "100m"},"storage-driver": "overlay2"
}
EOF# Start and enable Services
sudo systemctl daemon-reload && \
sudo systemctl restart docker && \
sudo systemctl enable docker
kubernetes的cgroup驱动默认是system的，而docker的cgroup驱动默认是cgroupfs，所以建议将docker的cgroup驱动改为system的，与kubernetes保持一致，否则易导致kubeadm
init失败
3.安装k8s
3.1 安装阿里源以及一些证书之类的
sudo apt update && \
sudo apt -y install curl apt-transport-https \
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - \
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
3.2 更新下载
#查看kubeadm kubelet kubectl有哪些版本，以及版本安装时的具体名称，例如1.23.6是错误的，要是1.23.6-00
apt-cache madison kubeadm kubelet kubectl
#安装指定版本的kubeadm kubelet kubectl
sudo apt update && \
sudo apt-get -y install kubelet=1.23.6-00 kubeadm=1.23.6-00 kubectl=1.23.6-00 && \ # 安装的时候需要指定版本，否则会安装最新版本，node节点可以不需要安装kubectl
sudo apt-mark hold kubelet kubeadm kubectl #阻止软件自动更新
systemctl start kubelet
systemctl enable kubelet查看安装的情况以及版本
kubectl version --client && kubeadm version
3.2 拉取k8s的镜像
root@master:~# kubeadm config images list
I0207 17:52:14.976303 56639 version.go:255] remote version is much newer: v1.26.1; falling back to: stable-1.23
k8s.gcr.io/kube-apiserver:v1.23.16
k8s.gcr.io/kube-controller-manager:v1.23.16
k8s.gcr.io/kube-scheduler:v1.23.16
k8s.gcr.io/kube-proxy:v1.23.16
k8s.gcr.io/pause:3.6
k8s.gcr.io/etcd:3.5.1-0
k8s.gcr.io/coredns/coredns:v1.8.6#kubeadm config images pull #本来应该直接拉镜像，但是因为这个需要翻墙，所以此命令无法执行#拉取阿里镜像
root@master:~# kubeadm config print init-defaults > kubeadm.conf
root@master:~# sed -i 's/k8s.gcr.io/registry.aliyuncs.com\/google_containers/g' kubeadm.conf
root@master:~# kubeadm config images list --config kubeadm.conf
root@master:~# kubeadm config images list --config kubeadm.conf #更改镜像名字
registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.0
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.23.0
registry.aliyuncs.com/google_containers/kube-scheduler:v1.23.0
registry.aliyuncs.com/google_containers/kube-proxy:v1.23.0
registry.aliyuncs.com/google_containers/pause:3.6
registry.aliyuncs.com/google_containers/etcd:3.5.1-0
registry.aliyuncs.com/google_containers/coredns:v1.8.6
#修改镜像名字为阿里镜像名字 前者为阿里云镜像名字 后者为谷歌镜像名字
#前者：sudo kubeadm config images list --config kubeadm.conf
#后者：sudo kubeadm config images listroot@master:~# kubeadm config images pull --config kubeadm.conf #拉取阿里镜像
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.6
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.5.1-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.8.6
4.kubeadm建立集群
4.1 首先是在master上的操作
#如果初始化集群失败后要记得先kubeadm reset，再继续kubeadm init
#初始化集群配置kubeadm init \--image-repository registry.aliyuncs.com/google_containers \--kubernetes-version v1.23.6 \--pod-network-cidr=192.168.0.0/16 \--service-cidr=10.96.0.0/12 \--apiserver-advertise-address=172.16.10.50# 初始化命令参数说明：--apiserver-advertise-addres=172.16.10.21 这个参数就是master主机的IP地址，例如我的Master主机的IP是：172.16.10.21
--image-repository=registry.aliyuncs.com/google_containers 这个是选择拉取 control plane images 的镜像repo这个是镜像地址，由于国外地址无法访问，故使用的阿里云仓库地址：registry.aliyuncs.com/google_containers，
--kubernetes-version=v1.23.6 这个参数是下载的k8s软件版本号
--service-cidr=10.96.0.0/12 这个参数后的IP地址直接就套用10.96.0.0/12 ,以后安装时也套用即可，不要更改
--pod-network-cidr=10.10.0.0/16 k8s内部的pod节点之间网络可以使用的IP段，不能和service-cidr写一样，如果不知道怎么配，就先用这个10.244.0.0/16，最好是和docker0处于同一个网段，如果后续安装的网络插件是Calico，那么kubeadm init时必须添加此参数#在kubeadm init命令之前，会执行一系列被称为pre-flight checks的系统与检查，以确保主机环境符合安装要求，如果检查失败就直接终止，不再进行init操作；用户可以通过kubeadm init phase preflight命令执行预检查操作，确保系统就绪后再执行init操作；或者也可以在执行kubeadm init命令时添加--ignore-preflight-errors参数关闭预检查# 输出如下 ......
Your Kubernetes control-plane has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:export KUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/Then you can join any number of worker nodes by running the following on each as root:kubeadm join 172.16.10.21:6443 --token 0t6kt4.e312po5zfm7xpt9y \--discovery-token-ca-cert-hash sha256:0f43ed44e4b628e96b27166b97b3b41a9fcc382b53a544e67e219addb25f5571
# 在master上执行下面操作(初始化成功才执行）
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
#查看集群状态
kubectl cluster-info
#查看节点状态，此时都是NotReady
root@master:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane,master 2d3h v1.23.6
node1 NotReady <none> 2d3h v1.23.6
node2 NotReady <none> 2d3h v1.23.6#master产生的token会失效，重新产生token的命令为
root@master:/lianxi/2.13# kubeadm token create --print-join-command
kubeadm join 172.16.10.21:6443 --token txj249.qbxqb4w5ro2lhgy8 --discovery-token-ca-cert-hash sha256:0f43ed44e4b628e96b27166b97b3b41a9fcc382b53a544e67e219addb25f5571
4.2 所有node节点分别上执行
kubeadm join 172.16.10.21:6443 --token 0t6kt4.e312po5zfm7xpt9y \--discovery-token-ca-cert-hash sha256:0f43ed44e4b628e96b27166b97b3b41a9fcc382b53a544e67e219addb25f5571 # 如果重新加入master，也需要先kubeadm reset
5.安装网络插件
# 第一种是安装calico插件，但是我安装之后有报错，所以我后面改成了flannel
# 只需要在master节点上执行
kubectl apply -f https://docs.projectcalico.org/v3.21/manifests/calico.yaml #，创建过程会要几分钟，完成后所有node节点的状态变为ready
kubectl get nodes #所有节点的状态会变成ready# 第二种是安装flannel插件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel.yml #记得修改配置文件里的net-conf.json下的Network地址，修改为kubeadm init时设置的--pod-network-cidr，然后再apply
#检查集群
root@master:~# kubectl get nodes #安装了网络插件后就都是Ready状态
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 2d3h v1.23.6
node1 Ready <none> 2d3h v1.23.6
node2 Ready <none> 2d3h v1.23.6
6.部署dashboard
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
mv recommended.yaml kubernetes-dashboard.yaml
vim kubernetes-dashboard.yaml
cat kubernetes-dashboard.yaml
kind: Service
apiVersion: v1
metadata:labels:k8s-app: kubernetes-dashboardname: kubernetes-dashboardnamespace: kubernetes-dashboard
spec:type: NodePortports:- port: 443targetPort: 8443nodePort: 30001selector:k8s-app: kubernetes-dashboardroot@master:~# kubectl apply -f kubernetes-dashboard.yaml
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
Warning: spec.template.metadata.annotations[seccomp.security.alpha.kubernetes.io/pod]: deprecated since v1.19, non-functional in v1.25+; use the "seccompProfile" field instead
deployment.apps/dashboard-metrics-scraper createdroot@master:~# kubectl get pods,svc -n kubernetes-dashboard -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/dashboard-metrics-scraper-577dc49767-tzjls 1/1 Running 0 25m 10.10.166.131 node1 <none> <none>
pod/kubernetes-dashboard-6bd77794f-4pqbh 1/1 Running 0 25m 10.10.104.2 node2 <none> <none>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/dashboard-metrics-scraper ClusterIP 10.111.188.114 <none> 8000/TCP 25m k8s-app=dashboard-metrics-scraper
service/kubernetes-dashboard NodePort 10.110.197.85 <none> 443:30001/TCP 25m k8s-app=kubernetes-dashboardroot@master:~# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created
root@master:~# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created
root@master:~# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
Name: dashboard-admin-token-4545w
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: dashboard-adminkubernetes.io/service-account.uid: 37c0d3a6-2c21-4cd7-aa7b-2709be5fd424Type: kubernetes.io/service-account-tokenData
====
ca.crt: 1099 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6Ikp2OUktLTZXWlpUV3g5aTBlOEwyckxWX3dPelc0RktaTnU3RFhKN1kzcUEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tNDU0NXciLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzdjMGQzYTYtMmMyMS00Y2Q3LWFhN2ItMjcwOWJlNWZkNDI0Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.QGfm8T1fNWvO85-SEYSwdRNTEnLOuvCrwetI8o6OO_PXYg1Ka_0vp4M6knlPB9a_c2SrMbLFfoBd_6NBj5E1ovywOtrKyM14vBPkRrbmbbWpA0niDNrEYq-OHo_8XWv3q5w3hLfX_K5_GKZNhxLBYiNL_4R7crqILiFK0_vPxm6QPrguGNomNl4PjpOuYSEvQUxYbKnx37Lxc1EW2ZuYnKLvohEs3Ib22znusug3CF624e8Rnh7fGShSh_BaO_nAn54nKl3PsCgUlprlkq7N9JiOoLsWEmXNaWh-Y9RglomFoJOUrE-CnkZX9gODgcJziIZQPzMIpbbd9DWbygUHVA#因为kubernetes-dashboard部署在node2上，所以通过https://node2IP:30001可以访问dashboard，token为上面所示