1. 为什么选择二进制方式部署Kubernetes?
在容器编排领域,Kubernetes(简称k8s)的部署方式主要有两种:基于kubeadm工具快速部署和手动二进制部署。二进制部署虽然步骤繁琐,但却是理解k8s架构最透彻的方式。我最近在生产环境用二进制方式部署了4节点k8s集群,过程中积累了不少实战经验。
二进制部署的核心优势在于:
- 完全掌控:每个组件的启动参数、日志路径、证书周期都可自定义
- 排障直观:组件间通过标准网络协议通信,问题定位更直接
- 升级灵活:可以逐个组件灰度升级,避免kubeadm的整体版本绑定
重要提示:生产环境建议至少准备3个控制平面节点(奇数个)和若干工作节点,本文以1个控制节点+3个工作节点为例说明。
2. 环境准备与组件规划
2.1 基础环境要求
所有节点需要满足:
- CentOS 7.9/Ubuntu 20.04 LTS
- 2核CPU/4GB内存/50GB磁盘(控制平面节点建议翻倍)
- 关闭swap:
swapoff -a && sed -i '/swap/s/^/#/' /etc/fstab - 时间同步:
yum install -y ntpdate && ntpdate time.windows.com
网络规划示例:
| 节点类型 | IP地址 | 主机名 |
|---|---|---|
| Master | 192.168.101.10 | k8s-master01 |
| Worker01 | 192.168.101.11 | k8s-node01 |
| Worker02 | 192.168.101.12 | k8s-node02 |
| Worker03 | 192.168.101.13 | k8s-node03 |
2.2 关键组件版本选择
建议版本组合:
- Kubernetes: v1.23.6(较稳定版本)
- etcd: v3.5.0
- CNI插件: Calico v3.21.4
- 容器运行时: containerd v1.6.4
组件下载地址:
# 下载k8s二进制文件 wget https://dl.k8s.io/v1.23.6/kubernetes-server-linux-amd64.tar.gz # 下载etcd wget https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz3. 证书体系构建
3.1 CA根证书生成
创建证书配置文件ca-config.json:
{ "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "usages": ["signing", "key encipherment", "server auth", "client auth"], "expiry": "87600h" } } } }生成CA证书(在master节点操作):
openssl genrsa -out ca.key 2048 openssl req -x509 -new -nodes -key ca.key -subj "/CN=kubernetes" -days 3650 -out ca.crt3.2 组件证书签发
以apiserver证书为例,创建CSR配置文件apiserver-csr.json:
{ "CN": "kube-apiserver", "hosts": [ "127.0.0.1", "192.168.101.10", "10.96.0.1", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 } }生成证书:
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json \ -profile=kubernetes apiserver-csr.json | cfssljson -bare apiserver4. 核心组件部署
4.1 etcd集群部署
解压etcd并配置服务(所有master节点):
tar -xvf etcd-v3.5.0-linux-amd64.tar.gz mv etcd-v3.5.0-linux-amd64/{etcd,etcdctl} /usr/local/bin/ # 创建配置文件 cat > /etc/etcd/etcd.conf <<EOF name=$(hostname)>systemctl daemon-reload systemctl enable etcd systemctl start etcd验证集群状态:
ETCDCTL_API=3 etcdctl \ --endpoints=https://192.168.101.10:2379 \ --cacert=/etc/kubernetes/pki/ca.crt \ --cert=/etc/kubernetes/pki/etcd/server.crt \ --key=/etc/kubernetes/pki/etcd/server.key \ endpoint health4.2 kube-apiserver部署
创建apiserver配置文件/etc/kubernetes/apiserver.conf:
apiVersion: v1 kind: Pod metadata: name: kube-apiserver namespace: kube-system spec: containers: - command: - kube-apiserver - --advertise-address=192.168.101.10 - --allow-privileged=true - --authorization-mode=Node,RBAC - --client-ca-file=/etc/kubernetes/pki/ca.crt - --enable-admission-plugins=NodeRestriction - --enable-bootstrap-token-auth=true - --etcd-cafile=/etc/kubernetes/pki/ca.crt - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key - --etcd-servers=https://192.168.101.10:2379 - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key - --requestheader-allowed-names=front-proxy-client - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt - --requestheader-extra-headers-prefix=X-Remote-Extra- - --requestheader-group-headers=X-Remote-Group - --requestheader-username-headers=X-Remote-User - --secure-port=6443 - --service-account-key-file=/etc/kubernetes/pki/sa.pub - --service-cluster-ip-range=10.96.0.0/12 - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key image: registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.6 name: kube-apiserver volumeMounts: - mountPath: /etc/kubernetes/pki name: k8s-certs readOnly: true - mountPath: /etc/ssl/certs name: ca-certs readOnly: true volumes: - hostPath: path: /etc/kubernetes/pki type: DirectoryOrCreate name: k8s-certs - hostPath: path: /etc/ssl/certs type: DirectoryOrCreate name: ca-certs5. 网络插件与节点加入
5.1 Calico网络部署
下载Calico manifest:
wget https://docs.projectcalico.org/manifests/calico.yaml修改配置关键参数:
- name: CALICO_IPV4POOL_CIDR value: "10.244.0.0/16" - name: IP_AUTODETECTION_METHOD value: "interface=eth.*"应用配置:
kubectl apply -f calico.yaml5.2 Worker节点加入集群
在worker节点执行:
kubeadm join 192.168.101.10:6443 \ --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:xxxxxxxxxx验证节点状态:
kubectl get nodes -o wide6. 常见问题排查
6.1 证书过期处理
查看证书有效期:
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -dates续期方法:
- 备份旧证书
- 使用相同CSR重新生成
- 重启apiserver
6.2 节点NotReady状态排查
检查步骤:
- 确认kubelet服务状态:
systemctl status kubelet - 查看容器运行时日志:
journalctl -u containerd -f - 检查网络插件Pod状态:
kubectl get pods -n kube-system
6.3 etcd集群故障恢复
单节点故障处理:
ETCDCTL_API=3 etcdctl --endpoints=https://192.168.101.10:2379 \ --cacert=/etc/kubernetes/pki/ca.crt \ --cert=/etc/kubernetes/pki/etcd/server.crt \ --key=/etc/kubernetes/pki/etcd/server.key \ snapshot save snapshot.db7. 维护与优化建议
- 证书管理:使用cert-manager自动管理证书续期
- 日志收集:部署Loki+Promtail+Grafana日志系统
- 监控告警:配置Prometheus监控关键指标
- 备份策略:定期备份etcd数据和应用PV
升级注意事项:
- 先升级kube-apiserver,再升级其他控制平面组件
- worker节点可以滚动升级
- 每次只升级一个minor版本(如1.23→1.24)