Centos7 配置 k8s 集群

我这里使用 ansible 配置，方便统一部署相同软件

[k8s-master]

kubernetes-1 ansible_ssh_host=10.173.32.34 ansible_ssh_user=root ansible_ssh_private_key_file=/Users/wenjun/.ssh/id_rsa

[k8s-node]

kubernetes-2 ansible_ssh_host=10.173.32.32 ansible_ssh_user=root ansible_ssh_private_key_file=/Users/wenjun/.ssh/id_rsa
kubernetes-3 ansible_ssh_host=10.173.32.33 ansible_ssh_user=root ansible_ssh_private_key_file=/Users/wenjun/.ssh/id_rsa

系统信息

➜  www ansible all -m command -a 'uname -a'
kubernetes-1 | SUCCESS | rc=0 >>
Linux kubernetes-1 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

kubernetes-2 | SUCCESS | rc=0 >>
Linux kubernetes-2.novalocal 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

kubernetes-3 | SUCCESS | rc=0 >>
Linux kubernetes-3.novalocal 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

➜  ~ ansible all -m command -a 'cat /etc/redhat-release'
kubernetes-2 | SUCCESS | rc=0 >>
CentOS Linux release 7.4.1708 (Core)

kubernetes-1 | SUCCESS | rc=0 >>
CentOS Linux release 7.4.1708 (Core)

kubernetes-3 | SUCCESS | rc=0 >>
CentOS Linux release 7.4.1708 (Core)

相同软件安装

通过 ansible 命令在三台机器上安装 docker etcd flannel kubernetes bridge-utils vim 等软件。

➜  ~ ansible k8s-master -m command -a 'yum -y  etcd install docker kubernetes bridge-utils redhat-lsb vim'

然后3台机器都需要写入 hosts

➜  ~ ansible all -m shell -a 'echo -e "10.173.32.34  k8s-master \n10.173.32.34   etcd \n10.173.32.34 registry \n10.173.32.32   k8s-nodeA \n10.173.32.33   k8s-nodeB" >> /etc/hosts'
kubernetes-2 | SUCCESS | rc=0 >>


kubernetes-3 | SUCCESS | rc=0 >>


kubernetes-1 | SUCCESS | rc=0 >>

这里使用 -m command 会有问题，需要使用 shell

并修改3台机器的主机名

master上运行：

[root@localhost ~]#  hostnamectl --static set-hostname  k8s-master

nodeA上运行：

[root@localhost ~]# hostnamectl --static set-hostname  k8s-nodeA

nodeB上运行：

[root@localhost ~]# hostnamectl --static set-hostname  k8s-nodeB

配置网桥，否则 docker 会启动失败

brctl show
brctl addbr docker0              #自定义网桥
ifconfig docker0 172.16.0.1/12   #给自定义网桥指定 IP 和子网
ifconfig docker0 up

Master 配置

[root@k8s-master ~]# vim /etc/etcd/etcd.conf

修改如下部分

ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379,http://0.0.0.0:4001"
ETCD_NAME="master"
ETCD_ADVERTISE_CLIENT_URLS="http://etcd:2379,http://etcd:4001"

1.配置 etcd

2.启动

[root@k8s-master ~]# systemctl start etcd
[root@k8s-master ~]# systemctl enable etcd
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.

3.测试

[root@k8s-master ~]# etcdctl -C http://etcd:2379 cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://10.173.32.34:2379
cluster is healthy
[root@k8s-master ~]# etcdctl -C http://etcd:4001 cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://10.173.32.34:2379
cluster is healthy

4.修改 docker 配置文件

[root@k8s-master ~]# vim /etc/sysconfig/docker
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'
if [ -z "${DOCKER_CERT_PATH}" ]; then
    DOCKER_CERT_PATH=/etc/docker
fi
OPTIONS='--insecure-registry registry:5000'

在kubernetes master 上运行需要以下组件：

• kubernetes api server

• kubernetes controller manager

• kubernetes scheduler

修改 apiserver

[root@k8s-master ~]# vim /etc/kubernetes/apiserver

###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#

# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

# The port on the local server to listen on.
 KUBE_API_PORT="--port=8080"

# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://etcd:2379"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

# Add your own!
KUBE_API_ARGS=""
~

修改

[root@k8s-master ~]# vim /etc/kubernetes/config

KUBE_MASTER="--master=http://k8s-master:8080"

启动

[root@k8s-master ~]# systemctl enable kube-apiserver
[root@k8s-master ~]# systemctl start kube-apiserver
[root@k8s-master ~]# systemctl enable kube-controller-manager
[root@k8s-master ~]# systemctl start kube-controller-manager
[root@k8s-master ~]# systemctl enable kube-scheduler
[root@k8s-master ~]# systemctl start kube-scheduler

Node 节点配置

在k8s-node上需要运行以下组件：

• kubelet

• kubernetes proxy

修改nodeA 机器

[root@k8s-nodeA ~]# cat /etc/kubernetes/kubelet
###
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
# KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=k8s-nodeA"

# location of the api-server
KUBELET_API_SERVER="--api-servers=http://k8s-master:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""

修改 nodeB 机器

[root@k8s-nodeA ~]# cat /etc/kubernetes/kubelet
###
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
# KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=k8s-nodeB"

# location of the api-server
KUBELET_API_SERVER="--api-servers=http://k8s-master:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""

启动

➜  www ansible k8s-node -m command -a 'systemctl enable kubelet'
kubernetes-2 | SUCCESS | rc=0 >>
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

kubernetes-3 | SUCCESS | rc=0 >>
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

➜  www ansible k8s-node -m command -a 'systemctl start kubelet'
kubernetes-2 | SUCCESS | rc=0 >>


kubernetes-3 | SUCCESS | rc=0 >>


➜  www ansible k8s-node -m command -a 'systemctl enable kube-proxy'
kubernetes-2 | SUCCESS | rc=0 >>
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.

kubernetes-3 | SUCCESS | rc=0 >>
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.

➜  www ansible k8s-node -m command -a 'systemctl start kube-proxy'
kubernetes-2 | SUCCESS | rc=0 >>


kubernetes-3 | SUCCESS | rc=0 >>

在 Master 节点查看状态

[root@k8s-master ~]# kubectl get node
NAME        STATUS    AGE
k8s-nodea   Ready     2m
k8s-nodeb   Ready     2m

配置 Flanneld

修改三台机器的 /etc/sysconfig/flanneld 配置文件，是有 ansible 配置如下

➜  www ansible all -m command -a 'sed -i s@127.0.0.1:2379@etcd:2379@g /etc/sysconfig/flanneld'
 [WARNING]: Consider using template or lineinfile module rather than running sed

kubernetes-2 | SUCCESS | rc=0 >>


kubernetes-3 | SUCCESS | rc=0 >>


kubernetes-1 | SUCCESS | rc=0 >>

然后在 MASTER 上执行

etcdctl mk /atomic.io/network/config '{"Network":"192.0.0.0/16"}'

查看配置

[root@k8s-master ~]# etcdctl get /atomic.io/network/config
{"Network":"192.0.0.0/16"}

重启 master 进程

systemctl enable flanneld.service 
systemctl start flanneld.service 
service docker restart
systemctl restart kube-apiserver.service
systemctl restart kube-controller-manager.service
systemctl restart kube-scheduler.service

重启 node 进程

systemctl enable flanneld.service 
systemctl start flanneld.service 
service docker restart
systemctl restart kubelet.service
systemctl restart kube-proxy.service

完成安装

[root@k8s-master ~]# kubectl --version
Kubernetes v1.5.2
[root@k8s-master ~]# kubectl get nodes
NAME        STATUS    AGE
k8s-nodea   Ready     16m
k8s-nodeb   Ready     16m
[root@k8s-master ~]#

到此为止，我们就搭建了一个 k8s 集群了，不过这里有很多问题，比如没有配置 https，etcd 没有配置集群等。