编写你自己的Kubernetes Operator

CRDs are only a mean to specify a configuration, though. The cluster still needs controllers to monitor its state and reconcile the resource to match with the configuration. This is where Operators come into play.

CRDS 其实就是一些对象的定义，我们最终还是需要controllers 来监控和调度资源来打到预期值，这就是为什么我们需要Operators

Operators are controllers working in association with custom resources to perform tasks that well… “human operators” have to take care of. Think about deploying a database cluster with a manageable number of nodes, managing upgrades and even performing backups. A custom resource would specify the version of the database, the number of nodes to deploy and the frequency of the backups as well as their target storage, and the controller would implement all the business logic needed to perform these operations. This is what the etcd-operator does, for example.

Operators其实就是一个控制器（controllers）来调配客户资源并执行一些特定的任务，一些原来必须有人工来处理的工作，Operators来代替人工，自动执行。例如，我们要部署一个拥有特定节点数量的数据库集群，管理数据库的升级，备份。而通过Kubernetes我们可以通过CRDs来定义数据库的版本，节点的数量，备份的频率等，然后controllers来对这些操作做逻辑上处理，这就是etcd-operator的工作模式 etcd-operator

Yet, writing a CRD schema and its accompanying controller can be a daunting task. Thankfully, the Operator SDK is here to help. Let’s see how it can be used to build a simple operator that scales up and down pods running a particular Docker image. In other words, let’s code a basic Kubernetes ReplicaSet!

但是，编写CRD已经对应的负责逻辑处理controllers是一个艰巨的任务。不过，我们有一个 工具 能让这个工作变得简单一些： Operator SDK ，下面我们就用个例子来实现Kubernetes的replicaSet 功能，不过我们是通过我们自己定义的CRD+controller

Please note that at the time of writing this article, the Operator SDK was in version  v0.4.0 , so things may have changed afterwards. Also, in this article, we’ll build a Go Operator, but keep in mind that operators can also be developed as Ansible playbooks or Helm charts.

请注意当前的Operator SDK 版本为v0.4.0 ，后续版本可能会有所不同

Primary resources and secondary resources主要资源和次要资源

Before diving into the Operator SDK and the source code, let’s take a step back and discuss about operators a little more. As we’ve seen before, operators are the CRDs associated with controllers which observe and act upon changes in the configuration or changes in the state of the cluster. But there are actually two kinds of resources that need to be monitored by the controller. The primary resource s and the  secondary resources .

在我们开始之前，让我们先进一步了解operators。正如我们前边介绍的，operator是 CRDS + conrollers 一个组合，CRDS负责定义资源，controllers负责监控并执行必须的操作让结果符合预期。但其实，我们有两种资源需要被controller监控，就是 primary resource s and the  secondary resources .

In the case of a ReplicaSet, the primary resource is the ReplicaSet itself: it specifies the Docker image to run and the number of pods in the set. The secondary resources are the pods themselves. Then, when a change occurs in the ReplicaSet spec itself (e.g., the version of the image was changed or the number of pods was updated) or when a change occurs in the pods (e.g.: a pod was deleted), then the controller is notified and acts in consequence to reconcile the state of the cluster by rolling out a new version, or just scaling up or down the pods.

在ReplicaSet的例子中， 主要资源是ReplicaSet本身，它指定了具体的Docker image版本和pod的数量，次要资源就是这些pod本身。然后当ReplicaSet 本身发生变化时（例如 docker 的镜像版本或者Pod的数量发生变化），或者pods发生了变化（pod被删掉）, 这个时候，controllers就会得到通知并作出相应的调整，例如，按照新的docker image版本重新部署指定数量的pod， 或者增加一个pod来达到预期的数量

In the case of a DaemonSet, the primary resources is the DaemonSet itself, while the secondary resources are once again the pods, but also the nodes of the cluster. The difference here is that the controller also monitors the nodes of the cluster to add or remove pods as the cluster grows or shrinks.

我们的例子DaemonSet, 主要资源就是DaemonSet本身，但是，次要资源除了pod之外，还有集群的节点。这与ReplicaSet不同的地方在于我们的controller会监控这个cluster节点的数量，并随着节点的数量的增加而增加我们的pod的数量，减少而减少pod的数量

Building a PodSet operator 创建PodSet operator

Enough with the theory, it’s now time to dive into the code and build a basic ReplicaSet -ish operator called  PodSet and which will take care of scaling up and down pods which run execute a  sleep 3600 in a busybox container. Nothing fancy here, but the focus of this article is building an operator, not diving too deep in the specificities of setting up a cluster of this or that application.

我们已经讲完了理论部分，让我们真正的实践起来，编写一个我们自己的ReplicaSet operator,我们叫它PodSet。它的功能就是增加和减少Pod的数量，它的基础镜像是busybox。

Installing the Operator SDK 安装Operator SDK

The first thing to do is download and install the SDK:

执行如下命令：

$ mkdir -p $GOPATH/src/github.com/operator-framework
$ cd $GOPATH/src/github.com/operator-framework
$ git clone <a class="markup--anchor markup--pre-anchor" href="https://github.com/operator-framework/operator-sdk" target="_blank" rel="nofollow noopener" data-href="https://github.com/operator-framework/operator-sdk">https://github.com/operator-framework/operator-sdk</a>
$ cd operator-sdk
$ git checkout v0.4.0
$ make dep
$ make install

Bootstrapping the Go project  初始化我们的代码

Next, let’s use the operator-sdk command to bootstrap the project:

我们用operator-sdk提供的命令来创建我们的Operator: podset-operator

# bootstrap the project and run "deps ensure" and "git init"
$ operator-sdk new podset-operator

# check the project structure
$ tree -I vendor
.
├── Gopkg.lock
├── Gopkg.toml
├── build
│   └── Dockerfile
├── cmd
│   └── manager
│       └── main.go
├── deploy
│   ├── operator.yaml
│   ├── role.yaml
│   ├── role_binding.yaml
│   └── service_account.yaml
├── pkg
│   ├── apis
│   │   └── apis.go
│   └── controller
│       └── controller.go
└── version
    └── version.go

Once this is done, we have the base layout for our project, which contains not only the go code to run the operator ( cmd/manager/main.go ), but also the Dockerfile to package the binary into a Docker image, and a set of YAML manifests to 1) deploy the Docker image and 2) create the Service Account to run the controller along with a role and role bindings to allow the operations (adding or removing pods, etc.).

我们执行完这个命令以后，我们就有一个基本的框架了，包含了必须的 go 代码以及Dockerfile ，以及一些镜像还有一些manifests ,这些manifest用来部署Docker 镜像，已经创建 执行我们controllers 的Service Account 以及相应的role bindings

Adding a CRD and a controller 增加CRD以及controller

Now, let’s create a CRD for the PodSet operator, as well as its associated controller. Since this will be the first version of the operator, it is a good practice to set the version to v1alpha1 :

现在我们来通过命令来增加我们PodSet 的CRD 和 与之配套的controllers，因为是版本1，我们就把版本设置为v1a1pha1

# Add a new API for the custom resource PodSet
$ operator-sdk add api --api-version=app.example.com/v1alpha1 --kind=PodSet
 
# Add a new controller that watches for PodSet
$ operator-sdk add controller --api-version=app.example.com/v1alpha1 --kind=PodSet

上边的两个命令又产生了一下go 代码和一系列YAML文件。最主要的改变如下：

Another set of YAML files and Go code were generated by these 2 commands, and the most noticeable changes are the following ones:

1. the new  deploy/crds  folder contains the Custom Resource Definition of the PodSet, along with an example of Custom Resource:  在文件夹deploy/crds下包含了Custom Resource Definition 的具体代码（Podset），例子如下：

$ cat deploy/crds/app_v1alpha1_podset_crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: podsets.app.example.com
spec:
  group: app.example.com
  names:
    kind: PodSet
    listKind: PodSetList
    plural: podsets
    singular: podset
  scope: Namespaced
  version: v1alpha1

$ cat deploy/crds/app_v1alpha1_podset_cr.yaml
apiVersion: app.example.com/v1alpha1
kind: PodSet
metadata:
  name: example-podset
spec:
  # Add fields here
  size: 3

The full name of the CRD is podsets.app.example.com but there are also various names associated with it ( PodSet ,  PodSetList ,  podsets and  podset ). They will be part of the extended cluster API and available in the  kubectl command line, as we’ll see later when deploying and running the operator.

我们CRD的完整名字： podsets.app.example.com 但是还是有一些自动生成的名字与之关联（PodSet,PodSetList,potsets, podset）,我们将会在后边用到这些

2. The pkg/apis/app/v1alpha1/podset_types.go defines the structure of the  PodSetSpec which is the expected state of the PodSet and which is specified by the user in the aforementioned  deploy/crds/app_v1alpha1_podset_cr.yaml file. It also defines the structure of the  PodSetStatus  which will be used to provide the observed state of the PodSet when the  kubectl describe command is executed. More on this later.

在pkg/apis/app/v1alpha1/podset_types.go这个文件中定义了PodSetSpec的架构，主要是指定了PodSet的期望状态，这部分是通过deploy/crds/app_v1alpha1_podset_cr.yaml这个文件用户自己设定的。文件中同时定义了PodSetStatus的架构，主要是为了提供当我们执行kubectl describe的时候显示给我们的内容

因为我们还没有增加我们的内容，所以现在架构内部是空的

type PodSetSpec struct {
        // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
        // Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
        // Add custom validation using kubebuilder tags: https://book.kubebuilder.io/beyond_basics/generating_crd.html
}
 
// PodSetStatus defines the observed state of PodSet
// +k8s:openapi-gen=true
type PodSetStatus struct {
        // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
        // Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
        // Add custom validation using kubebuilder tags: https://book.kubebuilder.io/beyond_basics/generating_crd.html
}

3. The scaffold file pkg/controller/podset/podset_controller.go is where we will put the business logic of the controller.

pkg/controller/podset/podset_controller.go文件中是我们存放controller逻辑代码的部分

The rest of the changes are the necessary plumbing to register the CRD and the controller.

下边我们来实际修改下如上两个文件

Implementing the business logic of the controller

为controller增加逻辑处理部分：

Out of the box, the controller code generated by the SDK creates a single pod if none with a given app label already exists. Although this is not exactly what we want here, it is a nonetheless a good starting point since it shows how to use the k8s API, whether to list the pods or to create new ones.

默认情况下，SDK给我们创建的controller创建了一个Pod,如果事先没有其它的pod存在的情况下（pod的标签为 app）

虽然这并不严格符合我们的需求，但是，这也是一个很好的例子让我们来了解如何使用k8s API ， 无论是List pod的列表或者创建一个新的Pod

The first thing we want to do is changing the PodSetSpec and  PodSetStatus Go structs by adding fields to store the number of replicas in the former and the name of the pods in the latter:

我们第一个要修改的就是PodSetSpec和 PodSetStatus Go的定义部分，增加存储副本数量的变量以及存储副本名称的变量类型：

type PodSetSpec struct {
  Replicas int32 `json:"replicas"`
}

type PodSetStatus struct {
  PodNames []string `json:"podNames"`
}

Each time we make a change in these structures, we need to run the operator-sdk generate k8s command to update the  pkg/apis/app/v1alpha1/zz_generated.deepcopy.go file accordingly.

每次我们修改如上两个结构体，我们需要执行命令：operator-sdk generate k8s 来更新pkg/apis/app/v1alpha1/zz_generated.deepcopy.go文件

Then, we need to configure the primary and secondary resources that the controller will monitor in the namespace. For our PodSet operator, the primary resource is the PodSet resource and the secondary resources are the pods in the namespace. By chance, we don’t have to do anything as this was already implemented in the generated code. Remember that by default, the controller operates on a PodSet resource and creates a pod.

接下来我们需要配置contoller需要监控主要和次要资源，对于我们这个例子来说，PodSet operator，主要资源就是PodSet 这个类型的资源，次要资源就是当前namespace下的pod。对于我们这个特例，我们不需要做任何的修改，因为默认的代码已经实现了这个功能

Lastly, we need to implement the logic of scaling up and down the pods and updating the custom resource status with the names of the pods. All of this happens in the Reconcile function of the controller.

最后，我们需要实现逻辑处理模块，如何增加和删除一个pod当客户定义的资源发生改变的时候，这个部分的实现代码全部在Reconcile部分

During the reconcile, the controller fetches the PodSet resource in the current namespace and compares the value of its Replica field with the actual number of Pods that match a specific set of labels (here,  app and  version ) to decide whether pods need to be created or deleted.

在reconcile部分，controller会获取当前namespace下所有的PodSet资源，然后对比他们的Replica字段和实际当前namespace下的Pod数量是否一致，然后决定是该增加还是减少

Instead of going into the details of the implementation of the controller’s Reconcile function ( the code is available on GitHub ), let’s focus on the key points to remember here:

代码详见： ( the code is available on GitHub )，我们这里只介绍一下关键的点：

Reconcile
controllerutil.SetControllerReference()

Building and publishing the operator

生成并发布我们的Operator

Let’s use the Operator SDK to build the Docker image containing the controller, and let’s push it to an online registry. We’ll use Quay.io in this case, but other registries would work as well:

我们用Operator SDK来创建包含controller 代码的Docker 镜像，然后让我们push到Quay.io 这个registry

# build the Docker image using the Operator SDK
$ operator-sdk build quay.io/xcoulon/podset-operator

# login to Quay.io
$ docker login -u xcoulon quay.io

# push the image to Docker Hub
$ docker push quay.io/xcoulon/podset-operator

Also, we need to update the operator.yaml manifest to use the new Docker image available on Quay.io:

# On Linux:
$ sed -i 's|REPLACE_IMAGE|quay.io/xcoulon/podset-operator|g' deploy/operator.yaml

# On OSX:
$ sed -i "" 's|REPLACE_IMAGE|quay.io/xcoulon/podset-operator|g' deploy/operator.yaml

原文地址：https://medium.com/faun/writing-your-first-kubernetes-operator-8f3df4453234