Kubernetes Secrets: A Secure Credential Store for Jenkins

栏目: IT技术 · 发布时间: 4年前

Cloud inResearch

Kubernetes Secrets: A Secure Credential Store for Jenkins

By: Vasumathy Seenuvasan

At eBay, we containerized Jenkins to provide a continuous build infrastructure on Kubernetes Clusters to power the ecommerce marketplace experience. Our goal was to leverage the capability of Kubernetes secrets, for managing the Jenkins credentials.

Kubernetes Secrets: A Secure Credential Store for Jenkins

Background

eBay.com is powered by applications that are built with Jenkins as the opinionated choice of build-machinery. We run Jenkins instances as a cloud native workload on the Kubernetes Clusters with server-agent mode, all of which are managed by the eBay cloud team.

The Jenkins servers provided to eBay development teams have a default credential plugin installed. The credentials stored on these Jenkins instances are used for various purposes, such as connecting to GitHub and third-party APIs. The credential plugin stores the user credentials, as well as the encryption key, on the disk in an encrypted format.

These credentials are very sensitive in nature and need to be stored safely and securely. However, storing the key and the credential on the same disk poses a risk. Jenkins instances are also generally shared across a team with ADMINISTER privileges , which increases the chances of team members knowing each other’s credentials.

Initially, we considered setting up a Vault 1 store, which is generally used in the community to secure the credentials in Jenkins. However, that would be a vendor lock-in. To remain vendor-neutral, we chose a different strategy. 

However, eBay has its own proprietary key management platform, which integrates with Kubernetes cluster’s control-plane and mounts secrets in a secure manner to the containers leveraging Kubernetes API standards. Since we are running Jenkins as workloads on Kubernetes, a simple solve was to standardize by using Kubernetes Secrets. Another benefit of this approach is that the same secrets can also be backed up to the in-house key management solution.

The challenge here was to integrate the Jenkins application with this proprietary key management platform. To solve the challenge, we developed a new plugin - Jenkins Credentials Secrets Plugin - which replaces the default credential plugin and is now available as an eBay open source project.

This article explains the design and process of a more secure way of storing Jenkins credentials as secrets on Kubernetes Clusters.

Scale of Operation

We have enabled this plugin to store credentials as Kubernetes Secrets on ~6000 Jenkins instances at eBay. We have also migrated all the credentials currently stored in these instances to Kubernetes secrets with ZERO down time for the end users - eBay application developers.

Assumptions

1.     The Jenkins master runs as a container in Kubernetes Cluster.

2.     The Jenkins master has a Kubernetes service account mounted to it.

Limitations

Only credentials under the Jenkins Global domain are supported by the plugin at this moment.

Design

When a credential is created by the user from a Jenkins UI or API call, a Kubernetes Secret is generated in a namespace (as specified by the plugin user) in the cluster with the required information from the credentials. The Secret specification’s “data” will hold the credential’s “sensitive” information. The Secret will have labels and annotations in the spec to store the below details.

Kubernetes Secrets: A Secure Credential Store for Jenkins

The secret name is generated as “sec”  + “-” + “UUID” + “-” + “JENKINS_INSTANCE_NAME”

Example : sec-<>-myCI.

We use UUID because the Kubernetes Secret object’s naming convention does not support all the characters allowed by Jenkins credentials (e.g. whitespaces and underscores). The credentials ID stored by the user is captured in the Secret’s labels, as shown in the above chart. We also capture other credential information, like description and type, and this list can be extended to attributes added in the future as well.

Secrets-To-Credentials Converter

The core component of this approach is the “Secrets-to-credentials converter” module. On Jenkins start up, the plugin gets the list of secrets that have the Jenkins name label selector. This is necessary so only the secrets belonging to the Jenkins in question are pulled. Converters for all of the below Jenkins supported credentials are implemented.

  • Username with password
  • Docker Host Certificate authentication
  • Kubernetes Service Account
  • OpenShift OAuth token
  • OpenShift Username and Password
  • SSH Username with Private Key
  • SSH Username with Private Key and Passphrase
  • Secret file
  • Secret text
  • Certificate

The credential type of a secret is identified from the label (jenkins.io/credentials-type), and the corresponding converter implementation is invoked to convert a secret to a credential. This credential is added to the Jenkins credentials Map offered by the Jenkins credentials plugin. This makes the approach transparent to the users, as they just see the same credential information on Jenkins UI, and all the secret conversion happens in the background.

Lifecycle Management of Secrets in Jenkins

Create: When users add credentials in the Jenkins UI, secrets will be created on the corresponding Kubernetes cluster in the namespace provided, and credentials will no longer be stored in `credentials.xml` on the disk.

Read: On Jenkins startup, credentials of the particular CI are loaded from the Kubernetes cluster (using label selector “jenkins.io/ci-name”).

Update: When a credential is updated, the corresponding secret is updated (using label selectors “jenkins.io/ci-name” and “jenkins.io/credential-id”).

Delete: When a credential is deleted, the corresponding secret is deleted (using label selectors “jenkins.io/ci-name” and “jenkins.io/credential-id”).

Secrets YAML

Username Password credential Type:

Kubernetes Secrets: A Secure Credential Store for Jenkins

The rest of the credential type YAML specs are available in the source code repository here:

https://github.com/eBay/kube-credentials-plugin/tree/master/credentialspecsamples

  • Base Credentials plugin - 2.1.19
  • Kubernetes plugin (dependency) - 1.1.4

We have open sourced this project, and the git repo can be found here:

https://github.com/eBay/kube-credentials-plugin

We welcome any PullRequests (PR) or github-issues on this repo. We have listed to-do items on the repo and welcome PRs to address them.

Vault is a product from HashiCorp:  https://www.vaultproject.io/

Tag: Cloud

Previous Article : Relation Embedding with Dihedral Group in Knowledge Graph


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

算法与数据结构

算法与数据结构

张乃孝 / 高等教育出版社 / 2006-1 / 31.00元

《算法与数据结构:C语言描述(第2版)》以数据结构为主线,算法为辅线组织教学内容。全书共分10章:绪论、线性表、字符串、栈与队列、二叉树与树、集合与字典、高级字典结构、排序、图和算法分析与设计。《算法与数据结构:C语言描述(第2版)》体系完整,概念清楚,内容充实,取材适当。第一版被列入“面向21世纪课程教材”,2004年被评为“北京市高等教育精品教材”,第二版被列入普通高等教育“十一五”国家级规划......一起来看看 《算法与数据结构》 这本书的介绍吧!

HTML 压缩/解压工具
HTML 压缩/解压工具

在线压缩/解压 HTML 代码

JS 压缩/解压工具
JS 压缩/解压工具

在线压缩/解压 JS 代码

RGB CMYK 转换工具
RGB CMYK 转换工具

RGB CMYK 互转工具