jib源码分析之Step实现

简介（持续更新）

• retrieveTargetRegistryCredentialsStep
• pullAndCacheBaseImageLayersStep
• PullAndCacheBaseImageLayerStep
• pushBaseImageLayersStep
• buildAndCacheApplicationLayers

阅读本文前，建议事先了解下jib源码分析及应用 jib源码分析之细节

以常用 的forBuildToDockerRegistry 包含的步骤来分析

retrieveTargetRegistryCredentialsStep

这个步骤的本质是 获取Credential ，也就是用户名和密码，无需远程访问。

class RetrieveRegistryCredentialsStep implements AsyncStep<Credential>, Callable<Credential> {
  	public Credential call() throws CredentialRetrievalException {
  		...
  		Optional<Credential> optionalCredential = credentialRetriever.retrieve();
  		...
  	}	
}

@FunctionalInterface
public interface CredentialRetriever {
  	Optional<Credential> retrieve() throws CredentialRetrievalException;
}

CredentialRetriever 是构建 RegistryImage 时拿到的

public class RegistryImage implements SourceImage, TargetImage {
	public RegistryImage addCredential(String username, String password) {
    	addCredentialRetriever(() -> Optional.of(Credential.basic(username, password)));
    	return this;
  	}
}

public class Credential {
	private final String username;
  		private final String password;
}

authenticatePushStep

Sends the authentication request and retrieves the Bearer authorization token.

这部分要参照官网 Token Authentication Specification

1. Attempt to begin a push/pull operation with the registry.
2. If the registry requires authorization it will return a 401 Unauthorized HTTP response with information on how to authenticate.
3. The registry client makes a request to the authorization service for a Bearer token.
4. The authorization service returns an opaque Bearer token representing the client’s authorized access.
5. The client retries the original request with the Bearer token embedded in the request’s Authorization header.
6. The Registry authorizes the client by validating the Bearer token and the claim set embedded within it and begins the push/pull session as usual.

authenticatePushStep 的结果是 得到一个 Authorization

public class Authorization {
  	private final String scheme;
  	private final String token;
}

Authorization 也可以根据Credential 直接构建。

PullBaseImageStep

先 try with no credentials ，行就直接结束了。If failed, then, retrieve base registry credentials and try with retrieved credentials. 构造一个retrieveBaseRegistryCredentialsStep，获取Credential，进而构建Authorization，然后再干活 pullBaseImage

对于返回值，schema version= 1时，则只是将得到的 Manifest 数据 转换为 Image<Layer> 。若schema version= 2，则需要拉取 一个blob（其中包括containerConfiguration），然后也转换为 Image<Layer> 并返回

可以看到，PullBaseImageStep 主要是拉取Image 元数据，如果本地没有base image 缓存，则要交给下一个Step

pullAndCacheBaseImageLayersStep

public ImmutableList<PullAndCacheBaseImageLayerStep> call() {
    BaseImageWithAuthorization pullBaseImageStepResult = NonBlockingSteps.get(pullBaseImageStep);
    ImmutableList<Layer> baseImageLayers = pullBaseImageStepResult.getBaseImage().getLayers();
    ...
  	ImmutableList.Builder<PullAndCacheBaseImageLayerStep> pullAndCacheBaseImageLayerStepsBuilder =
      	ImmutableList.builderWithExpectedSize(baseImageLayers.size());
  	for (Layer layer : baseImageLayers) {
    	pullAndCacheBaseImageLayerStepsBuilder.add(
        	new PullAndCacheBaseImageLayerStep(...,
            	layer.getBlobDescriptor().getDigest(),
            	pullBaseImageStepResult.getBaseImageAuthorization()));
  	}
  return pullAndCacheBaseImageLayerStepsBuilder.build();
  	}

pullAndCacheBaseImageLayersStep.call 真正干活的是PullAndCacheBaseImageLayerStep

PullAndCacheBaseImageLayerStep

public CachedLayer call() throws IOException, CacheCorruptedException {

...
    try (...) {
      	Cache cache = buildConfiguration.getBaseImageLayersCache();
      	// Checks if the layer already exists in the cache.
      	Optional<CachedLayer> optionalCachedLayer = cache.retrieve(layerDigest);
      	if (optionalCachedLayer.isPresent()) {
        	buildConfiguration.getEventDispatcher().dispatch(new ProgressEvent(progressAllocation, 1));
        	return optionalCachedLayer.get();
      	}
      	RegistryClient registryClient = ...
      	CachedLayer cachedLayer = cache.writeCompressedLayer(registryClient.pullBlob(layerDigest));
      	...
      	return cachedLayer;
    }
  	}

本地有则返回，无则下载

pushBaseImageLayersStep

实际工作的是 PushLayersStep

public ImmutableList<AsyncStep<PushBlobStep>> call() throws ExecutionException {
    try (...) {
        ImmutableList<? extends AsyncStep<? extends CachedLayer>> cachedLayer =
            NonBlockingSteps.get(cachedLayerStep);
        // Constructs a PushBlobStep for each layer.
        ImmutableList.Builder<AsyncStep<PushBlobStep>> pushBlobStepsBuilder = ImmutableList.builder();
        for (AsyncStep<? extends CachedLayer> cachedLayerStep : cachedLayer) {
          ListenableFuture<PushBlobStep> pushBlobStepFuture =
              Futures.whenAllSucceed(cachedLayerStep.getFuture())
                  .call(() -> makePushBlobStep(cachedLayerStep), listeningExecutorService);
          pushBlobStepsBuilder.add(() -> pushBlobStepFuture);
        }
        return pushBlobStepsBuilder.build();
    }
}

从 pullAndCacheBaseImageLayersStep 可以拿到一系列 PullAndCacheBaseImageLayerStep，对于每个PullAndCacheBaseImageLayerStep，拿到其执行结果 CachedLayer，然后构建 PushBlobStep

从这里也可以看到 ，jib 刚拉完base image，便又尝试重新push 了一下，意图何在呢？

buildAndCacheApplicationLayers

是一系列 BuildAndCacheApplicationLayerStep 的集合

jib 应用demo

Jib.from("busybox")
   .addLayer(Arrays.asList(Paths.get("helloworld.sh")), AbsoluteUnixPath.get("/")) 
   .setEntrypoint("sh", "/helloworld.sh")
   .containerize(
       Containerizer.to(RegistryImage.named("gcr.io/my-project/hello-from-jib")
                                     .addCredential("myusername", "mypassword")));

addLayer 的方法签名 JibContainerBuilder addLayer(List<Path> files, AbsoluteUnixPath pathInContainer) 表示将文件加入到 容器的特定目录下。最终一个 layerConfiguration 持有文件在host 上的地址、在容器内的地址和文件的访问权限(一个LayerEntry 可以表示的内容)

public CachedLayer call() throws IOException, CacheCorruptedException {
    ...
    try (...) {
        Cache cache = buildConfiguration.getApplicationLayersCache();
        // Don't build the layer if it exists already.
        Optional<CachedLayer> optionalCachedLayer = cache.retrieve(layerConfiguration.getLayerEntries());
        if (optionalCachedLayer.isPresent()) {
        	return optionalCachedLayer.get();
        }
        Blob layerBlob = new ReproducibleLayerBuilder(layerConfiguration.getLayerEntries()).build();
        CachedLayer cachedLayer =
          cache.writeUncompressedLayer(layerBlob, layerConfiguration.getLayerEntries());
        ...
        return cachedLayer;
    }
}

本地加入的layer 都是UncompressedLayer，先对其计算 digest 查询本地是否存在，如果已存在则直接返回对应的CachedLayer，否则写入到cache 目录 再返回。

对barge 流程的优化

不要 清理 jib-cache

个人微信订阅号