Modules Part 06: Vendoring

栏目: IT技术 · 发布时间: 4年前

内容简介:It’s no secret that I am a fan of vendoring when it’s reasonable and practical to use it for your application projects. I believe vendoring gives your application projects the most durability since the project owns every line of source code it needs to bui

Series Index

Introduction

It’s no secret that I am a fan of vendoring when it’s reasonable and practical to use it for your application projects. I believe vendoring gives your application projects the most durability since the project owns every line of source code it needs to build the applications. If you want a reproducible build without needing to rely on external services (like module mirrors) and being connected to the network, vendoring is the solution.

These are other benefits of vendoring:

  • If dependencies are removed from the VCS or somehow proxy servers lose modules, you are covered.
  • Upgrading dependencies can be seen by running diffs and you maintain a history.
  • You will have the ability to trace and debug your dependencies and test changes if necessary.
    • Once you run a go mod tidy and go mod vendor your changes will be replaced.

In this post, I will provide a history of Go’s support for vendoring and the changes in default behavior that have existed over time. I will also share how Go’s tooling is capable of maintaining backwards compatibility between versions. Finally, I will share how you may need to (over time) manually upgrade the version listed in the go.mod file to change the default behavior of future Go releases.

Running Different Versions Of Go

To show you the differences in default behavior between Go 1.13 and Go 1.14, I need to be able to run both versions of the tooling on my machine at the same time. I’ve already installed Go 1.14.2 on my machine at the time I published this post and I access that version using the traditional go front end. However for this post, I also need to run a Go 1.13 environment. So how can I do that without disrupting my current development environment?

Luckily, the Go team publishes version downloads that give you a specific binary for any version of Go you want to use, including Go Tip .

Figure 1

Modules Part 06: Vendoring

Figure 1 shows a screenshot of the Go 1.13.10 page from the download server. It shows the instructions for building a binary that can be used to build and test your Go code using Go 1.13.10.

Listing 1

$ cd $HOME
$ go get golang.org/dl/go1.13.10

OUTPUT
go: downloading golang.org/dl v0.0.0-20200408221700-d6f4cf58dce2
go: found golang.org/dl/go1.13.10 in golang.org/dl v0.0.0-20200408221700-d6f4cf58dce2


$ go1.13.10 download

OUTPUT
Downloaded   0.0% (    14448 / 121613848 bytes) ...
Downloaded   9.5% ( 11499632 / 121613848 bytes) ...
Downloaded  30.8% ( 37436528 / 121613848 bytes) ...
Downloaded  49.2% ( 59849840 / 121613848 bytes) ...
Downloaded  69.3% ( 84262000 / 121613848 bytes) ...
Downloaded  90.3% (109804656 / 121613848 bytes) ...
Downloaded 100.0% (121613848 / 121613848 bytes)
Unpacking /Users/bill/sdk/go1.13.10/go1.13.10.darwin-amd64.tar.gz ...
Success. You may now run 'go1.13.10'


$ go1.13.10 version

OUTPUT
go version go1.13.10 darwin/amd64


$ go version

OUTPUT
go version go1.14.2 darwin/amd64

Listing 1 shows how after running the go get command for version 1.13.10 of Go and performing the download call, I can now use Go 1.13.10 on my machine without any disruption to my Go 1.14.2 installation.

If you want to remove any version of Go from your machine, you can find the specific binaries in your $GOPATH/bin folder and all the supporting files will be found in $HOME/sdk .

Listing 2

$ cd $GOPATH/bin
$ l

OUTPUT
-rwxr-xr-x   1 bill  staff   7.0M Apr 11 10:51  go 1.13.10
-rwxr-xr-x   1 bill  staff   2.3M Jan  6 11:02 gotip


$ cd $HOME
$ l sdk/

OUTPUT
drwxr-xr-x  22 bill  staff   704B Apr 11 10:52 go1.13.10
drwxr-xr-x  24 bill  staff   768B Feb 26 01:59 gotip

Quick Vendoring Tutorial

The Go tooling did a great job minimizing the workflow impacts to manage and vendor an application project’s dependencies. It requires two commands: tidy and vendor .

Listing 3

$ go mod tidy

Listing 3 shows the tidy command that helps to keep the dependencies listed in your module files accurate. Some editors (like VS Code and GoLand) provide support to update the module files during development but that doesn’t mean the modules files will be clean and accurate once you have everything working. I recommend running the tidy command before you commit and push any code back to your VCS.

If you want to vendor those dependencies as well, then run the vendor command after tidy .

Listing 4

$ go mod vendor

Listing 4 shows the vendor command. This command creates a vendor folder inside your project that contains the source code for all the dependencies (direct and indirect) that the project requires to build and test the code. This command should be run after running tidy to keep your vendor folder in sync with your module files. Make sure to commit and push the vendor folder to your VCS.

GOPATH or Module Mode

In Go 1.11, a new mode was added to the Go tooling called Module-Mode. When the Go tooling is operating in Module-Mode, the module system is used to find and build code. When the Go tooling is operating in GOPATH-Mode, the traditional GOPATH system continues to be used to find and build code. One of the bigger struggles I have had with the Go tooling is knowing what mode will be used by default between the different versions. Then knowing what configuration changes and flags I need to keep my builds consistent.

To understand the history and the semantic changes that have occurred over the past 4 versions of Go, it’s good to have a refresher on these modes.

Go 1.11

A new environment variable was introduced called GO111MODULE whose default was auto . This variable would determine if the Go tooling would use Module-Mode or GOPATH-Mode depending on where the code was located (inside or outside of GOPATH). To force one mode or the other, you would set this variable to on or off . When it came to vendor folders, Module-Mode would ignore a vendor folder by default and build dependencies against the module cache.

Go 1.12

The default setting for GO111MODULE is still auto and the Go tooling continues to determine Module-Mode or GOPATH-Mode depending on where the code is located (inside or outside of GOPATH). When it comes to vendor folders, Module-Mode would still ignore a vendor folder by default and build dependencies against the module cache.

Go 1.13

The default setting for GO111MODULE is now on if a go.mod file is located in the project (regardless if the project is located inside or outside of GOPATH). Module-Mode would still ignore a vendor folder by default and build dependencies against the module cache.

Go 1.14

The default setting for GO111MODULE is still on if a go.mod file is located in the project (regardless if the project is located inside or outside of GOPATH). However, if a vendor folder exists, it will be used by default to build dependencies instead of the module cache. In addition, the go command verifies that the project’s vendor/modules.txt file is consistent with its go.mod file.

Backwards Compatibility Between Versions

The change in Go 1.14 to use the vendor folder by default over the module cache is the behavior I wanted for my projects. Initially I thought I could just use Go 1.14 to build against my existing projects and it would be enough, but I was wrong. After my first build with Go 1.14 and not seeing the vendor folder being respected, I learned that the Go tooling reads the go.mod file for version information and maintains backwards compatibility with that version listed. I had no idea, but it is clearly expressed in the release notes for Go 1.14.

https://golang.org/doc/go1.14#go-command

When the main module contains a top-level vendor directory and its go.mod file specifies Go 1.14 or higher, the go command now defaults to -mod=vendor for operations that accept that flag.

In order to use the new default behavior for vendoring, I was going to need to upgrade the version information in the go.mod file from Go 1.13 to Go 1.14. This is something I quickly did.

Small Demo

To show you the behavior of Go 1.13 and Go 1.14, and how the tooling maintains backwards compatibility, I am going to use the service project. I will show you how changing the version listed in go.mod will change the default behavior of the Go tooling.

To start, I will clone the service project outside of my GOPATH.

Listing 5

$ cd $HOME/code
$ git clone https://github.com/ardanlabs/service
$ cd service
$ code .

Listing 5 shows the commands to clone the project and open the project in VS Code.

Listing 6

$ ls -l vendor/

OUTPUT
total 8
drwxr-xr-x   3 bill  staff    96 Mar 26 16:01 contrib.go.opencensus.io
drwxr-xr-x  14 bill  staff   448 Mar 26 16:01 github.com
drwxr-xr-x  20 bill  staff   640 Mar 26 16:01 go.opencensus.io
drwxr-xr-x   3 bill  staff    96 Mar 26 16:01 golang.org
drwxr-xr-x   3 bill  staff    96 Mar 26 16:01 gopkg.in
-rw-r--r--   1 bill  staff  2860 Mar 26 16:01 modules.txt

Listing 6 shows the listing of the vendor folder for the service project. You can see directories for some of the popular VCS sites that exist today as well as several vanity domains. All the code the project depends on to build and test are located inside the vendor folder.

Next, I will manually change the go.mod file back to version 1.13. This will allow me to show you the behavior I experienced when I used Go 1.14 for the first time against this project.

Listing 7

module github.com/ardanlabs/service

go 1.13   // I just changed this from go 1.14 to go 1.13

Listing 7 shows the change I am making to the go.mod file (switching out go 1.14 for go 1.13 ).

Go 1.13

On this first build, I will use Go 1.13.10 to build the sales-api application. Remember, the go.mod file is listing Go 1.13 as the compatible version for this project.

Listing 8

$ cd service/cmd/sales-api
$ go1.13.10 clean -modcache
$ go1.13.10 build

OUTPUT
go: downloading contrib.go.opencensus.io/exporter/zipkin v0.1.1
. . .
go: finding github.com/leodido/go-urn v1.2.0

Listing 8 shows how I navigated to the application folder, cleaned out my local module cache and then performed a build using Go 1.13.10. Notice how the Go tooling downloaded all the dependencies back into my module cache in order to build the binary. The vendor folder was ignored.

To get Go 1.13 to respect the vendor folder, I need to use the -mod=vendor flag when building and testing.

Listing 9

$ go1.13.10 clean -modcache
$ go1.13.10 build -mod=vendor

OUTPUT

Listing 9 shows how I am now using the -mod=vendor flag on the build call. This time the module cache is not re-populated with the missing modules and the code in the vendor folder is respected.

Go 1.14

This time I will run the build command using Go 1.14.2 without the use of the -mod=vendor flag.

Listing 10

$ go clean -modcache
$ go build

OUTPUT
go: downloading github.com/openzipkin/zipkin-go v0.2.2
. . .
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e

Listing 10 shows what happens when I use Go 1.14 to build the project. The vendor folder is not being respected because the tooling is operating with Go 1.13 semantics. This is because the go.mod file is listing Go 1.13 as the compatible version for the project. When I saw this for the first time I was shocked. This is what started my investigation.

If I switch the go.mod file to version 1.14, the default mode of the Go 1.14 tooling will switch to respect the vendor folder by default.

Listing 11

module github.com/ardanlabs/service

go 1.14   // I just changed this from go 1.13 to go 1.14

Listing 11 shows the change to the go.mod file back to 1.14. I will clear the module cache again and run the build command again using Go 1.14.

Listing 12

$ go clean -modcache
$ go build

OUTPUT

Listing 12 shows that the module cache is not re-populated this time on the call to go build using Go 1.14. Which means the vendor folder is being respected, without the need of the -mod=vendor flag. The default behavior has changed because the module file is listing Go 1.14.

Future Changes For Vendoring and Modules

Thanks to John Reese , here is a link to a discussion about the tooling maintaining backwards compatibility between different versions of Go based on what is listed in the go.mod file. John was instrumental in making sure the post was accurate and flowed correctly.

https://github.com/golang/go/issues/30791

There is more support coming for vendoring that will follow in future releases. One such feature being discussed is about validating the code in the vendor folder to find situations where the code has been changed.

https://github.com/golang/go/issues/27348

I have to thank Chris Hines for reminding me about the default behaviors in the previous versions of Go and how that has been promoted with each new release. Chris also provided some interesting links that share some history and other cool things coming to the Go tooling for modules. Chris was instrumental in making sure the post was accurate and flowed correctly.

Conclusion

This post is a result of me being surprised that the version listed in go.mod was affecting the default behavior of the Go tooling. In order to gain access to the new default vendoring behavior in Go 1.14 that I wanted, I had to manually upgrade the version listed in go.mod from 1.13 to 1.14.

I haven’t formed any concrete opinions about the Go tooling using the version information in go.mod to maintain backwards compatibility between versions. The Go tooling has never been tied to the Go compatibility promise and so this was unexpected to me. Maybe this is the start of something great and moving forward the Go tooling can grow without the Go community worrying if their builds, tests and workflows will break when a new version of the Go tooling is released.

If you have any opinions, I’d love to hear them on Twitter.


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

文明之光 (第三册)

文明之光 (第三册)

吴军 / 人民邮电出版社 / 2015-1-1 / 59

【《文明之光》系列荣获由中宣部、中国图书评论学会和中央电视台联合推选的2014“中国好书”奖】 吴军博士从对人类文明产生了重大影响却在过去被忽略的历史故事里,选择了有意思的几十个片段特写,以人文和科技、经济结合的视角,有机地展现了一幅人类文明发展的宏大画卷。 《文明之光 》系列大致按照从地球诞生到近现代的顺序讲述了人类文明进程的各个阶段,每个章节相对独立,全景式地展现了人类文明发展历程......一起来看看 《文明之光 (第三册)》 这本书的介绍吧!

JS 压缩/解压工具
JS 压缩/解压工具

在线压缩/解压 JS 代码

在线进制转换器
在线进制转换器

各进制数互转换器

HSV CMYK 转换工具
HSV CMYK 转换工具

HSV CMYK互换工具