内容简介:Russ CoxBrad FitzpatrickJuly 2020
Go command support for embedded static assets (files) — Draft Design
Russ Cox
Brad Fitzpatrick
July 2020
This is a Draft Design , not a formal Go proposal, because it describes a potential large change that addresses the same need as many third-party packages and could affect their implementations (hopefully by simplifying them!). The goal of circulating this draft design is to collect feedback to shape an intended eventual proposal.
This design builds upon the file system interfaces draft design .
We are using this change to experiment with new ways to scale discussions about large changes. For this change, we will use a Go Reddit thread to manage Q&A, since Reddit’s threading support can easily match questions with answers and keep separate lines of discussion separate.
There is a video presentation of this draft design.
The prototype code is available for trying out.
Abstract
There are many tools to embed static assets (files) into Go binaries. All depend on a manual generation step followed by checking in the generated files to the source code repository. This draft design eliminates both of these steps by adding support for embedded static assets to the go
command itself.
Background
There are many tools to embed static assets (files) into Go binaries. One of the earliest and most popular was github.com/jteeuwen/go-bindata and its forks, but there are many more, including (but not limited to!):
- github.com/alecthomas/gobundle
- github.com/GeertJohan/go.rice
- github.com/go-playground/statics
- github.com/gobuffalo/packr
- github.com/knadh/stuffbin
- github.com/mjibson/esc
- github.com/omeid/go-resources
- github.com/phogolabs/parcello
- github.com/pyros2097/go-embed
- github.com/rakyll/statik
- github.com/shurcooL/vfsgen
- github.com/UnnoTed/fileb0x
- github.com/wlbr/templify
- perkeep.org/pkg/fileembed
Clearly there is a widespread need for this functionality.
The go
command is the way Go developers build Go programs. Adding direct support to the go
command for the basic functionality of embedding will eliminate the need for some of these tools and at least simplify the implementation of others.
It is an explicit goal to eliminate the need to generate new Go source files for the assets and commit those source files to version control.
Another explicit goal is to avoid a language change. To us, embedding static assets seems like a tooling issue, not a language issue. Avoiding a language change also means we avoid the need to update the many tools that process Go code, among them goimports, gopls, and staticcheck.
It is important to note that as a matter of both design and policy, the go
command never runs user-specified code during a build
. This improves the reproducibility, scalability, and security of builds. This is also the reason that go generate
is a separate manual step rather than an automatic one. Any new go
command support for embedded static assets is constrained by that design and policy choice.
Another goal is that the solution apply equally well to the main package and to its dependencies, recursively. For example, it would not work to require the developer to list all embeddings on the go build
command line, because that would require knowing the embeddings needed by all of the dependencies of the program being built.
Another goal is to avoid designing novel APIs for accessing files. The API for accessing embedded files should be as close as possible to *os.File,
the existing standard library API for accessing native operating-system files.
This design adds direct support for embedded static assets into the go command itself, building on the file system draft design.
That support consists of:
-
A new
//go:embed
comment directive naming the files to embed. -
A new
embed
package, which defines the typeembed.Files
, the public API for a set of embedded files. Theembed.Files
implementsfs.FS
from the file system interfaces draft design , making it directly usable with packages likenet/http
andhtml/template
. - Go command changes to process the directives.
-
Changes to
go/build
andgolang.org/x/tools/go/packages
to expose information about embedded files.
//go:embed directives
A new package embed
, described in detail below, provides the type embed.Files
. One or more //go:embed
directives above a variable declaration of that type specify which files to embed, in the form of a glob pattern. For example:
package server // content holds our static web server content. //go:embed image/* template/* //go:embed html/index.html var content embed.Files
The go
command will recognize the directives and arrange for the declared embed.Files
variable (in this case, content
) to be populated with the matching files from the file system.
The //go:embed
directive accepts multiple space-separated glob patterns for brevity, but it can also be repeated, to avoid very long lines when there are many patterns. The glob patterns are in the syntax of path.Match
; they must be unrooted, and they are interpreted relative to the package directory containing the source file. The path separator is a forward slash, even on Windows systems. To allow for naming files with spaces in their names, patterns can be written as Go double-quoted or back-quoted string literals.
If a pattern names a directory, all files in the subtree rooted at that directory are embedded (recursively), so the above example is equivalent to:
package server // content is our static web server content. //go:embed image template html/index.html var content embed.Files
An embed.Files
variable can be exported or unexported, depending on whether the package wants to make the file set available to other packages. Similarly, an embed.Files
variable can be a global or a local variable, depending on what is more convenient in context.
- When evaluating patterns, matches for empty directories are ignored (because empty directories are never packaged into a module).
- It is an error for a pattern not to match any file or non-empty directory.
- It is not an error to repeat a pattern or for multiple patterns to match a particular file; such a file will only be embedded once.
-
It is an error for a pattern to contain a
..
path element. -
It is an error for a pattern to contain a
.
path element (to match everything in the current directory, use*
). -
It is an error for a pattern to match files outside the current module or that cannot be packaged into a module, like
.git/*
or symbolic links (or, as noted above, empty directories). -
It is an error for a
//go:embed
directive to appear except before a declaration of anembed.Files
. (More specifically, each//go:embed
directive must be followed by avar
declaration of a variable of typeembed.Files
, with only blank lines and other//
-comment-only lines between the//go:embed
and the declaration.) -
It is an error to use
//go:embed
in a source file that does not import"embed"
(the only way to violate this rule involves type alias trickery). -
It is an error to use
//go:embed
in a module declaring a Go version before Go 1. N , where N is the Go version that adds this support. -
It is not
an error to use
//go:embed
with local variables declared in functions. -
It is not
an error to use
//go:embed
in tests. -
It is not
an error to declare an
embed.Files
without a//go:embed
directive. That variable simply contains no embedded files.
The embed package
The new package embed
defines the Files
type:
// A Files provides access to a set of files embedded in a package at build time. type Files struct { … }
The Files
type provides an Open
method that opens an embedded file, as an fs.File
:
func (f Files) Open(name string) (fs.File, error)
By providing this method, the Files
type implements fs.FS
and can be used with utility functions such as fs.ReadFile
, fs.ReadDir
, fs.Glob
, and fs.Walk
.
As a convenience for the most common operation on embedded files, the Files
type also provides a ReadFile
method:
func (f Files) ReadFile(name string) ([]byte, error)
Because Files
implements fs.FS
, a set of embedded files can also be passed to template.ParseFS
, to parse embedded templates, and to http.HandlerFS
, to serve a set of embedded files over HTTP.
Go command changes
The go
command will change to process //go:embed
directives and pass appropriate information to the compiler and linker to carry out the embedding.
The go
command will also add six new fields to the Package
struct exposed by go list
:
EmbedPatterns []string EmbedFiles []string TestEmbedPatterns []string TestEmbedFiles []string XTestEmbedPatterns []string XTestEmbedFiles []string
The EmbedPatterns
field lists all the patterns found on //go:embed
lines in the package’s non-test source files; TestEmbedPatterns
and XTestEmbedPatterns
list the patterns in the package’s test source files (internal and external tests, respectively).
The EmbedFiles
field lists all the files, relative to the package directory, matched by the EmbedPatterns
; it does not specify which files match which pattern, although that could be reconstructed using path.Match
. Similarly, TestEmbedFiles
and XTestEmbedFiles
list the files matched by TestEmbedPatterns
and XTestEmbedPatterns
. These file lists contain only files; if a pattern matches a directory, the file list includes all the files found in that directory subtree.
go/build and golang.org/x/tools/go/packages
In the go/build
package, the Package
struct adds only EmbedPatterns
, TestEmbedPatterns
, and XTestEmbedPatterns
, not EmbedFiles
, TestEmbedFiles
, or XTestEmbedFiles
, because the go/build
package does not take on the job of matching patterns against a file system.
In the golang.org/x/tools/go/packages
package, the Package
struct adds one new field: EmbedFiles
lists the embedded files. (If embedded files were added to OtherFiles
, it would not be possible to tell whether a file with a valid source extension in that list—for example, x.c
—was being built or embedded or both.)
Rationale
As noted above, the Go ecosystem has many tools for embedding static assets, too many for a direct comparison to each one. Instead, this section lays out the affirmative rationale in favor of each of the parts of the design. Each subsection also addresses the points raised in the helpful preliminary discussion on golang.org/issue/35950. (The Appendix at the end of this document makes direct comparisons with a few existing tools and examines how they might be simplified.)
It is worth repeating the goals and constraints mentioned in the background section:
- No generated Go source files.
- No language change, so no changes to tools processing Go code.
-
The
go
command does not run user code duringgo build
. - The solution must apply as well to dependency packages as it does to the main package.
- The APIs for accessing embedded files should be close to those for operating-system files.
Approach
The core of the design is the new embed.Files
type annotated at its use with the new //go:embed
directive:
//go:embed *.jpg var jpgs embed.Files
This is different from the two approaches mentioned at the start of the preliminary discussion on golang.org/issue/35950. In some ways it is a combination of the best parts of each.
The first approach mentioned was a directive along the lines of
//go:genembed Logo logo.jpg
that would be replaced by a generated func Logo() []byte
function, or some similar accessor.
A significant drawback of this approach is that it changes the way programs are type-checked: you can’t type-check a call to Logo
unless you know what that directive turns into. There is also no obvious place to write the documentation for the new Logo
function. In effect, this new directive ends up being a full language change: all tools processing Go code have to be updated to understand it.
The second approach mentioned was to have a new importable embed
package with standard Go function definitions, but the functions are in effect executed at compile time, as in:
var Static = embed.Dir("static") var Logo = embed.File("images/logo.jpg") var Words = embed.CompressedReader("dict/words")
This approach fixes the type-checking problem—it is not a full language change—but it still has significant implementation complexity. The go
command would need to parse the entire Go source file to understand which files need to be made available for embedding. Today it only parses up to the import block, never full Go expressions. It would also be unclear to users what constraints are placed on the arguments to these special calls: they look like ordinary Go calls but they can only take string literals, not strings computed by Go code, and probably not even named constants (or else the go
command would need a full Go expression evaluator).
Much of the preliminary discussion focused on deciding between these two approaches. This design combines the two and avoids the drawbacks of each.
The //go:embed
comment directive follows the established convention for Go build system and compiler directives. The directive is easy for the go
command to find, and it is clear immediately that the directive can’t refer to a string computed by a function call, nor to a named constant.
The embed.Files
type is plain Go code, defined in a plain Go package embed
. All tools that type-check Go code or run other analysis on it can understand the code without any special handling of the //go:embed
directive.
The explicit variable declaration provides a clear place to write documentation:
// jpgs holds the static images used on the home page. //go:embed *.jpg var jpgs embed.Files
(As of Go 1.15, the //go:embed
line is not considered part of the doc comment.)
The explicit variable declaration also provides a clear way to control whether the embed.Files
is exported. A data-only package might do nothing but export embedded files, like:
package web // Styles holds the CSS files shared among all our websites. //go:embed style/*.css var Styles embed.Files
Modules versus packages
In the preliminary discussion, a few people suggested specifying embedded files using a new directive in go.mod
.
The design of Go modules, however, is that go.mod
serves only to describe information about the module’s version requirements, not other details of a particular package. It is not a collection of general-purpose metadata. For example, compiler flags or build tags would be inappropriate in go.mod
. For the same reason, information about one package’s embedded files is also inappropriate in go.mod
: each package’s individual meaning should be defined by its Go sources. The go.mod
is only for deciding which versions of other packages are used to resolve imports.
Placing the embedding information in the package has benefits that using go.mod
would not, including the explicit declaration of the file set, control over exportedness, and so on.
Glob patterns
It is clear that there needs to be some way to give a pattern of files to include, such as *.jpg
. This design adopts glob patterns as the single way to name files for inclusion. Glob patterns are common to developers from command shells, and they are already well-defined in Go, in the APIs for path.Match
, filepath.Match
, and filepath.Glob
. Nearly all file names are valid glob patterns matching only themselves; using globs avoids the need for separate //go:embedfile
and //go:embedglob
directives. (This would not be the case if we used, say, Go regular expressions as provided by the regexp
package.)
Directories versus ** glob patterns
In some systems, the glob pattern **
is like *
but can match multiple path elements. For example images/**.jpg
matches all .jpg
files in the directory tree rooted at images/
. This syntax is not available in Go’s path.Match
or in filepath.Glob
, and it seems better to use the available syntax than to define a new one. The rule that matching a directory includes all files in that directory tree should address most of the need for **
patterns. For example, //go:embed images
instead of //go:embed images/**.jpg
. It’s not exactly the same, but hopefully good enough.
If at some point in the future it becomes clear that **
glob patterns are needed, the right way to support them would be to add them to path.Match
and filepath.Glob
; then the //go:embed
directives would get them for free.
Dot-dot, module boundaries, and file name restrictions
In order to build files embedded in a dependency, the raw files themselves must be included in module zip files. This implies that any embedded file must be in the module’s own file tree. It cannot be in a parent directory above the module root (like ../../../etc/passwd
), it cannot be in a subdirectory that contains a different module, and it cannot be in a directory that would be left out of the module (like .git
). Another implication is that it is not possible to embed two different files that differ only in the case of their file names, because those files would not be possible to extract on a case-insensitive system like Windows or macOS. So you can’t embed two files with different casings, like this:
//go:embed README readme
But //go:embed dir/README other/readme
is fine.
Because embed.Files
implements fs.FS
, it cannot provide access to files with names beginning with ..
, so files in parent directories are also disallowed entirely, even when the parent directory named by ..
does happen to be in the same module.
Codecs and other processing
The preliminary discussion raised a large number of possible transformations that might be applied to files before embedding, including: data compression, JavaScript minification, TypeScript compilation, image resizing, generation of sprite maps, UTF-8 normalization, and CR/LF normalization.
It is not feasible for the go
command to anticipate or include all the possible transformations that might be desirable. The go
command is also not a general build system; in particular, remember the design constraint that it never runs user programs during a build. These kinds of transformations are best left to an external build system, such as Make or Bazel, which can write out the exact bytes that the go
command should embed.
A more limited version of this suggestion was to gzip-compress the embedded data and then make that compressed form available for direct use in HTTP servers as gzipped response content. Doing this would force the use of (or at least support for) gzip and compressed content, making it harder to adjust the implementation in the future as we learn more about how well it works. Overall this seems like overfitting to a specific use case.
The simplest approach is for Go’s embedding feature to store plain files, let build systems or third-party packages take care of preprocessing before the build or postprocessing at runtime. That is, the design focuses on providing the core functionality of embedding raw bytes into the binary for use at run-time, leaving other tools and packages to build on a solid foundation.
Compression to reduce binary size
A popular question in the preliminary discussion was whether the embedded data should be stored in compressed or uncompressed form in the binary. This design carefully avoids assuming an answer to that question. Instead, whether to compress can be left as an implementation detail.
Compression carries the obvious benefit of smaller binaries. However, it also carries some less obvious costs. Most compression formats (in particular gzip and zip) do not support random access to the uncompressed data, but an http.File
needs random access ( ReadAt
, Seek
) to implement range requests. Other uses may need random access as well. For this reason, many of the popular embedding tools start by decompressing the embedded data at runtime. This imposes a startup CPU cost and a memory cost. In contrast, storing the embedded data uncompressed in the binary supports random access with no startup CPU cost. It also reduces memory cost: the file contents are never stored in the garbage-collected heap, and the operating system efficiently pages in necessary data from the executable as that data is accessed, instead of needing to load it all at once.
Most systems have more disk than RAM. On those systems, it makes very little sense to make binaries smaller at the cost of using more memory (and more CPU) at run time.
On the other hand, projects like TinyGo and U-root target systems with more RAM than disk or flash. For those projects, compressing assets and using incremental decompression at runtime could provide significant savings.
Again, this design allows compression to be left as an implementation detail. The detail is not decided by each package author but instead could be decided when building the final binary. Future work might be to add -embed=compress
as a go
build option for use in limited environments.
Go command changes
Other than support for //go:embed
itself, the only user-visible go
command change is new fields exposed in go list
output.
It is important for tools that process Go packages to be able to understand what files are needed for a build. The go list
command is the underlying mechanism used now, even by golang.org/x/tools/go/packages
. Exposing the embedded files as a new field in Package
struct used by go list
makes them available both for direct use and for use by higher level APIs.
Command-line configuration
In the preliminary discussion, a few people suggested that the list of embedded files could be specified on the go build
command line. This could potentially work for files embedded in the main package, perhaps with an appropriate Makefile. But it would fail badly for dependencies: if a dependency wanted to add a new embedded file, all programs built with that dependency would need to adjust their build command lines.
Potential confusion with go:generate
In the preliminary discussion, a few people pointed out that developers might be confused by the inconsistency that //go:embed
directives are processed during builds but //go:generate
directives are not.
There are other special comment directives as well: //go:noinline
, //go:noescape
, // +build
, //line
. All of these are processed during builds. The exception is //go:generate
, because of the design constraint that the go
command not run user code during builds. The //go:embed
is not the special case, nor does it make //go:generate
any more of a special case.
For more about go generate
, see the original proposal
and discussion
.
The embed package
Import path
The new embed
package provides access to embedded files. Previous additions to the standard library have been made in golang.org/x
first, to make them available to earlier versions of Go. However, it would not make sense to use golang.org/x/embed
instead of embed
: the older versions of Go could import golang.org/x/embed
but still not be able to embed files without the newer go
command support. It is clearer for a program using embed
to fail to compile than it would be to compile but not embed any files.
File API
Implementing fs.FS
enables hooking into net/http
, text/template
, and html/template
, without needing to make those packages aware of embed
.
Code that wants to change between using operating system files and embedded files can be written in terms of fs.FS
and fs.File
and then use os.DirFS
as an fs.FS
or use a *os.File
directly as an fs.File
.
Direct access to embedded data
An obvious extension would be to add to embed.Files
a ReadFileString
method that returns the file content as a string. If the embedded data were stored in the binary uncompressed, ReadFileString
would be very efficient: it could return a string pointing into the in-binary copy of the data. Callers expecting zero allocation in ReadFileString
might well preclude a future -embed=compress
mode that trades binary size for access time, which could not provide the same kind of efficient direct access to raw uncompressed data. An explicit ReadFileString
method would also make it more difficult to convert code using embed.Files
to use other fs.FS
implementations, including operating system files. For now, it seems best to omit a ReadFileString
method, to avoid exposing the underlying representation and also to avoid diverging from fs.FS
.
Another extension would be to add to the returned fs.File
a WriteTo
method. All the arguments against ReadFileString
apply equally well to WriteTo
. An additional reason to avoid WriteTo
is that it would expose the uncompressed data in a mutable form, []byte
instead of string
.
The price of this flexibility—both the flexibility to move easily between embed.Files
and other file systems and also the flexibility to add -embed=compress
later (perhaps that would useful for TinyGo)—is that access to data requires making a copy. This is at least no less efficient than reading from other file sources.
Writing embedded files to disk
In the preliminary discussion, one person asked about making it easy to write embedded files back to disk at runtime, to make them available for use with the HTTP server, template parsing, and so on. While this is certainly possible to do, we probably should avoid that as the suggested way to use embedded files: many programs run with limited or no access to writable disk. Instead, this design builds on the file system draft design to make the embedded files available to those APIs.
Compatibility
This is all new API. There are no conflicts with the compatibility guidelines .
It is worth noting that, as with all new API, this functionality cannot be adopted by a Go project until all developers building the project have updated to the version of Go that supports the API. This may be a particularly important concern for authors of libraries. If this functionality ships in Go 1.15, library authors may wish to wait to adopt it until they are confident that all their users have updated to Go 1.15.
Implementation
The implementation details are not user-visible and do not matter nearly as much as the rest of the design.
A prototype implementation is available.
Appendix: Comparison with other tools
A goal of this design is to eliminate much of the effort involved in embedding static assets in Go binaries. It should be able to replace the common uses of most of the available embedding tools. Replacing all possible uses is a non-goal. Replacing all possible embedding tools is also a non-goal.
This section examines a few popular embedding tools and compares and contrasts them with this design.
go-bindata
One of the earliest and simplest generators for static assets is
github.com/jteeuwen/go-bindata
. It is no longer maintained, so now there are many forks and derivatives, but this section examines the original.
Given an input file hello.txt
containing the single line hello, world
, go-bindata hello.txt
produces 235 lines of Go code. The generated code exposes this exported API (in the package where it is run):
func Asset(name string) ([]byte, error) Asset loads and returns the asset for the given name. It returns an error if the asset could not be found or could not be loaded. func AssetDir(name string) ([]string, error) AssetDir returns the file names below a certain directory embedded in the file by go-bindata. For example if you run go-bindata on data/... and data contains the following hierarchy: data/ foo.txt img/ a.png b.png then AssetDir("data") would return []string{"foo.txt", "img"} AssetDir("data/img") would return []string{"a.png", "b.png"} AssetDir("foo.txt") and AssetDir("notexist") would return an error AssetDir("") will return []string{"data"}. func AssetInfo(name string) (os.FileInfo, error) AssetInfo loads and returns the asset info for the given name. It returns an error if the asset could not be found or could not be loaded. func AssetNames() []string AssetNames returns the names of the assets. func MustAsset(name string) []byte MustAsset is like Asset but panics when Asset would return an error. It simplifies safe initialization of global variables. func RestoreAsset(dir, name string) error RestoreAsset restores an asset under the given directory func RestoreAssets(dir, name string) error RestoreAssets restores an asset under the given directory recursively
This code and exported API is duplicated in every package using go-bindata
-generated output. One benefit of this design is that the access code can be in a single package shared by all clients.
The registered data is gzipped. It must be decompressed when accessed.
The embed
API provides all this functionality except for “restoring” assets back to the local file system. See the “Writing embedded assets to disk” section above for more discussion about why it makes sense to leave that out.
Another venerable asset generator is github.com/rakyll/statik
. Given an input file public/hello.txt
containing the single line hello, world
, running statik
generates a subdirectory statik
containing an import-only package with a func init
containing a single call, to register the data for asset named "hello.txt"
with the access package github.com/rakyll/statik/fs
.
The use of a single shared registration introduces the possibility of naming conflicts: what if multiple packages want to embed different static hello.txt
assets? Users can specify a namespace when running statik
, but the default is that all assets end up in the same namespace.
This design avoids collisions and explicit namespaces by keeping each embed.Files
separate: there is no global state or registration.
The registered data in any given invocation is a string containing the bytes of a single zip file holding all the static assets.
Other than registration calls, the statik/fs
package includes this API:
func New() (http.FileSystem, error) New creates a new file system with the default registered zip contents data. It unzips all files and stores them in an in-memory map. func NewWithNamespace(assetNamespace string) (http.FileSystem, error) NewWithNamespace creates a new file system with the registered zip contents data. It unzips all files and stores them in an in-memory map. func ReadFile(hfs http.FileSystem, name string) ([]byte, error) ReadFile reads the contents of the file of hfs specified by name. Just as ioutil.ReadFile does. func Walk(hfs http.FileSystem, root string, walkFn filepath.WalkFunc) error Walk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root. All errors that arise visiting files and directories are filtered by walkFn. As with filepath.Walk, if the walkFn returns filepath.SkipDir, then the directory is skipped.
The embed
API provides all this functionality (converting to http.FileSystem
, reading a file, and walking the files).
Note that accessing any single file requires first decompressing all the embedded files. The decision in this design to avoid compression is discussed more above, in the “Compression to reduce binary size” section.
go.rice
Another venerable asset generator is github.com/GeertJohan/go.rice
. It presents a concept called a rice.Box
which is like an embed.Files
filled from a specific file system directory. Suppose box/hello.txt
contains hello world
and hello.go
is:
package main import rice "github.com/GeertJohan/go.rice" func main() { rice.FindBox("box") }
The command rice embed-go
generates a 44-line file rice-box.go
that calls embedded.RegisterEmbeddedBox
to registers a box named box
containing the single file hello.txt
. The data is uncompressed. The registration means that go.rice
has the same possible collisions as statik
.
The rice embed-go
command parses the Go source file hello.go
to find calls to rice.FindBox
and then uses the argument as both the name of the box and the local directory containing its contents. This approach is similar to the “second approach” identified in the preliminary discussion, and it demonstrates all the drawbacks suggested above. In particular, only the first of these variants works with the rice
command:
rice.FindBox("box") rice.FindBox("b" + "o" + "x") const box = "box" rice.FindBox(box) func box() string { return "box" } rice.FindBox(box())
As the Go language is defined, these should all do the same thing. The limitation to the first form is fine in an opt-in tool, but it would be problematic to impose in the standard toolchain, because it would break the orthogonality of language concepts.
The API provided by the rice
package is:
type Box struct { // Has unexported fields. } Box abstracts a directory for resources/files. It can either load files from disk, or from embedded code (when `rice --embed` was ran). func FindBox(name string) (*Box, error) FindBox returns a Box instance for given name. When the given name is a relative path, it’s base path will be the calling pkg/cmd’s source root. When the given name is absolute, it’s absolute. derp. Make sure the path doesn’t contain any sensitive information as it might be placed into generated go source (embedded). func MustFindBox(name string) *Box MustFindBox returns a Box instance for given name, like FindBox does. It does not return an error, instead it panics when an error occurs. func (b *Box) Bytes(name string) ([]byte, error) Bytes returns the content of the file with given name as []byte. func (b *Box) HTTPBox() *HTTPBox HTTPBox creates a new HTTPBox from an existing Box func (b *Box) IsAppended() bool IsAppended indicates whether this box was appended to the application func (b *Box) IsEmbedded() bool IsEmbedded indicates whether this box was embedded into the application func (b *Box) MustBytes(name string) []byte MustBytes returns the content of the file with given name as []byte. panic’s on error. func (b *Box) MustString(name string) string MustString returns the content of the file with given name as string. panic’s on error. func (b *Box) Name() string Name returns the name of the box func (b *Box) Open(name string) (*File, error) Open opens a File from the box If there is an error, it will be of type *os.PathError. func (b *Box) String(name string) (string, error) String returns the content of the file with given name as string. func (b *Box) Time() time.Time Time returns how actual the box is. When the box is embedded, it’s value is saved in the embedding code. When the box is live, this methods returns time.Now() func (b *Box) Walk(path string, walkFn filepath.WalkFunc) error Walk is like filepath.Walk() Visit http://golang.org/pkg/path/filepath/#Walk for more information
type File struct { // Has unexported fields. } File implements the io.Reader, io.Seeker, io.Closer and http.File interfaces func (f *File) Close() error Close is like (*os.File).Close() Visit http://golang.org/pkg/os/#File.Close for more information func (f *File) Read(bts []byte) (int, error) Read is like (*os.File).Read() Visit http://golang.org/pkg/os/#File.Read for more information func (f *File) Readdir(count int) ([]os.FileInfo, error) Readdir is like (*os.File).Readdir() Visit http://golang.org/pkg/os/#File.Readdir for more information func (f *File) Readdirnames(count int) ([]string, error) Readdirnames is like (*os.File).Readdirnames() Visit http://golang.org/pkg/os/#File.Readdirnames for more information func (f *File) Seek(offset int64, whence int) (int64, error) Seek is like (*os.File).Seek() Visit http://golang.org/pkg/os/#File.Seek for more information func (f *File) Stat() (os.FileInfo, error) Stat is like (*os.File).Stat() Visit http://golang.org/pkg/os/#File.Stat for more information
type HTTPBox struct { *Box } HTTPBox implements http.FileSystem which allows the use of Box with a http.FileServer. e.g.: http.Handle("/", http.FileServer(rice.MustFindBox("http-files").HTTPBox())) func (hb *HTTPBox) Open(name string) (http.File, error) Open returns a File using the http.File interface
As far as public API, go.rice
is very similar to this design. The Box
itself is like embed.Files
, and the File
is similar to fs.File
. This design avoids HTTPBox
by building on HTTP support for fs.FS
.
The Bazel build tool includes support for building Go, and its
go_embed_data
rule supports embedding a file as data in a Go program. It is used like:
go_embed_data( name = "rule_name", package = "main", var = "hello", src = "hello.txt", )
or
go_embed_data( name = "rule_name", package = "main", var = "files", srcs = [ "hello.txt", "gopher.txt", ], )
The first form generates a file like:
package main var hello = []byte("hello, world\n")
The second form generates a file like:
package main var files = map[string][]byte{ "hello.txt": []byte("hello, world\n"), "gopher.txt": []byte("ʕ◔ϖ◔ʔ\n"), }
That’s all. There are configuration knobs to generate string
instead of []byte
, and to expand zip and tar files into their contents, but there’s no richer API: just declared data.
Code using this form would likely keep using it: the embed
API is more complex.
However, it will still be important to support this //go:embed
design in Bazel. The way to do that would be to provide a go tool embed
that generates the right code and then either adjust the Bazel go_library
rule to invoke it or have Gazelle (the tool that reads Go files and generates Bazel rules) generate appropriate genrules
. The details would depend on the eventual Go implementation, but any Go implementation of //go:embed
needs to be able to be implemented in Bazel/Gazelle in some way.
以上所述就是小编给大家介绍的《Go command support for embedded static assets (files)》,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对 码农网 的支持!
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
大思维:集体智慧如何改变我们的世界
杰夫·摩根 / 郭莉玲、尹玮琦、徐强 / 中信出版集团股份有限公司 / 2018-8-1 / CNY 65.00
智能时代,我们如何与机器互联,利用技术来让我们变得更聪明?为什么智能技术不会自动导致智能结果呢?线上线下群体如何协作?社会、政府或管理系统如何解决复杂的问题?本书从哲学、计算机科学和生物学等领域收集见解,揭示了如何引导组织和社会充分利用人脑和数字技术进行大规模思考,从而提高整个集体的智力水平,以解决我们时代的巨大挑战。是英国社会创新之父的洞见之作,解析企业、群体、社会如何明智决策、协作进化。一起来看看 《大思维:集体智慧如何改变我们的世界》 这本书的介绍吧!