内容简介:With the release of 3.20, LXD was included into theAs canonical is developing the project, and they only targetI’m not old enough to have jammed with the console cowboys in cyberspace during
With the release of 3.20, LXD was included into the community repository of Arch Linux in January, and has currently been sitting there happily for the past months. LXD is a container manager from Canonical that manages containers as if they where independent machines in a cluster. I have somehow taken to calling them “containers-as-machines”. This is in contrast to podman and docker which would be “containers-as-applications”. Think of lxd as ganeti , but for containers.
As canonical is developing the project, and they only target snap packages downstream, it takes quite a few liberties with dependencies and vendored projects which makes for an interesting package challenge.
I’m not old enough to have jammed with the console cowboys in cyberspace during the 90s . Thus the linker magic presented here might be painly obvious for some people, however this is a learning exercise! Hopefully this encourages people to try their hands at packaging without thinking they need to know everything, or so my imposter syndrome keeps telling me.
Investigating the project
When you are looking to package something the ideal place to start is with the top-level package, and work your way down. The first step is to read the documentation.
Opening the lxd repository You end up somewhere midway in the README when you see “From Source: Building the latest version”, which is what we are after.
[…] a specific release of LXD which may not be offered by their Linux distribution. Source builds for integration into Linux distributions are not covered here and may be covered in detail in a separate document in the future.
Sure, just tell me the dependencies, and we can deal with that.
When building from source, it is customary to configure a GOPATH which contains the to-be-built source code. […] and with a little LD_LIBRARY_PATH magic (described later), these binaries can be run directly from the built source tree.
A little what-are-you-talking-about now?
/me scrolls
cd lxd-3.18 export GOPATH=$(pwd)/_dist
Uh…
make deps export CGO_CFLAGS="${CGO_CFLAGS} -I${GOPATH}/deps/sqlite/ -I${GOPATH}/deps/dqlite/include/ -I${GOPATH}/deps/raft/include/ -I${GOPATH}/deps/libco/" export CGO_LDFLAGS="${CGO_LDFLAGS} -L${GOPATH}/deps/sqlite/.libs/ -L${GOPATH}/deps/dqlite/.libs/ -L${GOPATH}/deps/raft/.libs -L${GOPATH}/deps/libco/" export LD_LIBRARY_PATH="${GOPATH}/deps/sqlite/.libs/:${GOPATH}/deps/dqlite/.libs/:${GOPATH}/deps/raft/.libs:${GOPATH}/deps/libco/:${LD_LIBRARY_PATH}" make
Wat ಠ_ಠ
This isn’t suppose to be easy. Is it[]?
Unesting this flag magic, we know the vendored dependencies LXD vendor is:
- sqlite
- raft
- libco
- dqlite
Vendoring is generally considered bad for a number of reasons. The most important one is security. Having Z number of upstream vendors with N versions of dependency X makes security patching and updating a nightmare for distributions. And you can’t rely on upstream tracking security issues on their dependencies, this is hard enough for security teams today.
The second one is dependencies themselves. If we where to package this as-is, dqlite, raft and libco would be provided by the LXD package. Inaccessible or hard to use for other packages and users. And what if two packages vendored dqlite? Which package should provide it?
What struck me as most interesting is the fact that sqlite is vendored. Why on earth is it vendored? Most distributions should depend on sqlite fairly early in their dependency chain.
sqlite, sqlite-replication and dqlite
Turns out canonical has forked sqlite to provide a patch adding Write-Ahead logging replication. The details are a bit beyond me as I don’t know C very well. The strange thing is that it still utilizes the same so-name and pkg-config file as upstream sqlite. Which is a problem considering they are ABI incompatible. I should also check, but it would be interesting to verify if there has been attempt at upstreaming these patches.
sqlite-replication
is used as a dependency for dqlite
which is essentially a
distributed sqlite database with a raft consensus
protocol
. Rest of the dependencies are somehow
unremarkable. raft
is a C implementation of said protocol, and libco
is an
maintained out-of-tree fork of byuus
libco
. Neither of these are
interesting to at look further in this post, and was fairly straight forward to
package.
Since sqlite-replication
built normally is going to have conflicting files
with sqlite
, we need to build any headers and libraries into it’s own path.
This can be done in the configure step of the software.
./configure --prefix=/usr \ --libdir=/usr/lib/$pkgname \ --includedir=/usr/include/$pkgname \ [snip] # Note: `$pkgname` is a shorthand for `sqlite-replication`.
The resulting header files would reside in /usr/include/sqlite-replication
,
so-names in /usr/lib/sqlite-replication
and the pkg-config
file is going to
reside in /usr/lib/sqlite-replication/pkgconfig
. dqlite
needs to be aware
of the extra path to look for linker files and libraries. We can abuse
environment files a little bit for this.
PKG_CONFIG_PATH="/usr/lib/$pkgname/pkgconfig" ./configure --prefix=/usr make LDFLAGS="-Wl,-R/usr/lib/$pkgname"
And this seems to work fine[]. As far as I got from the documentation is
that -Wl,-R<path>
rewrites the included search path as it’s interpreted as -rpath
when a complete directory is added. I think.
λ ~ » ldd /usr/lib/libdqlite.so.0.0.1 | grep replication libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 # with -rpath: libsqlite3.so.0 => /usr/lib/sqlite-replication/libsqlite3.so.0
Nice, this enables us to verify that we build dqlite
linked towards the
patched sqlite
provided by Canonical.
Golang and linkers
Now comes the fun part, lets build lxd
! In contrast to the other projects we
have looked at, lxd is written in go. Cgo is a special runtime in go which is
used to interface with C code, it supports it’s own
set of CGO_*
variables
that
mirrors the gcc/ld flags.
So lets try with the buildflags we used earlier. This should make pkg-config
capable of finding the linker flags needed, along with the -rpath
argument to
ensure we have the correct ld path.
export PKG_CONFIG_PATH="/usr/lib/sqlite-replication/pkgconfig" export CGO_LDFLAGS="-Wl,-R/usr/lib/sqlite-replication"
And the results are…
# github.com/canonical/go-dqlite/internal/bindings /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_frames' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_enabled' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_leader' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_undo' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_unregister' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_follower' /usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib/libdqlite.so: undefined reference to `sqlite3_wal_replication_register'
Interesting. We didn’t find the library! So admittedly I’m a bit unsure what is
going on. libdqlite.so
should be looking for the correct library, and the
library should be present. However considering we are dealing with some bindings
magic, I looked up the
code
which canonical/go-dqlite
utilizes and found the following LDFLAGS.
/* #cgo linux LDFLAGS: -lsqlite3 -lraft -lco -ldqlite */
What I think
is happening here is -lsqlite3
causes dqlite to assume the
correct so-name has been found, and disregards looking for it anywhere else. It
is a bit weird that we are dealing with pkg-config
inside
lxd
which behaves
differently. However, I did figure out something in the end!
export CGO_CFLAGS="-I/usr/include/sqlite-replication" export CGO_LDFLAGS="-L/usr/lib/sqlite-replication -Wl,-R/usr/lib/sqlite-replication"
What I wound up doing in the end is to include the -rpath
stuff from sqlite-replication
and riffing on the flags provided by upstream for their
vendoring. It seems to work correctly and I haven’t recieved any bugreports
suggesting otherwise. After getting this to work I did discover this is exactly
what Void Linux does, so oh-well. Should probably read other distribution
package files a bit better.
And a side note; libcap
introduced a hard
dependency
on an optional
libpsx
library. This file contained several flags which the
Golang compiler denies by default. You need to explicitly whitelist them when
building. This was removed in a later
release
I discovered while backtracing the how I packaged this. This was noted in the
build instruction
for LXD as both Void
Linux and us encountered this issue.
The whitelisting can be fixed by utilizing the CGO_LDFLAGS_ALLOW
.
export CGO_LDFLAGS_ALLOW='-Wl,-wrap,pthread_create'
After this fumbling, which is honestly is the result of two months on-and-off work now condensed into a 10 minute long read, it works! LXD was packaged properly and been a happy LXD user.
The resulting PKGBUILD can be found in the repository for LXD and sqlite-replication !
Reproducible Builds
I don’t think this is a complete blog post about packaging without looking a little on reproducible builds . The goal is to reproduce the bit-for-bit identical package which I build and distribute into the Arch Linux mirrors. For this we can use archlinux-repro , and the technical details can be read in the blog post I wrote last year.
Can we reproduce the packages from his blog post?
λ » repro sqlite-replication-3.31.1.4-1-x86_64.pkg.tar.zst [...snip....] ==> Finished making: sqlite-replication 3.31.1.4-1 (Mon 27 Apr 2020 11:59:03 PM CEST) [...] ==> Comparing hashes... ==> Package is reproducible!
λ » repro raft-0.9.18-1-x86_64.pkg.tar.zst [...snip....] ==> Finished making: raft 0.9.18-1 (Tue 28 Apr 2020 12:01:03 AM CEST) [...] ==> Comparing hashes... ==> Package is reproducible!
λ » repro libco-20-2-x86_64.pkg.tar.zst [...snip....] ==> Finished making: libco 20-2 (Tue 28 Apr 2020 12:03:02 AM CEST) [...] ==> Comparing hashes... ==> Package is reproducible!
λ » repro dqlite-1.4.1-1-x86_64.pkg.tar.zst [...snip....] ==> Finished making: dqlite 1.4.1-1 (Tue 28 Apr 2020 12:04:17 AM CEST) [...] ==> Comparing hashes... ==> Package is reproducible!
λ » repro lxd-4.0.1-1-x86_64.pkg.tar.zst [...snip....] ==> Finished making: lxd 4.0.1-1 (Tue 28 Apr 2020 12:10:55 AM CEST) [...] ==> Comparing hashes... ==> Package is reproducible!
Yes we can :dancers: ! Amazing!
I think it’s hard to convey the importance of this. This is the combined work between multiple distributions for well over 5 years now. We are capable of recreating bit-for-bit identical packages to the ones distributed in the Arch Linux repositories!
Other distributions!
The interesting part now is to figure out what different distributions are doing to package LXD.
- Debian doesn’t package LXD because of the Go dependency packaging .
- Alpine seems content with packaging LXD as-is.
- OpenSUSE seems also content with packaging it as-is.
- And Gentoo seems content with packaging it as-is.
- Void Linux and NixOS packages it correctly by splitting up all the dependencies.
- I where unable to locate the Fedora package which is suppose to exist somewhere .
The intention isn’t to name and shame. Different distributions has other priorities and philosophies. However it is also interesting to compare what you are getting with the distributed LXD package across repositories.
以上所述就是小编给大家介绍的《Packaging LXD for Arch Linux》,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对 码农网 的支持!
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。