raspberry pi 3 b+ cross compile on macos mojave

本文记录了如何在mojave上构建raspberry pi 3 b+的交叉编译环境。交叉编译的原因最主要的还是速度，当然还有目标平台没有直接的编译 工具 链等。在构建交叉编译环境之前，最好确认目标平台的相关信息，能直接确认目标平台的工具链是最好的，因为从CPU判断可能并不准确。

比如raspebrry pi 3 b+，自带gcc，相关信息如下

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/6/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Raspbian 6.3.0-18+rpi1+deb9u1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=arm-linux-gnueabihf- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-armhf/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-armhf --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-armhf --with-arch-directory=arm --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-sjlj-exceptions --with-arch=armv6 --with-fpu=vfp --with-float=hard --enable-hecking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 6.3.0 20170516 (Raspbian 6.3.0-18+rpi1+deb9u1)

注意其中target为arm-linux-gnueabihf。假如从raspberry pi 3 b+的CPU（Broadcom BCM2837 64bit CPU）判断的话，armv8，aarch64貌似都可以用，但实际用aarch64架构编译出来的程序无法在raspberry pi，准确来说是安装了raspbian上的raspberry pi上执行，这点请注意（假如想在raspberry pi 3 b+上执行aarch64架构的程序的话，个人觉得可能要换raspbian以外的系统，或者修改内核编译选项，这块等自己有空尝试并成功了再发出来）。

macos下交叉编译，首先要安装crosstool-ng

brew install crosstool-ng

个人安装的是版本1.23.0_3

➜  ~ brew info crosstool-ng
crosstool-ng: stable 1.23.0 (bottled), HEAD
Tool for building toolchains
https://crosstool-ng.github.io/
/usr/local/Cellar/crosstool-ng/1.23.0_3 (1,294 files, 7.7MB) *
  Poured from bottle on 2019-02-08 at 17:07:19
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/crosstool-ng.rb
==> Dependencies
Build: help2man ✔
Required: autoconf ✔, automake ✔, binutils ✔, coreutils ✔, flex ✔, gawk ✔, gnu-sed ✔, grep ✔, libtool ✔, m4 ✔, make ✔, ncurses ✔, xz ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 370 (30 days), 1,283 (90 days), 4,018 (365 days)
install_on_request: 342 (30 days), 1,106 (90 days), 3,409 (365 days)
build_error: 0 (30 days)

如果你在brew info时，发现help2man没有打勾的话，可以通过brew install安装。个人不确定对构建交叉编译环境是否有影响。

部分介绍交叉编译环境的文章会说再安装一个新版本的bison，个人没有安装，作为参考，本机的bison信息如下

➜  ~ brew info bison
bison: stable 3.3.2 (bottled) [keg-only]
Parser generator
https://www.gnu.org/software/bison/
/usr/local/Cellar/bison/3.0.4 (51 files, 2.1MB)
  Poured from bottle on 2016-01-25 at 19:23:54
/usr/local/Cellar/bison/3.0.5 (52 files, 2.1MB)
  Poured from bottle on 2018-06-19 at 11:25:38
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/bison.rb
==> Caveats
bison is keg-only, which means it was not symlinked into /usr/local,
because some formulae require a newer version of bison.
 
==> Analytics
install: 17,770 (30 days), 52,184 (90 days), 156,520 (365 days)
install_on_request: 10,829 (30 days), 31,225 (90 days), 92,262 (365 days)
build_error: 0 (30 days)

安装完crosstool-ng之后，macos上需要为交叉编译环境分配一个大约10G的字符大小写敏感的分区。具体要多大，看目标平台。一个平台的话10G够了，两个会超过10G。作为参考构建完arm-linux-gnueabihf之后使用空间8.7G。

打开Disk Utility，工具栏上添加volume，选择一个名字，格式选择APFS(Case-sensitive)。在Size options…中两个都填入10G。

创建完之后在命令行下进入这个新volume

cd /Volumes/xtool-build-env

用crosstool-ng列出可选的一些sample

ct-ng list-samples

其中应该包括arm-linux-gnueabihf。选择arm-linux-gnueabihf

ct-ng arm-linux-gnueabihf

ct-ng会在/Volumes/xtool-build-env下生成一个叫做 .config 的文件，里面包含了很多配置。其中需要关注的几个配置是路径相关的几个配置。把几个路径改成volume内的路径

CT_LOCAL_TARBALLS_DIR="/Volumes/xtool-build-env/src"
CT_SAVE_TARBALLS=y
CT_WORK_DIR="/Volumes/xtool-build-env/build"
CT_BUILD_TOP_DIR="${CT_WORK_DIR}/${CT_HOST:+HOST-${CT_HOST}/}${CT_TARGET}"
CT_PREFIX_DIR="/Volumes/xtool-build-env/${CT_TARGET}"

改好之后，备份这个 .config

cp .config config-bak

备份的原因是ct-ng build中途失败时处理方法导致的，具体之后会看到。

开始build

ct-ng build

如果一切正常的话，大约1个多小时能完全（不包括下载时间）。问题是，ct-ng build有大几率会碰到乱七八糟的问题而停下来。以下是我碰到的几个

build m4 failed

[INFO ]  Installing m4 for build
[EXTRA]    Configuring m4
[EXTRA]    Building m4
[24:09] / /usr/local/Cellar/crosstool-ng/1.23.0_3/lib/crosstool-ng-1.23.0/scripts/functions: line 297: 30256 Segmentation fault: 11  "${@}" 2>&1
     30257 Done                    | CT_DoLog "${level}"
[ERROR]
[ERROR]  >>
[ERROR]  >>  Build failed in step 'Installing m4 for build'
[ERROR]  >>        called in step '(top-level)'
[ERROR]  >>
[ERROR]  >>  Error happened in: CT_DoExecLog[scripts/functions@297]
[ERROR]  >>        called from: do_m4_backend[scripts/build/companion_tools/100-m4.sh@66]
[ERROR]  >>        called from: do_companion_tools_m4_for_build[scripts/build/companion_tools/100-m4.sh@17]
[ERROR]  >>        called from: do_companion_tools_for_build[scripts/build/companion_tools.sh@35]
[ERROR]  >>        called from: main[scripts/crosstool-NG.sh@653]
[ERROR]  >>
[ERROR]  >>  For more info on this error, look at the file: 'build.log'
[ERROR]  >>  There is a list of known issues, some with workarounds, in:
[ERROR]  >>      '/usr/local/Cellar/crosstool-ng/1.23.0_3/share/doc/crosstool-ng/crosstool-ng-1.23.0/B - Known issues.txt'
[ERROR]  >>
[ERROR]  >>  If you feel this is a bug in crosstool-NG, report it at:
[ERROR]  >>      https://github.com/crosstool-ng/crosstool-ng/issues/
[ERROR]  >>
[ERROR]  >>  Make sure your report includes all the information pertinent to this issue.
[ERROR]  >>  Read the bug reporting guidelines here:
[ERROR]  >>      http://crosstool-ng.github.io/support/
[ERROR]
[ERROR]  (elapsed: 24:07.00)
[24:09] / gmake: *** [/usr/local/bin/ct-ng:147: build] Error 1

根据 这个issue ，解决方法如下

brew uninstall --ignore-dependencies binutils
brew install binutils

但是此时如果你ct-ng build想继续build的话是不行的，具体原因不清楚，我的解决方法是

ct-ng distclean
cp config-bak .config

distclean会把.config删除掉，所以要事先备份。

automake not found

[INFO ]  Installing gettext for host
[EXTRA]    Configuring gettext
[EXTRA]    Building gettext
[ERROR]    make[5]: *** [Makefile:2561: /Volumes/xtool-build-env/build/src/gettext-0.19.8.1/gettext-tools/src/Makefile.in] Error 127
[ERROR]    make[4]: *** [Makefile:2026: all-recursive] Error 1
[ERROR]    make[3]: *** [Makefile:1892: all] Error 2
[ERROR]    make[2]: *** [Makefile:414: all-recursive] Error 1
[ERROR]    make[1]: *** [Makefile:370: all] Error 2
[ERROR]
[ERROR]  >>
[ERROR]  >>  Build failed in step 'Installing gettext for host'
[ERROR]  >>        called in step '(top-level)'
[ERROR]  >>
[ERROR]  >>  Error happened in: CT_DoExecLog[scripts/functions@297]
[ERROR]  >>        called from: do_gettext_backend[scripts/build/companion_libs/330-gettext.sh@136]
[ERROR]  >>        called from: do_gettext_for_host[scripts/build/companion_libs/330-gettext.sh@53]
[ERROR]  >>        called from: do_companion_libs_for_host[scripts/build/companion_libs.sh@36]
[ERROR]  >>        called from: main[scripts/crosstool-NG.sh@653]
[ERROR]  >>
[ERROR]  >>  For more info on this error, look at the file: 'build.log'
[ERROR]  >>  There is a list of known issues, some with workarounds, in:
[ERROR]  >>      '/usr/local/Cellar/crosstool-ng/1.23.0_3/share/doc/crosstool-ng/crosstool-ng-1.23.0/B - Known issues.txt'
[ERROR]  >>
[ERROR]  >>  If you feel this is a bug in crosstool-NG, report it at:
[ERROR]  >>      https://github.com/crosstool-ng/crosstool-ng/issues/
[ERROR]  >>
[ERROR]  >>  Make sure your report includes all the information pertinent to this issue.
[ERROR]  >>  Read the bug reporting guidelines here:
[ERROR]  >>      http://crosstool-ng.github.io/support/
[ERROR]
[ERROR]  (elapsed: 14:14.00)
[14:16] / gmake: *** [/usr/local/bin/ct-ng:147: build] Error 1

如果你tail一下build.log时，你会看到

[ALL  ]    /Volumes/xtool-build-env/build/src/gettext-0.19.8.1/build-aux/missing: line 81: automake-1.15: command not found
[ALL  ]    WARNING: 'automake-1.15' is missing on your system.
[ALL  ]             You should only need it if you modified 'Makefile.am' or
[ALL  ]             'configure.ac' or m4 files included by 'configure.ac'.
[ALL  ]             The 'automake' program is part of the GNU Automake package:
[ALL  ]             <http://www.gnu.org/software/automake>
[ALL  ]             It also requires GNU Autoconf, GNU m4 and Perl in order to run:
[ALL  ]             <http://www.gnu.org/software/autoconf>
[ALL  ]             <http://www.gnu.org/software/m4/>
[ALL  ]             <http://www.perl.org/>
[ERROR]    make[5]: *** [Makefile:2561: /Volumes/xtool-build-env/build/src/gettext-0.19.8.1/gettext-tools/src/Makefile.in] Error 127
[ALL  ]    make[5]: Leaving directory '/Volumes/xtool-build-env/build/aarch64-rpi3-linux-gnueabi/build/build-gettext-host-x86_64-build_apple-darwin18.2.0/gettext-tools/src'
[ERROR]    make[4]: *** [Makefile:2026: all-recursive] Error 1
[ALL  ]    make[4]: Leaving directory '/Volumes/xtool-build-env/build/aarch64-rpi3-linux-gnueabi/build/build-gettext-host-x86_64-build_apple-darwin18.2.0/gettext-tools'
[ERROR]    make[3]: *** [Makefile:1892: all] Error 2
[ALL  ]    make[3]: Leaving directory '/Volumes/xtool-build-env/build/aarch64-rpi3-linux-gnueabi/build/build-gettext-host-x86_64-build_apple-darwin18.2.0/gettext-tools'
[ERROR]    make[2]: *** [Makefile:414: all-recursive] Error 1
[ALL  ]    make[2]: Leaving directory '/Volumes/xtool-build-env/build/aarch64-rpi3-linux-gnueabi/build/build-gettext-host-x86_64-build_apple-darwin18.2.0'
[ERROR]    make[1]: *** [Makefile:370: all] Error 2
[ALL  ]    make[1]: Leaving directory '/Volumes/xtool-build-env/build/aarch64-rpi3-linux-gnueabi/build/build-gettext-host-x86_64-build_apple-darwin18.2.0'

即没有automake，所以解决方法是

ct-ng menuconfig

在companion tools中勾选automake。记得更新了 .config 之后备份。

在解决了上面两个问题之后，我的ct-ng build就顺利完成了。最后写一个hello world，cross compile之后复制到raspberry pi 3 b+上能执行的话就OK了。

小结

老实说，cross compile一直是个人的心结，因为cross compile的环境构建真得很麻烦。相比之下Write once，run everywhere是开发者的福音，不管是过去还是现在。