起因主要是想给我fork自convco的 git-cz 项目 release 那里增加一个Mac二进制文件方便使用Mac的人下载。

这是一个方便使用约定式提交记录的git工具.

git-cz-screen-record.gif

Environment

OS: Fedora 33 (Workstation Edition) x86_64
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz x86_64
rustc: 1.49.0 (e1884a8e3 2020-12-2)
cargo: 1.49.0 (d00d64df9 2020-12-05)

Requirements

# Install build dependencies

# CentOS Stream / Fedora 33
dnf group install "Development Tools"

dnf install -y \
        clang-devel clang \
        gcc \
        gcc-c++ \
        zlib-devel \
        libmpc-devel \
        mpfr-devel \
        gmp-devel

# Ubuntu 20.04
apt install \
    clang \
    gcc \
    g++ \
    zlib1g-dev \
    libmpc-dev \
    libmpfr-dev \
    libgmp-dev

# Add macOS Rust target
rustup target add x86_64-apple-darwin

Building OSXCross Toolchain

OSXCross 是什么? 是一个快速让你在Linux上安装Mac交叉编译toolchain的工具。

很多复杂或麻烦的事情都被它处理好了。

它甚至还带了一个包管理器:

OSXCross comes with a minimalistic MacPorts Packet Manager. See README.MACPORTS for more.

安装步骤:

1. 准备 SDK

这步骤主要参考: https://github.com/tpoechtrager/osxcross#packaging-the-sdk

– Xcode up to 12.5 Beta 3 is known to work. – Use Firefox if you have problems signing in.

构建这个toolchain需要苹果的SDK, 这个需要从Xcode提取。 去 https://developer.apple.com/download/all/ 下载最新版的Xcode (需要Apple ID登录), beta版的不需要从app store下载,下载回来是一个xip后缀的文件.

其实现在 12.5.1 正式版已经发布了, 应该也是支持的.

我这里下载当前最新版本 Xcode_12.5.1.xip

Released: June 20, 2021

https://download.developer.apple.com/Developer_Tools/Xcode_12.5.1/Xcode_12.5.1.xip

ps: 老灯还发现了一个下载 xcode 的好地方: https://xcodereleases.com/

Packing the SDK on Linux - Method 1 (Xcode > 8.0):

确保已经安装了打包SDK过程中需要的clang, make, libssl-devel, lzma-devel and libxml2-devel


# 如果之前有下载过, 记得删除 rm -rf osxcross , 不然连接的旧的库可能会导致 pbzx 等无法执行
git clone https://github.com/tpoechtrager/osxcross
cd osxcross

# ubuntu lzma.h: sudo apt install -y liblzma-dev

# /home/ttys3/Downloads/Xcode_12.5.1.xip 是下载好的 `Xcode_12.5.1.xip` 路径
./tools/gen_sdk_package_pbzx.sh /home/ttys3/Downloads/Xcode_12.5.1.xip

这里需要解包 xip 文档,再用xz 打包, 要一小会。

打包成功会在 OSXCross 根目录下生成 MacOSX11.3.sdk.tar.xz

将它移动到tarballs 目录下:

mv MacOSX11.3.sdk.tar.xz ./tarballs

2. 构建toolchain

确保你的系统已经安装了必要的组件,如 Clang 3.9+, cmake, git, patch, Python, libssl-devel (openssl) lzma-devel, libxml2-devel and the bash shell

可选: Optional:

llvm-devel: For Link Time Optimization support
llvm-devel: For ld64 -bitcode_bundle support
uuid-devel: For ld64 -random_uuid support

In clang there is no difference between cross-compilation and native compilation, so OSXCross can use a normal clang install for both. You can use either a clang installation you already have, or build your own from source.

由于我们已经安装了系统clang了,因此不用手动再编译.

我们接下来直接使用clang 编译 cross toolchain, 执行 ./build.sh

  ~/repo/toolchain/osxcross on  master 
❯ ./build.sh
found SDK version 11.3 at tarballs/MacOSX11.3.sdk.tar.xz
verified at tarballs/MacOSX11.3.sdk.tar.xz

Building OSXCross toolchain, Version: 1.4

macOS SDK Version: 11.3, Target: darwin20.4
Minimum targeted macOS Version: 10.9
Tarball Directory: /home/ttys3/repo/toolchain/osxcross/tarballs
Build Directory: /home/ttys3/repo/toolchain/osxcross/build
Install Directory: /home/ttys3/repo/toolchain/osxcross/target
SDK Install Directory: /home/ttys3/repo/toolchain/osxcross/target/SDK

Press enter to start building

# 或使用UNATTENDED=1跳过问题直接构建
UNATTENDED=1 ./build.sh

构建成功后会有类似如下提示:

    Do not forget to add

/home/ttys3/repo/rust/tools/osxcross/target/bin

to your PATH variable.

All done! Now you can use o32-clang(++) and o64-clang(++) like a normal compiler.

Example usage:

Example 1: CC=o32-clang ./configure --host=i386-apple-darwin20.2
Example 2: CC=i386-apple-darwin20.2-clang ./configure --host=i386-apple-darwin20.2
Example 3: o64-clang -Wall test.c -o test
Example 4: x86_64-apple-darwin20.2-strip -x test

!!! Use aarch64-apple-darwin20.2-* instead of arm64-* when dealing with Automake !!!
!!! CC=aarch64-apple-darwin20.2-clang ./configure --host=aarch64-apple-darwin20.2 !!!
!!! CC="aarch64-apple-darwin20.2-clang -arch arm64e" ./configure --host=aarch64-apple-darwin20.2 !!!

Your SDK does not support i386 anymore.
Use <= 10.13 SDK if you rely on i386 support.

Your SDK does not support libstdc++ anymore.
Use <= 10.13 SDK if you rely on libstdc++ support.

这里有关于32位支持和 libstdc++ 依赖的提示, 如果需要依赖这两个之一,就需要使用小于等于 10.13 的SDK

构建好的文件都在 target 目录下,直接放在这里显然不合适,我们把它移动到一个合适一点的地方, 比如 /usr/local/darwin-ndk-x86_64

sudo mkdir /usr/local/darwin-ndk-x86_64

sudo mv target/* /usr/local/darwin-ndk-x86_64/

如果还需要使用gcc作为交叉编译器的话,还需要行 ./build_gcc.sh, 不过这里我们是使用clang, 因此略过。

tips

链接错误问题:

x86_64-apple-darwin20.2-ld: error while loading shared libraries: libxar.so.1: cannot open shared object file: No such file or directory

❯ ls -l /usr/local/darwin-ndk-x86_64/lib
lrwxrwxrwx ttys3 ttys3  15 B  Wed Jan 27 10:46:58 2021  libtapi.so ⇒ libtapi.so.8svn
.rw-r--r-- ttys3 ttys3 3.2 MB Wed Jan 27 10:46:58 2021  libtapi.so.8svn
.rw-r--r-- ttys3 ttys3 205 KB Wed Jan 27 10:43:54 2021  libxar.a
.rw-r--r-- ttys3 ttys3 800 B  Wed Jan 27 10:43:54 2021  libxar.la
lrwxrwxrwx ttys3 ttys3  11 B  Wed Jan 27 10:43:54 2021  libxar.so ⇒ libxar.so.1
.rwxr-xr-x ttys3 ttys3 142 KB Wed Jan 27 10:43:54 2021  libxar.so.1

解决:

echo /usr/local/darwin-ndk-x86_64/lib | sudo tee /etc/ld.so.conf.d/darwin.conf
sudo ldconfig

ATTENTION:

OSXCross links libgcc and libstdc++ statically by default (this affects -foc-use-gcc-libstdc++ too). You can turn this behavior off with OSXCROSS_GCC_NO_STATIC_RUNTIME=1 (env).

The build also creates aliases *-g++-libc++ which link with the clang implementation of the C++ standard library instead of the GCC version. Don’t use these variants unless you know what you’re doing.

Configuring Cargo

编辑 .cargo/config, 增加

[target.x86_64-apple-darwin]
linker = "/usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin20.2-clang"
ar = "/usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin20.2-ar"

这里很重要,尤其是 linker, 如果没有配置, cargo 默认只会使用 cc, 而不会使用 clang, 即使你 export 了 CC=xxxxx, 在最后link的时候会报错( 报错类似于 https://gist.github.com/Edu4rdSHL/49b8b52bac916d88e32861a817d5f9a5 )

Building the project

PATH="/usr/local/darwin-ndk-x86_64/bin/:$PATH" \
CC=o64-clang \
CXX=o64-clang++ \
cargo build --target x86_64-apple-darwin
❯ file ./target/x86_64-apple-darwin/release/git-cz.darwin
./target/x86_64-apple-darwin/release/git-cz.darwin: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS>

OK, the binary: https://github.com/ttys3/git-cz/releases/download/v0.6.0/git-cz.darwin

Misc

o64-clang and x86_64-apple-darwin*-clang

link to the same wrapper: x86_64-apple-darwin20.2-wrapper

❯ ls -lh /usr/local/darwin-ndk-x86_64/bin/o64-clang
lrwxrwxrwx ttys3 ttys3 31 B Sun Jan 24 23:43:35 2021  /usr/local/darwin-ndk-x86_64/bin/o64-clang ⇒ x86_64-apple-darwin20.2-wrapper

❯ ls -lh /usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin20.2-clang
lrwxrwxrwx ttys3 ttys3 31 B Sun Jan 24 23:43:35 2021  /usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin20.2-clang ⇒ x86_64-apple-darwin20.2-wrapper

check all wrapper been linked to:

ls -l /usr/local/darwin-ndk-x86_64/bin/* | rg wrapper

Building *-sys crates

带C绑定的一些lib(一般命名为*-sys, 如 libz-sys )如果要交叉编译,你还得设置CC和CXX环境变量。 这样能编译成功了,但是最后链接阶段可能失败,原因是链接了错误的库.

有些库提供了环境变量可以方便使用交叉编译工具重新编译,比如 https://github.com/rust-lang/libz-sys/blob/master/build.rs#L25

如果设定了LIBZ_SYS_STATIC=1, 则会编译一个静态版本的, 由于我们指定了 CC 和 CXX 都是Mac的,因此可以成功链接。

PATH="/usr/local/darwin-ndk-x86_64/bin/:$PATH" \
CC=o64-clang \
CXX=o64-clang++ \
LIBZ_SYS_STATIC=1 \
cargo build --target x86_64-apple-darwin

About xip files

Unpacking XIP files on Linux

  1. Install xar from https://mackyle.github.io/xar/ (or my fork https://github.com/ttys3/xar)
  2. Install pbzx from https://github.com/NiklasRosenstein/pbzx (or my fork https://github.com/ttys3/pbzx/) (use gcc -llzma -lxar -I /usr/local/include pbzx.c -o pbzx and copy the binary into your PATH)
  3. use xar -xf XIP_FILE -C /path/to/extract/to
  4. Change to the directory where you extracted the file.
  5. Use pbzx -n Content | cpio -i to extract the contents.

ref: https://gist.github.com/phracker/1944ce190e01963c550566b749bd2b54

不过在这里,我们并不需要按上面的步骤手动提取,OSXCross 都已经有自动解包和打包脚本,很全.

xar 这个库, 原版已经很多年没人维护了, 所以,这里其实作者用了它自己的 fork 版: https://github.com/tpoechtrager/xar

老灯也 fork 了一个, 增加了 ArchLinux PKGBUILD https://github.com/ttys3/xar

参考文档

https://www.reddit.com/r/rust/comments/6rxoty/tutorial_cross_compiling_from_linux_for_osx/

https://wapl.es/rust/2019/02/17/rust-cross-compile-linux-to-macos.html

https://github.com/tpoechtrager/osxcross

https://github.com/rust-lang/backtrace-rs/issues/240

https://gist.github.com/Edu4rdSHL/49b8b52bac916d88e32861a817d5f9a5

https://john-millikin.com/notes-on-cross-compiling-rust

https://github.com/ttys3/xar/blob/master/xar.md

https://github.com/rust-embedded/cross

https://github.com/etrombly/gtk_osx_cross/blob/master/README.md

https://rust-lang.github.io/rustup-components-history/x86_64-apple-darwin.html