Rust Cross Compile OSX target under Linux

this article first post on 2021-01-25 updated on 2023-08-28

起因主要是想给我fork自convco的git-cz 项目 release 那里增加一个Mac二进制文件方便使用Mac的人下载。

这是一个方便使用约定式提交记录的git工具.

Environment

OS: Fedora 38 (Workstation Edition) x86_64
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz x86_64
rustc 1.72.0 (5680fa18f 2023-08-23)
cargo 1.72.0 (103a7ff2e 2023-08-15)

Requirements

# Install build dependencies

# CentOS Stream / Fedora 38
dnf group install "Development Tools"

dnf install -y \
        clang-devel clang \
        gcc \
        gcc-c++ \
        zlib-devel \
        libmpc-devel \
        mpfr-devel \
        gmp-devel

# Ubuntu 23.04
apt install \
    clang \
    gcc \
    g++ \
    zlib1g-dev \
    libmpc-dev \
    libmpfr-dev \
    libgmp-dev

# Add macOS Rust target
rustup target add x86_64-apple-darwin

Building OSXCross Toolchain

OSXCross 是什么？是一个快速让你在Linux上安装Mac交叉编译toolchain的工具。

很多复杂或麻烦的事情都被它处理好了。

它甚至还带了一个包管理器：

OSXCross comes with a minimalistic MacPorts Packet Manager. See README.MACPORTS for more.

安装步骤：

1. 准备 SDK

这步骤主要参考： https://github.com/tpoechtrager/osxcross#packaging-the-sdk

-- Xcode up to 15 Beta 6 is known to work. -- Use Firefox if you have problems signing in.

osxcross now support the latest Xcode 15 Beta 6, that's good.

构建这个toolchain需要苹果的SDK，这个需要从Xcode提取。去 https://developer.apple.com/download/all/ 下载最新版的Xcode (需要Apple ID登录), beta版的不需要从app store下载，下载回来是一个xip后缀的文件.

我这里下载当前最新的最低可以支持到较旧版本的 macOS 12 的 Xcode 版本 Xcode_14.2.xip

according to https://xcodereleases.com/?scope=release

the latest Xcode we can use to support macOS 12 is: Xcode 14.2

Version	Released	Requires	SDKs
Xcode 15.0 Beta 7	22 Aug 2023	macOS 13.4+	macOS 14.0 (23A5326c)
Xcode 14.3	30 Mar 2023	macOS 13.0+	macOS 13.3 (22E245)
Xcode 14.2	13 Dec 2022	macOS 12.5+	macOS 13.1 (22C55)
Xcode 13.2.1	17 Dec 2021	macOS 11.3+	macOS 12.1 (21C46)

https://download.developer.apple.com/Developer_Tools/Xcode_14.2/Xcode_14.2.xip

ps: 老灯还发现了一个下载 xcode 的好地方： https://xcodereleases.com/

Packing the SDK on Linux - Method 1 (Xcode > 8.0):

确保已经安装了打包SDK过程中需要的clang, make, libssl-devel, lzma-devel and libxml2-devel


# 如果之前有下载过, 记得删除 rm -rf osxcross , 不然连接的旧的库可能会导致 pbzx 等无法执行
git clone https://github.com/tpoechtrager/osxcross
cd osxcross

# ubuntu lzma.h: sudo apt install -y liblzma-dev

# /home/ttys3/Downloads/Compressed/Xcode_14.2.xip 是下载好的 `Xcode_14.2.xip` 路径
# size almost 7.1 GB
./tools/gen_sdk_package_pbzx.sh /home/ttys3/Downloads/Compressed/Xcode_14.2.xip

这里需要解包 xip 文档，再用xz 打包, 要一小会。

took 9m25s max disk usage: 29 GB

打包成功会在 OSXCross 根目录下生成 MacOSX13.1.sdk.tar.xz and MacOSX13.sdk.tar.xz

将 MacOSX13.1.sdk.tar.xz 移动到tarballs 目录下：

mv MacOSX13.1.sdk.tar.xz ./tarballs

#  we do not need MacOSX13.sdk.tar.xz
rm MacOSX13.sdk.tar.xz

2. 构建toolchain

确保你的系统已经安装了必要的组件，如 Clang 3.9+, cmake, git, patch, Python, libssl-devel (openssl) lzma-devel, libxml2-devel and the bash shell

可选： Optional:

llvm-devel: For Link Time Optimization support llvm-devel: For ld64 -bitcode_bundle support uuid-devel: For ld64 -random_uuid support

In clang there is no difference between cross-compilation and native compilation, so OSXCross can use a normal clang install for both. You can use either a clang installation you already have, or build your own from source.

由于我们已经安装了系统clang了，因此不用手动再编译.

我们接下来直接使用clang 编译 cross toolchain, 执行 ./build.sh ：

  ~/repo/toolchain/osxcross on  master 
❯ ./build.sh
found SDK version 13.1 at tarballs/MacOSX13.1.sdk.tar.xz
verified at /home/ttys3/repo/rust/macos/osxcross/tarballs/MacOSX13.1.sdk.tar.xz

Building OSXCross toolchain, Version: 1.4

macOS SDK Version: 13.1, Target: darwin22.2
Minimum targeted macOS Version: 10.9
Tarball Directory: /home/ttys3/repo/rust/macos/osxcross/tarballs
Build Directory: /home/ttys3/repo/rust/macos/osxcross/build
Install Directory: /home/ttys3/repo/rust/macos/osxcross/target
SDK Install Directory: /home/ttys3/repo/rust/macos/osxcross/target/SDK

Press enter to start building

# 或使用UNATTENDED=1跳过问题直接构建
UNATTENDED=1 ./build.sh

构建成功后会有类似如下提示：

Do not forget to add

/home/ttys3/repo/rust/macos/osxcross/target/bin

to your PATH variable.

All done! Now you can use o32-clang(++) and o64-clang(++) like a normal compiler.

Example usage:

Example 1: CC=o32-clang ./configure --host=i386-apple-darwin22.2
Example 2: CC=i386-apple-darwin22.2-clang ./configure --host=i386-apple-darwin22.2
Example 3: o64-clang -Wall test.c -o test
Example 4: x86_64-apple-darwin22.2-strip -x test

!!! Use aarch64-apple-darwin22.2-* instead of arm64-* when dealing with Automake !!!
!!! CC=aarch64-apple-darwin22.2-clang ./configure --host=aarch64-apple-darwin22.2 !!!
!!! CC="aarch64-apple-darwin22.2-clang -arch arm64e" ./configure --host=aarch64-apple-darwin22.2 !!!

Your SDK does not support i386 anymore.
Use <= 10.13 SDK if you rely on i386 support.

Your SDK does not support libstdc++ anymore.
Use <= 10.13 SDK if you rely on libstdc++ support.

这里有关于32位支持和 libstdc++ 依赖的提示, 如果需要依赖这两个之一，就需要使用小于等于 10.13 的SDK

构建好的文件都在 target 目录下，直接放在这里显然不合适，我们把它移动到一个合适一点的地方, 比如 /usr/local/darwin-ndk-x86_64：

sudo mkdir /usr/local/darwin-ndk-x86_64

sudo mv target/* /usr/local/darwin-ndk-x86_64/

如果还需要使用gcc作为交叉编译器的话，还需要行 ./build_gcc.sh，不过这里我们是使用clang, 因此略过。

tips

链接错误问题:

x86_64-apple-darwin22.2-ld: error while loading shared libraries: libxar.so.1: cannot open shared object file: No such file or directory

❯ ls -l /usr/local/darwin-ndk-x86_64/lib
lrwxrwxrwx ttys3 ttys3  15 B  Wed Jan 27 10:46:58 2021  libtapi.so ⇒ libtapi.so.8svn
.rw-r--r-- ttys3 ttys3 3.2 MB Wed Jan 27 10:46:58 2021  libtapi.so.8svn
.rw-r--r-- ttys3 ttys3 205 KB Wed Jan 27 10:43:54 2021  libxar.a
.rw-r--r-- ttys3 ttys3 800 B  Wed Jan 27 10:43:54 2021  libxar.la
lrwxrwxrwx ttys3 ttys3  11 B  Wed Jan 27 10:43:54 2021  libxar.so ⇒ libxar.so.1
.rwxr-xr-x ttys3 ttys3 142 KB Wed Jan 27 10:43:54 2021  libxar.so.1

解决：

echo /usr/local/darwin-ndk-x86_64/lib | sudo tee /etc/ld.so.conf.d/darwin.conf
sudo ldconfig

ATTENTION:

OSXCross links libgcc and libstdc++ statically by default (this affects -foc-use-gcc-libstdc++ too). You can turn this behavior off with OSXCROSS_GCC_NO_STATIC_RUNTIME=1 (env).

The build also creates aliases *-g++-libc++ which link with the clang implementation of the C++ standard library instead of the GCC version. Don't use these variants unless you know what you're doing.

Configuring Cargo

编辑 .cargo/config, 增加

[target.x86_64-apple-darwin]
linker = "/usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin22.2-clang"
ar = "/usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin22.2-ar"

这里很重要，尤其是 linker, 如果没有配置， cargo 默认只会使用 cc, 而不会使用 clang, 即使你 export 了 CC=xxxxx, 在最后link的时候会报错( 报错类似于 https://gist.github.com/Edu4rdSHL/49b8b52bac916d88e32861a817d5f9a5 )

Building the project

PATH="/usr/local/darwin-ndk-x86_64/bin/:$PATH" \
CC=o64-clang \
CXX=o64-clang++ \
cargo build --target x86_64-apple-darwin

❯ file ./target/x86_64-apple-darwin/release/git-cz.darwin
./target/x86_64-apple-darwin/release/git-cz.darwin: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS>

OK, the binary: https://github.com/ttys3/git-cz/releases/download/v0.6.0/git-cz.darwin

Misc

o64-clang and x86_64-apple-darwin*-clang

link to the same wrapper: x86_64-apple-darwin22.2-wrapper

❯ ls -lh /usr/local/darwin-ndk-x86_64/bin/o64-clang
lrwxrwxrwx ttys3 ttys3 31 B Sun Jan 24 23:43:35 2021  /usr/local/darwin-ndk-x86_64/bin/o64-clang ⇒ x86_64-apple-darwin22.2-wrapper

❯ ls -lh /usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin22.2-clang
lrwxrwxrwx ttys3 ttys3 31 B Sun Jan 24 23:43:35 2021  /usr/local/darwin-ndk-x86_64/bin/x86_64-apple-darwin22.2-clang ⇒ x86_64-apple-darwin22.2-wrapper

check all wrapper been linked to:

ls -l /usr/local/darwin-ndk-x86_64/bin/* | rg wrapper

Building *-sys crates

带C绑定的一些lib(一般命名为*-sys, 如 libz-sys )如果要交叉编译，你还得设置CC和CXX环境变量。这样能编译成功了，但是最后链接阶段可能失败，原因是链接了错误的库.

有些库提供了环境变量可以方便使用交叉编译工具重新编译，比如 https://github.com/rust-lang/libz-sys/blob/master/build.rs#L25

如果设定了LIBZ_SYS_STATIC=1, 则会编译一个静态版本的, 由于我们指定了 CC 和 CXX 都是Mac的，因此可以成功链接。

PATH="/usr/local/darwin-ndk-x86_64/bin/:$PATH" \
CC=o64-clang \
CXX=o64-clang++ \
LIBZ_SYS_STATIC=1 \
cargo build --target x86_64-apple-darwin

About xip files

Unpacking XIP files on Linux

Install xar from https://mackyle.github.io/xar/ (or my fork https://github.com/ttys3/xar )
Install pbzx from https://github.com/NiklasRosenstein/pbzx (or my fork https://github.com/ttys3/pbzx/ ) (use gcc -llzma -lxar -I /usr/local/include pbzx.c -o pbzx and copy the binary into your PATH)
use xar -xf XIP_FILE -C /path/to/extract/to
Change to the directory where you extracted the file.
Use pbzx -n Content | cpio -i to extract the contents.

ref: https://gist.github.com/phracker/1944ce190e01963c550566b749bd2b54

不过在这里，我们并不需要按上面的步骤手动提取，OSXCross 都已经有自动解包和打包脚本，很全.

xar 这个库, 原版已经很多年没人维护了, 所以,这里其实作者用了它自己的 fork 版: https://github.com/tpoechtrager/xar

老灯也 fork 了一个, 增加了 ArchLinux PKGBUILD https://github.com/ttys3/xar

参考文档

https://www.reddit.com/r/rust/comments/6rxoty/tutorial_cross_compiling_from_linux_for_osx/

https://wapl.es/rust/2019/02/17/rust-cross-compile-linux-to-macos.html

https://github.com/tpoechtrager/osxcross

https://github.com/rust-lang/backtrace-rs/issues/240

https://gist.github.com/Edu4rdSHL/49b8b52bac916d88e32861a817d5f9a5

https://john-millikin.com/notes-on-cross-compiling-rust

https://github.com/ttys3/xar/blob/master/xar.md

https://github.com/rust-embedded/cross

https://github.com/etrombly/gtk_osx_cross/blob/master/README.md

https://rust-lang.github.io/rustup-components-history/x86_64-apple-darwin.html