Fixing Rust Cross-Compilation in Yocto: Architecting Hermetic Build Layers for Embedded Linux

Our client needed to deploy their new Rust firmware to custom ARM-based embedded hardware using Yocto, an industrial-grade Linux build system. The challenge wasn't just technical complexity—it was a fundamental architectural conflict between two systems designed with opposing philosophies.

Cargo, Rust's build system, is designed for rapid iteration in modern software development. It dynamically downloads dependencies from crates.io during compilation, expects constant internet access, and resolves version constraints on-the-fly. This works brilliantly for web services and desktop applications where the build machine has unrestricted network access.

Yocto, by contrast, is designed for industrial reproducibility and security-critical deployments. It requires all source code to be fetched and cryptographically verified before compilation begins, then enforces complete network isolation during the build phase. This "fetch once, build offline" model is non-negotiable for clients in secure facilities, regulatory environments, or scenarios requiring reproducible builds for compliance audits.

The symptom appeared immediately in our CI pipeline: attempting to run cargo build inside Yocto's do_compile task resulted in "Network Unreachable" errors. Cargo was trying to access crates.io during the build phase, but Yocto had already disabled network access as part of its security model.

We initially tried the obvious workaround—temporarily re-enabling network access during compilation. This "worked" in the sense that builds completed, but the resulting binaries were fundamentally broken. The firmware would crash on the target ARM64 hardware with segmentation faults within seconds of boot. The root cause: the Rust build was linking against OpenSSL libraries from the build machine (x86_64 architecture) instead of the cross-compiled ARM64 libraries prepared by Yocto. This ABI incompatibility meant the firmware was trying to execute x86_64 machine code on an ARM processor—a guaranteed crash.

The stakes were significant. The client had already invested months in porting their firmware to Rust specifically to eliminate memory safety bugs. They had committed delivery timelines to their customers based on the Rust migration schedule. A blocked deployment would not only waste that investment but also delay critical product improvements by months while we found an alternative approach.

Our team needed to understand precisely why Cargo was ignoring Yocto's carefully prepared cross-compilation environment. We suspected "host contamination"—a common cross-compilation failure mode where the build system inadvertently links against libraries from the build machine instead of the target architecture.

To prove this hypothesis, we instrumented the build process with strace, a Linux system call tracing tool that logs every file access operation:

strace -f -e trace=openat,open cargo build --release 2>&1 | grep -E "libssl|pkgconfig"

This command followed every subprocess spawned by Cargo and logged which files they opened. The output immediately revealed the problem:

openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/pkgconfig/openssl.pc", O_RDONLY) = 3

openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", O_RDONLY) = 4

Expected: /home/builder/yocto/build/tmp/sysroots/aarch64-poky-linux/usr/lib/libssl.so

Actual:   /usr/lib/x86_64-linux-gnu/libssl.so.1.1

The build system was opening library files from /usr/lib/x86_64-linux-gnu/—the build host's native library directory—when it should have been accessing the Yocto-prepared sysroot containing ARM64 libraries. This explained the segmentation faults: the firmware binary contained references to x86_64 shared libraries that didn't exist on the ARM target device.

Digging deeper into the trace logs, we identified the culprit: the openssl-sys Rust crate uses pkg-config (a library metadata tool) during its build process to locate OpenSSL headers and libraries. Yocto exports environment variables like PKG_CONFIG_PATH and PKG_CONFIG_SYSROOT_DIR to tell pkg-config where to find cross-compiled libraries. However, these variables weren't propagating through Cargo's build script layer to the actual pkg-config invocation. The tool was falling back to its default behavior: searching the host system's library paths.

This was a textbook case of environment variable propagation failure across complex build system boundaries—the kind of issue that's obvious in hindsight but can consume days of debugging without proper instrumentation.

Why Standard Approaches Failed
Before architecting our final solution, our team evaluated several "obvious" fixes that ultimately proved inadequate.

The cargo vendor Approach: We initially attempted to use cargo vendor, a Cargo subcommand that downloads all dependencies into a local directory. The idea was to commit this vendored directory to git, eliminating the need for network access during builds.

Failure Mode: This bloated the repository size by over 400MB—the 150+ Rust dependencies included substantial code. More critically, it made code reviews practically impossible. Every time we updated dependencies, the pull request would show thousands of changed files in the vendor directory, drowning out the actual code changes we needed to review. Our automated diff-based patch management system also broke, as it couldn't distinguish between "real" changes and vendored code churn. For a team practicing rigorous code review, this approach was untenable.

The meta-rust Default Layer: Yocto has a community-maintained meta-rust layer that provides Rust support. We tried using it as-is, hoping it would handle our requirements.

Failure Mode: The meta-rust layer works well for simple Rust applications with minimal dependencies. Our firmware, however, had a complex dependency tree involving bindgen (a tool that generates Rust bindings from C header files) and extensive C interoperability for hardware control. The firmware needed to interface directly with custom ASIC registers via ioctl system calls, requiring precise control over how bindgen processed our hardware-specific C headers. The standard meta-rust layer lacked the configuration hooks to inject the specific Clang compiler arguments needed for our cross-compilation scenario. We could have forked and modified meta-rust, but that would have created a maintenance burden as we diverged from the upstream.

Rather than trying to force-fit our requirements into existing solutions, our team architected a dedicated Yocto layer designed specifically for hermetic Rust cross-compilation with complex dependencies.

The Architectural Decision: cargo-bitbake Integration
The core of our solution was cargo-bitbake, a tool that bridges the Cargo and BitBake ecosystems. It parses a Rust project's Cargo.lock file (which lists exact dependency versions) and automatically generates individual BitBake recipes for each crate.

Here's how it transforms the problem: instead of Cargo downloading dependencies at build time, we generate a .bb (BitBake recipe) file for each of the 150+ crates. Each recipe specifies the crate's download URL from crates.io and its SHA256 checksum. BitBake then handles these as it would any other source package—downloading during the do_fetch phase (when network access is allowed) and verifying checksums before proceeding.

The practical workflow:

# Run this during development when dependencies change

cargo-bitbake --output recipes-rust/ ./Cargo.toml

# This generates recipes like:

# recipes-rust/serde/serde_1.0.152.bb

# recipes-rust/tokio/tokio_1.25.0.bb

# ... (150+ total)

Each generated recipe is simple but effective:

SUMMARY = "serde: serialization framework"

LICENSE = "MIT | Apache-2.0"

SRC_URI = "[https://crates.io/api/v1/crates/serde/1.0.152/download](https://crates.io/api/v1/crates/serde/1.0.152/download)"

SRC_URI[sha256sum] = "bb7d1f0d3021d347a83e556fc4683dea2ea09d87bccdf88ff5c12545d89d5efb"

When BitBake processes these recipes, it downloads each crate tarball during do_fetch, verifies the checksum, and stores them in its download cache. Later, during do_compile, Cargo can access these pre-downloaded crates without needing network access—BitBake has already prepared everything.

Why cargo-bitbake vs. manual recipe maintenance? Maintaining 150+ recipes by hand would be prohibitive. Every time we update dependencies in Cargo.lock, we'd need to manually update version numbers and SHA256 checksums in the corresponding recipes—a process taking hours and prone to copy-paste errors. With cargo-bitbake, regenerating all recipes after a dependency update takes about 30 seconds of automated processing.

Why Yocto vs. alternative build systems? We evaluated Buildroot, which has simpler Rust integration. However, our client was using an NXP-provided Board Support Package (BSP) for their i.MX8 processor, which was tightly integrated with Yocto. The BSP included custom Yocto layers for GPU driver support, power management firmware, and hardware-accelerated graphics. Switching to Buildroot would have meant re-implementing or porting these layers—a 6-8 week effort with significant regression testing risk. Staying with Yocto and solving the Rust integration problem was the pragmatic choice.

Solving the bindgen Problem
The most technically challenging aspect was configuring bindgen, which the firmware used to generate Rust bindings for hardware control. The firmware communicated with custom ASIC registers via ioctl system calls, and bindgen auto-generated the necessary Rust code from C header files.

Before our fix, bindgen would fail with cryptic errors:

error[E0425]: cannot find type `__u64` in this scope

  --> src/ioctl.rs:12:1

   |

12 | ioctl_read!(read_device_status, 0x40, DeviceStatus);

   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope

This error occurred because bindgen (which uses libclang internally) was parsing the build host's C headers at /usr/include/asm/types.h. These headers define types for x86_64 architecture. On ARM64, the equivalent types are defined in /usr/include/asm-generic/types.h with different sizes and alignments. When bindgen generated Rust code using the wrong type definitions, the resulting structures had incorrect memory layouts, causing the "type not found" errors.

We needed to force bindgen to use Yocto's cross-compilation sysroot—a directory containing all headers and libraries for the target ARM64 system. We created a custom BitBake class that injected the necessary environment variables:

# meta-probots-layer/classes/probots-rust.bbclass

# Force bindgen to use the cross-compilation sysroot for header lookup

export BINDGEN_EXTRA_CLANG_ARGS = "--sysroot=${STAGING_DIR_TARGET} \

                                   -I${STAGING_DIR_TARGET}/usr/include"

# Enforce pkg-config isolation (fixes the original OpenSSL problem)

export PKG_CONFIG_ALLOW_CROSS = "1"

export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR_TARGET}"

# These variables are inherited by all Cargo build.rs scripts via the 

# cargo.bbclass wrapper, which exports them before invoking rustc

The BINDGEN_EXTRA_CLANG_ARGS variable tells bindgen's libclang backend to treat ${STAGING_DIR_TARGET} (Yocto's ARM64 sysroot) as the system root directory. This ensures it only sees ARM64 headers. The PKG_CONFIG_* variables solve the original OpenSSL problem we diagnosed with strace, ensuring pkg-config always searches the cross-compilation sysroot.

After this configuration, bindgen generated correct ARM64 bindings, and the firmware compiled successfully with proper type definitions for the target architecture.

The custom Yocto layer delivered measurable improvements across multiple dimensions that mattered to both the engineering team and business stakeholders.

Build Reproducibility: We achieved 100% SHA256 reproducibility—a critical requirement for security-conscious deployments. Our team validated this by performing 50 consecutive builds on different build machines; every resulting firmware binary had an identical cryptographic hash. This means every build produces bit-for-bit identical output regardless of which machine performs the build or when it occurs. For regulatory compliance and security audits, this is essential: it proves that the deployed firmware exactly matches the reviewed and approved source code, with no possibility of undocumented changes or supply chain tampering.

Build Performance: Incremental build times dropped dramatically from 15 minutes to 45 seconds after Yocto's shared state (sstate) cache warmed up. This happened because Yocto could now cache individual compiled Rust crates as it would any other build artifact. When a developer changed a single Rust source file, only that crate and its direct dependents needed recompilation—the other 140+ dependencies were retrieved from cache. For the development team, this meant faster iteration cycles: make a change, rebuild, test in under a minute rather than taking a coffee break during every rebuild. Over a typical development week, this saved approximately 8-10 hours per engineer in waiting time.

Repository Hygiene: We eliminated the 400MB vendor directory from git. Instead of storing dependency source code in version control, BitBake manages it as ephemeral build artifacts—downloaded during do_fetch, used during compilation, and discarded after. This kept the git repository focused on actual source code, making code reviews tractable again. Pull requests showed meaningful changes rather than thousands of lines of vendored dependency updates.

Air-Gap Validation: The system passed a 72-hour continuous integration stress test with the network interface physically disabled after the initial do_fetch phase completed. Our testing team ran the full CI pipeline—including firmware builds, unit tests, integration tests, and deployment verification—continuously for three days with zero network access. This proved the build system was truly hermetic and offline-capable, satisfying security requirements for clients deploying in isolated facilities or regulated environments where internet-connected build systems are prohibited.

These results addressed both technical requirements (reproducibility, build performance) and business needs (developer productivity, compliance readiness, security posture). The client could now deploy with confidence, knowing their build system met industrial standards for embedded Linux development.

Integrating modern languages like Rust into industrial embedded build systems requires expertise that spans multiple specialized domains. It's not enough to understand Rust or to understand Yocto—you need deep knowledge of both, plus the experience to navigate the impedance mismatches where they intersect.

Our team at Probots has production experience across embedded Linux build systems (Yocto, Buildroot), modern systems languages (Rust, modern C++), cross-compilation toolchains, and hardware bring-up for custom SoCs. This combination is rare in the industry. Most Rust developers work in cloud or web contexts and have never touched Yocto. Most embedded Linux engineers work primarily in C and view Rust as unproven in embedded contexts. The intersection—engineers who can architect production-grade solutions bridging both worlds—is small.

We've deployed similar solutions across multiple hardware platforms (i.MX8, Zynq SoC, Jetson) and application domains (industrial automation, mining equipment, robotics). Each project has unique constraints—some need real-time guarantees, others have strict code size limits, some operate in extreme environmental conditions. Our approach is to understand these constraints deeply, then architect solutions that work within them rather than trying to force-fit generic approaches.

Facing a similar challenge integrating Rust into your embedded Linux build system?

Whether you're working with Yocto, Buildroot, or custom build infrastructure, our team can help architect a solution that maintains the safety benefits of Rust while meeting your reproducibility, security, and compliance requirements. Contact our engineering team for a consultation.