Zig passes incorrect sysroot to Clang when combined with Nix

[winterqt]

Zig Version

0.10.1

Steps to Reproduce and Observed Behavior

This is once again a bug that I've only been able to observe when compiling 0.10.1, but I believe it is still present in master, and should be fixed.

std.zig.system.NativePaths.detect has special support for Nix, by searching NIX_CFLAGS_COMPILE. When this is done, it results in absolute include directories being returned.

There are two codepaths that stage3's stage1 compilation for Darwin under Nix can take -- the sandboxed case and the non-sandboxed case:

zig/src/main.zig

Lines 2510 to 2527 in a5b34a6

	const has_sysroot = if (comptime builtin.target.isDarwin()) outer: {
	if (std.zig.system.darwin.isDarwinSDKInstalled(arena)) {
	const sdk = std.zig.system.darwin.getDarwinSDK(arena, target_info.target) orelse
	break :outer false;
	native_darwin_sdk = sdk;
	try clang_argv.ensureUnusedCapacity(2);
	clang_argv.appendAssumeCapacity("-isysroot");
	clang_argv.appendAssumeCapacity(sdk.path);
	break :outer true;
	} else break :outer false;
	} else false;
	try clang_argv.ensureUnusedCapacity(paths.include_dirs.items.len * 2);
	const isystem_flag = if (has_sysroot) "-iwithsysroot" else "-isystem";
	for (paths.include_dirs.items) |include_dir| {
	clang_argv.appendAssumeCapacity(isystem_flag);
	clang_argv.appendAssumeCapacity(include_dir);
	}

The sandboxed case is when the macOS SDK cannot be found, resulting in -isystem being passed to specify include directories. The issue is when the sandbox is disabled, and Xcode is installed. This results in -isysroot specifying a path to the macOS SDK, and absolute include directories being specified with -iwithsysroot.

For example, this results in -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -iwithsysroot /nix/store/j88zc0bv69jpiyh3k2g6s67x3i3wgs24-llvm-15.0.7-dev/include being passed to Clang, which results in the incorrect search path /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//nix/store/j88zc0bv69jpiyh3k2g6s67x3i3wgs24-llvm-15.0.7-dev/include being used.

This manifests itself as failures to find headers such as stdlib.h when compiling stage1 from stage3, as @kubkon observed in #14559 (comment).

There's at least a few ways to fix this, but I'm unsure which is the most optimal. For example, we can return the is_nix value from NativePaths.detect and special-case has_sysroot to always be false if is_nix is true (which I've confirmed does fix the issue).

Expected Behavior

For it to pass the correct flags.

[kubkon]

Excellent! Thanks for investigating and submitting a bug report!

[ryansname]

I'm running into a similar problem with sysroot and not finding headers on macOS with nix when building a project linking with C libraries. After some searching I've found that a standard zig build will create a clang command line containing this (edited for clarity):

...
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk \
-iwithsysroot  /nix/store/5fg14klyax2ddmd5qy2fnag8wgzq3a47-curl-8.1.1-dev/include \
-iwithsysroot /nix/store/7z836lazw1glvjlzvv1ry1h16cd577my-brotli-1.0.9-dev/include \
... more iwithsysroots

I work around the errors by manually specifying the sys root on the zig invocation like so: zig build --sysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/
Which leads to the following flags given to clang (notably, no -iwithsysroot):

...
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/
...

$ zig version
0.11.0-dev.3191+fd213accb

[kubkon]

Hey @winterqt, I am currently looking into this issue as part of a different issue, and have a design question for you. What would you say is the correct behavior for an impure build: should Zig see native macOS SDK or strictly link against Nix provided headers and libs? In other words, if we detect Nix on macOS when building natively, should we detect the Apple SDK or not?

[kubkon]

For context, here's the changeset I am correctly working on: https://github.com/ziglang/zig/compare/macos-autodetect-sdk-tmp The idea is that if we are targeting an Apple platform from an Apple device (well, mostly from macOS), we try to link against the Apple SDK provided it exists. With Nix coming into play, do we then create the following decision matrix:

1. If Darwin-on-Darwin and if Nix detected, don't try using Apple SDK.
2. If Darwin-on-Darwin, try using Apple SDK
3. Else, use Zig shipped headers and libSystem stab

I will see if I can hack on nixpkgs on macOS and see if my changeset makes sense. I am also trying to make sure that zig libc reports correct paths on both macOS without Nix, and with Nix.