Thin LTO not working

[TCROC]

Zig Version

0.13.0

Steps to Reproduce and Observed Behavior

Attempting to use zig cc or zig c++ with the -flto=thin flag always results in using full lto. This is a problem for large projects. In my particular case, building the godot game engine from source: https://github.com/godotengine/godot

And this is the project I'm creating that leverages zig:

https://github.com/lange-studios/godot-src

Now I attempted to add this feature myself, but I currently can't build zig due to this issue:

#21941

Expected Behavior

Should use thin lto when passing -flto=thin rather than full lto.

[TCROC]

And for an explanation as to why thin lto is necessary, LLVM has docs here explaining: https://clang.llvm.org/docs/ThinLTO.html

The TL;DR; is full lto loads A LOT into RAM / memory. And when a program gets too big, it can outgrow memory capacity. Leaving one with 2 choices:

1. Download buy more ram
2. Thin LTO! :)

[BreadTom]

Attempting to use zig cc or zig c++ with the -flto=thin flag always results in using full lto. This is a problem for large projects. In my particular case, building the godot game engine from source: https://github.com/godotengine/godot

Can you please provide data on how much memory -flto uses on your system?

[TCROC]

Can you please provide data on how much memory -flto uses on your system?

Absolutely! :) I have 32 GB of RAM. It uses all of this + an extra 2 GB of swap memory. So 34 GB in total and then it crashes because it runs out.

Now this is on my local machine. The issue gets further exasperated when running on Microsoft Hosted Agents in Azure Piplines due to their hardware: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#hardware and time limitations: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#capabilities-and-limitations

The time limitation of "Private project: One free parallel job that can run for up to 60 minutes each time, until you've used 1,800". This is a problem when we are building for ios on mac agents and full lto. Full lto takes too long and times out, but Thin lto finishes significantly faster. I expect it would run out of RAM anyways on the self hosted agent even if Microsoft did increase their time limit.

Further details if interested:

So either way I try to look at it, full lto is a bottleneck for building godot (https://github.com/godotengine/godot). But Thin LTO works perfectly! I'm currently using clang directly on the MacOS agents rather than zig and it works very well :). However, my desire is to use zig so that I can leverage its cross compilation capabilities for this project we at Lange Studios started: https://github.com/lange-studios/godot-src. Long term vision being making cross compilation from any desktop to any godot target a staple feature of the godot game engine! :) Powered of course by zig!

And in case you are wondering why we need lto, it is because video games are performance intensive. And LTO provides significant performance benefits for real time games! Our game in particular here: https://store.steampowered.com/app/1343040/Blocky_Ball/. Which we are developing a sequel for in Godot called Blocky Ball OT + maintaining our own Godot fork and contributing back to Godot + using zig for the godot-src project we started :)

[TCROC]

@alexrp should I close this issue now that the PR went through? Or are there still things to do before closing?

[alexrp]

Oh, I hadn't noticed that it didn't close automatically.

[TCROC]

Haha I hadn't either. Was scrolling open issues I was working on and noticed this one was still open 😅