gh-143120: pixi builds for free-threading and TSAN

[crusaderky]

Follow-up to #142469

• Add pixi infrastructure for
  • freethreading: compiles with --disable-gil
  • tsan-freethreading: compiles with --disable-gil --with-thread-sanitizer
• Refactor recipes:
  • Make recipe.yaml and pixi.toml all identical symlinks
  • Remove clang-19 since we are not building the experimental jit
  • Remove ld_impl because we are not hacking the makefiles
  • Add site_packages_path
• Add support for MacOS Intel (specifically macos-15-intel github runners)
• Signal support for Linux ARM (it already worked but was insofar untested)
• Remove ASAN/TSAN DLLs hack on MacOS

Tested in CI on https://github.com/crusaderky/hdf5-pixi

• Issue: Add pixi builds for TSan + free-threading to aid downstream development. #143120

[crusaderky]

copy-paste of asan/recipe.yaml, except this one line

[crusaderky]

copy-paste of asan/pixi.toml

[crusaderky]

copy-paste of asan/recipe.yaml, except this one line

[lucascolley]

@kumaraditya303 @lucascolley any idea what's wrong here?

no clue sorry I haven't seen this before

[crusaderky]

The tsan crash is due to the default mmap_rnd_bits on Ubuntu 24.04; it goes away after lowering it:

$ sudo sysctl vm.mmap_rnd_bits
vm.mmap_rnd_bits = 32  # too high
$ sudo sysctl vm.mmap_rnd_bits=28  # reduce it
vm.mmap_rnd_bits = 28

ubuntu-latest github actions workers seems to be unaffected.

[hugovk]

There's a lot of copy/paste here, which can make maintenance harder. Does pixi have some concept of code reuse or parametrisation? Maybe not the best example, but something like GitHub Actions' reusable workflows?

[ngoldbaum]

@lucascolley is there anything we can do right now to reduce the duplication here? Or does that require fixes in pixi?

[lucascolley]

As mentioned in the README already, this is blocked on prefix-dev/pixi#4599.

[ngoldbaum]

I opened an issue which is needed to make this mergeable. I'm also playing with a NumPy recipe based on this.

[crusaderky]

Ready for final review

[crusaderky]

@isuruf the libasan fix works in my hdf5 recipe, but not in the python recipe:
https://github.com/crusaderky/hdf5-pixi/actions/runs/21245433717/job/61133422587?pr=8
This is as of fd6ca6d.

Reverting for now.

[isuruf]

Your hdf5 recipe is still not using fd6ca6d. Can we please stop the reverting back and forth and debug why the correct commit from cpython is not being used?

[crusaderky]

For the sake of test velocity and thoroughness, I've created https://github.com/crusaderky/cpython-pixi
Latest Python and Python+NumPy builds can be seen here.

@isuruf the latest macos-15-intel changes let the python build complete; thank you.
Now however I'm seeing similar failures when building numpy ASAN/TSAN on the same VMs (not blocking this PR).

Pending

• (blocking) Confirmation that symlinks in the repository are OK
• (not blocking) Investigate why the latest libcompiler-rt fixes macos ASAN/TSAN builds for my hdf5-pixi project, but not for python. I don't have a mac so my velocity here is very limited. Note that the changes to this PR have been reverted in the last two commits.

[lucascolley]

(blocking) Confirmation that symlinks in the repository are OK

@hugovk would you be able to advise?

[crusaderky]

Your hdf5 recipe is still not using fd6ca6d. Can we please stop the reverting back and forth and debug why the correct commit from cpython is not being used?

@isuruf I appreciate your frustration. I'm frustrated too by this issue. I'm fairly sure that the last CI run I pointed to shows the effect of the libcompiler-rt changes - because I checked - but I'm fallible, so I'll do it again this time making extra sure not to muddle any evidence afterwards.

For the sake of velocity and clarity I've moved my CI to https://github.com/crusaderky/cpython-pixi
It takes several hours for CI to finish. I'll report back.

[hugovk]

(blocking) Confirmation that symlinks in the repository are OK

@hugovk would you be able to advise?

Let's not add symlinks to the repo, they're harder to use on Windows and make cross-platform support generally more complex.

[crusaderky]

The issue was that c_compiler is not a defined variable, and comparisons to undefined variables always return false.
So these two would always evaluate to false:

      - if: c_compiler == "gcc" and "san" in variant
        then:
          - libsanitizer
      - if: c_compiler == "clang" and "san" in variant
        then:
          - libcompiler-rt

Linux happened to work because, at least on ubuntu-latest and on my desktop, libasan.so and libtsan.so are installed by default system-wide.

I've opened an improvement ticket upstream: prefix-dev/pixi#5393

[crusaderky]

I've removed the symlinks and replaced them with physical copies + a sync script.
This PR has no known outstanding issues and should be ready for final review and merge.