feat(processing_engine): improve package management for standalone python.

[jacksonrnewhouse]

This adds another package manager specifically for standalone python. I also added a default plugin dir in the docker image of /plugin, as folks had had permissions issues with our existing images, particularly on docker.

You can now run the docker image without any additional arguments and it'll run a plugin manager using the standalone python.

@jdstrand, I've only tested the images on OS X, but I don't think this will cause any issues on other platforms.

[jdstrand]

@jdstrand, I've only tested the images on OS X, but I don't think this will cause any issues on other platforms.

@jacksonrnewhouse - I'm taking that to mean only the docker images on OSX. Is this PR expected to also help with non-docker images built with python standalone where the python directory is a subdir under the binary? Eg:

$ tar -ztvf influxdb3-3.0.0+snapshot-8daccb7e_darwin_arm64.tar.gz
...
drwxrwxr-x 1001/1002         0 2025-02-13 19:43 influxdb3-3.0.0+snapshot-8daccb7e/python/
...
drwxrwxr-x 1001/1002         0 2025-02-13 19:43 influxdb3-3.0.0+snapshot-8daccb7e/python/bin/
...
lrwxrwxrwx 1001/1002         0 2023-12-31 18:00 influxdb3-3.0.0+snapshot-8daccb7e/python/bin/python -> python3.11
-rwxrwxr-x 1001/1002       286 2023-12-31 18:00 influxdb3-3.0.0+snapshot-8daccb7e/python/bin/pip3
...
-rwxrwxr-x 1001/1002     49968 2023-12-31 18:00 influxdb3-3.0.0+snapshot-8daccb7e/python/bin/python3.11
...
-rwxrwxr-x 1001/1002       286 2023-12-31 18:00 influxdb3-3.0.0+snapshot-8daccb7e/python/bin/pip
...
-rwxr-xr-x root/root  89723728 2025-02-13 20:14 influxdb3-3.0.0+snapshot-8daccb7e/influxdb3

https://github.com/influxdata/influxdb/blob/main/README_processing_engine.md#local-development-with-python-build-standalone has more info.

[jdstrand]

@jdstrand, I've only tested the images on OS X, but I don't think this will cause any issues on other platforms.

@jacksonrnewhouse - I'm taking that to mean only the docker images on OSX. Is this PR expected to also help with non-docker images built with python standalone where the python directory is a subdir under the binary?

@jacksonrnewhouse - I like the idea of adjusting the build to set the paths at build rather than runtime. Let's sync up on this and discuss how you want things to look long term. Since I think those changes will be more substantial, in the interest of time, I created #26029 to address the venv issues I left open when #25969 was committed. It simply builds a little bit on the runtime setup to remove the regressions quickly and give some breathing room on designing the proper solution for how you want the paths setup.

[jdstrand]

@jdstrand, I've only tested the images on OS X, but I don't think this will cause any issues on other platforms.

@jacksonrnewhouse - I'm taking that to mean only the docker images on OSX. Is this PR expected to also help with non-docker images built with python standalone where the python directory is a subdir under the binary?

@jacksonrnewhouse - I like the idea of adjusting the build to set the paths at build rather than runtime. Let's sync up on this and discuss how you want things to look long term. Since I think those changes will be more substantial, in the interest of time, I created #26029 to address the venv issues I left open when #25969 was committed. It simply builds a little bit on the runtime setup to remove the regressions quickly and give some breathing room on designing the proper solution for how you want the paths setup.

Ok, various PRs have landed and #26042 adds some cleanup that altogether MAY be enough for the immediate term (and thus you can close this PR), but I still think adjusting the build to set the paths at build rather than runtime is a good idea. Like before, let's sync up on this and discuss how you want things to look long term.

[jdstrand]

@jacksonrnewhouse - should this be closed in favor of recent merges, #26047 and the discussions we had yesterday?