Accessing build scripts stored in private pypi repos

[couling]

There's currently a bit of a gap with the design of PEP 517/518 for private package indexes. I'll admit this is quite niche.

From what I can read, the current design for hatch-build-scripts means that if we need to make other python packages available to a build script, we need to include those in the [build-system] requires = [...] in pyproject.toml.

The problem with this is that, AFAIK, there's no way to specify alternative package indexes. This means that it's very hard to create shared build scripts where your org uses it's own private pypi repo.

I was wondering if there are any thoughts specific to hatch-build-scrips, on how build scripts can access packages from private pypi repositories.

There's some discussion of this here:

• we need a way to specify the use of a custom package index for a build dependency pypa/pyproject-hooks#69
• https://github.com/orgs/python-poetry/discussions/8177
• https://discuss.python.org/t/is-there-any-standard-way-to-use-a-package-from-a-private-pypi-in-build-system-requires/93481

[rmorshea]

At most companies I've been at there's an internal mirror of pypi.org with the ability to upload your own packages to it. That way you don't have to support two separate public/private indexes. That'd be my recommended approach. There's a number of solutions out there for PyPI mirroring:

• managed: https://jfrog.com/help/r/jfrog-artifactory-documentation/pypi-repositories
• self-hosted: https://packaging.python.org/en/latest/guides/hosting-your-own-index/

[rmorshea]

I see this was suggested in one of the linked threads already.

I'm not entirely sure that this project would be the right place to solve this problem. Even if it was, I'm not planning to add any major features since I've moved on from using Hatch myself, so I'm just doing basic bug fixes.

[couling]

Alas! Reforming a big corp's internal setup around this one issue isn't going to happen. There's a reason poetry and others included an ability to specify dependency sources in their project config.

Our default position at this point is to abandon working sdist on a few libraries, but that's a bit frustrating, and more of a pain for the architecture specific ones.

[rmorshea]

Looking at one of your prior comments mentioning how there are a variety of indexes you have to work with, I'm wondering if there'd be a way to set up an index that just mirrors all of them. Or at least to start out, the ones you need. From what I can understand, at its core a PyPI is just a file server so doesn't seem too hard to setup.