PSF Hires PyPI Safety and Security Engineer

https://pyfound.blogspot.com/2023/08/announcing-our-new-pypi-safety-security.html

by miketheman • 3 years ago

65 34 3 years ago

pyfound.blogspot.com

alanfranz• 3 years ago

PyPI is just one gear in the quite broken Python packaging mess. But I hope it’s a good starting point!

aeurielesn• 3 years ago

I'm waiting for the day when setuptools and pip are a thing of the past.

simonw3• 3 years ago

What would you expect to replace pip?

crabbone5• 3 years ago

Military vehicle rolls through the desert, US soldiers in the desert wind, IED hits the vehicle, guitar solo starts playing...

Waiting for the one...

More guitar solo, then drums

day that never comes

demizer• 3 years ago

As someone that has dealt with PSF and PyPI at a product level for packaging at the intersection of Python and AI for compute hardware, we consider PyPI an immature ecosystem. It's absolutely great for python libraries, but that's it. It's not ready for the flood of AI hardware that is coming. Mojo looks promising, but not sure how it's going to interact with PyPI.

KRAKRISMOTT2• 3 years ago

My wishlist: a mirror metadata service like Yarn to force pypi to innovate more. Pip is slow because it doesn't store the dependency metadata and has to download all upstream dependencies during the resolution phase. They were too busy wasting time on minor issues like domain squatting than core dependency resolution concerns. I also want to see Python support simultaneously conflicting diamond dependencies like Npm and Cargo. Python doesn't even support shading. One old dependency and your whole build breaks, with no recourse aside from forking upstream.

thejcannon• 3 years ago

Your first request is granted: https://peps.python.org/pep-0658/

bombolo3• 3 years ago

If your software requires 2 different versions of the same library, there is something deeply wrong with it.

fooker2• 3 years ago

It's fairly common in the compilers/tools world to have automated workflows to test against different tools and libraries.

KRAKRISMOTT2• 3 years ago

Most of the time diamond dependencies are upstream, not because the software itself requires two versions.

mirashii• 3 years ago

Pip is slow because it doesn't store the dependency metadata and has to download all upstream dependencies during the resolution phase.

This isn’t really true and hasn’t been for a long time. It relies on package authors to include the dependency metadata, but pypi has published it and pip has used it going back to at least 2017, if not earlier. They could do more to enforce it, but there’s a lot more to the story of why pip is slow than this.

crabbone5• 3 years ago

Role mentions security. Candidate shows no experience with security.

It would've been weird... but then you remember it's Python. It's by amateurs for amateurs. Well, god speed and god bless. Who knows, maybe despite the counter-indications something good will come out of it.

kstrauser2• 3 years ago

"By amateurs" is a really bizarre label to hit Python with.

crabbone5• 3 years ago

You must have never looked at CPython code, nor have you ever read the documentation published with Python, nor have you ever read any PEPs, nor have you ever been on the core-dev mailing list / have read its archives.

It would've been really funny how so many people are using this bizarre nonsense, if it weren't for the fact that me and you might also have to use it, directly or indirectly.

woodruffw2• 3 years ago

Having done plenty of work on PyPI, including lots of security work: Mike is an extremely competent engineer, and has been a significant force[1] in PyPI's overall modernization and maintenance efforts. That kind of work is security work, and it's frequently thankless.

[1] https://github.com/pypi/warehouse/commits?author=miketheman

crabbone5• 3 years ago

has been a significant force[1] in PyPI's overall modernization and maintenance efforts.

PyPI is a pile of hot garbage. Being one of people contributing to it is a negative in my book. That's definitely not anything to brag about.

How's that a security work again? What's even the connection?

kapilvt2• 3 years ago

in the name of supply chain security, i just want verified package signatures (cosign, not the extant unused gpg), the new passwordless publication is good step towards (get humans and static credentials out of pushing assets). actually one more minor, support for poetry in pip-audit.. https://github.com/pypa/pip-audit/issues/84

bit_flipper• 3 years ago

Where is the root of trust for package signatures? Who is verifying signatures: the package index or end-users? How do you distribute public keys? PGP is mostly maligned because of its support for old cryptography standards, some needless cruft, and especially the poor usability of its defacto standardized implementation in GPG, but cosign by itself doesn't actually make any of the trust questions I mentioned go away. There are major tradeoffs to be made about who-trusts-who and what that actually means in terms of security beyond just theatre. I'm not convinced that there exists a good trust mechanism that a package index can enforce that actually moves the needle on supply chain security.

kapilvt2• 3 years ago

closer to TLS CA setup with ephemeral certificates, a public log of issuance, then pgp individual trust circles and semi static keys.

https://www.sigstore.dev/how-it-works

woodruffw2• 3 years ago

FWIW: We're unlikely to support Poetry directly in pip-audit (I said as much in that issue, but it's a little buried). Instead we'll probably devolve the auditing "core" of pip-audit into its own library, which the poetry folks can then use, if they'd like.

ordx• 3 years ago

The guy is a chair at the Women's Flat Track Roller Derby Association and previously worked at a cannabis company. Interesting career change.

miketheman• 3 years ago

Software's software ;)

alexk3073• 3 years ago

I wrote a few scripts years ago to automate the creation of packages, name them something similar to a popular project (typo squatting), and then verify them by creating a new pypi user with a new email address. The payload was just an http request to a server I ran.

Submitted this to pycon, and I let the PSF know. Neither were interested. Haven’t trusted a single Python package since.

simonw3• 3 years ago

What measures would you suggest to prevent people from doing that?

alexk3073• 3 years ago

Better email verification (at the time there wasn’t even a captcha to solve), detection of typo squatting (pip install djanho or pip install jango), limiting number of packages that can be registered in a time period by IP and/or user, prompting the user that they’re installing a package with a single Levenshtein distance away from a top 100 package. It feels crazy that I can setup a mine field of packages that trick people into thinking they’ve installed the correct packages. It just takes one typo or mistake to install a package that runs an arbitrary potentially malicious script without warning.

https://github.com/alexk307/pypi

benatkin• 3 years ago

Please be vendor neutral and don't launch something that is GitHub only and centralized and faun over GitHub's security. https://github.com/python-poetry/poetry/issues/7940#issuecom... https://news.ycombinator.com/item?id=35646436

PSF Hires PyPI Safety and Security Engineer

Related Stories