Making PyPI's test suite 81% faster

https://blog.trailofbits.com/2025/05/01/making-pypis-test-suite-81-faster/

by woodruffw • 1 year ago

11 2 1 year ago

blog.trailofbits.com

odie5533• 1 year ago

Great post! Regarding the database, would the fixture possibly benefit from creating a single database in postgres, and then making copies of it for each worker using https://www.postgresql.org/docs/current/manage-ag-templatedb...? Then it doesn't need to run the migration lots of times. Possibly even faster than the squashed migration?

miketheman• 1 year ago

Excellent idea, and something I tried a little while back. The `pytest-postgresql` plugin used has the ability to do this natively, but when we tried it out we found that we had other issues with developing on a Linux machine.

Attempt: https://github.com/pypi/warehouse/pull/15365

Revert: https://github.com/pypi/warehouse/pull/15444

If you've got experience with making this kind of thing work on Linux development machines, it'd be great to have some help getting that back.

Making PyPI's test suite 81% faster

Related Stories