How to Run Python in Production

https://ashishb.net/programming/python-in-production-2/

by ashishb • 1 year ago

34 30 1 year ago

ashishb.net

gnabgib• 1 year ago

But you don't recommend it? (33 points, 38 comments) https://news.ycombinator.com/item?id=43297671

ashishb11• 1 year ago

I don't recommend it.

But for certain use-cases, as mentioned in the article itself, it is unavoidable.

Further, you might be working at a company that has decided to use Python, and you can make Python more efficient and effective using these suggestions.

Hackbraten• 1 year ago

Multithreading is not a prerequisite for async in Python. The default task executor runs in a single thread. So using async APIs should be safe as long as you stick to the single-threaded executor.

ashishb11• 1 year ago

Good point. Have you seen any major advantage of using async API in practice?

phyrex• 1 year ago

It's... faster for IO bound workloads? That's literally the whole point of async

ibash• 1 year ago

If you don’t use `async def` and you are using fastapi it’s easy to accidentally block the event loop.

andrewstuart2• 1 year ago

Async programs can do multiple tasks.

Blocking python can’t.

An example is running multiprocesses and parsing their output and looking for patterns. Async python does this sort of thing well.

jb_gericke2• 1 year ago

It sounds like the author makes a blanket claim to avoid asynchronous functions, even when using an inherently asynchronous web framework (FastAPI), which would negate any FastAPI asynchronous concurrency gains.

ashishb11• 1 year ago

It sounds like the author makes a blanket claim to avoid asynchronous functions, even when using an inherently asynchronous web framework (FastAPI), which would negate any FastAPI asynchronous concurrency gains.

FastAPI is faster than the other popular framework, Flask (I posted data supporting that), even without using async.

I only suggested that one should use their discretion when using async.

When did I suggest not to write async code?

jb_gericke2• 1 year ago

So first off, don’t use Python in prod. Second off, don’t use async because of complexities in multi-threading?

A lot of the world runs just find on Python (see Django), async is mature and stable.

ashishb11• 1 year ago

A lot of the world runs just find on Python (see Django), async is mature and stable.

I never claimed that one should not use `async`, I am only suggesting that be careful about using `async`.

In my experience, a median Go programmer is more comfortable with Go routines than a median Python programmer is with async functions. YMMV.

its-summertime4• 1 year ago

"Consider using an unstable package manager" is a really really rough start.

additional: "However, writing your own code that involves async functions", your code does use async functions, its just monkey patched into your program because you used green threads instead.

ashishb11• 1 year ago

"Consider using an unstable package manager" is a really really rough start.

You can use poetry instead. Poetry is stable.

I wish `uv` was version 1.0 or later. However, despite it not being version 1.0 yet, I do see that it provides a superior experience for me to switch from `poetry` to `uv` and to make that recommendation after adding the pre-version 1.0 caveat.

Apreche2• 1 year ago

People get so hung up on the version number. These fools would rather use software with tons of bugs and security holes simply because the devs of that software dared to use a version number greater than 1.0.

I see the entire community gravitating towards uv because of its merits. I tried it, and I agree. I already started using it for every project. I’ve even converted older projects to it.

ropable• 1 year ago

Sounds like someone really needs to try out uv.

ashishb11• 1 year ago

I was skeptical too because of the version number and then uv won me with a quick migration and insane speed.

There docs around GitHub Actions and docker are also pretty succinct and usable.

andrewstuart2• 1 year ago

Perhaps better titled as something like “my personal preferences for how I like to run python in production”.

Presenting these opinions as truths will get the hackles up of those who have a different opinion.

ashishb11• 1 year ago

Perhaps better titled as something like “my personal preferences for how I like to run python in production”.

Isn't that always implicit?

zvr• 1 year ago

On the "formatters" and "linters" front, there is no need to use anything else than ruff, since it incorporates the functionality of all the other tools mentioned.

ashishb11• 1 year ago

Last I checked, 6 months back, it didn't.

However, ruff keeps evolving rapidly, so I am hopeful that I can get rid of all other linters and formatters over time. If you have a suggestion on streamlining my `make format` and `make lint` commands, I am all ears.

How to Run Python in Production

Related Stories