Python 3.15: features that didn't make the headlines

https://blog.changs.co.uk/python-315-features-that-didnt-make-the-headlines.html

by rbanffy • 2 months ago

432 186 2 months ago

blog.changs.co.uk

brianwawok• 2 months ago

I was so into Python for 10 years, was enjoyable to work in. But have deleted 100k+ lines this year already moving them to faster languages in a post AI codebot world. Mostly moving to go these days.

shankysingh2• 2 months ago

Thats very intersting, If I may ask was it from professional projects or personal projects?

mountainriver• 2 months ago

Same, I’m not sure how Python survives this outside of machine learning.

All of our services we were our are significantly faster and more reliable. We used Rust, it wasn’t hard to do

prodigycorp2• 2 months ago

the funny thing is that everyone, including myself, posited that python would be the winner of the ai coding wars, because of how much training data there is for it. My experience has been the opposite.

LtWorf2• 2 months ago

You can test on the device directly, without needing to recompile to try something.

amelius3• 2 months ago

We used Rust

The problem with Rust is that you have to rethink your memory management architecture. I think Go is an easier choice.

anthk• 2 months ago

Prolog and Lisp will survive because contrary to LLM they know what reproducibility means instead of what LLM's slop around were by design they are stacking errors over time.

AI can't compete against the classical AI where both Prolog and Lisp have tons of experience on contraint logic programming and expert systems. With AI you can throw up tons of RAM and VRAM (> $12000) and yet proper designs with Prolog with outperform these by a huge gap.

physicsguy• 2 months ago

Go is terrible for scientific/ML work though, the libraries just aren't there. The wrapping C API story is weak too even with LLMs to assist.

Try and write a signal processing thing with filters, windowing, overlap, etc. - there's no easy way to do it at all with the libraries that exist.

LtWorf2• 2 months ago

I think the purpose of go is to write CRUD. Stray from that and you're on your own.

stuaxo2• 2 months ago

This is straightforward in the first instance, but how do you see maintenance of those projects going forward - especially adding more complex features ?

I can see one way forward being to prototype them in python and convert.

nostrebored• 2 months ago

It’s completely the opposite. LLMs write awful python. Horrendous. They write pretty reasonable go and do it quite quickly.

We started in Python because of “the ecosystem”. It was a mistake. The amount of time we spend ripping out each dependency and pruning it to what we need is way higher than if we’d spent the month building out what we need. I miss compilers and LLMs will NOT generate config driven code or things that serde well. Everything has layers and layers of adapters by default and the domain model slowly erodes over time.

deppep• 2 months ago

i don’t really see it this way. the value of a token in Python is much higher than it is in lower-level language

BOOSTERHIDROGEN• 2 months ago

Interested in why you'd use Python in the first place? Advice for someone who knows nothing about programming - what would you suggest?

IshKebab5• 2 months ago

IMO the main reasons people use Python are:

1. The very first steps are quite simple. Hello world is literally just `print("hello world")`. In other languages it can be a lot more complex.

2. It got a reputation as a beginner-friendly language as a result.

3. It has a "REPL" which means you can type code into a prompt and it will execute it interactively. This is very helpful for research (think AI) where you're trying stuff out and want to plot graphs and so on.

IMO it is undeservedly popular, or at least was. Wind back 10 years to when it was rapidly gaining mindshare:

1. While "hello world" is simple, if you went further to more complex programs you would hit two roadblocks: a) the lack of static type checking means large programs are difficult to maintain, and b) it's really really slow.

2. While the language is reasonable, the tooling (how you install packages, manage the code and so on) was eye-bleedingly abysmal.

3. While the REPL did technically exist, it was really bare bones. It couldn't even handle things like pasting code into it if the code contained blank lines (which it usually does).

However since it has become arguably the most popular language in the world, a lot of people have been forced to use it and so it is actually getting quite decent now. It has decent static types (even if lots of people still don't use them), the REPL is actually decent now (this changed very recently), and there's a new third party tool called `uv` to manage your code that is actually good.

The biggest issue with it now is that it's still horrifically slow (around 50-200x slower than "fast" languages like C++, Rust etc). It is pretty unlikely that that will ever change. People always try to excuse this by saying Python is a "glue" language and you just use it to connect components written in faster languages, but a) that's pure "you're holding it wrong", and b) that only works in some cases where there are nicely separated "slow bits" that can be moved to another language. That's the case for AI for example, where it's all numerical, but for lots of things it isn't. Mercurial was a competitor to Git that was written in Python and lost partly because it was way too slow. They've started writing parts in Rust but it took them 10 years to even start doing that and by then it was far too late.

what would you suggest?

It really depends on what you want to make. I would pick something to make first and then pick the language based on that. Something like:

* AI: Python for sure. Make sure you use uv and Pyright.

* Web-based games: Typescript

* Web sites: Typescript, or maybe Go.

* Desktop GUI: Tbh I'd still use C++ with QtWidgets. Getting a bit old-school now tbf.

Also Rust is the best language of them all, but I dunno if I'd pick it as a beginner unless you really know you want to get into programming.

js24• 2 months ago

Programs have to run in a lot of different contexts, not just as servers, and for some of those contexts (especially say glueing together other programs), an interpreted language is more convenient and easier to work with. In fact, unless I care about performance, I'm going to use an interpreted language because having the source close at hand when something breaks just turns out to be super useful.

t435628• 2 months ago

Because it's quick and easy to radically alter and refactor your prototype as you learn the problem space. By the time it works you often find out that you don't need anything more. This is something that Perl had.

Once your program starts to get bigger you have abstractions that can cope fairly well and keep your code simple to use - this is what Perl didn't have.

If you need more speed then you can write extensions in some compiled language.I think TCL was better at this hybrid approach but Python is a nicer language in itself.

You can also just dump python and write everything in that other language but now you understand the problem space quite well and you won't be trying to learn about it using a language where change is "difficult."

NegativeLatency• 2 months ago

Try a few, pick what you like

zabzonk6• 2 months ago

Three things I find unlikely about this:

- You wrote 100K lines of code (I've worked on several large C++ projects that were far smaller)

- You wrote those lines in Python (surely the whole point of Python is to write less code)

- You deleted them (never delete anything, isn't this what modern VCS is all about?)

But whatever floats your boat.

dkersten6• 2 months ago

You deleted them (never delete anything, isn't this what modern VCS is all about?)

The person said: "deleted 100k+ lines this year already moving them to faster languages"

Are you saying that when you move code to another language/rewrite in another language, you leave the original languages code in your repo?

They didn't say they deleted it from their git history. I delete code all the time (doesn't mean its "gone", just that its not in my git head).

squirrellous• 2 months ago

Uhm what? All of those things are totally ordinary.

throwatdem12311• 2 months ago

100k lines is tiny what are you on about, especially in the monolithic app sass world where many Fyll stack apps that handle all business ops are probably written with Django.

Our entire business runs on 300k lines of Ruby (on Rails) and I can keep most of the business logic in my head. I would say our codebase is not exactly “tiny” and just cracking the ceiling into “smal” territory. And comparatively, people probably write even less code in equivalent rails apps to django ones. 100k lines of C++ is miniscule.

Obviously “deleting code” in this context doesn’t mean purging version control history but the current state of the codebase.

sinpif2• 2 months ago

I'm still on the lookout for a comprehensive Django-like web framework for go. That would be an instant hit for me.

pennomi• 2 months ago

Same here. Django is my last holdout for Python. Everything new is go.

seabrookmx• 2 months ago

Try another language? The Go ecosystem tends towards libraries as opposed to "frameworks."

I personally chose C# for this reason, because ASP.NET is mature and (IMO) well designed. But there's also Java/Spring and and lots of other options in different languages depending on your preferences.

eager_learner7• 2 months ago

why not Rails?

biglost• 2 months ago

Try gofiber, i love python, i eat thanks to python but Django...... Gofiber isnt perfect, but it's really great

sgt• 2 months ago

Or just use Django. You can later identify hot spots and bottlenecks and spin out Go services where applicable.

kokada• 2 months ago

From this example:

    lazy from typing import Iterator

    def stream_events(...) -> Iterator[str]:
        while True:
            yield blocking_get_event(...)

    events = stream_events(...)

    for event in events:
        consume(event)

Do we finally have "lazy imports" in Python? I think I missed this change. Is this also something from Python 3.15 or earlier?

boxed• 2 months ago

Yes, 3.15+

llimllib• 2 months ago

3.15: https://docs.python.org/3.15/whatsnew/3.15.html#whatsnew315-...

javcasas• 2 months ago

When an AttributeError on a builtin type has no close match via Levenshtein distance, the error message now checks a static table of common method names from other languages (JavaScript, Java, Ruby, C#) and suggests the Python equivalent

Oh, that is such a nice thing.

karpetrosyan• 2 months ago

Note that you can work around it by implementing `def __getattr__(name: str) -> object:` at the module level on earlier Python versions

saghm• 2 months ago

Somehow I have no trouble imagining this being used as a rationale to avoid unnecessary "magic" to the language for years

tehnub• 2 months ago

Opus 4.7 did this in my code and I had never seen it before

kzrdude2• 2 months ago

What benefit does the lazy import have here - if we use the value in a type hint at module scope anyway? Would that require Deferred evaluation of annotations -- which I don't think are enabled by default?

js24• 2 months ago

Type annotations are lazily evaluated by moving them behind a special annotations scope as of 3.14:

https://peps.python.org/pep-0649/

https://docs.python.org/3/reference/compound_stmts.html#anno...

With 3.15, using lazy typing imports is more or less an alternative to putting such imports behind an "if TYPE_CHECKING" guard.

alcazar• 2 months ago

This one is really nice. Imports in the middle of a file are very ugly but sometimes necessary. This fixes it in a nicer way.

However, it kind of goes against the principle that “explicit is better than implicit.”

ActorNightly• 2 months ago

Python has had lazy imports from like day one, where you could have an import statement in a function, and the library won't be imported until that function is hit.

Hamuko• 2 months ago

It's one of the headline features of Python 3.15 (hence why it's not in this article). It's even mentioned as the first thing on the What's New doc, so I'm definitely counting it as a "headline feature".

Personally, can't wait. It was just this week that I observed a Python process running out of memory because a module import that's not being used during the process was added to the application, and the memory usage went over a critical threshold because of that.

0cf8612b2e1e• 2 months ago

I am left wondering when to use it. Every import, all the time? It is an optimization that benefits some pathological scenarios, but not sure I want to introduce bikeshedding in what is/not lazy.

JohnKemeny• 2 months ago

I've left this one to the bonus section because I've never used set operations on Counters and I'm finding it extremely hard to think of a use case for xor specifically. But I do appreciate the devs adding it for completeness.

Check out symmetric difference

https://en.wikipedia.org/wiki/Symmetric_difference

qsort• 2 months ago

Yeah, but applied to counters it would be the symmetric difference between multisets, which doesn't have a natural definition. If I understood the proposal they'd be defining it as absolute value of the difference of the counts, which isn't even associative.

If they only considered parities it could be interpreted as addition in F_2, which is more natural, but I'd still agree that it's hard to see how you'd use something like this in practice.

gtheodor• 2 months ago

You can get the L_k distances between the two counters. E.g. if you sum the absolute value of the difference of the counts, you get the L_1 distance between the counters. If you raise them to the n^th power and then sum them, you get the L_n distance. For n=2, that's the Euclidean distance (squared).

sunshine-o2• 2 months ago

I am not a python dev but have the utmost respect for the ecosystem.

But damn, with all the supply chain attacks now in the news, could they just make a simple way (for non python insiders) to install python apps without fearing to be infected by a vermin with full access to my $HOME ...

surajrmal• 2 months ago

There is little that they can do short of running the programs in a VM. Linux distros aren't engineered to consider applications as something different from the user running them. You need a completely different security model to achieve that and the Python runtime isn't tackle that.

sunshine-o2• 2 months ago

In its inception 35 years ago the creator of python could not foresee how far python would go and how the environment would look like today. But nowadays there are a lot of security mechanisms they could leverage to adapt (from chroot by default to namespaces, cgroup, etc. on Linux, pledge, unveil on OpenBSD).

The very idea that you offer a (python) package installer that is gonna pull a tree of code published and updated by random people in an unvetted manner open the door to all the supply chain attacks we are seeing.

Around the same time (early 90s) Java was designed with high isolation in mind but the goal and vision was very different. And Java had its own problems.

I'm saying that because at some point the security problem is gonna really hurt the python ecosystem.

nyrikki2• 2 months ago

There is no security barrier at all in UNIX(-like) Os's between a caller/callee, this is not thing that python can just fix.

There are ways to harden and/or reduce privileges, but shells/scripting languages will always have this issue on any modern OS.

The UNIX way to help prevent that is really to run processes as another user, but people seem to refuse to do so. You should always expect any process running as your UID to be able to access any data owned or visible to your UID.

While it is possible to reduce the risk of disclosure, they are all wack-a-mole preventions protecting the low hanging fruit, not absolute guarantees.

That is purely due to how UNIX works [0]

[0] https://man7.org/linux/man-pages/man7/credentials.7.html

OutOfHere• 2 months ago

Never install or run a Python project without a "devcontainer" or Docker container or outside a dedicated VM. For the development laptop, a devcontainer is a good answer.

zmmmmm2• 2 months ago

it's honestly ironic that Java is removing its built in sandboxing right at the time when it would finally have been considered a high value feature. Between the need for AI agent sandboxing and the security apocalypse that is upon us now, it finally would have had its time.

krupan5• 2 months ago

There are plenty of python apps that you can just apt install. You probably don't even realize some of the apps you run are written in python?

And any app you run has access to $HOME, that's not a python specific thing. Look into apparmor or selinix if you want finer grained security

armanj• 2 months ago

funny how we may have to wait even longer for llms to pick up this update in their pre-training

Alifatisk• 2 months ago

Is there seriously no solution to this? Perhaps something we fan do post training? For example add the new features to SKILLS.md? But the trade-off here is of course tokens.

shankysingh2• 2 months ago

My working methodology has been , LLM output is a jumping point and to use my experience, knowledge and basic understanding to N+1 it.

So for bleeding edge stuff it works out well or in places where documentation is not great like Apache Flink.

BiteCode_dev2• 2 months ago

It's actually fantastic for job interviews. Use that trick with my clients when asked to review candidates.

kwon-young• 2 months ago

It's nice that python 3.15 added Iterator synchronization primitives: https://docs.python.org/3.15/library/threading.html#iterator.... These will nicely complement my threaded-generator package which is doing just this but using a thread/process+generator+queue: https://pypi.org/project/threaded-generator/

veqq• 2 months ago

There's a good interview about Python internals and management, particularly in relation to free-threading: https://alexalejandre.com/programming/interview-with-ngoldba...

syedMohib45• 2 months ago

Thread safe ittertors? really are we still on these topics

lazy from typing import Iterator

def stream_events(...) -> Iterator[str]: while True: yield blocking_get_event(...)

events = stream_events(...)

for event in events: consume(event)

drchaim• 2 months ago

Oh, my beloved Python, for nearly 15 years I wrote you. I miss you, but I no longer do — it's not your fault, life has changed.

speedster217• 2 months ago

I've really been enjoying modern Python both at work and on personal projects.

amelius3• 2 months ago

Is anyone working on a more capable Python-like language which also interfaces well with Python but without the baggage?

pansa22• 2 months ago

Mojo?

jwineinger• 2 months ago

One of the Counter examples is incorrect, tested on both 3.13 and 3.15.0a

  >>> from collections import Counter 
  >>> c = Counter(a=3, b=1)  
  >>> d = Counter(a=1, b=2)   
  >>> c-d  
  Counter({'a': 2})

zuminator• 2 months ago

I noticed that as well. Per the docs:

  Several mathematical operations are provided for combining Counter objects to produce multisets (counters that have counts greater than zero). Addition and subtraction combine counters by adding or subtracting the counts of corresponding elements. Intersection and union return the minimum and maximum of corresponding counts. Equality and inclusion compare corresponding counts. Each operation can accept inputs with signed counts, but the output will exclude results with counts of zero or less.

Anyway, nice Counter-example ;-)

xg152• 2 months ago

Iterators, async functions and async iterators don't work well here because they have different semantics to standard functions. When you call them they return immediately with a generator object, coroutine function and async generator object respectively. So the decorator completes immediately as opposed to the entire lifecycle what it's wrapping.
This is an unfortunate problem I've encountered many times, and it's often a problem for normal decorators too. But this has changed in 3.15, now the ContextDecorator will check the type of the function it's wrapping and ensure that the decorator covers the entire lifespan.

I very much like the idea of that change - but it also seems kind of dangerous, to do this with no "opt-in mechanism", as that quite subtly changes the behavior of existing usage sites.

This is a bit of a "spacebar heating" situation, because someone would have to intentionally use a decorator in the old, broken way, but if someone actually did that, things may unexpectedly break.

ameliaquining• 2 months ago

The Python core team seems to think it's unlikely that anyone's relying on the existing behavior: https://github.com/python/cpython/pull/136212#issuecomment-4...

xg152• 2 months ago

Ok, good to see that they checked that possibility. Looks like there was no situation in which the previous behavior could have been usable, so yeah, agreeing with the change then.

Drakim• 2 months ago

Eh, what's the worst that could happen? Developers opting to run an old version of Python due to incompatible changes? I can't see that happening.

aniou• 2 months ago

I come to Python around version 1.5, painfully tired by debugging CGI scripts, created by wannabe perl-golfers. Unfortunately, I feel like Python is losing more and more of the zen that once tempted me...

Lazy loading looks like a last nail in the coffin, where my love to Python was buried, although it was a long, tiresome process.

t435628• 2 months ago

It's now being used by a lot of people because they have to rather than want to so there is a desire to ** it up and reinvent their favorite language by adding all sorts of warts. That's the price of success.

A proper revenge would be to introduce "easy mode" to Rust - where the compiler takes a huge chill pill and you get non-enforced garbage collection. :-)

BiteCode_dev2• 2 months ago

Note that 3.15 is not released yet. It will come out in 4 months

vorsken• 2 months ago

The `except*` improvements are underrated. Been using ExceptionGroup in a CLI tool that wraps Semgrep — catching multiple subprocess errors cleanly in one block made the retry logic much simpler.

axpy906• 2 months ago

When Wes McKinney wrote about the transition away from python I knew it was real. https://wesmckinney.com/blog/agent-ergonomics/

I still have a special place in my heart for the language and think it’s still got a niche.

groundzeros20152• 2 months ago

Why does every language need every feature? Python has completely lost its charm.

t435628• 2 months ago

It's too popular and the people channeled into using it don't care about it other than to complain about why it isn't <insertlanguagehere>

frr1492• 2 months ago

I used to be obsessed about language design and now, since I almost never write code directly (it's always Claude), I completely lost interest.

It feels like a total waste of time and I wonder if other feel the same.

One of the consequences of the LLM tsunami might be the freezing of research and development in programming languages.

Maybe we'll be stuck with J's and python forever...

inejge• 2 months ago

I almost never write code directly (it's always Claude)

Who, then, understands the code? If the answer is "no one really", entropy will overwhelm your codebase sooner or later. Otherwise, you need to read the code, and for that the knowledge of language is still relevant.

NotOscarWilde• 2 months ago

If the answer is "no one really", entropy will overwhelm your codebase sooner or later. Otherwise, you need to read the code

I think about this on the regular -- I know the answer is currently "you own the code, so you have to understand it", but to unlock the true productivity multiplier, in the future, the answer has to be "no one really".

I think about it using the concepts from my job (academia) -- to actually have PhD student-level intelligence means that you have to trust that it does a good enough job that you can focus on other stuff. Professors often bring the correct ideas or intuitions, but they have to trust the PhD student to write the code and/or fill in the gaps in the proofs -- they can advise them on the high-level issues during a consultation, but that's about it.

I am pretty bad at working in the current LLM workflow -- it is tough for me to focus on reading a TCS paper for review, keeping all the details and invariants in my head, but every 5-10 minutes go to my PC, completely switch contexts/projects, read the code and think about the LLM's comments, suggest the next step, and then go back to reading.

frr1492• 2 months ago

I do, with different tools and most of the time at a higher level of abstraction. The same way we understood the machine code 2 years ago, even though we didn't write it directly. Just another layer, nothing more.

t435628• 2 months ago

Presumably you don't really enjoy programming but the mistake would be to think that everyone doesn't.

dhruv3006• 2 months ago

Immutable JSON Objects With the addition of frozendict in 3.15, we now have the ability to represent all the json types (array, boolean, float, null, string, object) in immutable (hashable) forms.

I actually love this last feature !!

danborn26• 2 months ago

These smaller features often end up being the most useful. I'm especially interested in testing the new standard library additions in my current projects.

GeorgeTirebiter2• 2 months ago

Absolutely serious question: is anybody actually writing code anymore? (Reviewing it, architecting it, sure -- we do that). But writing code? If NOT, then seems to me, what we want in our LLM output is CODE THAT IS TRIVIAL TO UNDERSTAND (I apologize for shouting from the rooftops.)

I'm not sure adding 'features' to Python anymore makes sense - UNLESS those features help humans understand LLM code. Part of the problem is, of course, that LLMs haven't been trained on the latest-and-greatest, so they won't output any of it (even new training events won't capture the latest very much, since there is so little of it relative to what is already out there).

But again -- do new features help HUMANS understand what the LLMs produce? If not, seems to me ... new features merely add complexity for no apparent gain. (Or am I confused about this?) THANKS for any helpful opinions.

GeorgeTirebiter2• 2 months ago

Wow, the Python folks do not want to entertain different goals for the language, it would seem. I still remain interested in whether it makes sense to add features to Python anymore, given that LLMs write all the code. Unless, as I said, such features enhance human comprehension (since we have to review the code the LLM produces).

t435628• 2 months ago

If you don't write code you won't understand what you're reviewing. Any manager discovers this - their ability to review atrophies.

Python 3.15: features that didn't make the headlines

Related Stories