wrmsr

If anyone's interested I've implemented a fairly user friendly lazy import mechanism in the form of context managers (auto_proxy_import/init) at https://pypi.org/project/lazyimp/ that I use fairly heavily. Syntactically it's just wrapping otherwise unmodified import statements in a with block, so tools 'just work' and it can be easily disabled or told to import eagerly for debugging. It's powered primarily by swapping out the frame's f_builtins in a cext (as it needs more power than importlib hooks provide), but has a lame attempt at a threadsafe pure python version, and a super dumb global hook version.

I was skeptical and cautious with it at first but I've since moved large chunks of my codebase to it - it's caused surprisingly few problems (honestly none besides forgetting to handle some import-time registration in some modules) and the speed boost is addictive.

If you're building anything past a hello world c ext you're already off the path of shit like poetry, in which case AIUI we're basically on our own as we've always been lol, for better and worse.

Again I don't think setup.py is going away. Given that distsetuputiltools is a jenga tower of decades old monkeypatches documentation is understandably sparse, but they're at least passively open about how gross it is to work within. Subclassing or otherwise manually overriding the behavior of distutils classes like build_* is just how stuff is done at this level and I don't expect that to change, beyond just updating your stuff to subclass or hook direct setuptools equivs of existing distutils classes. distutils is already vendored by setuptools anyway, so you're already not really hitting 'distutils' as much as 'setuptools._distutils' aliased to distutils upon importing setuptools (and specifically importing it first ( https://github.com/pypa/setuptools/blob/c121d289da5d19cf6df2... )), so all this really does is collapse a level of hack.

As long as py2 remains supported I expect whatever changes happen here to be py2 compat, but I do expect new setuptools to break compat with old setup.py's. But setup.py being what it is it wouldn't be out of character to make your setup.py manually support both old pre-distutils-removal setuptools and new post-distutils-removal setuptools, but that's just how setup.py maintenance goes lol.

ed: ok well not so much py2 being 'supported' as 'no showstopping incompatibilities for anyone still straddling that ever widening gap' :)

AIUI, as the official way setuptools is used is a setup.py calling setuptools.setup(), doing this will remain supported in perpetuity and will really still be powering everything under the hood, it's just no longer what people are told to do when they ask 'how do I make a new python library?' - newcomers and people with simple requirements will be directed to poetry or whatever and avoid that whole mess. For obvious historical reasons python's pretty averse to large breaking changes, and its support for all kinds of weird native interops and obscure platforms is one of its major assets (but also why setuptools is such a mess), so that's almost certainly not going anywhere, even if it's now kept even further back in the shadows. Regarding pyproject.toml if you already have a setup.py doing what you need then it's really only for specifying suported python versions and setup_requires (solving the problem of your setup.py itself needing Cython/wheel/etc).

Significant but as it's freethreaded you only run one large process. That alone allows for all kinds of additional optimizations that would be pointless with a lot of little singlethreaded processes in which sharing anything mutable between them has massive overhead.

Past some critical mass local dev indeed simply doesn't work, but the majority of codebases aren't ~that~ large. And until that point I feel there's an analogy between having a light laptop and remotely doing your heavy lifting with the whole working remote thing in general. In theory, on paper, and assuming reliable decent internet connectivity, with a little elbow grease there should be no discernable difference between thin client remote dev and an in-your-lap xeon, just like with high quality low latency video teleconferencing with a team that knows how to do it there should be nothing lost with physical distance. In practice though the overhead of doing any little thing, no matter how small the overhead may be optimized down to, is still real. Non-verbal communication is lost on zoom and the distance between you and your real UI grew by hundreds of miles and dozens of links, and the tradeoffs your environment makes when there's no such thing as a 'quick check' add up fast. But it's obviously not one dimensional, and there's probably irony in the weight shedding that offers you 20 hrs of battery life coming with a price tag of requiring reliable wifi to get any real work done.

I'm a fellow fan of antlr but it's not really an option for the base interpreter. They're (rightfully) very stingy about deps. cpython supports a wide range of platforms and has its own very permissive license - it's a pretty self-contained codebase and they prefer to keep it that way.

The whole point of python is to eschew 'magic' in favor of dumb explicitness, and this is about as big of a conflict as you can get to that. Python lacks statement lambdas not because they've never considered the possibility but explicitly because "if it's that important it deserves a name (as an inner function)". Type annotations work so well in python as compared to ruby or clojure exactly because of this philosophy. The PEP seems to make no mention of things like reproducibility, IO in macro expansion, static analysis like in mypy, or any of the other realities of things like this. If macro expansion is limited to something carefully statically interpretable like constexpr then I'm cautiously receptive to it, OR as it positions itself to be 'for experimenting with language features' and that's genuine then I'd be fine with it if it's only enabled in debug builds, but in lieu of those the instant it's released generally it's a python 4 flag day.

You could use this to do little stunts like adding an "unless x:" statement equivalent to "if not x:"

You say that like it's a good thing. Python is a language empowered by its constraints. You have indeed always been able to pull stunts like this but it's strongly frowned upon - python is not ruby, lambdas intentionally do not have statements, and terseness is not a virtue.

As far as I can tell functools.partial has been implemented in C since it was added sixteen years ago ( https://github.com/python/cpython/commit/9c323f8de4910dfc0ba... ), and it's faster than lambda ( https://gist.github.com/wrmsr/1e13eda3ed78288679c010acbe6d2b... ). Regarding docstrings you can access the underlying function via the __wrapped__ attribute, and signature forwarding is too brittle and magical for stdlib.

Built for Scientific Computing

I mean the thing's called 'numba' lol.

I always liken Pypy to HotSpot in that to this day the numerical performance of the latter isn't spectacular and nobody really cares - it's built to handle the harder job of making vast tangled codebases of non-numerical application code run fast, not just tight math loops which are already handled perfectly well by other more specialized tools.

It actually exceeds non-police theft these days: https://www.washingtonpost.com/news/wonk/wp/2015/11/23/cops-... ( https://outline.com/UX7nfW ).

As a language purist this one is surprisingly well put together. Its impl suffers from legacy (py2 support in general, predating builtin dataclasses, staying in the shallow end of type annotations) but it really does what it does better enough than its host language to be worth giving attention to. Check it out.

and greenlet, and stackless, and pypy's stm..

python's never really been about dogma. part of why its package management is such a mess is because it's been so successful at integrating with everything under the sun and still needs to support it all.

If your python files are not in a python package (do not have __init__.py in their directory) then they will be importing non-relatively and resolution depends on sys.path. If you have a multi-file python project in which your source files have inter-dependencies you should be structuring it as [nested] packages not as a directory of scripts that happen to try to talk to each other (and thus be doing 'from . import requests'). It's not that this is a 'way to avoid' a problem, it's just that this is the correct way to do this. I fully agree though that package management remains one of python's biggest pain points, made even worse by just how much bad and long-outdated information is out there, and by how 'easy' it is to just dump code into a .py and run it, not knowing just how wrong that is for anything but a temporary scratch file or a deliberately engineered self-contained single-file portable script like black.

Python doesn't treat 'paths on disk' as special language tokens, python chooses to handle certain tokens by possibly accessing files on disk. Classes in java (lacking 'modules') resolve to equally named files during compilation, as it goes in go, node, haskell, and pretty much everything I can think of with the sole exception of C/C++ preprocessor driven multi-file development (which is even more filesystem-coupled than the former as you literally type filenames).

As below node has the luxuries of both having been able to learn lessons from python and ruby ~and~ not being a victim of its own success of being already integrated with and a dependency of most major operating systems. (That said javascript the language definitely ~is~ a victim of its own success in browsers - there's probably some metaphorical parallel here..)

But the 'python version' problem (and really just any notion of a 'system python' in general) is still absolutely one of the bigger pain points of the ecosystem. I can say that for nix OS's pyenv ( https://github.com/pyenv/pyenv/ ) is a real godsend for this though. Pure-bash, no bootstrap problem, doesn't need to be 'installed' / can just be cloned and used, and has been able to reliably install any version of python I've needed into any of my relevant platforms in a well isolated ~/.pyenv/version/v3.x.y from which I can '~/.pyenv/versions/v3.x.y/bin/python -m venv .venv' in my project's dev bootstrap/Makefile. It's not part of my production-bound Dockerfiles which just base off of the official python images but it's invaluable for development and testing.

If you are going to use it I definitely recommend cloning it directly to ~/.pyenv rather than installing it via brew or somesuch as those tend to lag further behind the most up-to-date python versions than I'd like. And this is of course in no way a general solution to all of python's many packaging woes, and it'll probably never really be as simple as packages.json for many very valid if unfortunate reasons, but the 'python version' problem is at least pretty well handled by pyenv.

Name conflicts were basically fixed in 2003 with absolute/relative imports ( https://www.python.org/dev/peps/pep-0328/ ) - you don't ever have to include the name of your library in imports within your library using relative imports and via absolute imports they will never conflict with a base-level (site/system) package. Regarding hyphens and such in package names pretty much every language has restrictions on identifier names.

Imagine if it was the day of the Super Bowl, and you wanted to go out to a sports bar that was going to be airing the Super Bowl. So you go out and find a bar that says they're throwing a Super Bowl party, and go in and settle in, and when it comes time for kickoff you realize that the bar is actually airing the Super Bowl from 10 years ago. Like, it would have been fine if they had called the party a 'Super Bowl from 10 years ago' party, and for some reason there's this part of the bar that seems to be all on board with rewatching the same Super Bowl from 10 years ago every year and frankly seems confused why the rest of the bar is confused and angry about it (I mean it's still the Super Bowl right?).

loom ( https://jdk.java.net/loom/ ) is well on its way :)

Curing polio wouldn't be fair to everyone who already died of polio.

Was the cold war real?

They took a step towards this with https://docs.python.org/3/library/gc.html#gc.freeze but it doesn't go as far as disabling refcount touching outright. I've experimented with doing that, both per-object and just globally, and the results really were promising if your forkserver can keep up with providing the necessarily much shorter-lived worker processes.

It's not China's fault that the US isn't even testing its own healthcare workers displaying symptoms.

This is the most Hackernews post title I've ever seen.

In the real world it is inevitably a lot more than just those 3 tables - add 'groups', different types of groups, different privacy settings, different per-user feed preferences, experiments, and any number of other things which _can_ be expressed as pure, normalized, joined tables in a matview but make naive approaches a lot less likely to actually work in prod.

In my experience the most successful approach to this is a midpoint - you materialize/denormalize enough to feed your app endpoints and search engines but retain flexibility in searching those fat but instantly available docs, and relatedly you also don't always need to preemptively materialize absolutely everything in any particular view - see https://engineering.fb.com/data-infrastructure/dragon-a-dist... . Without being able to transparently operate on arbitrarily partially populated matviews you are locked into a self-defeating all-or-nothing system that is likely to culturally do more harm than good with its rigidity. Imagine for example if there were no 'caches', just a binary choice of precomputing everything ahead of time or recomputing everything every time. Neither extreme is sufficient for all cases and real applications are comprised of many different points on that spectrum.

eval(/exec) is how dataclasses (and namedtuples before them) work: https://github.com/python/cpython/blob/4c1b6a6f4fc46add0097e... . It is a big gun, but sometimes (usually deep within lib code) big guns are the right answer. Don't use reflection or runtime bytecode emission in Java.. unless you have to. Don't drop to inline asm in C.. unless you have to. And so on.

And, of course, if it _is_ the right decision to use such a tool be aware of just how easy it is to use big guns wrong. Even this very article's 'just do it' tone seems to convey a lack of respect for decorators, what I'd consider a 'medium gun' in python. So many intermediate python programmers write decorators that don't properly interoperate with the descriptor protocol and thus either fail to work on instancemethods (as here: https://repl.it/repls/ThreadbareCurlyKeyboard ) or hardcode a 'self' arg in their wrapper and thus don't work on global functions. I'm fine with simplifying it for an article but for production code this is a pet peeve of mine :p

I've seen so many times people going all in on DRY not understanding that just as dangerous as duplication is _coupling_ - the inevitable result being some ungodly $COMPANY_NAME_common lib with a thousand dependencies, and usually only depped in a codebase for a config parser and a string helper. See also node_modules and left-pad.io.

It's stated that llvm ir is not stable (https://llvm.org/docs/DeveloperPolicy.html#ir-backwards-comp...) and in my relatively limited experience with it there have been enough differences between every little release to make decoupling even official frontend and backend versions not safe or reliable (much less bolting on and maintaining an unofficial backend impl). If bundling/pinning the flang/clang versions with the supported llvm ir version supported by the alternate backend is feasible (which seems sane) then I definitely agree this is an awesome capability (clang-on-sulong) but given how unapologetically volatile llvm ir is I wouldn't exactly call it 'free' :p (but still a lot better than writing a c++ frontend)

I've found Optional<> to at least ~help~ with the null issue - as we still don't have value types it still does come with a runtime penalty but most of the code I write and deal with these days has a no-null policy: if it's optional it's Optional<>, everything else defaults to sane defaults or 'empty' (usually immutable) objects. There will of course always be nulls in legacy and performance sensitive code but it is at least ~something~.

Back in the day someone figured out that punkbuster blindly scanned physmem for illegal string literals and banned on detecting them no matter what process they belonged to. They then posted one of those strings to #findscrim on gamesurge (or whichever it was at the time) and the channel quickly exploded with hundreds of people saying they just got pb banned for no reason. It was magical.

Short of baking it directly into silicon clientside security is an oxymoron.

I really learned to code writing all kinds of hacks for half-life and its numerous mods (mostly counterstrike) and helping teach others to (but not releasing binaries as ruining other peoples' fun wasn't the real goal). I'm grateful for those years and how they formed my views about programming. My neglected personal homepage is still just a little crappy homage to it: http://wrmsr.com/ :)