Python’s Preprocessor

https://pydong.org/posts/PythonsPreprocessor/

by rbanffy • 2 years ago

402 95 2 years ago

Part of me says that python has not gone out of its way to expose preprocessor hooks for good reason, and reasonable adults should stay away from it.

The other part wants nothing to do with reasonable adults anyhow. Such fun could be had.

0823498723498729• 2 years ago

Growing old is mandatory; growing up is optional.

pryelluw• 2 years ago

You’ve managed to sum my whole approach to life in one beautiful sentence. Thank you.

gradschoolfail9• 2 years ago

Thank you for connecting 2 sayings of Murakami Haruki :)

1.痛みは避けがたいが、苦しみはオプショナル

(https://archive.ph/Ca9rE)

Now, I don’t know how representative this book is of Murakami’s novelistic style, but I wonder: Is this low-maintenance, attention-deficit prose part of Murakami’s attraction, especially among the young? Do people enjoy reading him for the same reason they persist in listening to music as blandly familiar as Clapton’s? If Martin Amis is engaged in a “war against cliché” a phrase in danger of becoming a cliché itself then Murakami, on the evidence of this book, is a serial appeaser. How much does his thigh hurt? “Like crazy.” How do we know the weather is nice? Because as he tells us (twice) there’s “not a cloud in the sky.”

  self-styled zero-percenter and book reviewer in the above

2.^_^？

(Norwegian Wood)

(Roughly, “growth is pain, nongrowth is unfortunate”)

throwaway894345• 2 years ago

Python has a whole bit about how it is for "consenting adults" and that's why it lacks public/private visibility modifiers and presumably why it exposes all other manner of metaprogramming magic. If they were worried about "reasonable adults", these other design decisions would be curious. :)

rbanffy• 2 years ago

Batteries are included. Nannies are optional.

extraduder_ire• 2 years ago

If you stopped people from doing this the easy/obvious way, they'd attempt it in a worse and more hacky way.

IshKebab• 2 years ago

Haha great hack. Something tells me I'll run across someone seriously using this in like 10 years...

kevindamm• 2 years ago

I'm assuming you've written 10 in binary.

raminf• 2 years ago

It won't take that long (I already have two use-cases for it).

The only concern is how brittle it might be vs. future Python releases.

btown• 2 years ago

    >>> from __future__ import braces
      File "<stdin>", line 1
    SyntaxError: not a chance

For those interested in how this (hilarious) error text came to be, it's been hardcoded in cpython since 2001!

https://github.com/python/cpython/commit/ad3d3f2f3f19833f59f...

The author, Jeremy Hylton, is now a Principal Engineer at Google working on AI search quality. It's quite remarkable that in 24 years, a single person's career has gone from a tongue-in-cheek memorialization of certain syntax being forbidden, to working on ubiquitous query systems that don't require dedicated syntax at all!

SkiFire132• 2 years ago

Reminds me of `break rust;` causing the Rust compiler to emit an internal compiler error. I wonder which other languages have similar easter eggs.

bch• 2 years ago

Tcl really became “Enterprise-ready”[0] at this time.

[0] https://stackoverflow.com/a/1026619

hsfzxjy• 2 years ago

`break rust;` causing the Rust compiler to emit an internal compiler error

Is this real?

TheMatten• 2 years ago

This one in Haskell is coincidental, but used as a joke:

``` import Data.Function

main = do let it = fix error print it ```

sans_souse• 2 years ago

If NLP counts, try xyzzy in gemini.google.com

brcmthrowaway• 2 years ago

20 years at google, now thats some F-U money!

plhannb• 2 years ago

Innocent times. Perhaps Hylton could join the "vote of no confidence" motion in order to get justice for Tim Peters:

https://news.ycombinator.com/item?id=41314393

JodieBenitez• 2 years ago

And just like that, I'm down the drama rabbit hole.

throw-the-towel• 2 years ago

It's always so cool to see this kind of Easter eggs. So sad that they're not as common as they used to be.

lhmadk• 2 years ago

There are all sorts of easter eggs and jokes in Python, for example a joke about blind people that GvR prevented from being reverted:

https://marc.info/?l=python-dev&m=130991276326041&w=2

In 2024 suddenly everyone is grown up and corporate and has never done anything wrong.

bonoboTP2• 2 years ago

How is it remarkable? Random people didn't get to add stuff to Python in 2001. It was a niche thing and someone who was contributing to it was likely to be a smart and dedicated person who would naturally have an impactful career ahead of them.

It's a misconception that the informal playful hobby hacking things are their separate world from real professional development.

dylanfw• 2 years ago

I think OP is commenting more so on the reversal of the developer’s relationship with rules of syntax than their personal capabilities.

photonthug2• 2 years ago

I thought getting cute with import-hooks was probably the most creative possible way to get fired, but I see now that this was naive. My only regret is that the codec regex probably prevents using stuff like "μtf8" to really troll people properly, so now I'm going to have to use import hooks, preprocessors, and sys.settrace to monkey-patch every function to the previously called one, while swapping stdout and stderr every 17 minutes.

ImaCake• 2 years ago

Make sure you enforce use of curly braces like all good languages

nerdponx• 2 years ago

Now that I know this is possible, it seems like it would be a fun addition to the Hy, Hissp, and Coconut projects: https://hylang.org/ https://github.com/gilch/hissp http://coconut-lang.org/

dang• 2 years ago

[stub for offtopicness]

mdanger007• 2 years ago

Tied with Coq for best name

surfingdino2• 2 years ago

Let's not forget this fine contribution to the Python's nest. https://github.com/ajalt/fuckitpy

spapas82• 2 years ago

I remember that I wanted to implement a python module that would improve upon the builtin Django groups. So following a tradition of adding ex to the end of the existing name, I named my module "groupsex".

After taking a peek at it I decided to rename it to groups2 instead :)

peterleiser• 2 years ago

You can always ask the folks at Experts Exchange for guidance on naming things.

jedberg• 2 years ago

As a community we really couldn't come up with a better name, eh?

giancarlostoro2• 2 years ago

PrePy could have worked... I guess lol maybe it was taken, cause Pydong... yeah.

carom• 2 years ago

Pydong is the blog name.

BeetleB• 2 years ago

Dong is a very common name in some countries.

lysace• 2 years ago

Not a prude but this name just reeks of immaturity.

AlanYx• 2 years ago

Isn’t it a play on words/reference to Pudong in China? It’s a nice central part of Shanghai.

nomdep• 2 years ago

* As you can see by opening the link, "pydong" is the name of the blog, not the Python preprocessor

* I'm glad that we are not ants. Deciding everything "as a community" would be hell

ziml77• 2 years ago

I think the title of this needs to be changed as it implies that this is a preprocessor project named Pydong. As far as I can tell that's simply the name of this blog and this post is about how python has a preprocessor built-in if you abuse the right mechanisms.

metadat• 2 years ago

Are the dependencies introduced via this coding hook strategy picked up by `pip freeze' or uv?

Otherwise, ..have fun with that :). Probably easier to rewrite any library rather than fight such dragons (because if someone put this in, what other traps await? Seems almost guaranteed.)

Tsche3• 2 years ago

I'm glad y'all like it, more to come soon :)

skeledrew• 2 years ago

For the first time in a very, very long time I'm running into something that totally blows my mind and sets off lightbulbs like this is...

est• 2 years ago

best use case I've found so far is pyxl, inspired by jsx https://github.com/dropbox/pyxl.

You can write code like

  # coding: pyxl
  print <html><body>Hello World!</body></html>

0xbadcafebee2• 2 years ago

the Python developers have strong opinions

except when it comes to downloading, installing, or running anything Python related. maybe by 2124 we can give users a python app and let them run it without 20 steps. (oh who am i kidding, by then we'll rewrite everything in whatever comes after Rust)

Leynos• 2 years ago

https://pyinstaller.org/en/stable/

0xbadcafebee2• 2 years ago

Yep, that proves my point, thank you

BiteCode_dev• 2 years ago

Tangential, but if all you want is inline code generation from Python and not a full preprocessor, you can use the excellent cog from Ned Batchelder:

http://nedbatchelder.com/code/cog#h_what_does_it_do

rbarrois• 2 years ago

That's quite interesting.

Now, I'm left to wonder if this could have been used to better handle the Python 2-to-3 transition, e.g '# coding: six.python2' would adjust Python2 code to be valid Python3, or '# coding: six.python3' would adjust Python3 code to run under Python2 - e.g adding/removing the b"..." or u"..." prefixes!

dgoldstein0• 2 years ago

It could help, but the parts it would help with are the easy parts. The challenges of py2 to py3 were the runtime behavior change: a Unicode and regular string containing ASCII were the same string in py2, as in you could use them as keys of a dict and they'd key the same entry. In py3 a bytes and str of the same ASCII would identify different entries in the same dict.

Some more nasty changes: various built ins like .keys() and .values() return lists in py2 but iterators in py3. Code gets very verbose if you use the six utilities or other workarounds to translate code safely - most times those are called they are used once, but every once in a while they are used twice.

Imo if you have such a tool that can rewrite at import time you should just commit the transformed code, and clean it up incrementally. The hard parts are the behavior changes that can cross long distances like the str v bytes behaving so different than py2 Unicode v str

zxexz2• 2 years ago

Huh, that’s convenient and really useful. When I do ridiculous import hacks, i usually just do it be importing a module, that trades the ast module to rewrite code, exec it and shim in an exit(). A preprocessor would be so much more ergonomic.

I used the ast rewriting stuff mostly before dicts were all ordered - replacing fist literals with an ordered dict call, which was actually useful.

I love how flexible python is. Most cursed thing I ever did was mutating strings in place, which led me eventually abusing mmap as well to write scripts that mutated themselves. Now I feel need to write lisp interpreter as a producer.

sgarland• 2 years ago

mutating strings in place

They’re immutable. Were you directly writing to the memory location with ctypes or something?

zxexz2• 2 years ago

Yep!

buildbot• 2 years ago

Super cool! It would be kinda funny to make a pseudocode python that get’s “decoded” by an llm. Obviously it would be hideous but probably somewhat fun :)

Python’s Preprocessor

Related Stories