Tauthon: Fork of Python 2.7 with new syntax, builtins, libraries from Python 3

https://github.com/naftaliharris/tauthon

by albertzeyer • 6 years ago

121 130 6 years ago

Mave83• 6 years ago

please get rid of the old python...

nomel2• 6 years ago

Justifying the conversion of a legacy project, that's still being maintained, is very difficult since the conversion doesn't necessarily bring anything tangible, without refactoring, especially if you have fixed dependencies. And, until python 3.7, the conversion would just make everything slower. That's something you never want on your justification list.

qwerty4561273• 6 years ago

There are many apps written in the old Python nobody is going to rewrite in Python 3 despite many people may need them occasionally. In many cases they can be used for decades as long as there is a compatible interpreter available - there are many contexts where security fixes are not really necessary.

There also are platforms where new Python is not and not going to be available, e.g. DOS.

sevanteri• 6 years ago

But why?

ur-whale• 6 years ago

print with brackets

michaelmior• 6 years ago

I assume this is sarcasm, but you could do this in Python 2.x already :)

unethical_ban• 6 years ago

In case you look back at this comment, the author has posted in the root of the thread. BLUF: A proof of concept that python3 minus unicode strings could have been done in python2 with relative ease.

TedDoesntTalk• 6 years ago

Why would I choose this over python 3? Is it for 2.x legacy code that can’t be upgraded to 3.x?

coliveira2• 6 years ago

Many people dislike Python 3 syntax. I think this is a great project, and given the mishandling of Python2/3, I wish we had a more systematic way to introduce new features. I hope this succeeds.

ebg132• 6 years ago

Many people dislike Python 3 syntax

Which aspects?

gnulinux• 6 years ago

But this is literally python 3 syntax. I'm absolutely not following. They adopted python 2 codebase and implemented `await`, `async` etc syntax in python 3. How is this any better than python 3 syntax-wise?

Myrmornis• 6 years ago

It’s the opposite. This project is making it possible to use Python 3 syntax with a Python 2 interpreter.

parsimo2010• 6 years ago

If you were starting a project now, you wouldn't choose this. It was done to prove a point that Python 2 could have gotten (nearly) all the Python 3 features without a hard/breaking change.

Their point has been made, but it's not like the Python community is going to unmake those changes. If you're starting a project, you should definitely use the current version of a language that is supported by a zillion developers, rather than picking a (practically experimental) fork of the old version.

mistrial9• 6 years ago

I will try this - thank you !

dochtman• 6 years ago

Last commit from 5 months ago. Maybe not a great choice.

rurban• 6 years ago

Stable projects only need bugfixes... Looks very stable to me

loeg3• 6 years ago

Shrug. The last commit on the Python.org 2.7 branch was 3 months ago, per the release notes from the other thread today. It wasn't especially active before that. By their own plan, Python.org 2.7's "last commit age" will continue increasing forever.

This isn't a great metric for evaluating Python 2 forks, that's all.

berti2• 6 years ago

The Python.org 2.7 branch is unmaintained so that’s hardly a strong point. That last commit, barring some exceptional circumstance, will be the last ever received by that branch.

erwinh• 6 years ago

Why tho

joshuamorton3• 6 years ago

According to the open issues, it doesn't currently support `async for`, nor is anything from 3.7 or 3.8 included (dataclasses, f-strings, breakpoint(), a number of performance improvements).

There hasn't been a main branch commit since python 2.7.17 was merged in ~6 months ago.

ltbarcly34• 6 years ago

It seems like a lot of effort to avoid the parens with print()

ATsch• 6 years ago

The bigger porting problem is bytes vs unicode. That can't be done automatically and is a lot of work.

ltbarcly34• 6 years ago

I guess it seems hard, but if you understand the basics it's just a matter of finding IO and encoding/decoding things appropriately. Python 2 str, unicode and Python 3 str are almost exactly the same as they ever were, and if you are doing binary IO bytes should work as expected.

It's not that big of a project, even for a moderately large codebase. If you think you can't get it done in a reasonable period of time feel free to hire me as a contractor and I'll knock it out.

nomel2• 6 years ago

Print is not the only difference between python 2 and 3.

Strings, most things are generators now, exceptions are different, relative imports changed, division, etc. On top of that, moving to python 3 requires updating your dependencies.

berti2• 6 years ago

If you're looking to continue running old codebases without testing and patching this isn't it. There are various incompatibilities with CPython 2.7, some noted in the readme, some noted in issues e.g. [0], and no doubt others unknown. It also doesn't appear to be actively maintained..

[0] https://github.com/naftaliharris/tauthon/issues/22

ehutch796• 6 years ago

It's too bad the python 3 transition was super short notice and no one had very much time to port their code at all...

RMPR2• 6 years ago

12 years isn't exactly what I will call a super short notice...

https://www.python.org/dev/peps/pep-0373/

Edit: I can't find a way to delete this, but I don't think it's fair to be downvoted because I didn't get it was a sarcasm :(

hackingthenews2• 6 years ago

(I think it was sarcasm)

tomwojcik• 6 years ago

That was sarcasm

oddlama• 6 years ago

This is probably what ehutch79 was referring to.

hackingthenews2• 6 years ago

I think we need to recognize that most people/companies who stick to python 2 doesn't do it because they are being stubborn. There is a real judgment call involved. It might look strange from the outside, but maybe it really is daunting to migrate some code-bases to python 3.

(I got the sarcasm)

acdha3• 6 years ago

In my experience, it's been a number of years since the judgement call was something other than “can we avoid paying for maintenance?”. Active projects had reasons 5+ years ago but by now it's really just a question of whether your team has a handle on technical debt management or is on an unsustainable feature treadmill.

oliwarner4• 6 years ago

Daunting? Sure, but it's also daunting jumping from branch to branch trying to find somebody to support your rusty old Python 2 installations. Or deciding to actively run on unsupported Python.

Modern software isn't fire and forget. Developers who have grown up around security exploits understand that. Higher management and FORTRAN77 programmers don't. To them it's developer busywork.

The real judgement being made is money and resources.

ehutch796• 6 years ago

I feel like if you're not moving to 3, you probably aren't going to be pushing to move parts of your codebase to async.

I have code bases that will be forever stuck on 2, because they're not active projects and are archival only.

strenholme3• 6 years ago

Here’s my real world example of some Python 2 code which will never get ported to Python 3 (and the resulting Ycombinator heated discussion): https://news.ycombinator.com/item?id=21258527

formercoder• 6 years ago

Firms should only embark on NPV positive projects. It’s possible that the efficient gains from the new python features outweigh the cost, but tough to prove.

a1369209993• 6 years ago

the python 3 transition was super short notice

No it wasn't; it was the better part of a decade. Look, we all know Python 3 sucks (even if some poeple, unfortunately including the former Python maintainers, refuse to admit it), but this sort of blatant falsehood helps noone.

ivalm• 6 years ago

OP was sarcastic.

ehutch796• 6 years ago

It was in fact sarcasm.

Everyone had 10 years of warning.

fastball2• 6 years ago

How does Python3 suck?

opportune• 6 years ago

Good thing python didn't have a package management system where it was easy to take an external dependency on something that would then later be a monumental task to update. Or, even if you did do that, at least the automatic 2to3 upgrade tool was so well polished you wouldn't need to spend much time figuring out why it wasn't working.

dang2• 6 years ago

"Don't be snarky."

https://news.ycombinator.com/newsguidelines.html

bigdict2• 6 years ago

Is this the most unnecessary project ever?

Edit: no, the author built it as a proof of concept.

naftaliharris• 6 years ago

Original author here. I built this a few years ago, and the main motivation at the time was that I'd heard people say that adding the new features in Python3 required breaking backwards compatibility, which I didn't believe. IMO the only feature that really required breaking backwards compatibility was the str/unicode consolidation and refactoring. This project was my way of proving that we could have gotten the other features that people tend to be most excited about (async/await, function annotations, new super(), etc) without breaking existing Python2 code. I think it was successful at that as a proof of concept.

It was a fun project; I learned a lot about how the CPython implementation works and have a lot of respect for the people that built it. It was surprisingly easy to implement Tauthon based off the work the core dev team did on Python3: https://www.naftaliharris.com/blog/nonlocal/

For what it's worth, I do believe that Python3 is a better language than Python2. We use Python 3.7 at my work (SentiLink) and we've had a good experience with it. (If you're starting a new project or can migrate, I'd recommend it). But I do think that the ~10 year saga of upgrading to Python3 from Python2 wasn't necessary when the main benefit was really the unicode refactoring.

I no longer maintain Tauthon personally but there are others who are excited about the project who occasionally add new features or bugfixes.

baq• 6 years ago

meanwhile exceptions just sob in a corner, forgotten.

the main benefit is actually sane exceptions. unicode is nice and all (i'm a native speaker of a non-english language) but it's a storm in a teacup IME.

toyg2• 6 years ago

I feel a bit like you’re trying to rewrite history here.

When you launched it, you called it “Python 2.8”. You posted it everywhere to gain traction, and didn’t rename it until the PSF and Guido got you by the ear, so to speak. There was no mention of “everything but str” or whatever, as far as I recall.

It was an outright (and hostile) attempt to fork - something that, going by your words here, I guess you now recognise as a mistake. I guess saying “I screwed up” is hard.

tyrust2• 6 years ago

Context, since I had never heard of this:

   * https://news.ycombinator.com/item?id=13144713

   * https://github.com/naftaliharris/tauthon/issues/47#issuecomment-277081725

loeg3• 6 years ago

You're coloring the history with a lot more hostility than the reality. And the condescension is really unnecessary.

This matches the pattern of Python.org developers and python 3 aficionados being unnecessarily hostile and condescending to the concerns of Python 2 language users. You saw that in 2010; you saw it again in 2015; and you can see it in these threads today.

fao_2• 6 years ago

Can you provide citations for your claims? The commentary in the github renaming issue from 2016(0) seems pretty jovial and has the tone of "oh, actually we should change that, oops" rather than "we WILL replace python!". The comments by guido and harris reinforce this perspective:

    > gvanrossum commented on Dec 10, 2016
    > Since I was asked: The project's name (and its binary name) need to 
    > change. They are misleading. The rest looks acceptable according to 
    > Python's license. This is not an endorsement (far from it).

    > naftaliharris commented on Dec 10, 2016
    > I don't mind renaming this project. Any other suggestions for good 
    > names? I personally like "Pythonesque (/usr/bin/pesque)" the best so 
    > far, thanks @dbohdan! :-)
    >
    > @VanL, not that I'm necessarily picking that, but would a name like
    > that be acceptable?

When you're in a community or a space, actions that are unintentionally hostile towards that community or space, can be seen as intentionally hostile by the members, and if you only hear about it second-hand, or you spend a lot of time around the group, that belief can be reinforced through the discussions and gripes the group has about it(1).

(0): https://github.com/naftaliharris/tauthon/issues/47 (1): That's not to say there aren't bona fide intentionally hostile actions that happen, but rather that's how actions that aren't intended to be hostile can be remembered and percieved as such.

Ensorceled• 6 years ago

Do you have any references that support the level of invective you are displaying? I feel you are projecting a great deal of ill will on the original author that I, personally, didn't feel or see.

lioeters• 6 years ago

I wouldn't call it a "hostile" fork. Imagine what motivated him to fork in the first place. He and many others were between a rock and a hard place, and a fork was a possible way out.

At the time, the author stated:

I picked [the name "Python 2.8"] initially since when talking with friends about this project it conveyed pretty darn immediately what the project is and does. I'd be very keen to hear people's suggestions for alternate names!

This was no mistake or screw up at all. In fact, the project served his purpose for the time, and even though he's moved on, there are others who like it enough to maintain or improve it.

A fork is (or can be) a healthy, natural thing that happens.

unixhero• 6 years ago

Let's go easy on the ad hominems here.

u801e• 6 years ago

As a general question, why is forking a project perceived as hostile? I always thought that forking an open source project was something that was encouraged if one's proposed contributions were not incorporated into the original project.

alfalfasprout• 6 years ago

First off, cool project Naftali! One of the issues that ultimately became more common as Python2 became EOL is that a lot of libraries began to release only wheels for Python 3.x variants. When there's C/C++ extensions this means that even running a compatible fork wouldn't suffice.

Do you think it would have been feasible to make wheels compiled against the python 3.x C APIs also compatible with Tauthon?

sandGorgon• 6 years ago

Cool project!

strenholme3• 6 years ago

I think there is some demand for legacy Python 2 to be maintained, even if the maintenance is limited only to security fixes, and I think it would be good if some company paid you or someone else a full time salary to continue maintaining this Python 2 fork.

It’s a tragedy of the commons: A lot of companies making good money are still using Python 2, but none of them are willing to pay programmers to maintain an open source currently maintained Python 2 fork.

int_19h4• 6 years ago

ActiveState is offering commercial support for Python 2, for those who need it. But places that are so conservative as to need it, are also precisely the ones that will want vanilla Python rather than a fork.

https://www.activestate.com/products/python/python-2-7/

takeda2• 6 years ago

I don't know what it supposed to prove. It was no secret that Unicode was what broke compatibility, the other changes were there to use that as an opportunity to fix warts that accumulated over years (like organization of system packages, print statement, division, classes etc. I don't believe Thauton fixed these things though)

It was no secret that everything else was portable, that's essentially all what python 2.7 was - backporting 3.x features to Python 2. It was a waste of resources for developers to maintain 2 forks of Python so that effort was stopped in 2015. There was 5 extra years to move application to Python 3.

5 years is frigging long time in computer terms, and your project felt still like giving f-you to the core developers for trying to improve the language. Trying to call the project Python 2.8 was very aggressive and would create a lot of confusion if Guido would accept that.

I'm glad you gave python 3 a try, I feel like the people that were against it didn't write any new application in it, and their experience was porting Python 2 code to 3. It can be very frustrating when Python 3 complains about bugs that Python 2 just ignored, but if you write a python 3 application from scratch you don't even notice the Unicode, and that was the goal.

ehsankia• 6 years ago

Some of the changes I feel were a step too far though. Completely removing iter{items,values,keys} just made the migration so much more painful. I would say I spend a significant amount of time figuring out if I want to rely on six or if the array is small enough that I can live with unoptimal .items() in py2 throughout the migration. What would've been the issue of keeping .iter*() calls around, with a warning maybe? In general, there's a lot of stuff that could've been kept for backward compatibility, with a warning. It's much easier to slowly fix issues like that over time as you maintain code than to have to convert it all in one go, which put a giant barrier for people want to switch.

tengbretson• 6 years ago

When are we going to start treating projects that are stuck on Python 2 the same way that we treated IE6 or Adobe Flash? I'm kind of sick of hearing the "well some people have to make technical tradeoffs in what they choose to support" argument. We're all trying to make better software here. Catch up.

wnoise• 6 years ago

This just sounds like Python 3 with extra steps.

KMnO42• 6 years ago

I couldn’t care less about support for libraries and builtins. If my boss came up to me and said “Hey, I think we should use this Python 3.x library”, my response would be “Great, let’s use Python 3”.

That said, I understand there are certain applications that are not compatible with 3.x and the company does not have the resources to dedicate to rewriting it. So let’s suppose there is a valid reason someone is forced to use Python 2.7. In that case, the number one priority of this fork should be back porting the security fixes.

You can live without async, type-hinting, and f-strings. But please be responsible when it comes to security vulnerabilities.

mattbillenstein• 6 years ago

I hope this is the last Python2/3 thread we have to endure - I fear it is not.

gjvc• 6 years ago

It is possible that the transition from Python 2 to Python 3 is the worst-handled of its kind in the history of open-source so far. Is it any wonder, then, that things like this spring up in their wake?

cesarb• 6 years ago

I believe the transition from Perl 5 to Perl 6 was worse. (A good overview of it: http://blogs.perl.org/users/ovid/2019/08/is-perl-6-being-ren...)

qwerty4561273• 6 years ago

Matrix Multiplication Operator

Does this use SIMD instructions?

bigdict2• 6 years ago

Yes, if you are using NumPy. But `@` is just syntax. An operator by itself doesn’t “use instructions”.

MatthewWilkes• 6 years ago

Please, if you read this and decide it's a great idea, don't start bugging open source maintainers to officially support Tauthon in their projects. The most disruptive voices I've encountered in the last 6 months have been doing this.

OptionX• 6 years ago

Let python2 die people. Move on. Isn't 10 years of tech debt enough?

tyingq• 6 years ago

Interesting. Is there anything here to deal with the three states[1] of "bytes" and "barray" vs "strings"? That was the hardest part, for me, to release an extension that was 2vs3 agnostic. Personally, resorted to some ugly hacks with version detection, barray detection, etc to make a 2/3 compatible extension.

[1] The three states being 2.x prior to recognizing bytes, 2.x sorta recognizing, and 3.x hard recognizing.

habosa• 6 years ago

Is there a simple guide to understanding the str/unicode changes from python 2 to 3? I've made mistakes with strings that ruined every single piece of code I've upgraded from 2 to 3 unless I just littered the code with defensive six() calls ... and even then some things still broke.

hartator2• 6 years ago

I still don’t fully get why we can’t have ‘print x’ in Python 3. We would have avoided 10 years of schism for something that was soooo backward incompatible.

maest• 6 years ago

There are ample discussions about the change in `print x` syntax in Python all over the web.

hartator2• 6 years ago

Well everyone understands the academic reasons. (Print is a function and now you can do ‘my_print_function = print’) But really it was not necessary.

What piece of code is taking advantage that print is a pure function nowadays?

[EDIT] I am genuinely curious.

Tauthon: Fork of Python 2.7 with new syntax, builtins, libraries from Python 3

Related Stories