Python 3 at Facebook

https://lwn.net/SubscriberLink/758159/f1f631e1535ab9d6/

by wyldfire • 8 years ago

339 98 8 years ago

lwn.net

aidos• 8 years ago

That’s a pretty large juggernaut to try to change. I’m surprised and pleased they managed to make it work.

I wasn’t aware that Facebook used python so heavily. Presumably much of their code is in Js these days?

holografix• 8 years ago

I’d love to know what an entry level Python coder would earn and need to know to get a job at Facebook.

I’m so tempted to try and get a job in software eng sometimes, it’s by far what I have the most fun doing.

kajecounterhack• 8 years ago

FWIW larger companies don't hire "x coders" where x is some language. They look for CS fundamentals, mastery of programming paradigms, and then you work in whatever language you need / makes sense given the task.

kej• 8 years ago

If it's what you enjoy and you're not tied into a different job for other reasons, go do it. I'm not sure what Facebook's requirements are like, but there are tons of places that would hire someone with Python skills at a decent salary.

https://www.payscale.com/research/US/Job=Software_Engineer/S...

outworlder4• 8 years ago

The best advice I can think of is to get involved with open-source. You'll learn a lot by working with your peers (some of which may be even at Facebook and other companies) and you'll figure out what you need to learn. You'll also end up with code you can show, and evidence of being able to work in a team and complete actual tasks. This is huge for prospective employers.

akhilcacharya• 8 years ago

They don’t care about language, they care about algorithm ability (specifically solving problems quickly) and system design ability at higher levels.

pushtheenvelope• 8 years ago

Facebook code kinda has three layers: 1. clients (Objective-C for iOS, Java for Android, JS for Web and React Native) 2. application logic (Hack) 3. services (c++ and python are most prominent and well-supported, but there's a long tail of languages like Java, Haskell, OCaml, Ruby, etc.)

The prominent languages in each layer have pretty substantial code-bases.

More importantly, there's an engineering culture of eschewing "code ownership" and encouraging "code upgradability" so that motivated folks like Jason can drive these large changes.

wodenokoto• 8 years ago

Did facebook drop php? I thought they were really vested in it, with their HipHop VM and whatnot.

Thrymr• 8 years ago

Jason Fried says [0] that Facebook has "tens of thousands" of python entry points in its codebase (55% and growing are Python 3).

[0] https://www.youtube.com/watch?v=H4SS9yVWJYA

piyh• 8 years ago

What is meant by python entry points?

ksec• 8 years ago

I had to google and double check. Because the first Jason Fried I have in my mind is Founder, CEO of basecamp, Ruby Rails. And I thought when did he became a python guy?

Turns out there is another person in Facebook with the same name.

shafte• 8 years ago

Much of Instagram's application code is written in Python. Python also powers some service infrastructure. For example, FB's release automation system is written in Python.

stigi• 8 years ago

Besides what other people have mentioned FB is also one of the biggest Mercurial (which is build in Python) users out there and has a few developers working on it exclusively.

chucksmash• 8 years ago

There's also Mononoke[0], a "Mercurial source control server, specifically designed to support large monorepos" which they're working on in Rust.

[0] https://github.com/facebookexperimental/mononoke

mark-r2• 8 years ago

When they ran the resulting code, they found it was 40% faster and used half the memory. This points to a persistent myth that Fried has heard: Python 3 is slower than Python 2. That may have been true for earlier releases of Python 3, but it is definitely not true now, he said.

I wonder why that is? My theory is that it has to do with the new Unicode encoding; if 99% of your strings are ASCII compatible, the new encoding will be twice as efficient. Seems unlikely that would account for all of it though.

pletnes• 8 years ago

Optimizations of the dict class (both memory and cpu wise) in python 3.5+ are pretty important, as it’s used a lot internally. E.g class members and methods are stored in a dict. This basically affects all python code.

rectangletangle2• 8 years ago

The dictionary implementation changes are the biggest speed up I can think of. They touch nearly everything in the language.

One of the interesting side-effects of this, is in 3.7+ dictionaries are now ordered by insertion, which I'm sure will help fix tons and tons of subtle non-deterministic bugs (and probably expose a few too).

Although I'm not 100% on the performance characteristics of the new unicode implementation. The unicode first string handling is a massive improvement when it comes to non-English languages. So IMO it's worth a hefty performance hit, based solely on it's own merits. In practice, it hasn't seemed to make a meaningful difference to me. So I'd wager it's good enough, unless you're hitting some sort of use case specific bind. In which case, it's probably time to leverage the C bindings anyway.

scardine• 8 years ago

Anything improving dict performance will have significant impact because lots of things in Python are implemented using dicts under the hood (including namespaces).

boltzmannbrain• 8 years ago

Here you go: https://speed.python.org/

hetman• 8 years ago

Unfortunately, one area where Python 3 is definitely slower is the startup time (specifically module importing). This can be very noticeable on embedded systems as I discovered after porting our codebase. The remaining language improvements still make this worth it though.

matt4077• 8 years ago

I always felt just a tiny bit of xenophobia, or at least ignorance, in those complaints. Yes, your strings have the beauty of containing no more than 127 different characters. But isn't it quite obvious that there are people "out there" that don't only speak english?

Being from Europe, my strings are still 99% ASCII compatible. But 1% is about four orders of magnitude too big for a failure rate. Every single string that goes through an application and isn't ISO-specified (bank account etc) will sooner or later be hit with unicode. Just wait 'till bank accounts allow emoji identifiers.

johnramsden2• 8 years ago

Repost of https://news.ycombinator.com/item?id=17413350

trevor-e• 8 years ago

Why does it say this was posted 2 hours ago and yours 1 hour ago?

std_throwaway• 8 years ago

Could this be a delay due to waiting for moderation approval?

johnramsden2• 8 years ago

I guess because of a moderator updating the post. As you can see it was posted 17 hours ago https://news.ycombinator.com/submitted?id=johnramsden

sctb• 8 years ago

https://news.ycombinator.com/item?id=17419144

It's rare that we rescue an older story and while it's waiting to get re-upped another repost gets traction, but that's what happened.

hyperbovine• 8 years ago

To save you the trouble of Googling: this is not the Jason Fried that more or less co-invented Rails.

cityzen• 8 years ago

I don't think Jason Fried co-invented Rails, I think that was all DHH.

NelsonMinar• 8 years ago

I was surprised there was no mention of six.py. In my experience it's been a key enabler for 2->3 transitions. But maybe not at Facebook?

AceJohnny2• 8 years ago

It's disappointing to me that six.py hasn't yet made it into 2.7's standard library. That would definitely help enormously.

rspeer• 8 years ago

The standard library is where code goes to die. If six were in the standard library, it would be as stagnant and impossible to fix as 2to3.

cup-of-tea• 8 years ago

There is also python-future which does similar things, I think.

andrewshadura• 8 years ago

Yes, and python-future does it in a more Pythonic was, it seems to me

martincmartin• 8 years ago

We definitely use it a Facebook, the linter suggests when you should use six instead.

trumped• 8 years ago

Python is really slow, but luckily, most python code can be optimized by improving the interpreter... maybe Facebook will release a better interpreter (there is already some faster interpreters, not sure why Python doesn't adopt them)...

blake_himself• 8 years ago

There was an article somewhere recently, of someone proposing changes to CPython for just that reason. Guido had already resisted several attempts at the kind of thing he was doing, from a desire to keep the CPython implementation simple.

eiji• 8 years ago

He wanted to tell the "story about how I and couple of engineers used our free time, with no authority whatsoever, and made Python 3 the dominant version at Facebook."

Isn't it great when engineers upgrade your codebase in their free time.

golergka• 8 years ago

It's great when you have the opportunity to use your free time to or something that impressive on your CV. I wouldn't pass on it, personally.

bananadonkey• 8 years ago

This.

I always take the opportunity to identify some long-term-beneficial changes which would look good on my CV, and run with them even if it means investing my free time before I can present the potential to the business.

ipsum2• 8 years ago

This is the quote you focus on? Free time here here is probably defined as time in between their main projects, or when code is building, before meetings, etc.

ilammy• 8 years ago

But you're still not supposed to get paid for that.

alexgartrell• 8 years ago

I'm actually fairly certain it was "within working hours" free time. Facebook doesn't have 20% time or anything like that, but, in practice, there's lots of extra time for whatever you want. This was my experience as an Engineer at Facebook and I certainly "pay it forward" as a manager.

Probably worth adding that we ultimately created a python foundation team, of which Jason is a member.

Here's something dumb I created in my "within working hours" free time that Facebook let me open source: https://github.com/facebookresearch/py2bpf

_sdegutis• 8 years ago

I've always had the drive to do things like this in my spare time. I'd love to get paid for solving technical problems for an employer. I think besides the obvious, I'd make a good candidate for an employee at something like Facebook.

More companies should encourage hiring developers who would solve problems in their spare time. When I applied for an apprenticeship at 8th Light, they had me write a tic tac toe game, and I wrote it in 4 or 5 different languages, just for fun. I think that's why they hired me.

Man, it would be good to have an employer. Any takers?

metabagel• 8 years ago

I don’t believe it’s kosher to provide a subscriber link to everyone on HN. I believe it is intended to be used to communicate with a few colleagues, not to be posted on a news aggregator site.

Per LWN.net:

“The "subscriber link" mechanism allows an LWN.net subscriber to generate a special URL for a subscription-only article. That URL can then be given to others, who will be able to access the article regardless of whether they are subscribed. This feature is made available as a service to LWN subscribers, and in the hope that they will use it to spread the word about their favorite LWN articles.

If this feature is abused, it will hurt LWN's subscription revenues and defeat the whole point. Subscriber links may go away if that comes about.”

dmurray• 8 years ago

Sharing a single subscriber link and having it hit the front page of HN is unquestionably good for LWN and its revenue. Which is, as your quote says, the whole point.

xapata2• 8 years ago

There's probably some parabolic curve of revenue vs frequency of sharing subscriber links.

otterlicious• 8 years ago

They're okay with it, in moderation:

https://news.ycombinator.com/item?id=10236304

bertr4nd• 8 years ago

Ha! The 2013-era linter he mentions that required “from __future__” imports was actually one of my bootcamp tasks! Hilarious and kind of gratifying to stumble across it in a random HN post all these years later.

Python 3 at Facebook

Related Stories