How PayPal is being revolutionized by Node.js and Lean UX

http://www.nearform.com/nodecrunch/?p=109

by darsee • 13 years ago

121 100 13 years ago

it's interesting how some established startups (if paypal could even be referred to as that anymore) have been getting gains by leveraging node.js into existing stacks.

ericcholis• 13 years ago

To me, it's more interesting to see the willingness of large companies like PayPal to leverage "new" technologies in any way. The common narrative of large companies suggests otherwise.

widdershins• 13 years ago

Interesting article, but I'm in the UK and Paypal still looks and works exactly the same for me as it has for years. It's very slow, rather ugly, and very inconsistent - the splash page seems styled differently to the login page seems styled differently to the help page... I could go on.

I know this is mostly superficial stuff, but it doesn't contribute to a good experience at all.

Slightly OT, but why is it suddenly all the rage for sites to have persistent nav-bars that follow you down the page? The one on this blog is obnoxiously huge, and obscures a lot of content. What's wrong with scrolling to the top of the page?

orr94• 13 years ago

I totally agree about the persistent navbars. If they are really slim, I can tolerate it. That one is obnoxious.

KiwiCoder4• 13 years ago

I put a persistent nav bar at the top of my latest site (http://bosscheck.me) so that potential customers are always looking at a "book now" option. Shallow and obvious, perhaps, but my assumption is that this is optimal for conversion.

rattray• 13 years ago

Conversion in particular is where you should never assume what's optimal =)

QuasarLogic• 13 years ago

Your navbar is much nicer and less intrusive though, I like it. Have you thought about differentiating the booking button from the rest of your navbar elements?

hackula14• 13 years ago

I genuinely wonder how it is possible for paypal to have such terrible design for so long? Everyone knows its an issue, and it seems like a couple 100k in design work would clear up the issue for the next 4-5 years.

gamache3• 13 years ago

The move toward Node as a platform will make things like cosmetic redesigns much easier to accomplish. I don't think many design firms could, or would want to, work within PayPal's older stack, which requires knowledge of not only Java but dozens of internal services.

Keep in mind that PayPal is a gut-wrenching edge case of internationalization and localization. If you're an Israeli person transferring Thai baht while you're in Chile, the website must account for that while respecting your user preferences and all local, national, and international law. It's a gross problem no matter how you handle it, and it utterly prevents drive-by redesigns.

jonknee• 13 years ago

Maybe it's not terrible? Amazon tests everything to death and their site is very "ugly" too.

CmonDev5• 13 years ago

Once you have been acquired there is no longer a point trying hard.

k-mcgrady• 13 years ago

How come? Wouldn't the company that acquired you want you to succeed so their acquisition wasn't wasted? If you're point is that the founder has 'succeeded' and won't try as hard his new boss will fire him.

dcc13• 13 years ago

Who cares about Paypal? they have screwed so many people and companies one has to wonder how they get away with it and still are in business

Bitcoin (or bitcoinlike replacement) is the future and the most interesting thing to come out (Technologywise) since email, bittorent or social networks

conradfr2• 13 years ago

Business who accept payment online. People who use eBay.

Let's compare that to the number of people who know what Bitcoin is.

bpicolo• 13 years ago

Not to mention that the vast majority of the cases of paypal "screwing the customer" are because something actually violates their ToS.

dcc13• 13 years ago

See the comment above comparing Paypal vs Bitcoin to Microsoft vs Linux

in the long run (10 years) either bitcoin or bitcoinlike replacement is going to trash Paypal in same manner that Linux, android etc is trashing Microsoft now, yes Microsoft still make billions but bitcoin is growing rapidly, many people wrote off Linux years ago too for very similar sounding reasons.

chaz2• 13 years ago

PayPal processed $44 billion in 2013 Q3 alone. It's miles ahead of everyone else right now. That can obviously change, but when a the slow giant gets a little quicker on it's feet, everyone notices.

skrebbel• 13 years ago

PayPal is to worldwide online payments what Microsoft was to OSes in the nineties. Bitcoin is Linux. It's free, and by some measures better, yet nobody wants to use it.

They're essentially a monopolist.

Nobody else has what they have: access to nearly every country, worldwide.

Cthulhu_• 13 years ago

Except that the analogy is off, given that paypal deals in established currency that you don't first have to generate or buy (via a convoluted process that involves handing over your ID etc to trading websites, or selling post-order drugs).

leephillips• 13 years ago

I don't know much about bitcoin. If I pay you with that, and you fail to ship me the product, can I get a chargeback? If not, I think that answers your question.

outworlder• 13 years ago

Q) Who cares about Paypal?

A) People who are outside the US.

arscan• 13 years ago

I'm not so sure that it was node.js that revolutionized paypal... seems like the gains were from switching to a more agile process (and adding some supporting tools). A lot of the fixes seem pretty agnostic to the actual platform that they switched to.

Note that all those open source modules that they talk about aren't released yet. krakenjs.com doesn't even resolve (for me at least).

pcunite• 13 years ago

I think it was Stripe that got a fire underneath them.

chaz2• 13 years ago

Braintree and Square, too. Maybe more so than Stripe.

jlebrech• 13 years ago

a new programming language is a perfect excuse to drop old ways of doing things, that way there's no legacy code pulling you back into old ways of doing things.

pdwetz• 13 years ago

Unfortunately in practice that's not always the case. People are often so set in their ways they'll still try to do direct ports of "the old ways" even if the new language/platform offer easier/more elegant solutions. It can be very frustrating.

bryanlarsen• 13 years ago

That's seriously going to reduce adoption of their open source modules. They're getting a lot of hits from this PR-piece, it's too bad they can't use any of that momentum for kraken or lusca...

sanderjd• 13 years ago

There is some brief lip-service paid to using the same language on the front-end and the back-end, but mostly you're right - they fixed processes, not technologies.

billwscott8• 13 years ago

I can assure you that I had to attack multiple fronts. Process is a big part of it (I brought in LeanUX methodology), but I can tell you (and painfully so) that if we had not changed the technology we would not be moving at anywhere near the speed we can now.

Mind you that what you see on the PayPal sites is not all updated. This is a testament to just how deep the technical stack has been.

Other colleagues have been attacking the services layer and they are doing just as much change in that layer as my team has done on the app/frontend layer.

kamaal• 13 years ago

Its will be a little saying too much, to say node.js revolutionized paypal. But speaking as a matter of fact programming languages do play significant role in the overall productivity of the teams.

I wouldn't be too surprised if they are saying using Java was making it difficult to turnaround issues. The problem isn't Java the language itself, but the processes involved in taking changes to production with these languages are generally so much ceremony large turnaround times become the norm.

Add to this issues like fixing build failures, test failures, code reviews that span meta aspects of code(config changes, xmls etc), running around to get libraries standardized per your company standards(getting it into your local maven repo etc); and much more. These sort of things really slow you down. With these if you are tending to multiples issues, or if you have dependencies on your colleagues- I wouldn't be surprised with the 6-month delays mentioned in the article are actually true.

smrtinsert• 13 years ago

Better said, since there is no compilation step, so you can push non working js through and it will look "green".

Java isn't defined by a process, that's sad most people don't seem to understand that. Look at projects like dropwizard, happily throwing away what they dont like.

fabioyy• 13 years ago

and if you look at the http headers, every single one is apache, or apache tomcat.. ( when not akamai ). where they really using node.js?

leokun3• 13 years ago

A little off topic but that giant fixed header that obscures 30% of the vertical space I have available on my 15" macbook pro retina so I can see a lot of whitespace. This is terrible and made me not want to read the post.

aytekin• 13 years ago

Same on iPad. I had to save to instapaper to be able to read it.

mromanuk2• 13 years ago

Absurdly large and static navbar syndrome

taf2• 13 years ago

right click - inspect element - select "<nav>" element following body tag. right click -> "delete".

leokun3• 13 years ago

Command+W to close the tab is just so much more convenient though.

Xophmeister• 13 years ago

It took up about 2/3s of the screen on my phone; it made me stop reading quite quickly!

josefresco• 13 years ago

You mean navigating to another page on the site is not your first priority when reading an article? /sarcasm

vaadu• 13 years ago

Why would anyone let an unregulated internet company have control of their money? Especially one with a track record like PayPal.

hackula14• 13 years ago

Convenience mostly.

conradfr2• 13 years ago

While it does not prevent horror stories and frustrating experiences, it's not unregulated in Europe at least.

icebraining• 13 years ago

unregulated

PayPayl is licensed and regulated as a money transmitter in multiple US states and as a bank in the EU¹. The idea that states would allow it to transfer the amounts it does without being regulated is funny, but not realistic (and not true).

¹ https://www.paypal-media.com/licenses

filipedeschamps• 13 years ago

Here in our company we are also changing everything to Node.js (currently abandoning C#) and it's being an amazing experience.

hackula14• 13 years ago

I did the same a couple years ago. Development speed has sped up drastically.

CmonDev5• 13 years ago

What kind of application is that? Is it more than DB + GUI?

PS: I switched from JS to C# - it is an amazing experience!

cliffcrosland• 13 years ago

We've considered doing the same thing at our startup. I'd love to hear more about your experience. Did you simply build a node.js UI server that made calls to a C# service that did the database accessing and business logic? Or did you cut out C# all together and access the database and perform business logic directly in node? We're worried that re-writing major components in node would slow us down too much at the moment, but I'd love to hear your thoughts on the productivity boosts you found.

ScottWhigham2• 13 years ago

Another "What - how does that work?" question here. What you've said just isn't making sense - you're replacing a server side programming language with a client-side JS framework. There's obviously other "things" going on - I'd be interested to hear much more. C# is such a great language but ASP.NET is so slow...

themoonbus• 13 years ago

Well for starters, node.js isn't a client-side JS framework, it's a server-side JavaScript platform.

KiwiCoder4• 13 years ago

Content (text only) mirrored here: http://pastebin.com/aFPtzPpD

CmonDev5• 13 years ago

"Node.js and javascript all the way up from just above the services layer to the frontend layer"

Sounds like a glorified web GUI layer. Good old statically-typed Java is still used for the stuff that does the actual work.

I propose a new article title "How PayPal switched to a more reasonable web GUI approach".

Already__Taken• 13 years ago

I propose you read the article past step 5

CmonDev5• 13 years ago

I did. That quote is from step 6. Read again - everything revolves around front-end.

smackfu• 13 years ago

As of today, Paypal still shows an interstitial progress screen after you enter your userid and password and hit sign-in.

mromanuk2• 13 years ago

"Lines of code: In general they have seen code size shrink by factor of 3-5 by moving from the Java stack to the Node stack. Even the number of files shrunk by a similar factor."

How do they went to 3-5 times less files without recurring to several classes, controllers, etc in the same file?

jffry2• 13 years ago

I'm guessing that a good chunk of that came from either reducing the actual complexity of their UI, or from simplifying horribly-fragmented old code. And I would guess more of the former than the latter.

billwscott8• 13 years ago

We actually had the luxury of comparing parallel efforts. While Node was being readied for production we were writing our new consumer/wallet app with the PayPal Java framework (based on Spring). Meanwhile we started writing the Node version. Once we got to a certain point the Java team insisted that they stop the Java development and start on Node with the node team. At this point we had two versions of the app built in two stacks and thus could compare code size, developer productivity & scale.

Of course, if we compare the new node apps vs the old PayPal code... well that would be a poor comparison as they really were old and monolithic.

lightblade• 13 years ago

It could also means the functionalities of those files had been moved over to libraries. NodeJS does have an impressive amount of libraries, and dare I say probably more than Java.

Touche• 13 years ago

In most cases position: fixed is unnecessary. But it's especially unnecessary when your navbar takes up a quarter of the height of the screen. Please reconsider this approach, it's really not needed for users to see your sitemap at all times while reading a technical article.

zabraxias• 13 years ago

PayPal is still running on something they call XPT and their admin tools are built with something called Maxcode (Max Levchin wrote it).

As of 2 years ago they revolutionized CSS by having it being generated from Java because their fellowship of the ring wanted control over which properties you can use and how. Everything they do from a tech perspective has been to allow them to hire cheap labor while restricting which HTML/CSS they have access to. It's probably a good business strategy (like making sure they don't get classified as a bank) but it fosters an abysmal tech culture.

I have no way to prove the points above you'd just have to trust that I worked there as a UI dev.

gamache3• 13 years ago

I worked in PayPal's mobile team for a few months last year (acqui-hire). I can confirm that as of two years ago, everything you said is true, but as of last year the groundwork was laid for everything in this article being true today.

I used the PayPal Node stuff in its infancy (i.e., before anything was working). At that point it was a thirdhand tarball being passed around. Even in its preliminary state it was about 500% less shitty than using the old stuff (Sparta, JSP, etc).

I am glad to see that the (small, relatively insulated) team could deliver. I expect this migration will make many PP engineers obsolete, but I don't have a problem with that.

yla92• 13 years ago

Is there anywhere I can read about the technologies that PayPal is using in production ?

dangrossman• 13 years ago

PayPal has so much legacy UI cruft even their own employees can't figure it out. I was talking to a support rep on the phone last week who was telling me to click something on my home screen that wasn't there. It wasn't there because she couldn't figure out which of the 4 home screen interfaces my account was on -- having a business account with PP Pro subscription means I have extra reporting menus but none of the updated design of the past few years. Going from their public site to my logged-in profile page exposes me to 3 totally different page layouts.

jebblue• 13 years ago

I read through the article, hard to believe they will be processing billions of dollars on top of JavaScript. Maybe I've been wrong and now it's time to dump Java and go all script.

edwinnathaniel• 13 years ago

I wouldn't do anything reckless until I see the codebase, the system architecture, and pretty much everything.

If it's just a front-end of the actual processing then meh.

chillax• 13 years ago

I'm guessing this is just the front-end of paypal. The backend is most likely Java-based and is never going to be changed to Node.

pkmishra• 13 years ago

I do not agree that node.js revolutionized the company and everyone should start using node.js everywhere. Rather a better approach is what suggested by Nicholas - http://www.nczonline.net/blog/2013/10/07/node-js-and-the-new...

billwscott8• 13 years ago

Hmmm... That is kind of humorous, given that Nicholas was a consultant for us early last year as we started this effort. His team was also at PayPal today (from Box). That article was influenced to some degree by me giving a talk at Box HQ on this very topic.

Also, maybe you misunderstand what we are doing. We are using nodejs for the web app layer. But it could be used for services (like Walmart Labs is doing). You really have to determine the context, the people, the technology, the amount of risk, and so on to decide.

But I do agree, no one should see nodejs as a silver bullet.

tootie2• 13 years ago

"Head count: Node.js. Typically they are seeing anywhere from ⅓ to 1/10 the number of developers needed on the new stack vs the old stack."

That's ridiculously implausible. I know node is more concise, but 300-1000% productivity increase doesn't come from less typing. Even if we assume they had a lengthy build/restart cycle on some J2EE web container, that could only plausibly increase output by 10-20%. Sounds like they had an absolutely shitty application architecture and he convinced them to do something both reasonable and modern with the former being the biggest improvement.

astral3032• 13 years ago

First of all, J2EE web container lengthy restart is a huge overhead alone, and in case of iterating over a feature, I can easily see a 30-50% difference.

But second of all, from the article: "You didn’t write html, css or javascript – you wrote css in java, you wrote html in java and you wrote javascript in java." Can you imagine developing that?

A Google search reveals: "the only thing worse than XML/XSLT is a proprietary form of XML/XSLT -- which of course is what PayPal was built on (C++ on the backend)." (from http://looksgoodworkswell.blogspot.com/2012/09/why-you-shoul...)

I've been involved with writing Javascript in Java using GWT and it has poor developer productivity, especially as the project gets to anything beyond a medium size. At a medium size, compilation times get well above 5 minutes, and I can't even imagine a large app the size of a PayPal.

Imagine having to figure out the right XML/XSLT to transform into the right Javascript.

So it's not at all ridiculously implausible. At previous jobs, I worked with some really arcane stacks and you can't imagine the number of unnecessary layers that get in the way of good old GSD.

jmagoon• 13 years ago

So it still has little specifically to do with Java vs. node.js. Just a ridiculously bad old process to a better new process.

eeperson• 13 years ago

Servlet container restarts don't have to have a huge overhead for development (not sure about using a full j2ee container). If you use SBT[1] plus the the xsbt-web-plugin[2] then restarts are just as fast as they are for node projects.

Disclaimer - I contribute to the xsbt-web-plugin

[1] http://www.scala-sbt.org/ [2] https://github.com/JamesEarlDouglas/xsbt-web-plugin

larsmak• 13 years ago

J2EE-containers can take a while to start indeed (for large projects), but in most cases they can be tuned to restart in less than 10sec - at least during development.

Using GWT you don't "write Javascript in Java", the java-code is written in java and then compiled to optimized, cross-browser, javascript. Compile-times can be quite long - but again, during development you will use the hosted-mode (or even the new super-dev-mode) version, which compiles on the fly. Full java-to-javascript compilation is typically only needed when new versions are deployed to stage/prod.

tootie2• 13 years ago

Poor separation of concerns can be easily solved in J2EE and build/restart can be solved with something like JRebel or just using a lighter contain like Jetty for dev. I'm just saying Java to Node is not going to give everyone a huge increase in productivity. Programming in itself doesn't even solve most of the worst bottlenecks.

Joeri• 13 years ago

I read that another way. The whole article sounded like they replaced engineering with hacking/prototyping and measuring the productivity of a prototype vs the productivity of battle-hardened code. There is an admission that they have no specs and i also got the impression that they have no tests. I seriously doubt that at the claimed turnaround times they have a proper code review process. It sounded to me like they were getting false productivity and eventually would program their way back into a corner.

Having been on a team where the pendulum has swung both ways I know firsthand that you can be too agile and end up regretting it later.

mooreds• 13 years ago

I wondered about that too--I didn't see any mention of any software testing. Definitely user/outcome testing, but no unit tests or functional tests were mentioned.

ilaksh• 13 years ago

If they had that many developers I am not sure why they invented such a complex alternative to a private npm repo with the full replication. I mean it only took half a day to set up and then two days to load. Heh.

But I glad they did it. I really feel like someone should take some code like the core if Kappa and integrate it into npm so you don't have to wait two days or whatever to replicate a giant database just so you can have private modules. I know that will make a few peoples business model go away but I hope that's not what is stopping it from happening.

neovive• 13 years ago

This is interesting. We researched Node.js a bit and found many arguments on StackOverflow against using it for apps that require heavy DB interaction and/or non-real-time apps. Also, there seemed to be many people pointing to the single-process model as risky. Is this no longer the case or have these issues been addressed through configuration? I would love to give Node.js a try again.

baudehlo• 13 years ago

It never was the case. Ignore the stack overflow dribble.

puppetmaster3• 13 years ago

PayPal sux! that's all.

singingfish• 13 years ago

Article really confirms my suspicion that node.js is pretty much the new perl. (Although modern perl is still an excellent choice for new projects if you need small team high impact stuff).

How PayPal is being revolutionized by Node.js and Lean UX

Related Stories