YouPorn: Symfony2, Redis, Varnish, HA Proxy... (Keynote at ConFoo 2012)

https://joind.in/6123

by Sujan • 14 years ago

67 49 14 years ago

MarkPNeyer• 14 years ago

working on porn, in any respect, supposedly locks you into that industry. it makes no sense to me.

any thoughts on why?

Sujan3• 14 years ago

Why should it lock you in?

EliRivers• 14 years ago

Because you've been a bad, bad boy and now you're going to be punished... :p

kajecounterhack• 14 years ago

This is purely conjecture but the industry might still face prejudice due to issues of "morality" based on peoples' religions or personal values. A person working in the industry is perhaps establishing a link between themselves and something "dirty" or unscrupulous, and so they appear to potential employers to be less than desirable for that reason sometimes?

It seems likely this is the case to me, but again it's all conjecture. Personally I don't think it matters either. Experience is experience.

(And I'm not sure your supposition is at all true to begin with. The above assumes your supposition has some truth value.)

c0ff• 14 years ago

That may be true of actors and directors. But for software? I can't imagine.

Look at the presenter's experience:

> Over a six month period, I lead the project to rewrite a top 100 website using a new software stack. Doing so, we used HAProxy, Varnish, Nginx, PHP-FPM, Symfony2, Syslog-ng, Redis and MySQL to create a platform that handles 100 million page views per day and has room to grow.

There are tons of companies out there eager to hire someone with that experience.

perigrin• 14 years ago

This actually isn't true. I know several people who worked on the Perl version of YP and none of them are locked into the industry.

paulhauggis• 14 years ago

I would think it would help you. Many porn sites get so much traffic, the engineers rival those of Facebook and Google.

EricPickupYP3• 14 years ago

As much as I might like to think so, they are several orders of magnitude beyond us.

ma2rten4• 14 years ago

It's not just about traffic, you know. Facebook is different then most sites, because it generates different content for almost every pageview. This means much less possibilities for caching. And Google, well...

batista• 14 years ago

>working on porn, in any respect, supposedly locks you into that industry.

"Supposedly" by who? I don't think it's real at all.

nostrademons• 14 years ago

Not true at all. I know a few current and former Googlers who worked on porn. The skillset is fairly transferrable to any high-traffic website.

At least in Silicon Valley, people don't really care what you've worked on as long as you have the technical chops. Hell, look at all the folks working for defense contractors - their biggest problem is that it tends to lock them into a proprietary software stack, not that they've written software used to blow people up.

herge• 14 years ago

It's a funny world where selling porn is considered less honorable that selling your user's information to advertisers.

ma2rten4• 14 years ago

Why do people keep saying that? As far as I know no respected company is doing that.

TheHegemon• 14 years ago

Are you kidding?

I worked for gay porn company for a number of years and I've never had issues finding work outside of that industry. At least not for companies I'd be interested in working with in the first place.

If anything it helps considering there is a lot of big scaling issues that need to be solved when the viewer base gets big enough.

xal2• 14 years ago

Heh, this is true. However, everyone always misunderstands why. I can speak to this from two perspectives. I worked in Porn early in my career and made the transition. I also actively try to recruit people with this background in a city which has lots of great engineers work in this area (montreal).

The reason why people say that they get locked into porn ( at least on the software producing side of the fence, mind you ) is very similar to the reason why people get locked into working for the finance industry. There are nuances but it essentially comes down to two things: money and attitude.

If you are good at engineering and if you are willing to work for this industry ( a small minority ) you can make crazy money. The industry is pretty meritocratic, long before that became a buzz word in Silicon Valley. If you generate millions of dollars in value for your company, you can take millions of dollars home. Very few industries allow you to make this kind of money, and since the people who work there are self selected to not mind, it's tough for them to leave. Some social factors sometimes compel them to move to other industries anyway ( such as starting a family ) at which point the second problem comes into play.

The second problem is your mindset. The porn industry ( sometimes similar to financial ) has the opposite relationship with it's customers from other industries. You treat the customers as bags of money that ought to be exploited. "After all, they are all wankers" is what you hear a lot. This may sound basic but if that's the reality you have spend a decade in, you will find it tough to adjust for startup life.

Especially startups are usually very good at finding a business model where their own success depends on the success of their customers ( Square, Stripe, my own Shopify, etc). Life is much happier in a setup like this but it's totally alien to people from the Industry.

These things are huge contributors at keeping people from the industry in the industry.

moe• 14 years ago

You treat the customers as bags of money that ought to be exploited.

I honestly don't see a difference to most other industries - give or take a few idealistic startups.

Unless you meant to say the difference is that you're upfront and honest about it...

jasonlotito3• 14 years ago

It does not, in any way, lock you into the industry. And, in my mind, working in porn helps you find employment with good companies in a way no other company or industry can.

chrisacky• 14 years ago

I couldn't open the Presentation. Don't have anything that can open .pptx so I uploaded to Google Docs.

Link here: http://tinyurl.com/7bckqm8

After going through the 32 slides, I think this is one of those presentations you really needed to be there for. The archiecture stack is by and large quite standard for high scale applications. The previous HN submission from YouPorn last month contained more descriptive information.

fdb• 14 years ago

Could you link to that presentation? I can't seem to find it.

Sujan3• 14 years ago

"Slides" link on joind.in: http://www.manwin.com/confoo/eric2012.pptx

EricPickupYP3• 14 years ago

The video should on Youtube next week, It contains a lot more information. The slides are just bullet points.

wickedchicken• 14 years ago

> The video should be on Youtube next week,

Youtube or...?

cheald• 14 years ago

OpenOffice/LibreOffice Impress handle it nicely, FWIW.

bonsai• 14 years ago

Thnx for upload.

mthreat• 14 years ago

Does anyone know what YP is using for their fulltext search?

givan• 14 years ago

It doesn't say anything about the number of servers, that is important to know how efficient the software stack is.

antirez• 14 years ago

True, Eric provided some info in the Redis google group some time ago:

"I just want to let the devs know that Youporn.com relaunched two weeks ago with Redis as our primary database. With @100 million page views per day, our cluster of Redis slaves are handling over 300k queries per second.

After the switchover we had to add some additional Redis nodes but not because Redis was overworked but because the network cards couldn't keep up with Redis."

So at least the Redis part is I/O bound apparently, that suggests an high degree of efficiency for this use case.

mhd2• 14 years ago

So, a pretty massive rewrite and then it's 10% faster? Wonder how much of the reason for it in the first place is that a bunch of PHP/standard component programmers are probably easier to get (and replace) than competent Perl hackers…

Sujan3• 14 years ago

10% faster is quite an achievement for a site, that was fast before.

Additionally, I don't think speed was the primary drover behind the rewrite. I'm still waiting for the video/audio of the presentation, but he mentions the 'very complex architecture' of the PERL backend. So switching to a 'standard' architecture that fits on one PP slide seems liek a very good idea.

EricPickupYP3• 14 years ago

I love Perl (worked with it for 10 years) but we decided early on that finding enough good Perl devs would be too hard. There are still a few Perl parts of the rewrite, you need to use the right tool for the job.

mhd2• 14 years ago

Sure, and I certainly agree with that choice. In my (limited) experience, a Perl->PHP rewrite either indicates a desire for more developers or a seriously weird or outdated Perl architecture. The latter was quite common in the early days of the web, where basically everyone's first web page was a mess of Perl CGI (or mod_perl a bit later), but as YP isn't that old and performed well enough before, I wouldn't have thought that this was the case here. Thanks for confirming my suspicion, wasn't intended as criticism.

badclient• 14 years ago

Funny. When I was running my music startup with couple mil users, it was originally coded in perl by my perl-genius cofounder. When my cofounder dropped, it was next to impossible to find solid perl developers(there are many who tinker with it). I remember recoding it in php and life getting so much easier.

mst2• 14 years ago

"Finding enough good Perl devs" is actually very doable, it just tends to involve engaging them through the CPAN community. We've helped customers out with it a few times, with good results overall.

On the other hand, when trying to de-convolute an architecture a change of primary language often helps.

Does "we were substantially more confident of our ability to recruit sufficient PHP talent than Perl" seem like a reasonable rephrasing?

ma2rten4• 14 years ago

I wonder if they considered HipHop instead of php-fpm.

kijin• 14 years ago

Maybe they were focusing more on scalability than individual page load speeds?

jiggy20113• 14 years ago

So, if one were so inclined how would you get a job working in the porn industry (on the technical side)?

I've always wanted to tell people I work in porn but I somehow doubt that any of their job adverts directly say "This is porn".

corford2• 14 years ago

These guys are probably who you're looking for: http://wwww.manwin.com

8ig8• 14 years ago

I think there's an extra 'w' in the above link. This one works: http://www.manwin.com