Ask PG: Would you accept donations towards buying a monster server for HN?

https://news.ycombinator.com/item?id=2116941

by jacquesm • 16 years ago

26 17 16 years ago

news.ycombinator.com

HN is pretty slow again, and has been for some time now, connections timing out the rule rather than the exception when posting an article or commenting.

I can see why it's hard to scale HN using multiple servers, what with ARC not being the most cluster friendly platform, but there is a different solution, simply getting a really badass server.

I'm sure that you're not wanting for cash, but since we all use this resource and we all profit from it in various ways would you accept donations that would go to buying a server large enough to see HN through the next phase of growth?

pg• 16 years ago

We have the fastest server we could conveniently get about a year ago. But I just asked Rtm if we could get something better.

Unfortunately processors don't seem to get faster at the rate they used to. Now they just have more cores, and I believe MzScheme can only use one core.

I think the solution is to make the software faster. There's probably some small hack we could make somewhere in the guts of the system that would make it 10x faster. I'm thinking of having a make-HN-faster contest, but first I have to release a more current version of the code.

rick_20472• 16 years ago

but first I have to release a more current version of the code.

Please do that, some people are dying to see some of the stuff that goes under the hood. Of course things like hell banning may not be released, but some of the ranking and "ask" page stuff would be awesome to look into.

projectileboy2• 16 years ago

http://ycombinator.com/arc/arc3.1.tar

btilly• 16 years ago

I believe MzScheme can only use one core.

How annoying. http://osdir.com/ml/lisp.scheme.plt/2003-01/msg00216.html suggests that you're right, but MzScheme _used_ to support multiple cores.

How difficult would it be to divide HN up into multiple cooperating processes that each get their own core? With a little caching, it should be possible to get a lot more headroom.

jacquesm2• 16 years ago

Would it be possible to layer arc on top of clojure or is that too far distant from each other?

alnayyir2• 16 years ago

That's a bit much to resort to when AFAIK caching hasn't been pursued aggressively as it could be.

gus_massa• 16 years ago

Since version 4.2.5 MzScheme can use many cores with future and touch (http://docs.racket-lang.org/guide/performance.html#(part._ef... ) But they apparently they are tricky to use.

For example the manual says that to run the computations in parallel, you should no use operations that mix exact ans inexact numbers. It is possible to do this by hand writing the scheme code, but I think that it is more difficult to make the arc->scheme compiler do this automatically.

awwx• 16 years ago

I agree, given some representative sample data and a goal ("make rendering this page go fast"), finding speed improvements likely wouldn't be very hard.

coderdude• 16 years ago

HN is becoming crazy mainstream. Might as well start figuring out how to use multiple servers now before we're really in a bind.

roschdal• 16 years ago

PG: Have you considered setting up Varnish on HN? I think this would solve most of the current issues with the performance.

http://www.varnish-cache.org/

jason_slack• 16 years ago

I learned about Varnish a few weeks ago and it looks promising. I am setting it up now on one of my sites to test properly.

sigil• 16 years ago

Varnish is indeed crazy fast...for things you can cache. The complaints I've been seeing center on slowness and timeouts for posting stuff, in which case Varnish (like the recent pagination change) might not help.

I think PG opening up the latest code and giving people a chance to actually diagnose the causes of the slowness is a great a idea. Anything else -- premature optimization.

Ask PG: Would you accept donations towards buying a monster server for HN?

Related Stories