HN user

livando

80 karma
Posts2
Comments18
View on HN

"Multi-threading is a must, when scraping at scale."

I disagree on this point. Starting with a single threaded model allowed my team to scale quickly and with little additional overhead. What we have lost with performance we gained in simplicity and developer productivity. That being said tuning and porting portions of the app to a multi-threaded system is slotted to take place within the next year.

Start with single threaded and simple, move to multi-threaded scrapers when the juice is worth the squeeze.

With this article I was really hoping for some counter points of how JEE is doing some new and exciting things. Unfortunately, it looks like the only innovation was to throw some of the dead weight off the back of the truck. I can understand how corporate developers (i.e. can't use open source and it needs a support plan) would be happy about the lighter alternative approaches. I just don't see how given the whole technical landscape, that JEE is still a viable front runner.

WakeMate review 16 years ago

Fair review, but I'm still pumped for mine to arrive. I don't mind supporting smart people trying to innovate, and I'm betting the app will be getting better all the time.

I concur with your advice. BigTweet is a nice little add on that the author should be proud of, but few would pay for. But if you keep rolling out apps, the port and exp will grow. Before you know it you have google jet money!