HN user

treo

358 karma

Location: Germany

Posts6
Comments51
View on HN

Do you have any pointer on how you've profiled Neanderthal during development?

When I originally set out to compare ND4J and Neanderthal, I've ran into the issue that I bottomed out at: they basically both call MKL (or Openblas) for BLAS operations.

Awesome! I didn't expect you to write that blog post so soon. It will be interesting to find out where the differences (beyond the 'f' ordering thing) are that make Neanderthal so fast, since when I originally started the comparison, both basically boiled down to calling MKL through JNI.

For completeness sake, could you also provide some information about your machine specs and operating system (and if on linux your glibc and kernel version)?

It looks like while converting from my benchmarking code you've dropped the 'f' when creating the resulting array.

https://github.com/treo/benchmarking_nd4j/blob/master/src/ma...

The difference is rather huge with the newer versions of nd4j.

While the numbers in the following gists do not contain the measurements I took for neanderthal, they do contain the numbers that I got for ND4J.

Without f ordering: https://gist.github.com/treo/1fab39f213da26255cf4f75e383ff90...

With f ordering: https://gist.github.com/treo/94fe92c9417b5c8b24baa12924a35b0...

As you can see something happened in the time between the 0.4 release (I took that as the comparison point since that was when I ran my own benchmarks the last time) and the 0.9.1 release that introduced additional overhead.

Originally I planned to create my own write-up on this, but I wanted to first to find out what happened there.

Given that ND4J is mainly used inside of DL4J and the matrix sizes it is used with usually are rather large, the performance overhead difference that I've observed there for tiny multiplications isn't necessarily that bad, as the newer version performs much better on larger matrices.

TDD django Tutorial 14 years ago

Your tests, or the output they produce, look awesome. Can you elaborate on your setup? Just today I was looking for an up to date introduction for testing in django, but could only find some older blog posts. So I would be glad if you could detail your setup a bit.

That would take forever, at least on my system, as I don't really have a lot of entropy available. Just for kicks I have started this when I began writing this reply. And now I have the huge amount of 200 Byte in this test file. So thats about 10 bytes per second.

But if you take /dev/urandom you can get quite a bit more. On this system it is about 7Mbyte/s.

My Thinkpad T400s runs at 8W/h when tuned with powertop2 (3g disabled, wlan disabled). But as the battery only has a capacity of 43Wh it only comes up to a bit more then 5 hours.

I could however replace the disk drive for another battery and have about 9 hours of run time with it.

> We're very sorry, but while we would love to let you in and rock out with us, we need to currently restrict turntable access to only the United States due to licensing constraints

It looks like they have killed everything but USA access.

No, tpc-uva is a bit more than what I need right now. I might use it later on when I have decided on any single algorithm that I want to test in a more realistic environment. Because changing the caching algorithm that postgres uses isn't as easy as doing so in a standalone python simulator, I will have to be sure that I want to do that. I have already tried that before and it is a lot harder and takes a lot more time.

That 3x speed up is about the same that I have seen with my code. I'm currently writing a database cache simulator to try different algorithms with it, and if I want to have anywhere near realistic results I have to use realistic access traces.

Tried it today with a tpc-c trace which has about 500 million accesses. The result: CPython would have run for about 90 minutes (I stopped it after 30 minutes, and began to look for a speedier possibility), PyPy only took 22 minutes.

It's a shame that nobody has a chassis like the sun fire 4500 anymore. The closest I could find is a 5U Thinkmate STX XA48-2510 which would work out at about 32 cents/GB.

Nginx 1.0.0 is out 15 years ago

I ran into a problem with a tcp connection to php-fpm just some weeks ago: The Server served requests at a rate of about 400req/s for some time without a single problem, until I ran out of useable ports. You won't have this problem with sockets.

The big difference is that on Swoopo you lose your investment if you don't win. But on 99designs you can still keep the design and get the feedback that it wasn't the right thing, and what was. Even if you don't get the job, you still get the practise and feedback. So I think there is at least some benefit.

I always like to say that you could beat somebody with a x41 over the head and then continue working. Sometime ago a friend of mine actually had to use his X41 to defend himself. He did just that and it worked just fine for some more months until it died from an unrelated problem.

HackerBuddy 15 years ago

It's directly on the about page:

What is HackerBuddy? HackerBuddy is a weekend project built using Ruby on Rails. It was built as a way to learn Rails, there is a very large chance that this site will collapse under the weight of it's own awkward code. If it does - sorry - I plan to improve it as I get better at coding in Ruby, please bear with me.