Ask HN: How do you benchmark your programming languages?

https://news.ycombinator.com/item?id=38512288

by xqb64 • 3 years ago

3 5 3 years ago

news.ycombinator.com

During the development of my tiny (less than 4k LoC) virtual machine, I continuously measured how long it took to execute recursive fibonacci function (fib(40)) as a way to estimate how performant it is.

What other benchmarks do you use for your programming languages?

npalli• 3 years ago

If you are using fibonacci to test out performance in the small, you might want to stick with the Computer Language Benchmark Game. It gives you the universe of languages and sample programs to you can see what works best with your VM.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

xqb642• 3 years ago

Thanks, this looks like it could be useful.

DemocracyFTW22• 3 years ago

Using a single function, and then recursive Fibonacci of all, is certainly not enough to give estimates about The Language. Everyone should know that although recursive Fibonacci is like that one poster function that comes right after Hello World it is not a recommended implementation strategy for that functionality in most languages (but of course if you can make it run as fast as the linearized version that would be a good thing).

I haven't implemented any language / VM of my own, but when I write a library then even for very specialized purposes I often write several tests when I want to be on the safe side as far as performance goes (correctness being the more important concern most of the time). Also, I typically want to have tests involving other libraries (or other VMs in your case) to get an idea of how fast I am in comparison, otherwise I'd be stuck with oh wow 1000 times doing something takes just under a moment kind of wisdom.

xqb642• 3 years ago

Yeah, I'm aware that recursive way is not the fastest way to compute fibonacci, but it still serves as a good way to simulate CPU bound work. When it comes to the comparison to other, in this case, virtual machines, I've compared my language to Python (since they're conceptually similar), and I'm beating Python in this particular benchmark (18s vs 28s).

DemocracyFTW22• 3 years ago

OK I'm not saying that recursive Fibonacci is a bad way to test your language, but

* it's not a very good one either, because a sensible implementation won't be recursive in many current languages

* even if it's faster it's just 1 case so it doesn't speak for the language as a a hole set of possibilities; it's a bit as though the Python people were all like, hey look, our hello world runs 1.6 times faster than when using language Y!! It's just not convincing.

* your goal may be to beat other languages for this particular use case, then that's fine, most people won't be interested in your language if that's the case, but of course you do you

Ask HN: How do you benchmark your programming languages?

Related Stories