Fast high-level programming languages

https://lh3.github.io/2020/05/17/fast-high-level-programming-languages

by stanislavb • 6 years ago

124 171 6 years ago

lh3.github.io

maxk42• 6 years ago

Interesting and roughly in-line with what I've read elsewhere. One must note, however, that performance on a couple of benchmarks does not generalize to all problems. Still - perfect is the enemy of good, and it may be that when looking for a high-level language one might have to settle for "good enough."

Anyway - good job!

RcouF1uZ4gsC• 6 years ago

One option may be C++17. With the right libraries, C++ code can be very high-level.

Perhaps a well-written bio informatics library would be a nice solution.

api• 6 years ago

C++ can be a nice language if you are disciplined with it, but being disciplined means not using every feature and using features only when they make code clearer or higher quality.

throwaway894345• 6 years ago

Unfortunately, it's not only you who must be disciplined, but all of your colleagues/collaborators and to a lesser extent the authors of any libraries you happen to depend upon (or will depend upon). Which is a shame, because writing C++ can be a whole lot of fun, but the aforementioned invariants rarely hold.

MiroF• 6 years ago

I think C++ is a quite good language in its modern form that gets such a bad rap.

nerdponx5• 6 years ago

Where does one go to learn modern C++? Is there a "JavaScript: The Good Parts" for C++?

madhadron2• 6 years ago

I kind of wonder if modern FORTRAN would be the right option... Or pushing towards something like Oberon or a Pascal dialect.

neutronicus• 6 years ago

Biologists are text-mungers extraordinaire so I don't think Fortran will ever be right for them

kjs33• 6 years ago

Modern Fortran does a vastly better job on text than classical Fortran, but you have a very good point.

ngcc_hk2• 6 years ago

Pl/1 ?

madhadron2• 6 years ago

Treating sequences as strings of characters is a peculiarity, but it really isn't much like text. The operations you would expect to perform on text like uppercase, lowercase, tokenize at word boundaries, etc. are irrelevant. The operations that you perform on sequences like reverse, reverse complement, align, edit distance are separate. No reason why FORTRAN couldn't do great for that.

dnautics• 6 years ago

I was in the lab when Ham Smith wrote a sequence search program in FORTRAN about 7 years ago.

kjs33• 6 years ago

I actually got to the end and said "never heard of Fortran?".

Modern Fortran would be a good option; 'right' is a different matter. Certainly, Fortran 2003/2008/2018 are expressive languages (way more than F77), has an extensive ecosystem with good integration with C, has compilers that generate very fast code, handles parallel computations and SIMD and first-class language features, is already well established in the STEM world, etc., etc.

The biggest downside of Fortran is it's called 'Fortran' and so many people are unwilling to believe it's changed since the 70s.

nerdponx5• 6 years ago

Where does one go to learn modern Fortran? Is there a particular book or reference source that's recommended? What's the tooling like?

TheRealKing3• 6 years ago

It is the right option for anything numerical and array-based. Any super-cool array-based feature that people see in MATLAB and Python was inherited from Fortran 90. Yes, that is true. Sadly, modern Fortran is treated quite unfairly within the field of computer science and engineering because of people's lack of knowledge of modern Fortran, which reminds me of the famous quote: "People hate each other because they do not know each other, and they do not want to know each other, because they hate each other".

SeeTheTruth• 6 years ago

I wonder what the author would think of F Sharp.

blast• 6 years ago

Common Lisp is another obvious candidate. SBCL, for example, is known for performance.

Rochus• 6 years ago

I would rather vote for CLASP for which there are many applications in chemistry and molecular biology, see https://github.com/clasp-developers/clasp

mdgrech23• 6 years ago

what do you think of f sharp is the question?

ryan-allen• 6 years ago

Crystal is really exciting, it somehow 'fell under the radar' but when I heard 'fast, and ruby-like with types' I became very excited.

There's even a Rails-like [0] framework being developed!

Last weekend I spend a couple of hours getting Lucky up and running (it took some doing, I had to borrow a lot from people's Docker images to get it booted and working).

It's a big 'watch this space' situation. The macro system for metaprogramming is very easy to understand in Crystal [1] as well.

Can't wait for 1.0!

[0] https://luckyframework.org/ [1] https://crystal-lang.org/

etherio• 6 years ago

I'd recommend you check out https://amberframework.org too. I also like Crystal but Lucky kind of turned me off considering the steep learning curve coming from Rails whereas amber is much more similar.

carapace• 6 years ago

I wonder how Common Lisp compiled to native code would stack up?

dunefox3• 6 years ago

Probably really well.

zozbot234• 6 years ago

It's so sad that OP bothered to benchmark Nim and Crystal, but no Rust.

stolen_biscuit• 6 years ago

Is Rust really high level?

pasquinelli5• 6 years ago

to me a language is low level if variables are memory addresses and flow is controlled with jumps.

sfkdjf9j3j• 6 years ago

I think you could make that argument, although it might involve using Arc<T> for everything

amalcon• 6 years ago

Once upon a time, "high level" unambiguously meant higher than assembly language. A smallish fraction of the population still uses it that way.

eximius2• 6 years ago

I would qualify a language as high level if you're not expected to deal with pointers in 'normal' use. Yes, Rust has pointers, but using them is NOT normal.

C/C++/C#/D, etc are not high level by this criteria.

There are also of course degrees to all this. Rust is lower level than a lot of languages by virtue of having native pointer uses, even if it's frowned on.

There's also something to be said for language features, but I can't quite put my finger on it.

ginko• 6 years ago

Even C is a high level language.

Measter• 6 years ago

It can really depend how you write it. In some instances you can just throw things together and it can look a fair bit like you might write it in what many would consider high level languages.

On the other hand, there are times when you really need to get into the nitty gritty of things for performance reasons, and it ends up feeling quite low level.

For example, here's a Rust program which does the same as fqcnt_py1_4l.py: https://gist.github.com/Measter/d31abe88b5e318ba98856bf6f047...

It's longer than the Python version[0], I'm going out of my way to not allocate for every string. But, and maybe I'm biased here, it doesn't really feel like it's low-level code.

[0] https://github.com/lh3/biofast/blob/master/fqcnt/fqcnt_py1_4...

0xDEEPFAC• 6 years ago

I second you and raise you Ada and C++. C is the furthest thing from a high-level programming languages there is...

Comparing scripting languages to compiled languages is apples and oranges. The advantage of scripting is supposed to be the convenience - not speed.

VWWHFSfQ• 6 years ago

C is the furthest thing from a high-level programming languages there is

how times have changed

SAI_Peregrinus• 6 years ago

C is the furthest thing from a high-level programming languages there is...

C (especially C99 and up) is far, far higher level than something like FORTH.

hawski3• 6 years ago

OP had a specific high-levelness in mind, that Rust does not fall into. C is provided as performance baseline, which is a common benchmark. The languages are compared against Python - Nim and Crystal are closer to Python than Rust. For example GC is a requirement at language level implied by the author.

Rust is cool, but it does not have a horse in this race I'm afraid.

enricozb• 6 years ago

Would love to see an FP language on there, preferably OCaml

dunefox3• 6 years ago

F# might be interesting as well.

DonaldPShimoda• 6 years ago

It's perhaps worth pointing out that programming languages are not themselves fast or slow; it is the programming language implementations that are fast or slow. (This is a particularly sore point in the programming languages research community.)

ngcc_hk2• 6 years ago

I thought the pure python has issue as it assume sync on one global thread. Well if you mutatable global var ... or it is is not

TylerE3• 6 years ago

Nothing in the spec requires the GIL. Not having it would slow the vast majority of workload.

Jython does not have a GIL.

https://wiki.python.org/moin/GlobalInterpreterLock

TylerE3• 6 years ago

There are certainly languages that it is essentially impossible to implement to run fast.

Like, one can certainly write a slow C... but writing a fast Python (real Python, I mean, with full reflection, run-time introspection, etc..) would be...near impossible.

Yes, I know there are projects that make large subsets of Python run fast, but it's that last 5% that kills you.

nerdponx5• 6 years ago

FWIW PyPy is signfificantly faster than CPython in a lot of tasks.

hawski3• 6 years ago

It's usually not worth pointing out.

Most languages have a single implementation and definition of the language and especially its ecosystem is married to it. Some have more, but usually an ecosystem is popular for only one of them, because they are not entirely compatible.

Nowadays by a language people usually mean the whole package.

georgeg• 6 years ago

Would be nice to see your take on D-lang https://dlang.org/

exabyte• 6 years ago

Forgive my lack of specificity, as I am quite new to Julia, however, I believe that after the initial compile takes place, you do experience significantly faster runtimes, so you can "forgive" the one time delay.

I think you can also have VS Code just load up the packages you expect to use on a regular basis at startup.

zwaps3• 6 years ago

He did take out the start-up time.

but I agree. If you complain that compile takes 11 seconds when the program runs in 30, then I wonder if that's really the use-case where you need every last bit of performance.

Now, on a program that runs for two days straight in Python or Matlab, and Julia reduces that time by half, I can deal with a bit of compile time.

aduitsis• 6 years ago

Very interesting. If I understand correctly, this article is mainly focused on bioinformatics and performance of high level languages in that area.

I wonder why BioPerl (https://bioperl.org/, https://en.wikipedia.org/wiki/BioPerl) has not been included, it would have been interesting. Perl is considered surprisingly fast for certain classes of tasks (for an interpreted language of course, no point in comparing to C for example).

My info could be outdated of course.

DougWebb• 6 years ago

Perl isn't really an interpreted language. It's compiled to a sort of bytecode when you run the program, and executed on a sort of virtual machine. Similar to Java or C#, but much faster to compile. So fast that most people don't even know it's happening.

As for performance, it really helps if you write idiomatic Perl code. There may be more than one way to do it, but some ways are better. For example, an explicit for loop over a list has to be compiled to byte code that steps through the looping, but if you use map or grep the byte code calls a pre-written optimized function that loops as fast as code written in C would be. The more idiomatic your code is, the more optimized calls like that you get, and the faster your code will run.

nerdponx5• 6 years ago

Perl isn't really an interpreted language. It's compiled to a sort of bytecode when you run the program, and executed on a sort of virtual machine.

Same is true for both CPython and certain sections of R code.

cygx• 6 years ago

'Interpreted' is often used to mean 'not compiled to machine code'. Perl fits that bill. So do, say, CPython and Lua, in contrast to JIT-compiling runtime systems like PyPy or LuaJIT or AOT-compiled languages like C or Rust.

timClicks• 6 years ago

Perl used to be the standard for bioninformatics during the 90s iirc, but lost ground to R and Python over time.

laserson• 6 years ago

I always thought Go might be a good candidate for some of these tasks.

sk0g• 6 years ago

There's a go file or two in the repository, but no mention of it in the article.

I would make a pull request, but their setup uses 1TB of RAM...

outlace• 6 years ago

I think it would be much better if someone asked a group of experts for each language to come up with their best implementation for the tasks. If I just spent a few hours implementing a benchmark in a handful of languages I wasn't proficient with then I think the benchmark rankings would be nearly random or merely dependent on my priors.

zwaps3• 6 years ago

This seems to be a very specific benchmark. The second Julia benchmark is based on a truly obscure lib that has not been updated for two years?

jryb• 6 years ago

Should libraries like these need constant updates though? What change are they adapting to?

cbkeller• 6 years ago

Without having dug into the specifics of this particular package, I can say generally speaking that average package quality has in my view been improving a lot over the past few years; two years ago the language was pre-1.0 and people were still figuring a lot of things out. Some packages of course were excellent even then, but those tend to still be actively maintained.

zwaps3• 6 years ago

This does not seem to be a "real" library. It has a github, but no readme. It seems to be a julia implementation of some other c library, probably experimental.

jakobnissen4• 6 years ago

I reviewed the first Julia benchmark. The second one is based on this not-very-thorough "experimental" lib I've never heard of from a person not very well-versed in Julia, so I don't think that's representative at all.

For the uncompressed FASTQ benchmark, my relative timings differ from his, with Julia being ~30% faster. It's probably because he installed an old version of FASTX.jl, that would also explain his seemingly outdated comment about the FASTX.jl source code. Another 30% time can be shaven off by disabling bounds checks, which would put Julia as the fastest program on the list, around the speed of C. However, I don't think the speed increase is worth it, since FASTQ files are basically aæways compressed in real life.

For the compressed FASTQ file, it seems this is entirely explained by CodecZlib.jl being 2x slower than whatever his C solution is using. However, when profiling CodecZlib.jl, it just spends all its time calling `zlib` written in C (calling C has near-zero overhead in Julia). So I have no idea why his benchmark is slower there.

lostmsu3• 6 years ago

He complains about compilation time, and does not try neither C# nor Java?

The choice of languages for this benchmark is laughable.

dcftoapv• 6 years ago

I was confused by this as well. Java is my go to language when I'm trying to do things the right way.

Results in clear code, easy implementation, and good performance 95% of the time.

jmchuster• 6 years ago

I guess such languages, which are defaults in the realm of web development, don't even come up for consideration in the realm of biology?

grawprog2• 6 years ago

Trying to get biologists to care about computers is like telling someone to be excited that they need to get all their wisdom teeth pulled and have a cavity filled on the same day. I was pretty much the oddity throughout all my schooling and any of the biologists I met while working that actually knew much of anything about computers and programming beyond what needed to be done to write reports and enter data.

Most biologists I met were far more comfortable in the pissing rain, up the mountain, in the middle of nowhere collecting animal shit than in front of a keyboard.

If there were a Venn diagram of biologists and computer and technology enthusiasts the overlap would need a micrometer to be read.

Disclaimer: Please take this extremely generalized and likely offensive to one of those small few in that tiny overlap I mentioned, statement with a grain of salt. Please don't take this too seriously, it's just from my own narrow sampling of people i've interacted with which is may or may not be representative of the overall population.

bjonnh2• 6 years ago

Bioinformaticians use Python,R,Java,Julia,Groovy,Js and more (I use kotlin and python mostly but that's in a niche of bioinformatics I don't touch genes nor proteins)... Same as in software dev, there are a lot of beliefs. So you have the club of the js people that hate java, the club of pythoners that hate R, and people that just use what allow them to solve their problem and they are the ones that achieve the most but also that you hear about the less.

lostmsu3• 6 years ago

And that is the problem. I understand Julia and Python there. I'd understand Haskell (as every biologist is a mathematician too). But to list JS and Lua, and not have C# and Java?

AFAIK a lot of chemistry equipment actually is built with Java and C#.

pjmlp• 6 years ago

Sure they are, life sciences companies use tons of Java and .NET.

In fact you might encounter some biologists using VB.NET after they outgrown their Excel VBA macros.

strzibny2• 6 years ago

Not every article has to include $YOUR_FAVOURITE_LANGUAGE. Author probably chose what he knew or wanted to learn. To each its own.

lostmsu3• 6 years ago

IMO, that just speaks about the author as being prone to hype.

dang• 6 years ago

Can you please make your substantive points without being a jerk on HN?

https://news.ycombinator.com/newsguidelines.html

chadash4• 6 years ago

Both of these are good development languages, particularly for large projects. However, for bioinformatics or other science/academic projects, they have neither the wide range of libraries and ease of writing quick scripts that python has, nor the speed of something like C.

I think that for most academic uses, ease of development is really important. Remember, most of these people don't have CS backgrounds, which makes python a good choice because it's easy to learn and doesn't require much setup. It's main drawback is that it's slow, but this doesn't matter in many cases. When I was in grad school, I'd use a small dataset to try to figure out the best algorithm and then if I had to leave my computer running overnight to run on a larger dataset, it usually wasn't a big deal.

On the other hand, sometimes you just need something really fast. In that case, C is your darling.

Java/C# make lots of tradeoffs on both sides, which works well for software developers but not necessarily researchers (who don't have to worry about code being maintainable).

bjonnh2• 6 years ago

I wonder how the jvm (be it with kotlin, java,...) would behave on these algorithms.

kthielen2• 6 years ago

Shameless plug, I made hobbes and used it in high volume trading systems at Morgan Stanley:

https://github.com/Morgan-Stanley/hobbes

It’s kind of a structurally-typed variant of Haskell, integrates closely with C++, produces very fast code.

oddthink• 6 years ago

That looks super fun! Are they still using this? I was at MS a few years before that, still dealing with a few legacy things in A+. There were hopes that we'd switch to something like F#, but we ended up just re-doing most of the mortgage analytics in kdb/q, which was fun in its own way.

kthielen2• 6 years ago

Yep still used (pre and post trade, processing millions of orders and billions of market data events per day).

I'd like to grow it into a successor to kdb, though kdb is very entrenched in finance and the company is extremely litigious (they threatened to sue me).

F# is great too, and all of the other ML variants. We'll get there eventually, it's inevitable, but there's a lot of institutional inertia.

hopia• 6 years ago

What an odd bunch of languages to benchmark.

Why wasn't Go, Haskell, Java or OCaml even considered here? Seems much more appropriate than the likes of Javascript for scientific computations.

dunefox3• 6 years ago

The inclusion of javascript makes me want to dismiss this whole article. If that's a serious candidate then I know he didn't look very thoroughly. But no, go would be just as horrible.

pansa2• 6 years ago

If the goal here is maximum performance, that's going to require an AOT-compiled language, which implies static typing. Is that acceptable to scientists used to writing Python and R scripts?

jiofih• 6 years ago

Crystal has pretty decent type inference which helps lower the learning curve.

Types are child’s play for the crowd using Matlab and worrying about low-level performance like this though.

jakobnissen4• 6 years ago

Bioinformatician here. It's certainly a drawback. There is no quesiton that Python etc. are more expressive and quick to write in than C are. Some bioinformaticians swear to static languages for reasons other than speed (e.g. static type checking).

There is always a need for scripting languages. Not just for speed of development, but for interactive data manipulation and visualization. Static languages are a no-go in that regard.

My bet in on Julia. Although slower than C by itself, I think in practice, Julia code written by bioinformaticians will be faster than C code wrapped in Python, or C libraries used in inefficient workflows from the shell. That's certianly been my experience so far.

thu21112• 6 years ago

Why not take a look at Kotlin? High level, statically typed, fast iteration, good runtime performance, great support for parallelism (via Java's APIs for it).

ohyes• 6 years ago

Nah, that’s not true. Something like Common Lisp (AOT compiled, btw) you can write without types and then add them in later to bump performance by facilitating unboxing and removal of inline type checks. It is a tradeoff because you lose some safety, but it can lead to some pretty fast code.

gintery2• 6 years ago

This is sort of interesting but isn't informative without any idea as to why some languages are slower. What is the difference between the LLVM-IR/assembly produced by C and the one produced by Crystal/Nim/Julia?

filleduchaos3• 6 years ago

Compilers aren't different from pretty much every other kind of software, for which it's generally painfully obvious that two programs that target the same data _format_ will not necessarily output the exact same _data_, especially when they have wildly differing end goals.

I don't think every article discussing benchmarks has to restate that the differences between programming languages are not just syntactical to be informative.

gintery2• 6 years ago

I'm not sure I follow. What I meant is that a benchmark over a specific task is not really informative without a comparison of why it is slower in some languages compare to other.

wh-uws• 6 years ago

We've been using Crystal on an open source project I work on.

https://github.com/cncf/cnf-conformance/

Its been a dream.

You can pretty much write the ruby you are used to but it performs like a compiled language and you can make binaries.

Was SUPER skeptical at first when the team decided on it but its been a pleasant surprise.

lmilcin2• 6 years ago

I know many people don't consider Common Lisp as particularly fast language, but actually in my experience you can make it quite fast depending on what you need.

When using SBCL, for example, you get your application to be compiled to native instruction set. Moreover, you get control over optimization level for each piece of the code separately. Even more, you get complete control over the resulting native assembly. Something you don't get to do with other higher level languages.

One of the production applications I did in Common Lisp was parsing a stream of XDP messages (https://www.nyse.com/publicdocs/nyse/data/XDP_Common_Client_...) with a requirement for very low latency. I made the parser in Common Lisp so that it generates optimal binary code from XML specification of message types, fields, field types, etc. using a bunch of macros.

The goal of application was to proxy messages to the actual client of the stream. The proxy was there so that it was possible to introduce changes to the stream in real time without requiring to restart any components. Using REPL I was able to "deploy" any arbitrary transformation on the messages. The actual client of the messages was a black box application that we had no control over but we sometimes had problems with when it received something it did not like.

I liked Common Lisp in particular because it does not force you to make your performance decision upfront. You can develop your application using very high level constructs and then you get the option to focus on parts that are critical for your performance. Macros allow you to present DSL to your application but then have full control over code that is actually working beneath the DSL.

If everything fails, calling C code is a breeze in Common Lisp compared to other languages.

logicchains2• 6 years ago

I explored Common Lisp for high performance code once (I work in HFT), the biggest issue was that it didn't have native support for "arrays of structs" (where all the structs would be values stored "unboxed" adjacent in memory). I know it'd probably be possible to write a library to do that, but that'd be a huge amount of work compared to just using a language with existing support for unboxed arrays.

lmilcin2• 6 years ago

Well, I did some algorithmic trading (not really HFT but still monitoring the market and responding within 5 microseconds of incoming messages).

I would not use Common Lisp on the critical path because I would end up basically rewriting everything to ensure I have control over what is happening so that some kind of lazy logic does not suddenly interrupt the flow to do some lazy thing.

A large part of the application was basically about controlling the memory layout, cache usage, messaging between different cores, ensuring branch predictor is happy, etc which would be really awkward in Common Lisp (technically possible, practically you would have to redo almost everything). We have also experimented with Java with the end result being that the code looked like C but was much more awkward.

I have, however, successfully used Common Lisp to build the higher layer of the application that was orchestrating a bunch of compiled C code and also did things like optimizing and compiling decision trees to machine code or giving us REPL to interact with the application during trading session.

charlesdaniels4• 6 years ago

I would be curious to see what the performance is like in Go. I've been experimenting with it a bit lately, and coming from a mostly C and Python background, I have thus far found it easy to pick up. I haven't tested the performance for my own use cases very much yet, but I am told it compares very well.

I think the real challenge for scientific computing (I'm a graduate student, so this is most of the programming I do) is that there is already a huge network effect around NumPy + SciPy + matplotlib and friends. Golang just doesn't quite have the library ecosystem yet, although gonum[0] shows some potential.

In my limited experience so far, I think Go is good in an environment where most people are going to have experience with C and/or Python. It also makes it much harder to write truly crappy code, and it's much easier to get a Go codebase to build on n different people's workstations than C.

Having written a lot of Python, and relatively little Go, I think I would prefer to write scientific code in Go if the libraries are available for whatever I'm trying to do.

It's also much easier to integrate Go and C code, compared to integrating C and Python.

0 - https://www.gonum.org/

orbifold2• 6 years ago

Personally I think numerical code should still mostly be written in C++, right now it still has by far the widest choice of options for doing so. It is also relatively easy to interface with python. For example both xtensor, libtorch/ATen, arrayfire have straightforward python interoperability via pybind11.

Finally no other language except for maybe FORTRAN has seamless parallelisation support and first class low level numerical primitives developed by vendors. Sometimes you will get a massive performance increase by #pragma omp parallel for.

Even for visualization some python libraries will suddenly fall off a cliff (Altair) once you reach a moderately large number of datapoints.

charlesdaniels4• 6 years ago

I would definitely agree that it depends on what kind of scientific computing you are doing.

For big numerical stuff and things that need to run on supercomputers, C/C++/FORTRAN are definitely very relevant and I don't see that changing. Likewise for edge stuff that has to run on bare metal or embedded, I think we're still going to be using C/C++ for a long time to come.

"Scientific computing" is a huge range of different use cases with very different levels of numerical intensity and amounts of data. I doubt very much that there would ever be a one-size-fits-all approach.

However in the context of the OP, I'm arguing that Go would be preferable to Python for the purpose of writing bioinformatics models, and certainly more suitable than Lua or JavaScript.

Of course Python can sometimes be very performant if you leverage NumPy/SciPy, since those are ultimately bindings into the FORTRAN numeric computing libraries of yore. But if we're talking about writing the inner loop, and the choices are Go, Python, Lua, and JavaScript, I think Go is going to win that on the performance and interoperability fronts handily (I omit Crystal, as I am not familiar with it).

metreo• 6 years ago

Performance in Go? Far less than what you would get from Rust.

charlesdaniels4• 6 years ago

I would expect Rust to be faster. I never claimed Go would be faster than Rust, but that it would be faster than Python.

I think for rust, the barrier of entry is high in that it is a difficult language to learn. Admittedly, I don't know rust, but people I know who do have said as much.

I think Go strikes a good balance of being easy to learn and use, having a rich standard library, and also being performant.

daralthus• 6 years ago

Is go really high level? Imho the type system is so botched you cannot really say that.

One cannot even write a generic `reduce` function...

totalperspectiv• 6 years ago

Someone has a PR merged already for Go. The benchmark numbers need updating

ausbah• 6 years ago

has anyone used Kotlin for similar purposes?

jghn• 6 years ago

The author has a lot of familiarity with JVM based tooling for the operations he describes in the blog. I'm not aware of a Kotlin implementation but have seen him commenting on both Java and Scala implementations over the years. My assumption is the performance would be similar to those.

mindB3• 6 years ago

The computer language benchmarks game[1][2][3] may be of interest here. It benchmarks C, Python, Javascript, and Julia, in several tasks involving FASTA input (regex-redux, k-nucleotide, and reverse-complement), but implementations are bespoke rather than relying on libraries. Relative timing is much more favorable for Julia in these benchmarks. Looks worse for Python outside regex-redux.

[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[3] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

anonymoushn6• 6 years ago

I recently spent some time in the Lua section of the benchmarks game. It is a sad place for a few reasons:

- Lua programs cannot use shared memory concurrency or subprocesses with 2-way communication with the master process.

- Lua programs run on a very slow runtime compared to the fastest Lua runtime.

My impression after this is that for languages that aren't super fast and don't include all the primitives one could want, benchmarks like reverse-compliment are mainly measuring whether the language's standard library includes some C function that does the bulk of the work.

mindB3• 6 years ago

I would love it if Isaac included LuaJit and pypy in the benchmarks game, but ultimately I get it; it's just one guy's project, and he doesn't want to spend the time to maintain it across the entire incredible diversity of programming languages/implementations[1].

To a great extent any "language" benchmark (for languages that don't compile to efficient machine code) is certainly a benchmark of the language's standard library. I'm not sure there's a way around that reality. Are there external Lua libraries that allow shared-memory concurrency? If so, it's probably worth opening an issue asking whether those libraries could be allowed[2]; it might just be that nobody has submitted a program making use of Lua shared concurrency.

[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[2] https://salsa.debian.org/benchmarksgame-team/benchmarksgame/...

igouy8• 6 years ago

… shared memory concurrency or subprocesses with 2-way communication with the master proc

Isn't that the same situation as Python, Perl, PHP, Ruby… except for those languages, programmers have converted the programs to use multicore ?

burntsushi3• 6 years ago

Looks worse for Python outside regex-redux

Take a look at the source code for the top Python program for regex-redux.

mindB3• 6 years ago

I'm aware. To a large extent both regex-redux and pidigits are measures of the overhead of FFI for all non-C/C++/Rust languages. Rust is very cool in that one for actually using a regex engine implemented in Rust; definitely has my admiration.

anonymoushn6• 6 years ago

The Luajit implementations should perhaps use FFI C structs, especially if the main issue for one of them is the lack of arrays of structs (other than FFI C structs)

sho2• 6 years ago

Hard to install crystal? It's

  brew install crystal

filleduchaos3• 6 years ago

This might come as something of a shock, but not everybody uses or wants to use Homebrew.

sho2• 6 years ago

It doesn't come as a shock but more of a sigh - there's always one. 90% of devs I know use macs. Of them, basically 100% use homebrew. So for 90% of developers, installing crystal is as easy as I said. So the criticism that it's hard to install is, for the most part, invalid.

Of course the build-everything-from-scratch gentoo linux crowd is going to have a harder time but isn't that part of the masochistic appeal?

chadash4• 6 years ago

"I don’t see Julia a good replacement of Python. Julia has a long startup time. When you use a large package like Bio.jl, Julia may take 30 seconds to compile the code, longer than the actual running time of your scripts."

I think this is the crux of it. Python may not be the fastest language out there, but it's definitely one of the fastest languages to develop with, which is why it's such a hit with the academic research crowd.

Static typing and compilation are great features for many use cases, but if you are just prototyping something (and most academics are never going to write "production" code), it's nice to be able to try something, test it and then iterate and try again. When you have to compile, that iteration loop takes longer.

A few people here mentioned GoLang as a good alternative. Personally, I love the language, but it's really not very good for quick prototyping. As an example, the code won't compile with unused variables. This is great for production code but very impractical when you are testing some new algorithm.

idoubtit• 6 years ago

A "prototype" software is sometimes a nice way to name some junk code. When I read that most academics are never going to write production code, they're prototyping, they need something to develop fast... it looks like most academics value fast writing more than correctness, which is not my experience.

As an anecdote, a friend of mine is an applied mathematician specializing in PDE and fluid dynamics. He used to code with C++, because that was what was used in the labs where he worked. Then he discovered Python and he thought it was so much easier, and still quite fast thanks to optimized libraries (written in C++ with a Python interface). But after a few months his enthusiasm had disappeared because of the runtime errors of his Python code. He didn't want to go back to full C++, though, and he still codes mostly with Python.

chadash4• 6 years ago

"He didn't want to go back to full C++, though, and he still codes mostly with Python."

Exactly. It seems like he understands the tradeoffs and prefers quick iteration over "correctness".

In my experience, for computational "academic" code, the hard part is often coming up with the correct algorithm, not implementing that algorithm correctly. In software engineering, it's often the opposite.

teraku• 6 years ago

You said it's not your experience, but then you give an anecdote which says where because fast writing was prioritized, he ended up with runtime errors...

Tarq0n• 6 years ago

Maybe the two-language problem is not a problem after all. I've been coming around to the idea of writing the core algorithms in a fast language with a lot of features around correctness, and the rest in whatever language is most productive. Learning another language isn't so bad as long as it's not an utter beast like C++.

chadash4• 6 years ago

Ideal if you can swing it. One of my professors used to prototype in Python for development speed and then re-implement in C++ for runtime speed. You get the best of both worlds in a sense.

jakobnissen4• 6 years ago

I don't think this is entirely accurate. When you develop software, you don't repeatedly unload and load your dependencies (which is what causes Julia's startup time). You probably only develop one package, which you then reload one in a while. That causes very little lag in Julia. Similarly when working interactively, there you only suffer from startup time when you launch your session and use your heavy dependencies the first time.

Also, for what it's worth, I recreated his benchmarks in Julia and found a startup time of 5-6 seconds on my laptop. Not nothing, but compared to the time any FASTQ-processing operation usually takes, it's insignificant.

Julia' compile time latency does prevent you from e.g. calling a short Julia script in a loop, or from using Julia scripts that both have large dependencies and do very short-timed tasks. But I find that to be not very common in my workflow - especially since I usually don't use Julia as a small glue layer between C programs the way Python is often used.

vmchale• 6 years ago

A good high-level high-performance programming language would be a blessing to the field of bioinformatics. It could extend the reach of biologists, shorten the development time for experienced programmers and save the running time of numerous python scripts by many folds.

The author might like to look at J for specific calculations (and Futhark for similar tasks).

I use Haskell, it's not quite on par with C though. Other MLs are wonderful too.

ibiza• 6 years ago

Note, Heng Li[0] is a significant figure in Bioinformatics software. Most notably, he is the author of BWA[1] (Burrows-Wheeler Alignment). BWA performs a large percent of all sequence alignments worldwide.

[0] http://www.liheng.org

[1] https://github.com/lh3/bwa

hpcjoe• 6 years ago

Looking through the julia code, I saw quite a bit of memory management, which did surprise me. Generally with Julia, you want to allocate memory, once, to avoid penalties of reallocating memory and gc.

I pulled the code down, placed the data on a ramdisk so as to not impact benchmark measurements, with physical issues.

I built the C code, and ran the two julia codes. My timing looked like this:

version t(raw) t(gz)

c1 1.47s 8.31s

jl1 3.80s 15.82s

jl2x 5.85s 17.86s

py1 fails

py2 6.92s 29.62s

I don't have lua, nim, or crystal on my machine. This is Julia 1.4.1 BTW. Running Linux Mint 19.3 on my laptop, 5.3.0-51-generic kernel.

Beyond putting the data and compressing it with pigz for the compressed version, no optimizations were done. Putting the data on ramdisk optimizes all codes.

My thoughts: The author noted that Julia has long startup/run times. This is true, for the first compilation of modules you use. As a reflex these days I (and I am guessing most Julia users) do a "using $MODULE" after adding it. This makes the startup times less painful, for most modules used. Plotting, with the Plots module, is still a problem, though it has gotten dramatically better over time.

Basically, if you run your code more than once, with the modules being compiled into your cache, the nice part is that startup time is significantly better. Such that Python reverts to its lower performance than Julia. If the startup time on first run is important (think of it like a PyPy compilation step along with a run of the code), and you'll only ever run a code once, and for the less than 1/2 minute that it will take for this example, use whatever it is you are comfortable with.

FWIW, the author noted, with implied disdain, that Julia users are telling them that they are "holding the phone wrong." Looking over the code, specifically all the memory allocation bits, I could see that. Basically, I'm not sure how much, if any of that, is actually needed.

That said, very limited critique of the tests. I like to see "real world" examples of use. Kudos to the author for sharing!

[edited to "fix" table ... not sure how to do real tables here]

TeMPOraL• 6 years ago

Nim supporters advised me to run a profiler. I am not sure biologists would enjoy that.

We need better profilers! Ones that don't require anything more complicated to use than passing a --profile-me parameter to the interpreter/compiler, and whose output can just be dragged into a pretty, user-friendly and fast application (included with the language runtime), and where reported results are both trustworthy and correspond to actual locations in the source code.

leephillips• 6 years ago

The author got good results from the language that he says he’s proficient in, and bad results from those he says he’s a beginner with. (Read some comments here below explaining his Julia mistakes.) He ran each benchmark one time only—not a very good methodology.

hawski3• 6 years ago

Yeah, because his benchmark is also his ability to quickly write efficient code.

I am equally new to Julia, Nim and Crystal.

He might be more familiar with a certain style or paradigm, but he wants to switch from Python to something significantly faster jumping through as little hoops as possible.

einpoklum• 6 years ago

A post comparing programming languages which makes a reference to "C/C++" is already problematic IMHO.

TheRealKing3• 6 years ago

There has been one around for the past 7 decades and still evolving: Fortran

Fast high-level programming languages

Related Stories