Golang vs. C# (.NET 5.0) at Benchmarks Game

https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go-csharpcore.html

by iamdual • 5 years ago

101 110 5 years ago

benchmarksgame-team.pages.debian.net

The focus on performance since Core is really nice to see, dotnet 6 is continuing the trend as well: https://devblogs.microsoft.com/dotnet/performance-improvemen...

Thaxll• 5 years ago

When you see how much effort it takes to C# and Java to optimize the runtime, there are a lot of people working on that. C# is fast but you see that it uses between 2 and 32 times the memory that Go needs.

Overall you can see how fast Go is, it has little optimization compare to C# and it's as fast. Compare this: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... and overly complicated C# version: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... ( avx, Intrinsics etc ... )

JamesSwift3• 5 years ago

I definitely have run into this, even when using 'server mode' in asp.net core. I never was able to figure out why the C# version of my POC was using so much memory, but rewriting to golang ended up using a very predictable, minimal amount of memory in comparison.

azth• 5 years ago

My guess is that golang's GC is optimized for latency at the expense of throughput, limiting the max memory size.

Mattish• 5 years ago

Server mode is much less likely to incur GC. Were you causing enough memory usage to force your app to actually free memory?

It will intentionally use more memory for the sake of throughput, hence why this post has all .NET program flag for it, as it's a _speed_ benchmark primarily.

agumonkey2• 5 years ago

and afaik, .net has record types unlike the jvm (yet) which means java is even worse

kaba05• 5 years ago

I think you mean structs (value types. Will be called primitive types in Java). Records are not too interesting from a performance pov (and java has them, and I think they actually predate c#’s), though java will likely be able to optimize serialization/deserialization of records better.

ternaryoperator• 5 years ago

Java recently added record types.

igouy21• 5 years ago

Compare this and C# version:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

joelfolksy• 5 years ago

"( avx, Intrinsics, etc ... )"

I have to give you credit for trying to apply the Rule of Three to a single criticism.

Of course, I don't really understand how the fact that someone took the time to vectorize the C# submission is supposed to be a mark against C#...

melling• 5 years ago

I was always under the impression that Go never had a great optimizing compiler. It was never a primary focus given the limited developer resources.

I couldn’t find a direct C# to Rust comparison but Rust trying to compete with C++ means performance is a goal, if that’s what you are after.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

throwaway89434517• 5 years ago

I don't think it was "limited developer resources" so much as a desire to preserve blazing-fast compile times. The very rough rule-of-thumb that I've heard is that optimizations must pay for themselves (a compiler which is itself compiled with the given optimization must not be slower than the previous version).

igouy21• 5 years ago

"He used the compiler's self-compilation speed as a measure of the compiler's quality. Considering that Wirth's compilers were written in the languages they compiled, and that compilers are substantial and non-trivial pieces of software in their own right, this introduced a highly practical benchmarks that directly contested a compiler's complexity against its performance."

p44 "Oberon — The Overlooked Jewel" Michael Franz, in "The School of Niklaus Wirth".

https://www.google.com/books/edition/The_School_of_Niklaus_W...

the_duke• 5 years ago

There are both GCC [1] and LLVM [2] backends for Go, but I don't think they see much usage compared to the default.

[1] https://golang.org/doc/install/gccgo

[2] https://go.googlesource.com/gollvm/

gwp• 5 years ago

I was always under the impression that Go never had a great optimizing compiler. It was never a primary focus given the limited developer resources.

It's an intentional choice, that's why it compiles code so fast. Also because of that it's a lot simpler than say GCC. Before Go, the Plan 9 C compiler was designed in a similar manner too (I think the Go compiler was forked from it).

I think the simplicity aspect is even more important than the compiling speed. It's easier and cleaner to keep the compiler simple and write optimized assembly code by hand when it's needed. That way, the compiler doesn't get so messy (fewer bugs, easier to maintain...) and the written program is of better quality (humans can produce better code than compilers).

kaba05• 5 years ago

How often have you met compiler bugs? Also, humans absolutely can’t produce better code than compilers, at least not reliably. All those minor improvements that compilers routinely do does add up. And noone wants to write arcane hacks that make code unmaintainable and may not even provide a performance benefit (CPUs are finicky beasts, sometimes the seemingly worth option is faster)

I very much want my computer to work as hard as it can to make my code more performant for free. What I would like to see more is a separate debug and release build mode, where the former can go as fast as it can without optimizations, while the latter can be as slow as it wants and result in the most optimal binary it can produce. Zig does that for example.

flyinglizard• 5 years ago

I say modern .NET is a marvel of features, development tools, interoperability, performance and even ships with its own cloud environment (Azure).

Unfortunately developers who don’t know better judge it by it’s historical association with Windows rather than how powerful it is today.

umvi• 5 years ago

Can you now get the full .NET development experience in a Linux-only environment? I haven't used it in a while, but from using Unity (game engine) it seemed like C# on Linux was a bit crippled

moonchrome3• 5 years ago

Yes, use Rider and .net core - everything works on non-windows platforms (working on .net from osx right now, deploying to linux)

Avoid vscode for C# development, unlike TS/JS (which is top of the line), the support for C# even in core is toy level.

cyral• 5 years ago

Yes, JetBrains rider is IMO superior to VisualStudio these days, and I run a huge amount of C# code in production on Linux. It was definitely crippled in the past but it's a first class citizen in the .NET ecosystem now.

na852• 5 years ago

Especially with lsp-mode, C# on Linux (or Mac) is a great experience.

https://0x85.org/csharp-emacs.html

adwww2• 5 years ago

Also relevant... can you learn it, build it and deploy it for free?

I've always thought the whole ecosystem looked really productive, and the code I've had to review occasionally looked well structured and readable. But when I was starting out MSDN cost a fortune and I've never really considered trying to learn it.

jakearmitage• 5 years ago

I use Sublime and it works well.

throwaway89434517• 5 years ago

even ships with its own cloud environment (Azure)

Can you elaborate on this? What does it mean that ".Net ships with Azure"?

lambda_dn2• 5 years ago

Microsoft created .NET and Azure and a lot of the tooling so the integration is tight, now that it's multi platform and with Rider (multiplatform .NET IDE) being as good as if not better than Visual Studio there is nothing stopping it.

bob10292• 5 years ago

Unfortunately developers who don’t know better judge it by it’s historical association with Windows rather than how powerful it is today.

Some of us actually got to experience the entire journey from the old to new world first-hand. We started out as a .NET 3.5 Framework solution (windows only), and are now looking at a .NET 6 upgrade (any platform). Over the course of 7+ years, we went through all of the following frameworks:

3.5 => 4.0 => 4.5 => 4.6.2 => [netcore convert]

2.0 => 2.2 => 3.0 => 3.1 => 5.0 => ...

Some of the transitions were a little painful, but the same fundamental product survived the entire trip.

I don't know of many other development ecosystems where you can get away with something like this. If we didn't have the stability this ecosystem has to offer, we would not be in business today.

alkonaut3• 5 years ago

I’m on a project I created in .NET 1 and then migrated to 2.0 (to generics) early on. It’s moved through VSS via svn to git. It has gone from VS 2002 on CD’s to VS2019/2022. It has over 100k commits. Still work full time on it today. I share your experience with the migrations. Never an issue. The new sdk (“core”) style project system was probably the largest blessing of this whole journey.

kaba05• 5 years ago

Well, without starting a flamewar (.NET is an impressive runtime), the JVM is better at backwards compatibility, I think.

EMM_386• 5 years ago

I actually created a .Net Enterprise app that went on to become an industry leading application.

At the time I was hired, it was to modernize an Access application used internally, to try to sell it as a product.

.Net was still in beta at the time (this was early in 2001). I figured I might as well go with the flow and try it out.

It was a crazy ride, and I've since left the company ... but we went through every version from beta through 4.8 before I left.

I'm now using .Net 5 in Azure to power my new company's REST APIs.

caeril• 5 years ago

Not really. And the allure of Go over .NET has very little to do with performance.

Most applications I develop these days are web services. I write most things today in Go, but I used to work quite a bit with C#/Asp.NET applications.

Here's what I do to build a Go web application:

  - go build .

What I get out of it is a single, statically-linked, self-contained ELF binary for which deployment is as simple as scp, if I want to. It will run on any x64 linux box, without dependencies, since it contains its own webserver. I don't need to dump it into IIS to make sure the build works.

Here's what I used to do with .NET:

  - Open VS, wait about 30 seconds for it to finally start working
  - Set the target, Rebuild All
  - Publish to file, wait
  - Eventually get a packaging error, predicated on some obscure tools dependency issue somewhere inside my 98KB .csproj file, which I have to fix by closing down VS, manually editing in Notepad, and re-opening VS
  - Finally get a working build+publish, ok, let's take a look
  - Oh, very nice, the publish directory weighs in at nearly a QUARTER GIGABYTE, and contains about 60 dll dependencies and a ton of entirely useless descriptor files.
  - Well, okay, let's at least get this deployed to the test server to make sure it still plays nice with IIS, xcopy this over.
  - Oh, IIS doesn't like this at all, now let's spend the next couple hours tracking down this insane .net framework dependency hell.
  - Screw it, where's my whiskey?

sbelskie• 5 years ago

dotnet publish -r linux-x64

Seems pretty simple. Though the binary size is definitely not on par with go.

snotrockets• 5 years ago

C# was always the better Java, F# the better Scala.

But for many years, the per seat cost was higher, which scared many away

kaba05• 5 years ago

In what way do you consider F# better than scala? Especially with the recently released, revamped Scala 3?

popotamonga• 5 years ago

Is there one similar for c# vs scala?

igouy21• 5 years ago

Many years ago — but from version-to-version too many of the Scala programs suffered bad bitrot, failed and were not updated.

kaba05• 5 years ago

Scala code is backwards compatible, only the class files are not - so at most dependencies could have become stale. With a recompile, the programs should work just fine.

throwaway89434517• 5 years ago

I can't imagine a better setup for a language flame war :). I really like debating languages, so I hope it doesn't go that direction.

One of the standard caveats with this particular benchmark game with respect to Go is idiomatic optimizations are prohibited. To use the btree example, Go's memory management is low latency and non-moving, so allocations are expensive--any Go programmer writing a performance-sensitive btree implementation would pre-allocate the nodes in a single allocation--an absolutely idiomatic and trivial optimization--but the benchmark game requires that the nodes are allocated one at a time. In other words, the C# version is idiomatic, but the Go version is expressly contrived to be slower--not a very useful comparison.

Mad respect for .Net though; it's really impressive, I like the direction it's going, I'm glad it exists, etc.

abledon• 5 years ago

reminds me of the 'im tired of being a hipster' post on frontpage recently.... "go learn 'unhip' tech/languages and live your life"

na852• 5 years ago

This one?

https://news.ycombinator.com/item?id=28274485

igouy21• 5 years ago

absolutely idiomatic and trivial optimization

Which is not accepted for the C# programs either.

sync.Pool is accepted —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

the Go version is expressly contrived to be slower

The requirements were contrived in April 2008.

afaict Go initial release was March 2012.

throwaway89434517• 5 years ago

Which is not accepted for the C# programs either.

Because C# doesn't benefit from this kind of optimization. Its GC is generational, which means that it has very fast allocations at the expense of high latency. In most applications, lower latency is more important than slower allocations (not least of all because these batch-allocating optimizations are nearly trivial), but these benchmarks don't reflect that at all.

The requirements were contrived in April 2008. afaict Go initial release was March 2012.

Contrived = "the rules artificially prohibit idiomatic optimizations". It doesn't require that the maintainers have a prejudice against Go (although as you point out, the maintainers have had a decade to revisit their rules).

lalaithion2• 5 years ago

The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned. It's a valid argument that this is a rare real-world requirement, but it's not contrived to be slower.

throwaway89434517• 5 years ago

The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned.

Forcing allocations for every node isn't justified by a desire to demonstrate dynamically sized binary trees. A naive dynamically-sized tree would just keep a list of node buffers and allocate a new node buffer every time the previous one fills up (perhaps with subsequent buffers doubling in size). The benchmark is, by all appearances, contrived to be slower.

_ph_• 5 years ago

That is one reason I don't consider the language benchmark to be really relevant: a lot of benchmarks are taited by the exact rules of the competition.

throwaway89434517• 5 years ago

And these rules are particularly bizarre. Rust, C, and C++ are all allowed to use custom allocators while Java and C# have GCs which are optimized for this particular micro benchmark but not for real world applications (although I hear Java’s GCs are making good headway on latency lately, and clearly all of the GCs are suitable for general application development). So it’s really just Go which is forbidden from an idiomatic optimization as far as I can tell.

scanr• 5 years ago

I enjoy using both languages. The significant performance difference between the two for me is compilation speed. Size of binaries produced is also important if you’re shipping them around.

nedsma• 5 years ago

It's 2021 and .NET developers still argue which coding style they should adopt and how they should enforce it. It's totally mind numbing that team members are split between implicit and explicit variable naming. Contrary, in Go you just write code because those nuances should not matter.

codenesium• 5 years ago

That nuance does matter which is why we are still having the discussion. Teams can pick which style they want but mixing is unnecessary overhead and when you're talking about 10s or 100s of solutions I'd like the style to match across the board.

Salgat2• 5 years ago

That's more of a failure of management. Companies like Google outline style guides for company-wide usage. It might not be "the best" but it's consistent and the company enforces that.

booleandilemma• 5 years ago

I'm not really concerned with such small differences in speed, to be honest. The thing that I look for is: how productive am I when using language X?

Koshkin• 5 years ago

... and nothing can compare with C# (under Visual Studio) in this regard, it seems. Not the most efficient at run time, but a good, knowledgeable developer's productivity is insane.

Salgat2• 5 years ago

Especially with the .NET Standard Library. It has damn near everything already out of the box with 1st class support, and if not there's almost certainly a nuget for it. C# is an extremely productive language to develop for.

keewee7• 5 years ago

I prefer C#. But here the Go code actually looks like normal production code while the C# examples look like something made by a low-level optimization wizard.

alkonaut3• 5 years ago

I looked at the k-nucleotide, n-body, etc and saw nothing out of the ordinary in terms of C#.

There wasn’t even a lot of “modern C#” low level optimization like ref-structs/spans and similar. It actually looks like there is quite a bit of performance from C#8, 9, 10 left on the table.

coder543• 5 years ago

Are we looking at the same file?[0]

Tons of very "interesting" attributes like this:

    // prevent inlining into main to decrease JIT time to generate main
    [SkipLocalsInit][MethodImpl(NoInlining)]

    [SkipLocalsInit][StructLayout(LayoutKind.Explicit, Pack = 32)]

    [SkipLocalsInit][MethodImpl(AggressiveOptimization | NoInlining)]

    [FieldOffset(32)]

and tons of "unchecked" blocks.

Not to mention that the entire file is using explicit vectorization, which I consider to be a very high degree of optimization -- tons of software never bothers to implement explicit vectorization, and does just fine.

If all of this is "nothing out of the ordinary", then "ordinary" C# has changed a lot since I last spent much time with it.

[0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

notdang• 5 years ago

For C# they went too far, it has this comment in one of the files: > Using GOTO which currently causes dotnet to generate better machine code

matttproud4• 5 years ago

If what's in the Benchmarks Game is normal Go code to you, then I feel pretty bad for you in terms of what your colleagues are exposing you to on a day-to-day basis quality-wise.

fulafel• 5 years ago

Please fix the title (the original title spells Go correctly too).

matttproud4• 5 years ago

I've never understood why folks treat the Benchmarks Game results as indicative nor representative of anything useful. The code specimens they use are often unpolished nor idiomatic, without even commenting on whether they could be made to perform better through Byzantine, careful by-hand optimization.

Why does their web site have no contact nor link to where the source code for the project can be checked out, contributed to, or amended?

igouy21• 5 years ago

I've never understood why…

Perhaps they don't read the website text?

… no contact nor link…

Search works.

matttproud4• 5 years ago

I ran multiple search queries. I wouldn't be so dumb to post a comment like this here without having done my homework. The best I found after trying numerous keyword permutations was https://salsa.debian.org/benchmarksgame-team/benchmarksgame, but this did not appear to contain all of the benchmarks' source, just the source embedded in HTML, which is specious at best. This repository looks mostly like frontend HTML and chrome, not a SUT, executor, nor even the sub-test code.

At the very least, I couldn't realistically re-run some of the example benchmarks from the source embedded in the HTML, because they did not include vendoring/version information for external packages they depend on. That made me doubt the provenance of https://salsa.debian.org/benchmarksgame-team/benchmarksgame.

Golang vs. C# (.NET 5.0) at Benchmarks Game

Related Stories