Guide to the CoreCLR Source Code

http://mattwarren.org/2017/03/23/Hitchhikers-Guide-to-the-CoreCLR-Source-Code/

by adamnemecek • 9 years ago

229 45 9 years ago

mattwarren.org

cm21872• 9 years ago

gc.cpp, 37,000 lines of code. And some want to remove Regions from the .net framework!

matthewwarren9• 9 years ago

Yeah people always find that file surprising.

I think the reason for it being soo large is that the GC is shared across all the .NET versions, so having it in one file has some advantages.

See https://github.com/dotnet/coreclr/issues/408 for a discussion on reorganizing it.

LeanderK• 9 years ago

sadly, no. Excerpt from the issue

In #401 @cnblogs-dudu referenced a post explaining that the GC was machine-generated from LISP. Since gc.cpp is not a real source code (just intermediate code), but the lisp code is, should it be more useful to publish the lisp source and the transpiler (Source-to-source compiler) for LISP -> CPP?

UnoriginalGuy2• 9 years ago

Regions aren't a .Net Framework feature, they're a language feature.

There's absolutely no reason they should be in C# and encourage bad coding practices (like overly long .cs files). Even if you need all your code in a single class, you can shard that class across several .cs files to make organisation and maintainability easier.

Regions are just a crutch people use to allow them to create 10K+ line .cs files that will eventually become a maintainability nightmare (as well as making merges/check-ins/finding things more annoying in general).

Only region I'll use is the built DEBUG one and I only use that to create:

     private bool IsDebug()
     {
     #if DEBUG
       return true;
     #else
       return false;
     #endif
     }

Stackoverflow answer for why regions are bad: http://softwareengineering.stackexchange.com/questions/53086...

taco_emoji• 9 years ago

I agree about regions, but `#if` et al are preprocessor directives, not regions. They actually influence how your code is compiled.

lazyman75• 9 years ago

Well, just because YOU cannot write good code with regions doesn't mean it's a bad feature. It's actually very very useful thing, but as with everything, you have to know when and how to use it.

tombert• 9 years ago

I find these runtime VMs fascinating to ogle at after looking at how Erlang's GC worked. I think you have provided me reading material for this weekend.

matthewwarren9• 9 years ago

Glad to be of service!!

BTW if it's reading material you want, you should also check out me other post 'Research papers in the .NET source' http://mattwarren.org/2016/12/12/Research-papers-in-the-.NET...

NeverTrump• 9 years ago

I want to note something that my interest some:

The second biggest fear which made Sun sue Microsoft was that when MS licensed Java, their VM was faster than Sun's. Microsoft is (was?) the king of JIT. Technically biggest since if theirs was faster, they wouldn't have cared for Microsoft's anyway. Even now CLR is generally considered much faster than JRE.

I don't know how Chakra compares against v8 but then it also depends on what makes money for MS.

chrisseaton3• 9 years ago

Microsoft is (was?) the king of JIT.

Hmmm really? I'm not sure I can think of many technological breakthroughs in the area of JITs that Microsoft have been responsible for? Google and Mozilla have made lots of advances, as have Sun and Oracle, and lots of academics like the PyPy people, HP with Dynamo.

What have Microsoft done in the area of JITs that so notable as to earn them the title 'king' of JITs? Can you say anything specific?

cm21872• 9 years ago

Mostly I never understood the point of JIT in the .net framework. You pay a price in term of startup time, but the reality is that you can only run .net on x86 and x64. So we might as well have saved the trouble. And the first time .net was given an opportunity to run on ARM (WinRT), they disallowed regular .net applications in order to introduce a toll through the windows store.

We would have been better off without JIT. Even now with ARM. You compile to 3 architectures,... so what?

kuschku• 9 years ago

Even now CLR is generally considered much faster than JRE.

That’s... not really true. JRE’s HotSpot is much faster than CLR, even today.

I’d be interested to see benchmarks proving your argument, because basically every benchmark out there proves you wrong.

Sun’s HotSpot does recompilation and profiling, to reoptimize depending on use case, while CLR only JIT’s once, so the base performance is very low. This is also why HotSpot can reach better-than-C performance in some cases (because it can optimize at runtime for actually taken code paths, and jump to interpreted code for rarely taken paths), while .NET (and native code) can not. (This is what gives HotSpot the name – it recognizes hot spots, and swaps them out for more and more optimized versions at runtime)

chrisseaton3• 9 years ago

I tried to find any studies because the claim surprised me, but I didn't find any, so I wasn't going to try to refute that without any data.

But I can say that the number of research papers around the JIT in the JVM is probably and order of magnitude more than around the CLR. Someone else linked to 'Research papers in the .NET source', and guess what? Several of them are really JVM papers, talking about how they've used the same technique.

pjmlp• 9 years ago

The CLR has some APIs to help with that, but they aren't at the same level as what JVM JIT compilers are capable of.

Additionally there is PGO support between runs and explicit SIMD support.

IBM J9 also does support caching JIT code thought.

JoachimSchipper• 9 years ago

Yes, that's an odd claim. You can construct a synthetic benchmark where C#'s support for value types lets .Net beat a pointer-chasing JVM, and I'm sure there are other corners where .Net wins; but that's not the general rule.

abiox2• 9 years ago

Even now CLR is generally considered much faster than JRE

really? since when? can't say i've ever heard this claim before.

ledgerdev• 9 years ago

Got to agree with you.

The only thing the CoreCLR might be quicker at is starting up, and maybe a little better memory usage though I don't have any numbers and this is just a feeling from using both. Anyone know for sure on this?

giancarlostoro7• 9 years ago

> Just over 2 years ago Microsoft open-sourced the entire .NET framework,

Except they open-sourced a new .NET stack, not really the entire .NET Framework, .NET Core and .NET Framework are similar but not fully backwards compatible. I've been porting .NET Framework code to .NET Core and depending on how specialized your project is you may not always find the same libraries supporting your project. A project that perfectly compiles with .NET 2.0 will not just compile for .NET Core, at least not as easily as using a .NET Framework 2.0 project in .NET Framework 4.6. I love Microsoft and .NET but I think Core is executed poorly. You also have EntityFramework vs. EntityFramework Core which lacks features from EF which were deemed "unused" which bites some people in the face.

UK-AL• 9 years ago

I think it needed to be done. There is a lot of stuff .net framework that doesn't really make sense anymore.

matthewwarren9• 9 years ago

Well, it depends how you look at it, but you have a point.

I guess I could've written 'Microsoft open-sourced the entire .NET stack'. I wanted to make it clear that they open-sourced everything, i.e. runtime, JIT, GC, base-class libraries, etc

giancarlostoro7• 9 years ago

I understood what you meant, but I figure some might not understand what you meant, overall I appreciate the write up, I will take the time to read through it when I'm not working. It also inspired me to want to read through the source code behind .NET's CoreCLR.

abiox2• 9 years ago

being 100% backwards compatible may be nice, but it would also be severely restrictive. this is a good a moment as any to clean up (where possible) bad choices made in .net's younger days.

justin66• 9 years ago

.NET Framework 2.0

It'd be a shame if the progress they are making were stymied in order to be compatible with the .Net of a decade ago.

I've been porting .NET Framework code to .NET Core

Yeah, but you probably don't need to do that. .Net Framework is still supported, and will be for a long time.

giancarlostoro7• 9 years ago

I feel like .NET Framework is going to be replaced by .NET Core eventually as its use rises more. I wish I didn't need to port to .NET Core, but I don't pay my own salary.

manigandham• 9 years ago

.NET Standard/Core 2.0 will add back many APIs. It'll be the real release of the newer framework, similar to how the original .NET Framework took until v2 to get things into shape.

.NET Standard is all about compatibility - so you can use that to determine which framework you can/need to use. However .NET Core is a distinctly designed for cross-platform reach so it will be a break from .NET Framework in some ways - this is just a given, otherwise they would've just released another .NET Framework version instead of doing all this.

giancarlostoro7• 9 years ago

Yeah I know this, and hopefully they get it right, unfortunately I'm not there yet. To be fair .NET Core has been great otherwise, I have been able to develop from Ubuntu (used Sublime Text, which I didn't use the plugins since they seem abandoned or something) gave Rider a try but it was buggy at times. I wish Microsoft releases either a plugin for MonoDevelop, or "Visual Studio for Linux" at some point.

WorldMaker2• 9 years ago

About the same time the .NET Framework "Reference Source" for the old .NET Framework moved to a much more permissive open source license, but not open source/community-driven development (partly because development has been focused on .NET Core). It includes a lot more than people think it might (including most/all of SWF and WPF).

You still might quibble over "entire" .NET Framework, but a lot of the .NET Framework is open source in addition to .NET Core.

matthewwarren9• 9 years ago

Oh yeah, I'd forgotten about 'Reference Source', although AFAIK that's only the equivalent of CoreFX, not CoreCLR

UnoriginalGuy2• 9 years ago

You also have EntityFramework vs. EntityFramework Core which lacks features from EF which were deemed "unused" which bites some people in the face.

Where did you get this "unused" claim? I've been following EF Core and EF6 for quite a while and I've never heard them say this about most of the major features missing from EF Core.

EF Core is a complete re-write from the ground up of Entity Framework. They're attempting to fix some design issues that cropped up in the original Entity Framework, improve performance, and improve the underlying architecture (the APIs are similar, but the "under the hood" layout/design has shifted).

They've been extremely clear that the current release isn't on feature parity to Entity Framework 6, and they've told people to avoid it until it is. If you look at their EF Core TODO list it will come back into EF 6 feature parity eventually and with some new functionality provided by the underlying changes they're making in EF Core.

Honestly your whole post is full is misleading claims, this is just one example. You've started using brand new technology (which is a rewrite) and are complaining because you ignored their very specific warnings about the state of it. Then you justify your own lack of research by making wild outlandish claims about how it is everyone else's fault.

To quote EF Core Team's Roadmap[0]:

Because EF Core is a new code base, the presence of a feature in Entity Framework 6.x does not mean that the feature is implemented in EF Core.
We have provided a list of features that we think are important but are not yet implemented. This is by no means an exhaustive list, but calls out some of the important features that are not yet implemented in EF Core.
The things we think we need before we say EF Core is the recommended version of EF. Until we implement these features EF Core will be a valid option for many applications, especially on platforms such as UWP and .NET Core where EF6.x does not work, but for many applications the lack of these features will make EF6.x a better option. [Big List of missing features]
There are many features on our backlog and this is by no means an exhaustive list. These features are high priority but we think EF Core would be a compelling release for the vast majority of applications without them [Big List of missing features]
Our team is currently working on the EF Core 2.0 release. It's early days, so our plans will likely change as the release progresses, but here are the major features we are planning to address in this release. [Big List of missing features]

The truth is that YOU started using a piece of technology they expressly told you not to use and told you why you shouldn't. Then created fictional justifications for why it is their fault you screwed up.

But I welcome you to cite your "[won't implement features] which were deemed 'unused'" claim. Somehow I don't think you will...

[0] https://github.com/aspnet/EntityFramework/wiki/Roadmap

giancarlostoro7• 9 years ago

1) I didn't pick .NET Core because I'm not in a leadership position, I'm probably the developer in my job who gets paid the least, and has the lowest of job titles, unless we at some point have an intern.

2) My Team Lead picked .NET Core, we are porting from Silverlight, not from ASP .NET which I have a feeling eventually will stop being included as a project template for Visual Studio as more people swap to ASP .NET Core so if that becomes the case I could see why I wouldn't be asked to develop a project in ASP .NET just to recode it later on down the road.

3) I might of mistaken EF Core not implementing features with ASP .NET Core, they seem to be shifting a lot of things for ASP .NET Core, especially with Identity being a new way of doing things. ASP .NET has always "lost" features along the way, need we remember when it was not even called .NET and was just "ASP"? Not sure why I need citations for that, you could form your own conclusions. I'm just a "Junior" anyway what do I know?

youdontknowtho• 9 years ago

The GC Sample is really interesting.

matthewwarren9• 9 years ago

Yeah it's great, I learnt so much about the GC from looking at that sample.

BTW if you want a bit more info on it, I wrote a whole blog post, see http://mattwarren.org/2016/02/04/learning-how-garbage-collec...

eiji• 9 years ago

What's with the Smalltalk portion? Anyone knows why that is?

desdiv• 9 years ago

There's no actual Smalltalk code in there. It's just a Github display bug.

https://github.com/dotnet/coreclr/search?l=smalltalk

matthewwarren9• 9 years ago

I didn't notice that before, I wonder what file extension it's incorrectly assuming are smalltalk? There is a whole lot of '.S' files in there, I wonder if it's those?

Guide to the CoreCLR Source Code

Related Stories