Comptime – C# meta-programming with compile-time code generation and evaluation

https://github.com/sebastienros/comptime

by bj-rn • 7 months ago

150 66 7 months ago

github.com

mfro• 7 months ago

This seems like the kind of feature that should be built into MSBuild.

eterm3• 7 months ago

It's a lot less ergonomic but there are source generators in C# :

https://devblogs.microsoft.com/dotnet/introducing-c-source-g...

That said, for more complex results, you'd typically load a serialization on start.

I can see the value in this tool, but there must be a fairly limited niche which is too expensive to just have as static and run on start-up and cache, but not so large you'd prefer to just serialize, store and load.

It also needs to be something that is dynamic at compile time but not at runtime.

So it's very niche, but it's an interesting take on the concept, and it looks easier to use than the default source generators.

piskov• 7 months ago

Also t4 templates before that for at least a decade

richardw• 7 months ago

Not terribly niche. All config that isn’t environment-specific and is used in inner loops or at startup. It’s even got a test for serialised values so can be used to speed your case up:

https://github.com/sebastienros/comptime/blob/main/test/Comp...

But you need to be sure you won’t want to change without compiling.

mexicocitinluez2• 7 months ago

I use source generators pretty extensively but don't really understand how this is different (or what it's solving that source generators can't).

I'm pretty new in the source generation area and only do enterprise dev, so I'm sure I'm missing something or just don't have the use cases.

Const-me• 7 months ago

there are source generators

Last time I tried them discovered source generators in the current .NET 10 SDK are broken beyond repair, because Microsoft does not support dependencies between source generators.

Want to auto-generate COM proxies or similar? Impossible because library import and export are implemented with another source generators. Want to generate something JSON serializable? Impossible because in modern .NET JSON serializer is implemented with another source generator. Generate regular expressions? Another SDK provided source generator, as long as you want good runtime performance.

andix11• 7 months ago

Please don't. Nobody wants even more complexity in MSBuild ;)

ygra3• 7 months ago

The use cases are different. While MSBuild tasks run during build (and partially when loading a project), typically the IDE is oblivious what happens there. The source generator runs directly inside the compiler infrastructure and thus you didn't get error highlights for code that would otherwise be only generated during build but not as you type. This makes it much more friendly than pure build-time generation of code.

mgaunard• 7 months ago

So it's like C++ consteval?

CharlieDigital• 7 months ago

C# meta programming game is strong. Source generators are :chefs_kiss:

daeken3• 7 months ago

I love (and heavily use) source generators, but the development experience is godawful. Working with the raw Roslyn types is painful at best and this is compounded by them having to be written against .NET Standard, severely limiting the use of newer .NET functionality.

Eventually I want to write a good baseline library to use for my source generators -- simplifying finding definitions with attributes, mapping types to System.Type, adding some basic pattern matching for structures -- but haven't found a way to do it that's general enough while being very useful.

pjmlp3• 7 months ago

Yeah, it is kind of sad that it was the community that had to step up for some T4 like experience instead of string concatenation.

That isn't as cool as Aspire and AI features.

mexicocitinluez2• 7 months ago

I feel the exact same way. They can be insanely powerful, but the syntax is insanely off-putting.

And the documentation is still pretty sparse.

Eventually I want to write a good baseline library to use for my source generators -- simplifying finding definitions with attributes, mapping types to System.Type, adding some basic pattern matching for structures -- but haven't found a way to do it that's general enough while being very useful.

I'd use it in a heartbeat. The new ForAttributeWithMetadataName function has been a big help, but everything else feels like a totally different language to me.

zigzag312• 7 months ago

I agree, .NET Standard limitation unnecessarily complicates development experience. I think it's because some tools (Visual Studio) is still use legacy .NET Framework. I don't understand why they didn't integrate them via out of process architecture into these tools, since source generators didn't exist in the legacy framework anyway.

I sometimes generate code from plain CLI projects (avoiding source generators altogether), as whole debugging and DX is so much better.

olidb2• 7 months ago

Add "<LangVersion>latest</LangVersion>" to the csproj to get more useful features like pattern matching and raw string literals.

torginus2• 7 months ago

I used to think so too, but I do have to complain about Microsoft doing the Microsoft thing - they have a brilliant idea (and execution), but they bury it in so much boilerplate that 99% of people who would use it are put off by it.

The amount of stuff you have to wade through here compared to something like comptime in Zig (Roslyn API, setting up the project, having VS recognize it and inject it in the compiler, debugging etc) makes usage of these an absolute pain.

smcnc• 7 months ago

I think Zig really shines here: https://ziglang.org/documentation/master/#comptime

Zambyte• 7 months ago

A key difference is that in this C# package, `[Comptime]` is an attribute (annotation? not sure on the C# term) applied to methods. In Zig, the `comptime` keyword can be applied to pretty much any expression. In the C# package, if you want to do factorial at runtime and at compile time, (I think, from reading the README) you need to define the same function twice, one with `[Comptime]` and once without. Contrast this to Zig, where if you have a regular runtime factorial function, you can just execute it at compile time like:

    const x = comptime factorial(n);

Another limitation of the C# package is it only works with primitive types and collections. Zig comptime works on any arbitrary types.

hahn-kev• 7 months ago

You don't. The way it works is that it intercepts the call site when the input args are constant. If they're not then it won't be replaced and it will call the original method. C# source generators can't replace method definitions, however a call site can be changed to another method via source generators.

estimator7292• 7 months ago

Decorations in [square brackets] are Attributes in C#

Tiberium• 7 months ago

There's also Nim, it shines even more in compile time and has proper compile time macros :)

pjmlp3• 7 months ago

D was already doing it in 2010 thereabouts, an then there is the whole reader macros in Lisp and Scheme.

jibal• 7 months ago

And Nim, which has multiple levels of metaprogramming.

pyrolistical• 7 months ago

I’m annoyed they called it comptime when it isn’t the same as Zig’s more powerful comptime.

You can think of zig’s comptime as partial evaluation. Zig doesnt have a runtime type reflection system, but with comptime it makes it feel like you do.

jauntywundrkind• 7 months ago

Makes me think of Boo language; Boo was so good at metaprogramming and multi-phasr programming! A very fine .NET language that was so far ahead of the curve, with having the tools of that language be usable at runtime.

Alas many of the docs are offline now. But it had great quasiquotes, which let you write code that gets turned into AST that you can then process. Good macros. A programmable compiler pipeline. So much. Alas, obscured now. https://boo-language.github.io/

daeken3• 7 months ago

Boo and Nemerle both were really showing what was possible in .NET back in the early days. I still miss the metaprogramming they had, not to mention their pattern matching (which C# has closed the gap on, but is still way, way short.)

andix11• 7 months ago

The syntax looks a bit like F#. But F# only has a few features that generous people might consider meta programming.

There was a lot of fuss about meta programming around 10-15 years ago, but it never got a lot of traction. Maybe for a good reason? I think a lot of the problems it solved, were also solved by functional programming features that slowly appeared in C# over the years.

rubenvanwyk• 7 months ago

How is Comptime different from default AOT compilation? I assume it can work with third-party libraries that AOT don’t work with yet?

betaporter• 7 months ago

I believe this is like `constexpr` for C#.

andix11• 7 months ago

It's a code generator that runs during compile time. It's a source generator that adds some generated code files to the project. So it runs way before AOT or JIT. Once AOT/JIT run, Comptime is already invisible to them, they only see the generated code from Comptime.

spicyusername• 7 months ago

I started using C# recently for a hobby project writing a game engine on top of Monogame, and I have been very surprised at how nice of a language C# is to use.

It has a clean syntax, decent package management, and basically every language feature I regularly reach for except algebraic data types, which are probably coming eventually.

I think the association of .NET to Microsoft tarnished my expectations.

andix11• 7 months ago

Modern C# and .NET are great. It still suffers from the bad reputation of the Windows-only .NET Framework. It's still a quite heavy platform with a lot of features, but the .NET team invested a lot of time to make it more approachable recently.

With top level Programs and file-based apps[1] it can be used as a scripting language now, just add a shebang (#!/usr/local/share/dotnet/dotnet run) to the first line and make it executable. It will still compile to a temporary file (slow on first run), but it doesn't require a build step anymore for smaller scripts.

[1] https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals...

actionfromafar2• 7 months ago

Also, if you compile Ahead Of Time (AOT) you can cut down on the features and get basically as small a subset as you want of the libraries. IMHO C# and dotnet are really starting to become very impressive.

thdrtol• 7 months ago

I love C# and .NET and use it every day.

Sometimes I wonder if the .NET department is totally separated from the rest of Microsoft. Microsoft is so bad on all fronts I stopped using everything that has to do with it. Windows, Xbox, the Microsoft account experience, the Microsoft store, for me it has been one big trip of frustration.

andix11• 7 months ago

Microsoft is huge, it's many companies inside one company.

.NET seems to be somewhere close to Azure, but now far away from Windows or the business applications (Office/Teams, Dynamics, Power Platform). Things like GitHub, LinkedIn or Xbox seem to be de facto separate companies.

Edit: .NET used to be tied closely to Windows, which gave it the horrible reputation. The dark age of .NET ;)

vips7L• 7 months ago

Unions have a proposal: https://github.com/dotnet/csharplang/issues/9662

faithlv• 7 months ago

This is amazing, after using Zig, I started to write exactly this for C# too but never really had motivation to finish it.

orphea4• 7 months ago

  • Supported return types:
     ◦ Collections: ..., List<T>, ...
     ◦ Note: Arrays are not allowed as return types because they are mutable. Use IReadOnlyList<T> instead.

I don't understand. Why is List<T> allowed then if it's mutable?

andix11• 7 months ago

Array also implements IReadOnlyList if I'm not mistaken.

I think C# doesn't really have immutable collections, they just can be typecasted to IReadonly* to hide the mutable operations. But they can always be typecasted back to their mutable base implementation.

The only real immutable collections I know of, are F#s linked lists.

maltalex• 7 months ago

Immutable collections exit, they were just added later. See System.Collections.Immutable:

https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

randomNumber7• 7 months ago

One man's constant is another man's variable.

orphea4• 7 months ago

Adding to the sibling comment, there are also Frozen{Dictionary,Set}: https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

torginus2• 7 months ago

While this looks cool, I don't see any code generation capability in the examples or the tests, only compile time evaluation.

This is more like constrexpr than a macro system.

andix11• 7 months ago

Does this finally allow reading string literals from files and include them into the binaries?

I've seen a Go project that was heavily using https://pkg.go.dev/embed for loading some template files, and got a bit jealous. Back in the days of .NET Framework it was common to compile file contents into resource files, but it was always quite cumbersome.

floucky• 7 months ago

Can't you already do that with embedded resources?

andix11• 7 months ago

Yes, you can. But it's a bit cumbersome. The API to read them is not really intuitive, they are only accessible as a Stream, so they need to be either read every time into a string (new allocation, slow), or you need a helper that reads them once and keeps them in memory. I think there are also a lot of gotchas around naming and listing them.

In modern code I don't see them that often anymore.

eterm3• 7 months ago

C# / .NET has always had that abililty.

  <EmbeddedResource Include="/path/to/file.txt" />

Then you can read that file from the assembly with:

  assembly.GetManifestResourceStream

I'm not sure what's awkward about that, but it's nothing new.

Discoverability for this is in visual studio, if you go to file properties there's a drop-down where you choose between Content, EmbeddedResource, Ignore, etc.

That determines what happens in compile, whether it is copied to output directory, compiled into the binary, etc.

andix11• 7 months ago

The Stream is awkward. It needs to be processed either once at startup or lazy on access, or every time it's used (allocations!). Also discovering multiple files is awkward.

Comptime – C# meta-programming with compile-time code generation and evaluation

Related Stories