LINQ and Learning to Be Declarative

https://www.nickstambaugh.dev/posts/LINQ-and-being-declarative

by sieep • 10 months ago

59 57 10 months ago

bob1029• 10 months ago

Functional programming isn’t an afterthought in C#, it’s effective and you should learn it if you haven’t already.

I think C# is the best functional programming language because you always have access to a procedural code safety valve if the situation calls for it.

100% purity down the entire vertical is a very strong anti-pattern. You want to focus on putting the functional code where it is most likely to be wrong or cause trouble (your business logic). Worrying about making the underlying infrastructure functional is where I start to zone out.

Does F# care if a DLL it references was coded in a functional style?

louthy4• 9 months ago

100% purity down the entire vertical is a very strong anti-pattern.

It really isn't. The benefit of pure all the way down is that later you can replace bits with something more performant if necessary. But starting out with the idea that some bits should never be pure just means none of it is. The beauty of pure functional programming is that its very compositional nature means that you can replace a component without having a detrimental effect to everything its composed with: as long as you maintain referential transparency for that component.

What we gain from imposing the pure functional constraints on ourselves is genuine composition. If we compose two pure functions into a new function, that resulting function will also be pure. This is the pure functional programming super power that leads to fewer bugs, easier refactoring, easier optimisation, faster feature addition, improved code clarity, parallelisation for free, and reduced cognitive load. Opting out for arbitrary reasons at certain stages of the vertical threatens all of that.

mrkeen2• 9 months ago

All of this!

"Pure functional" isn't a style choice. The judges won't hold up big placards with 10 on them because you executed your business logic in style.

It's a contract that your code will give the same output given the same input. The contract goes both ways: your ability to supply a caller with functional code is made easier by your callees supplying you with functional code.

Someone decides that that's draconian and says "what's the worse that can happen?" and starts mutating in a library dependency. Well now your backtracking parser may or may not be able to backtrack. Your transactional code may or may not be able to be rolled back. Your multi-threaded code may or may not be free of races. `true || f()` no longer means the same thing as `true`. You might need to start scaffolding all your unit tests with @Before and @After to setup and teardown state, and give up running them in parallel.

Maybe you really, really, really need to fire the missiles at the bottom of the call chain. Fine, just mark that method as 'red' so I get a compiler error when I try to serve up 'blue' code to my callers.

As long as you're showing off the syntax (matter of taste) on blog posts you might as well get some mileage out of the semantics (guarantees).

pjc503• 9 months ago

The problem is this tends to collide hard with things which are both stateful and mandatory ubiquitous, like logging.

"Functional core, imperative shell" is a great tradeoff position though.

> Does F# care if a DLL it references was coded in a functional style

Deeper problem: it can't know. It can only assume. I'd have to check how the loader works but it may be the case that "first call to a function in an external DLL" is not stateless (and can error!) because it triggers the linker.

jayd162• 9 months ago

There are a lot of patterns you get when you can be 100% certain of something. For certain functional languages, you could actually safely assume immutable patterns and such.

I really love C# but I will also say that 100% purity is a super power that C# will never have (can't have 'em all).

bcrosby95• 9 months ago

C# isn't unique in that, and it also doesn't offer the tools required to help avoid having to dip into procedural code.

Here's things Clojure, for example, offers:

1. Structural sharing which makes you less likely to have to dig in to procedural code.

2. The transient function which takes an immutable data structure and makes it... not immutable... you can now do your procedural code, then switch back into immutability land with persistent!.

3. If the above fails, you can dip into Java.

So yeah, I vehemently disagree with C# being the best. I could kinda see it if you said F# (type safety), but C# itself? Uhhh.

xnorswap• 10 months ago

If you're going to write code snippets, please make sure they compile!

The final example has signature List<Product> but tries to return IOrderedEnumerable<Product>.

I recognise that this is very much a taster rather than even a full introduction, so the author didn't want to explain IOrderedEnumerable<T>, but please when writing blogs, run through your code examples and make sure they compile.

This means that your audience can follow along.

( It just needs a .ToList() on the end. )

sieep7• 10 months ago

Thanks for the heads up, must've slipped my mind. Edit: fixed

NetMageSCW• 9 months ago

Isn’t that also true for the second to last example with the query syntax code?

abstractspoon• 10 months ago

Fwiw in the final iteration I would still place each clause on a seperate line aligned by their leading '.' because I find it much easier to scan.

sieep7• 10 months ago

Agreed. That's how I would write it in a professional codebase. I added it to show the transformation from separate functions -> LINQ one-liner. Cheers!

Etheryte• 9 months ago

Having things on one line is not an upside. Just think of all the Bash one-liners that are write once edit never. Doubly so in what's essentially a tutorial.

wslh• 9 months ago

I think the future of programming will be mostly declarative and parametric since most professional software is redundant.

cjonas2• 9 months ago

The second SQL-like version is far more readable. There is just less "syntax noise" and it's far more "declarative" by definition.

The author stating that the lambda is "better" because it's less lines is also silly. It's been a while since I've written C#, but pretty sure the SQL-like version can be formatted to a single line as well:

List<Product> GetExclusiveProducts(List<Product> source) => (from p in source where p.ProductTitle == "iPhone" orderby p.TypeOfPhone select p).ToList();

z500• 9 months ago

You can format it on one line if you want. Please don't do that, though. Personally I prefer the method syntax with one method per line, it reads more linear than the query syntax.

cjonas2• 9 months ago

Agree and disagree. I prefer the second approach, but I wouldn't format either as a single line.

magicalhippo• 9 months ago

Never understood why a lot of programmers are so obsessed with reducing lines of code at the expense of all else.

What's important to me is that I can understand the intent of the code, that I can reason about how that code will be executed at runtime, and that I can easily debug the code if needed.

Of course, there's no need to go full enterprise Java. Never go full enterprise Java.

But having ten lines of clear code that's easily debuggable and which runtime characteristics is predictable is much better than one dense line that's difficult to untangle, or a few that's hard to predict what will do etc.

anonymars• 9 months ago

Not to mention it's much easier to put a breakpoint on one of those specific lines

the_other• 9 months ago

Years ago on the Ruby Rogues podcast, someone mentioned they aim to do just one operation per line. I’ve aimed to do that with ny TS/JS ever since. My files are longer but they’re usually easier to read.

Kuinox• 9 months ago

Over all, peoples prefer writing with lambda, somes use the SQL-like, but there is a major preference towards lambdas.

louthy4• 9 months ago

I'm the author of language-ext [1] a pure functional framework for C# and have been pushing the declarative programming thing in C# for over decade. C# is a great language for declarative programming and LINQ is awesome. You need to constrain yourself (and your team) so you don't fall into bad habits, but it can really pay dividends in terms of code stability/maintainability.

I have a couple of samples that I think might surprise you if you think C# must look like Java...

* A game of pontoon [2] - I create a Game monad which is an alias for a monad-transformer stack of StateT > OptionT > IO. This shows how terse you can get, where the code almost turns into a narrative.

* Newsletter sender [3] (which I use for my blog) [4] - This generalises over any monad as long as the trait requirements are met. This is about as declarative as you can get in C#. It's certainly pushing the language, but is still elegant (in my eyes anyway).

[1] https://github.com/louthy/language-ext

[2] https://github.com/louthy/language-ext/blob/main/Samples/Car...

[3] https://github.com/louthy/language-ext/blob/main/Samples/New...

[4] https://paullouth.com

Digit-Al• 9 months ago

Your documentation appears to be a bit broken. When I try to click on pipes I get a 404. Just to let you know.

louthy4• 9 months ago

Thanks for that, think I've fixed them all. But it is 1 am, so the chances I've missed something isn't zero!

The API docs are all auto-generated [1] so sometimes my repo markdowns get out of sync when I refactor!

[1] https://louthy.github.io/language-ext

yCombLinks• 9 months ago

I think the final version is far worse than the second version. This is simpler in what way ?

shortrounddev2• 9 months ago

I like to return IEnumerable instead of List<>:

    IEnumerable<Product> GetExclusiveProducts(List<Product> source) =>
        source
            .Where(p => p.ProductTitle == "iPhone")
            .OrderBy(p => p.TypeOfPhone);

That way the user can decide if they want a List or Array or Set or whatever, and you can also add additional queries to this

Also better to pass IEnumerable to the function instead of List, for the same reasons

Also I forget the syntax but you can use an extension method to add it to linq I think

mlhpdx• 9 months ago

Given the use case I’d consider IGrouping<PhoneType,Product>. My point being LINQ is a wide subject and most code I see barely touches the surface.

nlawalker• 9 months ago

The only problem with IEnumerable's open-endedness is that it's so open-ended. It makes no implicit guarantees about order, finiteness, side effects, speed/efficiency, idempotency etc. It's easy to assume those things until you accidentally find a situation where one or more are not in your favor.

pjc503• 9 months ago

This is generally better yes, especially if it's going to end up inside another LINQ operation - you can avoid materializing the list.

Cases where it isn't:

- risk of multiple execution (there is a CA warning for this)

- when returning data protected by a mutex, you should always materialize a copy of the returned list/array rather than return an enumerable

jayd162• 9 months ago

It really depends. IEnumerable could have a lambda that leaks the world and makes a web call on every iteration for all the caller knows. You know iterating a List will be fairly banal.

So from an API perspective, I think returning IEnumerables can end up being too cute. The caller has to deal with this unknown thing.

In general you should be clear about what you return and loose about what you accept. If your whole API is returning IEnumerables and some are expensive to iterate and others are not its actually less clear what I'm getting. And this isn't to say List is always the answer either.

jasonthorsness• 9 months ago

I love LINQ (the object method syntax not the keyword syntax) and used it extensively in a production application in ~2015-2017. It is easier in many cases to write and read than manual loops.

In performance-critical code at the time it had to be avoided due to allocations and poor performance compared to loop-based implementations.

However, the new versions of .NET have been reducing this penalty [1], to the point where it might make sense to try LINQ first if it's more concise/clear then only rewrite after profiling!

[1] https://devblogs.microsoft.com/dotnet/performance-improvemen...

HackerThemAll• 9 months ago

"But what if I told you we could make it simpler"

... and then the author proceeds to presenting clunky, unreadable fluent syntax

That was a good joke for the afternoon.

steego2• 9 months ago

Are you saying that you’re unable to read a fluent syntax that’s been supported by most mainstream programming languages for the past 10 years?

Or are you suggesting that the majority of programmers struggle to read and understand fluent method chaining?

I don’t have a dog in this fight because this blog post is very novice oriented. I’m just genuinely confused why you think it’s unreadable or “clunky”. What is it about the fluent example that you find clunky?

sieep7• 9 months ago

Can you show me how you would write it? I don't see how this contributes to the conversation unless you show me how or why

These patterns are commonplace for many years now in many languages.

The intention was to show how declarative code can come from imperative with a 'true' one-liner.

4rt• 9 months ago

i think his point was that the actual LINQ (language integrated query) is already superior to transposing it back to the functional fluent version. i largely agree, especially when you're doing joins and group by.

i've worked with people in the past who refused to allow any actual LINQ in the codebase (use resharper to convert it to fluent!) even though it essentially became obfuscated.

vivalahn• 9 months ago

I’m going to reserve a thread on this post for folks who want to share horror stories trying to implement their own LINQ providers.

steego2• 9 months ago

Why?

Writing your own LINQ provider is a very niche activity done by people who want to translate or “transpile” C# expression trees into something else.

It is fundamentally a difficult endeavor because you’re trying to construct a mapping between two languages AND you’re trying to do it in a way that produces efficient target code/query AND you’re trying to do that in a way that has reasonable runtime efficiency.

Granted, on top of that, I’m sure LINQ provider SDKs probably add their own complexity, but this isn’t an activity that C# developers typically encourage.

stanac• 9 months ago

Not a horror story. I like how marten worked with postgres and wanted something similar for sqlite, so I made a library that stores data as json, and translates linq to sql using json query functions. It wasn't very fast, but it was fun experience. For next attempt (once I have more time) I will probably include source generators to precompile queries and skip if not all, then most translation at runtime.

Kwpolska• 9 months ago

The example with lambdas should be written on multiple lines, too. At the same time, you can leverage the => syntax to avoid braces and end up with five lines:

    List<Product> GetExclusiveProducts(List<Product> source)
      => source
        .Where(p => p.ProductTitle == "iPhone")
        .OrderBy(p => p.TypeOfPhone)
        .ToList();

(You could join the first two lines, but I think that’s ugly for multi-line expressions.)

Also, less lines is not a good argument for SQL-style syntax vs method-call syntax. The good argument is that the SQL-style syntax is limited to only a few basic operations, when there are many more useful methods available.

Another reason is that this does not compile:

    List<Product> GetExclusiveProducts(List<Product> source)
    {
      return from p in source
             where p.ProductTitle == "iPhone"
             orderby p.TypeOfPhone
             select p;
    }

This method returns IOrderedEnumerable<Product>, not a list. To fix it, you would need to either change the return type, or go outside of the SQL-style syntax and call the ToList method:

    return (from ... select p).ToList();

recursive• 9 months ago

There are some things that are harder to write in extension methods too. Such as `let` declarations. Or multiple from clauses referencing iterating variables from multiple levels. I use both.

tracker1• 9 months ago

I'm largely with you on this... I prefer the extension methods myself most of the time.

Also adjacently worth mentioning that I tend to use Dapper instead of EF, so there's even less use of this type of expression all around unless it's actual SQL or assembled SQL via a query builder.

mau• 9 months ago

Your coworkers and QA will thank you for learning LINQ and ditching the imperative methods that plague your Python brain.

This is a very unfortunate joke: Python has list (and generator) comprehension expression for a long time (2.3?) which are similar to LINQ. At some point in the history many languages stole useful expressions from other paradigms.

Let’s joke on BASIC, it always works.

pjc503• 9 months ago

List comprehension is pretty good, but I prefer LINQ method-style because it's executed left-to-right, whereas I keep having to look up the order of Python.

dragonwriter• 9 months ago

This is a very unfortunate joke: Python has list (and generator) comprehension expression for a long time (2.3?) which are similar to LINQ.

I love Python, its my main daily driver, both at work and by preference for most of my personal coding, but Python comprehensions and genexps are much more limited than LINQ language level query syntax (Scala’s visually-similar construct is more like LINQ in capabilities) and Python—purely because of core and stdlib convention which also drive convention for the ecosystem, not actual structural features—lacks anything like the method syntax as a common API (unlike, say, Ruby).

EDIT: Thinking about it a little bit, though, it should be possible in theory to implement LINQ in Python without language level changes (including providing something close to but not quite as clean as the language level query syntax[0]) as a library via creative use of inspect.getsource and ast.parse, both for providing the query syntax and for building the underlying expression tree functionality around which providers are built (support for future python versions would require implementing translation layers for the ASTs and rejecting unsupported new constructs). Conceptually, this is similar to how a lot of embedded DSLs in Python for numeric JIT, compiling GPU kernels, etc., from (subsets of) normal Python coded are done.

[0] existing comprehension/genexp syntax looks similar, but relies on simple iteration, not pushing code execution out to a provider which may be doing something very different behind the scenes, like mapping "if..." clauses into SQL WHERE clauses for a database query.

zihotki• 9 months ago

with a one-liner for the actual logic

Looking at that 'one-liner' I get strong perl vibes, or is it chills?..

LarsDu883• 9 months ago

LINQ definitely helped me get my Unity gamedev sideproject get completed faster

sieep7• 9 months ago

I only have ever used LINQ in the context of enterprise. What are your thoughts on using it in game dev?

LarsDu883• 9 months ago

Very useful when you need to do logic on Iterables. Feels almost like python.

For example, in a game you may have a situation where you maintain a runtime list of objects with a certain property but you want to filter by some other property

dfee2• 9 months ago

I worked in a C# shop for a bit, but never wrote LINQ and saw very little of it.

However, in my Java days, I've very much enjoyed using streams. Looking back at LINQ now, it seems like a nice DSL around streams (which probably exist in C# land, too).

Interested in commentary around this!

mrsmrtss• 9 months ago

You worked in a C# shop and never wrote LINQ? How is that possible? That's like driving a manual car and never shifting out of first gear.

dfee2• 9 months ago

In retrospect, I imagine you’re right!

bytefish• 9 months ago

Once you start with LINQ you basically see it everywhere. It makes you think different about structuring your code, which is a good thing. It makes you think about immutability and pure functions, leading to more robust code.

But it’s also a fine line, when to use it and when not to. While LINQ is easy to read and make sense of, it is far from being easy to debug. Especially on inexperienced teams I tend to limit my LINQ usage and try to write more “debuggable” code. But that’s my approach, I would love to hear other peoples thoughts on it.

LINQ and Learning to Be Declarative

Related Stories