The case for dynamic, functional programming

https://www.onebigfluke.com/2022/11/the-case-for-dynamic-functional.html

by todsacerdoti • 4 years ago

101 115 4 years ago

onebigfluke.com

functional_pyth• 4 years ago

Would have been a stronger case if Julia, Erlang/Elixir, or Clojure/Racket was used.

Python is one of the worst dynamic/FP languages I've ever had the displeasure to utilize.

I think the OP needs to step out of their comfort zone of Python.

qsort• 4 years ago

The point of the article isn't to say that python is better, all it's saying is that a more functional style leads to cleaner code. This is irrespective of python's technical merits.

With a dynamic, functional language you could enjoy all of this simplicity. But you don't even need that to benefit from this perspective today! You can adopt the techniques of functional programming right now in your favorite language (such as Python, used above). What's important is your mindset and how you approach problems, minimizing state and maximizing clarity in the code you write.

z9znz6• 4 years ago

I suspect the point of the article was to be seen again and remind people of his book.

bslatkin9• 4 years ago

I agree that another language that's better for functional programming would make this more compelling. Python has a lot of limitations in this area. I think all of those alternatives you listed (Julia, Clojure, etc) include turn-offs for Python programmers that are deal breakers. So I'm seeking another language that appeals to Python programmers but has more of the attributes that are amenable to functional programming.

z9znz6• 4 years ago

IMO, Ruby is a better general purpose "scripting" language like Python that has cleaner and more consistent features - including ones that lend themselves to functional approaches. Outside of web development it doesn't have the same appeal as Python due to the larger Python ecosystem... (many companies think they want to and will do "data science!")

Also, in my experience, Pythonistas tend to be more resistent to paradigm changes. So often I have heard, "Why would you want to do that?", or "This works fine as it is.". Not that I'm a fan of list comprehensions in Python (I think they're awkward and ugly compared to Ruby collection operations), but Python codebases I have had the displeasure of working in had lots of nested loops, mutations everywhere, copy-pasted code, and 30+ line methods. Trying to encourage single-responsibility, composability, less OOP, and more pure functions is like shoveling water uphill.

With both Ruby and Python you do need to be a bit thoughtful about unnecessary collection copies (which you would tend to favor when writing pure functions); but often you have a good sense of how large a volume of data you are handling and where copies will be particularly slow or heavy to do. When necessary, you can have an impure function that mutates input data, and at least in Ruby you can warn callers of the impending mutation by adding the ! to the function name. update_order!(order).

revskill• 4 years ago

Agree. But Python is not good example for me. I think Python developers are genius because they could get things done in a language with 1-line lambda !

Jtsummers• 4 years ago

Single expression lambdas. Lambdas can span multiple lines but they specifically contain a single expression, no statements. And since Python isn't really an expression oriented language this is still pretty constraining.

bjd2385• 4 years ago

Wait where is self.count set, and is it 0 or 1?

bslatkin9• 4 years ago

Fixed now, thanks!

xedrac• 4 years ago

With Python it's much easier to become proficient and remember everything you need to know. That translates to more time being productive writing code instead of reading documentation

Until your python program reaches a certain size where the lack of type safety and other protections becomes a real liability. Contrast that to something like Rust, where it's easy to have confidence in very large code bases.

falcolas5• 4 years ago

As someone who's worked on a huge Python program (10M+ LOC), type safety was the last thing on our mind when maintaining it.

Refactoring might not have been quite as simple, but there are plenty of tools available for refactoring Python code, and we were never afraid of doing it.

Though, working on a project right now in Java, the type system isn't exactly saving us from refactoring failures, because the type system doesn't ensure that the String or Integer or <low level interface> being passed in is the right one.

Of course, I'm sure we're just "doing it wrong".

yakshaving_jgt2• 4 years ago

There’s a significant delta between Java types and Haskell types.

cosmic_quanta• 4 years ago

As someone who's worked on a huge Python program (10M+ LOC)

Holy moly. I can't imagine working on a codebase like this in Python.

I maintain several small codebases (<10k LOC) at work. Even at this size, I definitely feel the lack of type safety -- specifically, at the boundaries of logical blocks of code. When refactoring, the lack of thorough type-checking means that I need to mentally propagate my changes throughout the codebase. This is exhausting!

harveywi• 4 years ago

As someone who's worked on a huge Python program (10M+ LOC), type safety was the last thing on our mind when maintaining it.

This is true. Python dictionaries are extremely powerful, and they obviate the need to use elaborate types throughout the code. There is even a special syntax for making dictionaries in Python. In a code base with lots of dictionaries, type errors are a thing of the past.

bslatkin9• 4 years ago

I've had a similar experience in a large Python codebase!

perrygeo• 4 years ago

So, what was the first thing on your mind when maintaining it? Python gives you plenty else to worry about besides types.

linguae• 4 years ago

Judging by the title, I was expecting to read about the advantages of dynamically-typed functional programming languages like Scheme and Clojure over statically-typed functional languages like Standard ML and Haskell. It seems to me that large segments of the functional programming community love static typing, and the research community has done (and continues to do) a lot of work applying type theory to the design of programming languages. However, there are still many advocates of dynamically-typed functional programming languages, generally from the Lisp family.

I don’t think of functional programming when I use Python, even though it’s possible to program in a functional style. I’m also unaware of any Python development environments that offer Smalltalk or Common Lisp levels of interactivity, though this is less of a criticism of the language and more of a matter of tooling.

falcolas5• 4 years ago

Yup, I would love to see a bit more love for Lisp style languages. They're incredibly powerful and it's a shame they're relegated to the fringes of professional programming.

bslatkin9• 4 years ago

100% agree with both of these comments. I'd love to see a resurgence of interest in this area.

grumpyprole9• 4 years ago

Lisp evolved into ML and Haskell and these have influenced professional programming enormously over the last few decades. Or are you advocating we ditch all progress and go back to Lisp? Have you tried an ML such as OCaml? It has built-in cons lists, convenient symbol manipulation and macros.

AtlasBarfed2• 4 years ago

It's because LISP has a "great filter" for high IQs.

Programming has many such filters:

Understanding pointers / references separates the python/script kiddies from the "real" C programmers and understanding von neumann architecture.

Recursion separates the programming-is-a-job with the beginnings of algorithmic understanding of computation.

LISP is heavy recursion, non-infix notation, lambda calculus, and other concepts that are simply too much for a garden variety programmer.

z9znz6• 4 years ago

Unfortunately this toy example isn't even pure functional because it raises an exception as a way of providing an alternative return from the function.

A better example, if one wanted to keep a single function pure and still keep (two) states would be to return a tuple of (current_count, is_expired). Further, since it uses zero as the implicit boundary to determine when the counter is complete by counting down from the provided value, the function should be named countdown_timer() or something more descriptive.

Honestly I'm not sure who this little article was written for. The references to C++ were irrelevant. The OOP Python example, however, was a pretty common example of how Python people write code. It's a shame, really. We should (edit: not) have to deprogram people to get them to choose something more like the latter "functional" approach over the former pointlessly-OOPy example.

nlitened3• 4 years ago

Why do you consider raising an exception not being pure functional? In my opinion, it is exactly equivalent to propagating error return value up the call chain until it is handled (but is faster if no exception is thrown, and contains a stack trace of where the original error value was produced for programmer's convenience).

jerf• 4 years ago

If you take that view, yes, it's functional.

However, if you take that sort of view, everything is functional. Writing to the screen can be viewed as having returned instructions to write to the screen, combined with an interpreter that performs it. Writing to the network is like having returned an instruction to write the network, along with an interpreter to do it. (And a certain laissez faire attitude about laziness, perhaps.)

This view is not necessarily wrong. There is a certain truth to it; for one thing it's not a terrible gateway to understanding how "pure functional" languages interact with the real world in practice, because this is pretty close to how they do it.

However, it is useless, because it means every language is a pure functional language, and the utility of a term is its ability to split the universe under discussion into multiple pieces. Or to put it another way, a decision procedure that puts 100% of the items under decision into the same category conveys exactly zero bits of information.

So for a definition of "pure functional" to have any utility, it needs to exclude some things. Exceptions are borderline. Technically Haskell does indeed have them, but it has a complicated relationship with them. It is certainly the case that we are discussing something that is not in the function signature anymore, which people generally consider not "pure functional". But generally it definitely has the "well, we could pretend that we returned a special value and had a special interpreter wrapped around it for every call" copout feel to it in general.

z9znz6• 4 years ago

Raising an exception is not returning a value; it is essentially a GOTO.

A pure function has no side effects. Maybe it takes an input, and maybe it returns a result. But whatever it does internally has no affect on the outside world. Raising an exception is definitely a side effect.

Worse yet, in TFA example it wasn't even necessary to use an exception in place of a returned state. I would argue that the exception example was, irrespective of functional vs imperative principles, a bad use of exceptions.

substation135• 4 years ago

You can adopt the techniques of functional programming right now in your favorite language (such as Python, used above).

No tail call optimization, no expression orientation, mutability by default, no variable shadowing, single expression lambdas, no do-notation, reference semantics everywhere, dog slow function calls... functional programming in Python doesn't work in my experience.

Now, there is a great case for functional lisps, like Clojure, since they support all the fancy patterns in cutting-edge Haskell etc. but without an extremely complex compiler. Of course you sacrifice some type-safety.

bslatkin9• 4 years ago

Author here: I agree that another dynamic language with all of those attributes would be great.

throwawaymaths2• 4 years ago

Elixir. Lets you do stateful declarative stuff (talking to databases, communication, disk) with no fuss. No mutability, single line lambdas, fast function calls.

Plus, rediculously easy concurrency, customizable failure domains, first class concurrent testing...

substation135• 4 years ago

Could Google adopt such a language? They were early on Python, which was great! Perhaps they could "bless" Clojure - or more likely build their own.

discarded1023• 4 years ago

Like the strict MLs (ocaml, Standard ML, F#, ...), while Lisp and Scheme can express the Haskell idioms it lacks the pervasive laziness that makes them (somewhat) efficient. For instance one might write `zip [0..] xs` to number the list `xs` in a single pass. This lack of laziness (even on code that does notionally work without it) manifests in blowing stacks, in Scala at least. Also the lack of types (type classes) precludes at least some things such as the `deriving` mechanism.

For all that you're dead right about Python.

rekado• 4 years ago

With SRFI 41 (streams) this can be expressed lazily in Scheme, but it is more verbose than Haskell:

    (import (srfi srfi-41))
    (define xs '(a b c))
    (stream-zip (stream-from 0) (list->stream xs))

This returns a lazy stream whose values can be evaluated later.

substation135• 4 years ago

The strict MLs often have a `lazy` keyword, but it's definitely not as tidy as Haskell for this use-case. I'm sure a clever Clojure macro could be devised for the same!

still_grokking• 4 years ago

This lack of laziness (even on code that does notionally work without it) manifests in blowing stacks, in Scala at least.

Please don't spread FUD.

The std. lib functions in Scala are stack safe. (Simple, fast while loops under the hood when it comes to collections).

To pick up the `zip` example:

  val r = (1 to 100_000_000).zip(1 to 100_000_000)

won't blow up with a stack overflow.

https://scastie.scala-lang.org/a4v5w1bKTeadQaPtK9CC5A

Also custom recursive functions can be made safe (for example by using trampolining).

You blow stack in Scala only when you explicitly ask for it by writing unsafe code. (Which is BTW something that applies to more or less all languages as almost nobody ever repeated the mistake to build a lazy by default language).

https://news.ycombinator.com/item?id=1924061

omginternets2• 4 years ago

Hell, even JS is a better functional language than Python.

KolenCh• 4 years ago

https://github.com/baruchel/tco

bmitc2• 4 years ago

I'm not sure what Python has to do with functional programming, even remotely. It basically contains none of the typical building blocks for functional programming (immutable data, real lambdas, pattern matching, sane scoping, etc.). Python has weird scoping, weird lambdas, and fully on mutability.

Regarding static versus dynamic functional programming, I think dynamic typing is probably oversold in that most dynamic programs could be static programs with very little effort. Where dynamic types really shine are at boundaries of programs. Dynamic types are very useful for handling messages, I/O, data formats, etc.

I actually think an ideal language would be one that is at its core a statically typed functional language, but then it has dynamic types allowed at the above stated boundaries. Then there could be some explicit conversion layer or solution that moves values from one side to the other.

Basically all other uses of dynamic typing can be solved in a statically typed language that has anonymous types like anonymous records and anonymous discriminated unions and some sort of "umbrella" or "shared" type such as interfaces, generics, and/or typeclasses, for example.

somewhereoutth• 4 years ago

Indeed, and I would further argue that business data be dynamically typed, because business data is often messy and changeable, whereas the plumbing that carries it can be statically typed to ensure you are connecting the pipes correctly.

grumpyprole9• 4 years ago

I would further argue that business data be dynamically typed, because business data is often messy and changeabl

Huh? Static types make change and refactoring much easier.

rrradical• 4 years ago

Haskell is this. It is static with a dynamic type: https://downloads.haskell.org/~ghc/9.2.4/docs/html/libraries...

fiedzia2• 4 years ago

immutable data

sort-of possible now (frozen*)

pattern matching

That has been added recently

bmitc2• 4 years ago

Neither are foundational in the language and have all sorts of quirks. If anything, it makes Python akin to C++'s bloat, one of the complaints and ultimately ironic comparison at the beginning of the article.

quickthrower23• 4 years ago

I think Typescript is close. It does suffer from JS heritage.

A TS like language that also had a way to runtime check types (for those data boundaries) but mix in some “any” when needed would be a hell of a pragmatic language.

substation135• 4 years ago

A robust approach for data-boundaries is to write parsers or decoders. The Elm community has done some nice work here.

imaltont• 4 years ago

Common Lisp is idd pretty nice.

eru4• 4 years ago

What do you mean by 'real lambdas'?

The functions you can create with Python's lambda syntax are a bit limited, but that's a purely syntactical limitation: you can create named function just fine and pass them around.

(I agree with most of the rest of what you are saying.)

tsukikage• 4 years ago

They are likely referring to the hoops you have to jump through to get early instead of late binding. The boilerplate becomes automatic after a while, but that doesn't make it pleasant.

bitwize2• 4 years ago

Functional programming can be attempted in any language that offers higher-order functions that can close over free variables. In other words, languages isomorphic to untyped lambda calculus with additions. Python, Perl, JavaScript, and Go all qualify.

However, it seems to be generally accepted that you're really only doing baby FP unless you also program with types. Types, and the algebras that can be imposed over them, are where the really interesting FP work is done that yields new control structures (of which monads are just one), that are difficult and cumbersome to express in untyped LC, or in a dynamic programming language.

bslatkin9• 4 years ago

Author here. Part of my point here is 1) you get a lot of benefit from dynamic languages, and 2) you also get a lot of benefit from functional programming. And 3) you'd get the most benefit from both together. When you add types into the mix then you're taking two steps forward and one step back in terms of simplicity and understandability.

What you said is also perpetuating a common reason that people are turned off to FP in general these days:

https://twitter.com/haxor/status/1584725493513072640

"At some point functional programming became equivalent in people's minds to complex type systems (e.g., monads), and that caused the value of Lisp to be lost to history. It's a shame."

bitwize2• 4 years ago

I don't know what you're getting at. Static typing makes your code easier to understand and reason about. It's pretty much an unmitigated win.

Dynamic languages may have made some sense back when people wanted to cruft something up and get it running without declaring the type of each object before it is used (which declarations can get verbose and cumbersome in the presence of parameterized types). But today we have powerful computers that can quickly check and infer types in ever more elaborate type systems, and the risks of dynamic languages seem less and less worth the development speed gains. You can see this in perennial language popularity contests. Consider how thoroughly the programming community embraced TypeScript over JavaScript because of the guarantees static type checking provides before your code even runs. And while Lisp used to be the smart kids' favored programming language, it was displaced by Haskell and later by Rust once those languages matured to where people were building real world stuff in them. (As it turns out, baking object lifetimes and ownership into your type system means you don't even need a GC to get Lisp levels of power and abstraction!)

grumpyprole9• 4 years ago

"At some point functional programming became equivalent in people's minds to complex type systems (e.g., monads), and that caused the value of Lisp to be lost to history. It's a shame."

This is nonsense. Types help us name, categorise, discuss and share abstractions. These abstractions are useful in any language, typed or otherwise. Lisp lives on in its descendents ML and Haskell. If someone doesn't want to learn anything new, then they are themselves "lost to history".

epgui2• 4 years ago

I'm a big fan of Clojure (dynamic FP) and have worked with it professionally, but I really don't see why types seem to be so cumbersome.

I hate languages like C#/Java, but not because types are cumbersome. You play around with something like Haskell (and its type system) and it's a whole other story.

whateveracct• 4 years ago

Nix is my favorite dynamic functional language. So much so that I'll probably use it (via hnix) as a general scripting language in projects that benefit from that.

mayoff• 4 years ago

dynamic languages don't require you to "emulate the compiler" in your head

On the contrary. Your ‘functional’ timer function takes an argument and applies the - operator to it.

You must be sure, every time you call the function, that you pass an argument to which - can be applied. If you don't, your user might see a TypeError. So you, the human author of the code, must ‘emulate the compiler’ by performing type inference and type checking in your head.

In my statically-typed language, the compiler ensures that my program only ever passes arguments for which the - operator is applicable. I don't need to ‘emulate the compiler’, because I'm actually using a compiler.

bslatkin9• 4 years ago

I wouldn't do the type checking in my head, I'd just assume it's going to work as intended. If that's not the case, then an error will happen. What's wrong with the user seeing an error? There's always a chance my program will have a bug and fail. So I have to handle that case no matter what and properly deal with unexpected errors, error reporting, etc.

stank345• 4 years ago

Bugs are inevitable but we should strive to have as few as possible. Your program not working right is obviously bad UX (as we've all experienced using buggy software).

The reason we have type checking is same reason we have other tooling in our editors to do static analysis like highlighting syntax errors and linting: catching bugs during development is better than finding them in production.

lgas• 4 years ago

Sure, you always have to handle the case that an unexpected error occurs, but every unexpected error that occurs in production has a cost. What that cost is depends on what your software does and how your users are using it, but there's always a cost. You may decide in your particular context that there is no amount or severity of errors that is too costly for your users to encounter, but most successful software endeavours have some limits.

sidlls3• 4 years ago

Usually this is "solved" by creating unit tests to emulate what a compiler in a statically typed language would otherwise do. Which is (another) reason that dynamic languages shouldn't be the choice for anything but scripting.

pieresqi• 4 years ago

Dynamic typing is great for small to med projects or for prototyping but in bigger codebases it becomes quite messy trying to keep in mind what is what and can lead to logical errors or use of "bad" data.

Static typing let's compiler keep tabs on what variable can contain and you can define safe fully defined conversions between types and removes mental overhead of keeping types in mind.

omginternets2• 4 years ago

This entire post reminds me of that satirical O’Riley book called “upsetting your coworkers with Python”.

https://pics.me.me/thumb_using-list-map-lambda-and-filter-on...

BulgarianIdiot• 4 years ago

I'm working on a language/platform that combines pure core (no direct side-effects) with some procedural techniques (allow internal/local mutations) and no shared mutability (100% pass by value, copy-on-write is used to optimize quick deep snapshots of value trees).

I'm convinced this is the best combination of OOP and FP. I tried to find other languages like this, but I couldn't. Ironically the closest thing is how an SQL database works.

Does my description ring a bell to anyone?

quickthrower23• 4 years ago

Sounds like Haskell but making things that are awful monad stacks of readers, writers, state (yuk) into sublime language features.

qazxcvbnm• 4 years ago

Yes, I agree with your conception. Likewise, I am working on a language with no shared mutability, and something like a pure core and often find myself solving problems that feel like putting a database in the middle of the application.

Any links to your language, perhaps?

sampo• 4 years ago

This is what Firefox shows me:

Warning: Potential Security Risk Ahead
This site is blocked due to a phishing threat.

https://phish.opendns.com/main?url=www.onebigfluke.com%2F202...

bslatkin9• 4 years ago

Thanks for reporting that — I have no idea how that could be happening. Site is hosted on Blogger so it's a pretty vanilla setup.

The case for dynamic, functional programming

Related Stories