Learning Elm has gotten me confused about static vs. dynamic typing

http://rundis.github.io/blog/2016/type_confused.html

by mrundberget • 10 years ago

67 99 10 years ago

rundis.github.io

It’s hard to describe the experience of doing a major refactoring, having a really helpful and friendly compiler guide you along step by step and when finally everything compiles it just works.

This is my experience in a nutshell, my "home language" has been Python since high school after trying to learn C/C++ and persisted through learning Java in college. It was easy to use and get started with, and didn't require a ton of extraneous typing in Vim or looking up class types and returns every time you tried to use something.

For simple projects as a beginner dynamic/implicit typing is wonderful because you need fewer references. Having gone the way of the IDE, however, having good autocomplete, safe refactoring, error flagging before compilation/runtime, and an easy reference for your libraries has made me incredibly efficient in explicitly typed languages. I'm not sure how much longer I'll be using Python as my home language...

mordocai• 10 years ago

To be fair, at least good autocomplete + easy reference is quite possible in an IDE with dynamic languages.

I have it for Common Lisp in emacs for example(it requires running a live repl and having the editor connect to it).

Pre compilation errors are harder, and I personally don't feel safe refactoring in any language unless there are good automated tests. Static typing makes it better, but nowhere near full proof.

Programmatic4• 10 years ago

It is, but it has a lot harder time deciding what type of object I'm working with in a function vs. being able to specify the object's class in the function arguments. You can get that with docstrings but it's not as straightforward as just saying "def frobozinator(FooBar thing):"

RodericDay• 10 years ago

What do you use now?

Programmatic4• 10 years ago

I haven't yet replaced Python, but I really enjoyed Dart using JetBrains' WebStorm IDE for a side project.

Python maps really well to a lot of the problems that I'm tackling right now professionally, but I'm considering trying to use Java for a number of them. C# may wind up being viable for me as well; it has a chance of being a little more cross-platform with Xamarin's purchase by MS and overtures toward OSS and Linux by integrating Ubuntu.

pekk2• 10 years ago

Python is apparently the first language you used in anger and now you want to use other things, but that simply doesn't mean that it is unsuitable for experienced people or complex projects.

kazinator16• 10 years ago

It’s hard to describe the experience of doing a major refactoring, having a really helpful and friendly compiler guide you along step by step and when finally everything compiles it just works.

A truly large refactoring is one that you want to try long before getting everything to compile.

"Hmm, I have 75 operations involving data structure A. I want it to be B. Let's try it with five operations; I know that such and such features of the program only use those. Let's build the inconsistent program and try it out with with the B structure to the extent that it is possible."

The dynamic world lets us have inconsistent code that partially works, rather than forcing us to follow up on all the places reached by the ripple effect of a change. Maybe the change turns out to be a bad idea in the end; then it was a waste of effort.

Psychologically, having to deal with a big ripple effect is discouraging; it creates an "activation potential barrier" which opposes change.

tikhonj• 10 years ago

This is why Haskell recently got a flag to turn type errors into warnings: now you came make a change, get a bunch of type errors (as warnings!) and fix them one at a time, loading and testing each part as you change it. You don't have to update everything to play around with your module.

This isn't dynamic typing though: if you tried to use an expression that did not typecheck, you will get a runtime value immediately. It's the moral equivalent of automatically replacing the broken expression with an `undefined` exception.

talles2• 10 years ago

That's very, very nice. Is this exclusive to Haskell?

kornish• 10 years ago

For anyone reading and interested in checking out the docs, the flag is -fdefer-type-errors: https://downloads.haskell.org/~ghc/7.8.2/docs/html/users_gui...

hacker_9• 10 years ago

The dynamic world lets us have inconsistent code that partially works

This is precisely why I have nightmares about refactoring dynamic codebases.

Touche3• 10 years ago

If it's tested, you shouldn't.

vertex-four• 10 years ago

A truly large refactoring is one that you want to try long before getting everything to compile.

So I thought this for a good long while - and then I started coding in Rust on a major project. Turns out that once I'd written my serialization/deserialization functions on the boundaries of my program, it's remarkably difficult to get the rest of my code to compile and not do what I was intending it to do.

It's not magic - the compiler doesn't guess at what I want - but it's close enough that I'm comfortable making larger changes than with Python.

Buttons840• 10 years ago

I've had the same feeling. I have often though to myself "I do not know how to break this code. Even if I purposely tried to get this code to throw a run time error, I could not."

pka6• 10 years ago

Leaving Haskell's deferred type errors aside for a moment; after you are done with your partially working code, what then? Relying on (the surely preexisting) 100% test coverage?

kazinator16• 10 years ago

Say you have 65% test coverage over the refactored code. You have that before the refactoring and you have it after.

What does it mean? It means 35% of the code is being refactored without coverage. Its status is "unknown" before refactoring and "unknown" after.

Oh, but it compiles!

ionforce• 10 years ago

If lots of things change though... Couldn't that be a smell of bad boundaries? Like the change would not have been so drastic with the right abstractions?

kazinator16• 10 years ago

It could be the smell of bad boundaries. But sometimes a data type can have many functions that work with its internals. The 75 functions might not sounds so bad if there is a good justification for their existence, and there are 7500 other functions which don't have to change which rely those functions.

At some level we need to actually implement things; we can't have abstraction all the way down!

I'm also reminded of Alan Perlis Epigram #9: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."

skybrian• 10 years ago

You can still do experiments with some adjustment to your work habits: create the new data structure with a different name, try it it out in some places via renames, and if necessary comment things out. Delete the old one and rename back again when you're done.

The one that still annoys me, though, is having to comment out unused imports in Go. I agree they should always be removed before submit, but in the meantime it's annoying. (Tools that automatic remove unused imports are also annoying in their own way.)

moosingin3space• 10 years ago

`goimports` both adds and removes imports. It has been excellent to run on save in emacs for me.

naasking• 10 years ago

The dynamic world lets us have inconsistent code that partially works, rather than forcing us to follow up on all the places reached by the ripple effect of a change. Maybe the change turns out to be a bad idea in the end; then it was a waste of effort.

Except you can mostly do this reasoning just using the types. So you're rarely updating everything, you're just updating a few type definitions to see if your idea meshes well, and then you have to do the large refactoring.

dllthomas12• 10 years ago

I find, when I'm refactoring, that well designed types often lets the compiler help me think ahead of my code, and spot subtle contradiction before I've "reached" them in terms.

MaulingMonkey• 10 years ago

A truly large refactoring is one that you want to try long before getting everything to compile.

For me, from experience, this is a sign that I or my coworker is creating an unreviewable tarball of a diff that will break something, can't be incrementally rolled back, and may not be easily fixed.

A truly large refactoring is one that I want to break up into individual smaller refactorings, each with a more reviewable diff, aided by automatic refactoring tools that help offload my workload. While it's possible to perform enough static analysis of dynamically typed source code for such tools to exist and aid the programmer, they're less likely to be as rock solid as statically typed source code refactoring tools, for something as simple as a method rename.

igouy• 10 years ago

A truly large refactoring is one that you want to try long before getting everything to compile.

That's a luxury that may not be available :-)

Here's a case study of industrial use of a refactoring tool provided with a dynamically typed language:

…

"A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns.

…

Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary.

…

A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer.

…

A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation.

…

Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period.

…

If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules.

…

The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36."

…

from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002

pron• 10 years ago

I’m convinced that functional programming vs imperative programming is a much more important concern...

Except that you can be functional and imperative, like Clojure, Erlang and even OCaml. Those are functional-imperative languages (you can even be pure and imperative yet not functional, like Esterel and maybe Céu).

gelisam• 10 years ago

Does this mean I’m a static typing zealot now ?

That part made me chuckle, because it made me realize that a typing zealot is exactly what I've become! I used to be a huge fan of Ruby and I was using it for everything, and now 10 years later I'm a huge fan of Haskell instead, I give talks about how JavaScript is horribly error-prone and how Elm is so much better because not only is it statically-typed, it's also purely-functional, and yadi-yadi-ya.

The transition occurred so smoothly that I didn't notice it. Beware! You might not be a zealot now, but if you continue on this path, you might unintentionally become one too :)

jlgray• 10 years ago

As you said, this is slowly happening to me. I love being able to prototype systems in languages like python and matlab, but at the same time it feels like building on sand, especially when other developers get involved.

Recently while writing some matlab found a great example of the sort of trouble a decent type checking system will save you. The predict method takes a machine learning model, and applies it to some data. The output however, is up to the model, and not all models return the same data type (e.g. column vector of doubles, or cell array of strings), making it a pain in the butt to do any abstraction over models.

I am excited for languages like lbstanza that let you have your cake and eat it too.

cutler2• 10 years ago

Whatever Scala devotees may believe, I think FP and OOP are irreconcilable opposites which means if you really get into FP you're going to hate doing what the industry demands of you in your day job, ie. OOP. Maybe FP will one day become the dominant paradigm but until then treat it like alcohol - too much and you cease to function in the "real" world. I exposed myself to Rich Hickey's Sermons From The Hammock a few years ago and Clojure just blew my mind. Since then I get a churning feeling in the stomach when I have to dealt with OOP, ie. Ruby, PHP, Python. The effects are irreversible so beware - immutable data mixed with pure functions is a potent drug.

acjohnson552• 10 years ago

FP and OOP are totally not irreconcilable. Most modern languages offer both, including the most highly used dynamic languages. Javascript really cracked this nut with its function expressions. Scala's main contribution here is applying static typing to that mixture in a largely coherent way.

bunderbunder4• 10 years ago

It might be the particular OOP languages you're used to. Ruby, PHP and Python all have pretty strong cultures of relying on mutation and side effects. Part of this might be that the languages are so focused on dynamicism that you really have to go out of your way to even suggest that a type should be immutable. And even then it's only a matter of time until someone mucks it up by applying a mutable mixin to the type.

In other languages, though, it's perfectly possible to create immutable classes that can only be interacted with using pure functions. You could argue that it's questionable that these do anything you can't do better with records and algebraic data types, and there's something to that. But there's also something to taking the pragmatic approach and not letting the (presumably) remote odds of your employer adopting a niche language prevent you from writing better code.

z1mm32m4n• 10 years ago

Whatever Scala devotees may believe [...]

It's not just Scala devotees. You should read [this post][1] by Alan Kay (Turing Award winner, designer of Smalltalk). He arrives at a conclusion that might surprise you:

I think of "objects" and "functions" as being complementary ideas and not at odds at all.

Largely what you may be referring to is the idea of purity and mutation being at odds, but these are orthogonal from the ideas of OOP and FP.

A very interesting read.

[1] https://news.ycombinator.com/item?id=11808551

Learning Elm has gotten me confused about static vs. dynamic typing

Related Stories