Scala Best Practices

https://github.com/alexandru/scala-best-practices/

by century19 • 12 years ago

83 82 12 years ago

bad_user19• 12 years ago

Hello, I'm the author of this list - I compiled it primarily for my colleagues, but hopefully it will help others.

It isn't complete yet and I have more stuff to add and some stuff still needs clarification, like Rule 4.3 (just created an issue for it).

If you have suggestions, feedback, things you disagree with, I'm happy to make edits or to accept issues and pull requests :-)

century193• 12 years ago

Hi bad_user,

I seen this on Twitter and just by scanning it quickly I learned some things, so that's why I posted it here and thanks for sharing.

Can I ask about rule 5.2. SHOULD mutate state in actors only with "context.become"?

How would you use become to avoid using vars to keep state in an Actor like this?

class Account(var account: AccountDetails) extends Actor { def receive = { case msg: UpdateAccount => { account = msg.account } } }

wernerb• 12 years ago

Actors are good at encapsulating state, therefore using mutable variales inside an actor through is very natural, and one might even question if an actor is needed if it does not have any mutable state inside it.

However, 5.2 does not concern the elimination of vars, but rather better management of multiple states (Finite state machines)

Your example, just like the example in 5.2, only has a single state, so no context.become is strictly necessary. The author makes clear that when you have multiple states, the collection of top-level mutable vars will be hard to maintain. I had this problem a few times and I am glad this gets mentioned in the guide.

I agree though, that a better example would include at least two states.

bad_user19• 12 years ago

As @wernerb already pointed out, the idea is to use context.become such that inside your actor the only side-effects happening are due to context.become, otherwise the data structures are left immutable, the helper functions are left pure, etc... and it becomes easier (less error prone) to manage complex actors that way.

Another side-effect of this for example is that you're not going to worry about vars that leak into Future computations, which is a mistake that even experienced developers are doing, because it's an easy one to make.

Here's your sample translated into a state-machine with 2 states: https://gist.github.com/alexandru/63eebe8e49e796f31e73

netcraft• 12 years ago

Just as a matter of style I would put a comment like //dont do this or //bad practice in the negative examples, so that it stands out better.

bad_user19• 12 years ago

Thanks for the feedback, will do :-)

https://github.com/alexandru/scala-best-practices/issues/14

lmm3• 12 years ago

1.2 I disagree with your stance (and your misuse of "NP-complete" in justifying it is a pet peeve; it is a technical term that has a specific meaning). IMO manually maintained formatting is not worth the cost; being able to run an autoformatter whenever code is unclear is a huge benefit.

2.4 I would caveat that useless traits are useless in application code, but make sense in libraries published for use by other teams.

2.5 Disagree. The general contract of equals and hashCode does not demand that they are immutable; there are a wide variety of mutable java classes that offer similar equals and hashCode. Other aspects of the case class sugar can be very useful even for a mutable type. The correct solution is to avoid equals and hashCode in general (they're broken by default anyway); use something like scalaz's Equal, and don't define Equal instances for mutable types.

2.7 You should explain a bit about how to achieve the same results using e.g. scalaz.\/ and for/yield.

3.1 I find the arguments here very weak.

3.4 Stream has many problems and Iterable is hard to reason about. I think the suggestion to use lazy collections is bad advice. Temporary intermediate values ought to be caught by escape analysis and not cause GC pressure at all. I have never seen hard evidence of such values causing a problem in a real-world program. 3.3 applies.

4.2 It's worth mentioning the advantages of scalaz Future (more sensible handling of ExecutionContexts, also theoretically higher performance because of having fewer task boundaries).

Also I would emphasise that an actor with no mutable state is almost always a mistake (it should usually be replaced with use of Future or similar), as this is a common anti-pattern I see in newcomers' code.

4.3 It's reasonable to put actually heavy computation on dedicated threads (so that I/O threads can pre-empt compute threads). So there are times when this does make sense. You would certainly expect an explicit ExecutionContext in that case though.

5.2 Disagree. become is harder to reason about than ordinary mutable state (var and friends), as your example demonstrates. Actors encapsulate mutable state, that's their raison d'etre, if you're using them at all you've already given up on purity or referential transparency.

bad_user19• 12 years ago

1.2. I do not agree and I tried justifying that with samples. If you want to use auto-formatters, that's fine, but I find it disrespectful to reformat the code of other people that know what they are doing.

2.4. You missed my point. I didn't say traits are useless, but that people shouldn't define traits that are useless.

2.5. That rule includes a reference to a paper from 1993. And yes, the contract of equals and hashCode is broken. People should strive for sane equality, not a sane Object.equals, because there is not such thing. If that's not enough to convince you of it, because syntactic sugar, we'll have to agree to disagree but to be honest I refuse to work with code violating such a fundamental rule.

2.7. Thanks for the feedback, will do.

3.1. Point taken, will add more details.

3.4. What problems does Stream have? Also for me, those operators make much more sense in the context of lazy collections - that Scala's library does such a shitty job on lazy collections, that's a pet peeve of mine. On "escape analysis", I hope you're not relying on it because it doesn't work. The algorithm implemented by the JVM is very limited.

On evidence, I have, countless of times. But you know, search for the "mechanical sympathy" google group and ask there. Rule 3.3 applies to doing optimizations, but that doesn't mean one has a free pass to do stupid shit, ignoring CS 101. I personally don't see a conflict there.

4.2. I'm not familiar with Scalaz' Future. Will take a look.

On actors with no mutable state, I agree, but I also see another problem in that simple actors with mutable state could be modeled without state, as Futures (or similar) as well. I find it tough to explain to people when Futures should be used instead of Actors. I find it tough to make that choice myself. That's why this rule doesn't try to impose something specific - its purpose being to make people think about tradeoffs and not use something blindly.

4.3. This rule already has an issue started by me, as a reminder to myself that I did a shitty job - https://github.com/alexandru/scala-best-practices/issues/8

5.2. I think you miss-understood that rule. I'm not saying that actors should not mutate their state, but I'm saying that it is better to control the scope of those variables by means of context.become in combination with pure functions. You know, the Erlang way.

Also, actors are really NOT about mutating state willy-nilly, sorry but that misses the forrest from the trees. Actors are about bidirectional asynchronous communications, non-determinism and state machines. And as always, just because mutation happens, that doesn't mean you can't limit its effects. And it's actually really, really advised to do so on the JVM, in the presence of Futures that can capture the actor's internal state and so on and so forth.

orp• 12 years ago

Hi,

Quick comment on item 2.3: SHOULD NOT update a var using loops or conditions.

you conclude that this expression is the best option: val sum = elements.map(_.value).sum

This expression is not equivalent to the previous options in performance and memory, as the map expression will create a entire new sequence.

For large sequences, a better option is to write: val sum = elements.view.map(_.value).sum

jxm2623• 12 years ago

This is so awesome! I've been slowly learning Scala and was looking for a "Effective Scala" type book.

Thanks for making this!

rbehrends5• 12 years ago

Just keep in mind that it's an opinionated list. I routinely and intentionally violate several of these rules and have no intention of stopping that.

Much of it will depend on your application domain and needs. For example, if you're dealing with matrices, avoiding destructive updates will only give you a performance hit without an increase in maintainability.

bad_user19• 12 years ago

Hello @rbehrends, I marked rules that can be broken with SHOULD, in case you know what you're doing, versus rules that shouldn't be broken that are marked with a MUST. So optional versus non-optional.

I'm not against destructive updates, but I'm against doing destructive updates without knowing what you're doing. I'm also against usage of shared state without proper encapsulation and synchronization.

And when a developer makes such a problematic choice, it has to be able to defend it with arguments other than "couldn't think of anything better and we've got deadlines". I hate that argument with a passion, time is somehow always used as an argument for doing stupid shit, technical debt is then increased and then sprint after sprint you're left cleaning up stupid shit from previous sprints.

Yes, I'm very opinionated :-)

jxm2623• 12 years ago

Which section are you referring to? 2.1, 2.2 maybe?

Sorry of for the simple question, but I think I'm missing something. Wouldn't avoiding a destructive update give better performance? And the trade-off would be higher maintainability? With a mutable structure, you're 'destroying' just reusing the same locations in memory and just replacing, versus say an immutable structure returning a completely new structure (new memory allocations).

But to your main point, yes I won't take this as the canonical source of coding standards. I just like to read through stuff like this because it gives me alot of areas to explore and research. Like why do this vs that, what's the tradeoffs, etc..

<<edit>> Whoop's looks like i have my logic/terminology flipped! Saw this on SO which cleared things up a bit.

http://stackoverflow.com/questions/6964233/what-is-a-destruc...

Also just noticed the + vs += in some of the mutable collections, which helps to explain.

brudgers3• 12 years ago

Scala affords imperative place-based Von-Neumann style programming by design, because it is intended to ease the pain of working Java programmers not create more. See Odersky's Scala by Example [1].

He follows "show don't tell"[2] to meet people where they are. Odersky recognises that marginally better is still better Scala was designed pragmatically.

To an outsider, lists like this make it appear that entering Scala's language community is hard. There are just so many mores. [3] This is unfortunate because via Odersky's Coursera class, Scala has one of the best onboarding processes among contemporary languages.

[1] http://www.scala-lang.org/docu/files/ScalaByExample.pdf

[2] https://en.m.wikipedia.org/wiki/Show,_don%27t_tell

[3] http://learncodethehardway.org/blog/AUG_19_2012.html

talonx• 12 years ago

Um...It's "Odersky"

bad_user19• 12 years ago

Hello @brudgers,

The intent of this list isn't to scare people or to make them follow my conventions.

When working within larger teams, regardless of language, you need a list of do's and don'ts. And powerful or unfamiliar languages are especially problematic for rookies - because they need to make choices (e.g. what abstraction should I use to model this or that and before anybody mentions Python's philosophy - as an ex-Python developer I must say that it has led to horrible non-orthogonal choices).

I do not believe in imposed choices - mimicking what other people's are doing without understanding the reasoning behind it can lead to really awful hairballs, as anybody that worked within an enterprise-ish team can attest to. For example in this rule I'm trying to undo the damage done by another "best practice" that people are mindlessly following - https://github.com/alexandru/scala-best-practices/blob/maste...

Also many rules in that list are not necessarily related to Scala - does a language protect you from horrible, convoluted and big functions that do too many things? Of course not. Does a language keep from storing dates without timezone info? Of course not.

It's a work in progress though, feedback appreciated.

brudgers3• 12 years ago

I know you are, as Hanselman says, "helping people fall into the pit of success". I guess the question is, can this list really be highly opinionated and canonical and remain practically useful?

The criticism of "best practices" lists is that everything becomes a "core principle". By which I am suggesting that "best practices for Scala" encompasses both Scala and everything in programming. Everything except, unfortunately, the prime directive: get something working first. Which in the moment often means ignoring best practices and just writing code.

There are places where Java date/time are appropriate because that is both sufficient and the convention...and because the ways in which JODA time can accommodate obscurities derived from esoteric abstractions are manifest. JODA is a best practice for people who know JODA well and who can reasonably predict that future maintenance will be done by qualified persons, ie. people who know Java well, and not necessarily Scala.

Being a library on top of Java, Scala inherits best practices from it. Acknowledging these as such and handling them with a pointer will keep the list it DRY...or rather DRO [don't repeat others].

I am thinking that in programming, the form in which best practices are embodied is not a list because lists evolve to wiki's as corollary to Greenspun's 10th rule [1]. I think best practices are better implemented as frameworks because frameworks can be highly opinionated without being open for debate.

bkeroack3• 12 years ago

Seems like an awful lot of rules to hold in one's head. Scala just looks like a giant tumor of complexity to me.

sumnulu• 12 years ago

To be fair these are not rules, just best practices, like naming conventions.

i.e.

   > 2.1. MUST NOT use "return"
   > 1.1. SHOULD enforce a reasonable line length
   > 1.5. Names MUST be meaningful 
   *(x, xs, xss, ns, are meaningful names by the way)*
   > 3.3. SHOULD NOT apply optimizations without profiling

...

bad_user19• 12 years ago

What other people find complex, I personally find refreshing. Could you please describe why it looks like a giant tumor of complexity?

There's a difference between true complexity (i.e. complecting or that lead to accidental bugs) and things just being unfamiliar. I try to stay away from the former, while not shying away from the later - since I've picked up Scala about 3 years ago, my knowledge expanded with a lot of useful abstractions that help me do my job better, abstractions that weren't natural or popular or properly implemented in the other languages I worked with. This isn't to say that Scala doesn't have true complexity in it, but then again, what language doesn't?

bkeroack3• 12 years ago

a) Martin Odersky has admitted as much[1].

b) The compiler internals are reportedly a mess[2]--likely due at least in part to the extreme number of different syntax variants.

c) I find rule 4.1 in OP interesting ("SHOULD avoid concurrency like the plague it is"). According to Odersky's Coursera course[3], one of the main advantages of functional programming as enabled by Scala is easy, safer concurrency (reduced/absent mutable shared state). Yet here we see a presumably seasoned Scala veteran basically saying that this is a failure and we should avoid concurrency whenever possible because it results in too many bugs.

1. http://www.infoworld.com/article/2609013/java/scala-founder-...

2. https://www.youtube.com/watch?v=TS1lpKBMkgg

3. https://www.coursera.org/course/progfun

ionforce3• 12 years ago

As you get more experienced with Scala, you begin to develop better habits and notice more patterns and you will throw away a lot of the unneeded complexity.

Scala can be complex (on paper), but in practice it doesn't have to be.

the_af3• 12 years ago

When you think about them, a lot of those rules aren't Scala-specific (though some are). So you could just as well say "most programming languages are giant tumors of complexity".

For example, if you write Java, most (or similar, or even more complex) rules apply.

bkeroack3• 12 years ago

"Java is just is bad" is not a ringing endorsement.

danielbarla3• 12 years ago

I'm not very current on the Scala tooling - is there a reasonable lint tool which enforces these types of suggestions?

tempodox3• 12 years ago

I think it's commonly called your brain.

virtualwhys• 12 years ago

Brian McKenna's Wart Remover[1] is an excellent tool for capturing silent (but deadly) errors at compile time.

Can also turn on various scalac -X flags to, for example, turn warnings into compile time errors. Invoke scala REPL with -X to see available flags.

[1] https://github.com/typelevel/wartremover

danielbarla3• 12 years ago

Thanks, very useful.

It's nice to see that both Wart Remover and OP's post had reasonable explanations and examples to justify each rule. This makes the rule all that much better as a learning experience (contrast with, e.g. "I have never seen a piece of code that was not improved by refactoring it to remove the continue statement" from JavaScript: The Good Parts, which leaves me baffled and no wiser).

adriaanm2• 12 years ago

We've been working on https://github.com/scala/scala-abide. Our aim is to provide a nice platform for others to develop and distribute style checking rules. We (the core scalac devs) intend to focus on keeping the abide platform up to date with the compiler, letting the community evolve the set of rules.

Due to resource constraints, we haven't made a big push in advertising it yet, but we're happy to work with you if you'd like to start implementing rules. Have a look at recent PRs for some examples.

danielbarla3• 12 years ago

Thanks for the info, and kudos on the good work in general!

Abide looks quite nice, although I'm a very occasional user of Scala so far, so I'm not sure I'd be very good at coming up with and implementing rules. I'll definitely spread the word to some friends who use Scala more frequently, and may be able to contribute.

ionforce3• 12 years ago

IntelliJ's Scala plug-in has some support for things like these.

tempodox3• 12 years ago

I'm not so sure what to think of Scala as a language, but I think the naming and formatting recommendations in this document have merit beyond a single language.

There are only two hard things in Computer Science: cache invalidation and naming things.

Truer words were seldom written. Just compare the naming culture in Common Lisp with that of OCaml, for example. Worlds of difference.

And don't even get me started on the “standard” way of formatting in C family languages...

nvader• 12 years ago

There are only two hard things in Computer Science: cache invalidation and naming things and off-by-one errors.

rtpg• 12 years ago

I've heard this a lot but is cache invalidation as hard of a problem as , say, distributed systems like bitcoin?

tempodox3• 12 years ago

I thought of a single process as the frame of reference. Once you enter distributed systems, complexity (and consequently hardness) can achieve arbitrary levels.

jrobn2• 12 years ago

Scala Best Practices:

1. Don't use Scala 2. Rule #1 is blatant opinion.

bad_user19• 12 years ago

Hello, I'm the author of this list, which I hope to grow to be something more than a single-author thing.

Until now I've done non-trivial amounts of work on the job in Java, C#, C++, PHP, Python, Perl, Ruby, Javascript and Scala and I've played in my free time with half a dozen others, like Scheme, Clojure, Rust and Haskell.

Take what you will from this, but I found Scala to be the most sane thus far. My opinion may change in the future, since I don't really have loyalty for programming languages, but regardless - I find it more constructive and better for your knowledge to verbalize your dislikes with clear arguments, maybe there's something you're missing.

jrobn2• 12 years ago

I don't have any particular loyalty either. But, in my opinion, and after taking considerable time looking into scala, it's not my preferred language. The syntax can quickly devolve into soup. It's entangled with Java too closely (unlike say clojure). And last time I tried it compile times were...awful. All of this is my opinion and if scala works for you that's great. This guide is a great start for scala devs not to shoot themselves in the foot.

ajones• 12 years ago

I'm a big fan of the guide that Twitter put together titled Effective Scala: http://twitter.github.io/effectivescala/

For those who want to learn Scala, Twitter has also put together a set of lessons called Scala School: https://twitter.github.io/scala_school/

bad_user19• 12 years ago

Useful resource, but as often happens with lists of best practices, I do not agree with some things Twitter is giving advice on.

For example - https://twitter.github.io/effectivescala/#Control structures-Returns

They are saying that "return" can be used for "reducing nesting". First of all, that's a lie - if a return exists in that code, doesn't mean that the branch no longer exists, it only means that it isn't explicit and far more effective for reducing nesting would be to break those functions in smaller ones. I happen to believe that usage of "return" in Scala is never appropriate and it was a design mistake; at the very least they abstained from adding "break" or "continue". Scala is an expression oriented language and really, if you're going to write Scala as if you'd write Java, then just pick Java because Scala, contrary to other people's opinions, is not a better Java in the sense that it actively works against you if you're doing things like that.

I tried outlining the rationale in this rule, which I consider to be non-optional (hence the usage of MUST, instead of SHOULD :)) - https://github.com/alexandru/scala-best-practices/blob/maste...

chrisloy• 12 years ago

This is a really helpful guide, which I can see myself coming back to a lot.

I can't express how relieved I am to see someone else directing Scala developers away from the Cake Pattern. I've only ever seen it result in messy, confusing code.

jfb7• 12 years ago

Just as C++11 looks worlds different (and better) than C++ circa 1990, I imagine that the same dynamic will play out with Scala, which I tend to analogize as Java++. Certain idiom will develop, certain language features will be deprecated but by nature of the platform will not be able to be removed; &c.

I find some idiom used by the more experienced Scala devs on my team to be distasteful: I hate hate hate the semantic confusion between the Option type and sequences that leads people to map over Options (or the dual use of is/nonEmpty). I would always prefer pattern matching, or getOrElse, for instance.

I think I'm just a fan of smaller languages; I certainly understand -- and commend -- Dr Odersky's approach to a pragmatic extension of a (bad) industry language, just as I appreciated Stoustroup's; but given my druthers, I'd just as soon be working in something perhaps less pragmatic but certainly smaller and less warty.

If anybody knows what language that is, please tell me. Or invent it.

bkirwi2• 12 years ago

You might prefer Kotlin?[0] It's going in the same direction as Scala, but not quite so far.

By the way, I'm curious what you mean by "the semantic confusion between the Option type and sequences that leads people to map over Options". Mapping and 'flatMapping' are very general things in Scala, and apply to many types that are not collections, like Futures / Observables / etc.

[0] http://kotlinlang.org/

llemiengre• 12 years ago

Honestly Kotlin has 80% of the complexity of Scala, 20% of the functionality, almost no library ecosystem and 0 stable releases. I don't know why so many people consider it a serious choice.

If you want java with type inference you can use Xtend they have had many stable releases and it has excellent tool support.

jfb7• 12 years ago

I think I'd be more interested in Haskell-- than Kotlin; I am not a fan of statically dispatched OO language implementations. I am (perhaps unsurprisingly, given my employer) much more a fan of Smalltalk derived languages when writing in an OO style.

jfb7• 12 years ago

It's unfamiliarity with the idiom that makes me uncomfortable, to be perfectly frank. I'll get used to it.

I won't get used to how alien and ultimately weak Scala's support for algebraic data types is, and that's not changing. This comes down to the affordances that Odersky and the Scala team have decided to favor, which is OO flavored in a way that I don't like. I vastly prefer the ML style:

  data Foo = Bar s | Baz

to the sealed class/case nightmare of Scala. Again, this is not a technical nit -- this is an idiomatic and personal one.

bad_user19• 12 years ago

I hate hate hate the semantic confusion between the Option type and sequences

Option is a Monad in Scala and could also be modeled as an Applicative Functor - which are very generic abstractions that transcend sequences. Don't let the naming scare you, you can view them as design patterns, it just so happens that people aren't familiar with them.

In Scala mapping, filtering and flatMap-ing are generic and for example Scala's for-comprehensions work on all things implementing map, filter and flatMap including things that are not sequences and for good reason.

And most importantly - `Option` would be completely useless without mapping, filtering or flatMap-ing on it. Because often you have to combine multiple options and what happens in practice, without an easy to use API, people would just do "Option.get" and trade a null pointer exception for a NoSuchElementException, which would defeat the purpose of using Option.

The semantic confusion between an Option and a sequence is the same confusion between a sequence and a Set - none and if there is, then you need to go back to the basics. First of all, because Option really is a collection of zero or one elements and second because all of them are Monads.

I imagine that the same dynamic will play out with Scala, which I tend to analogize as Java++

I don't know what makes people say things like this. Scala is nothing like Java and I worked with C++ and with Perl too, in production. Scala has some warts in it, but such comparisons are unwarranted.

jfb7• 12 years ago

Thank you, this is very helpful. I tend to think of the Option/Maybe type as signaling to the reader a SQL-like 3VL value, where the distinction between having a thing and not having a thing is not contingent on the thing-in-itself. Clearly, I need to lift my level of thinking up at least one level of abstraction.

I don't know what makes people say things like this.

It's a social and not a technical analogy (although I think the technical analogy is stronger than you apparently do). C++ took an established industry language and opened the door to a whole new level of abstraction; just as you can write C in C++ (missing the whole point), you can write Java in Scala (again, missing the point). C++'s affordances to prevent this were weaker than Scala's, which means that the acceptance of Scala's idiomatic way will probably happen much faster than that of C++'s did.

It wasn't intended to be a derogatory analogy; I admire modern C++, even as I am happy not to be writing it.

ionforce3• 12 years ago

`Option` would be completely useless without mapping, filtering or flatMap-ing on it.

This position is a bit extreme. It's fine for pedestrian use to use pattern matching or isDefined.

lmm3• 12 years ago

Just as C++11 looks worlds different (and better) than C++ circa 1990, I imagine that the same dynamic will play out with Scala, which I tend to analogize as Java++. Certain idiom will develop, certain language features will be deprecated but by nature of the platform will not be able to be removed; &c.

I think popularity (particularly with the enterprise) ossifies a language; it's too late to fix certain mistakes. Certainly this is the case for Scala already. But don't underestimate the difference between 5 years of cruft and 25 years of cruft.

I find some idiom used by the more experienced Scala devs on my team to be distasteful: I hate hate hate the semantic confusion between the Option type and sequences that leads people to map over Options

That's not confusion but enlightenment. Options aren't Sequences, but both are Monads.

I think I'm just a fan of smaller languages; I certainly understand -- and commend -- Dr Odersky's approach to a pragmatic extension of a (bad) industry language, just as I appreciated Stoustroup's; but given my druthers, I'd just as soon be working in something perhaps less pragmatic but certainly smaller and less warty.

You might like Idris - it looks a lot like a slimmed-down, more principled Scala with the benefits of lessons learned. Of course the price for that is that it's less mature and has a much smaller community.

devondjones• 12 years ago

First best practice? Don't use Scala.

vkjv• 12 years ago

"And if a public API is not thread-safe for some reason (like the usual compromises made in software development), then state this fact in BOLD CAPITAL LETTERS."

I prefer to use the `Unsafe` prefix for this, to make it abundantly clear without looking at the call. E.g., `updateUnsafe`.

I also use the `Unsafe` suffix when defining an HTML template rendering method that does not do escaping.

century193• 12 years ago

Hi again bad_user,

Glad your collection of best practices got the attention and discussion they deserved today. I notice you posted them to HN 8 days ago and nothing. Unpredictable eh?

One more request, with your best practice "3.2. MUST NOT put things in Play's Global" - could you add an example of the correct way to "come up with your own freaking namespace"?

I assume putting authentication in there like SecureSocial does is ok?

Scala Best Practices

Related Stories