Learning Go by porting a medium-sized web back end from Python

http://benhoyt.com/writings/learning-go/

by benhoyt • 9 years ago

242 192 9 years ago

I'm at the same spot myself, trying to learn Go by building a simple web app. Your article was a refreshing read to me.

One extra thing that is the major motivation for me to go to Go is the ability to deploy as a single God damn binary and not deal with dependancies and pip. pip and counting on OS repositories may be the right thing to do, but it is just too cumbersome. I have an air gapped setup at work, and there is no easy way for me to deploy Python applications - because all projects just say pip instal this and pip install that. It would make my life so much easier building something on my laptop, scp the binary to servers and expect them to "just work".

pletnes• 9 years ago

I don’t disagree, but FYI pip has a —cache flag to use packages from file, and the files could be on a usb stick of course. Same approach can be used to speed up CI setups.

proto-n3• 9 years ago

Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install anaconda and setup / clone environments, also it can manage python versions independently from the system's python.

[1] https://conda.io/docs/user-guide/tasks/manage-environments.h...

reacharavindh5• 9 years ago

Thanks for your comment.

I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one.

Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?

jnwatson• 9 years ago

I'm in a similar situation with Python and deployment that has no internet. We have a solution that works well.

The philosophy is to think of Python as a compiled language and then everything works.

The CI system make the "image" and that gets pushed. In our case we make a virtualenv but you could also use pyfreeze.

Works like a charm.

TheAceOfHearts2• 9 years ago

Surely there's tools to bundle python apps? I'm not as familiarized with the python ecosystem, but with node you can use browserify or webpack to bundle an app into a single JS file, or pkg [0] to generate a single executable.

Some languages have even more powerful tools for that sort of deployment. One of the best examples I've seen is distillery [0], an Elixir package which allows for hot upgrades. That means you can just copy a single a file and upgrade without having to take anything down.

[0] https://github.com/zeit/pkg

[1] https://hexdocs.pm/distillery/getting-started.html

trumpeta• 9 years ago

Recently I have had the opportunity to write a microservice in Go. It was a very refreshing experience switching from Scala. Go is a lot faster to compile and runs with a far smaller footprint. The channels are nice. On the other hand the testing feels wrong because its so annoying to do mocks. And don't even get me started on the dependency management.

Overall Go feels to me like some version compiled PHP with better concurrency support.

egeozcan4• 9 years ago

compiled PHP with better concurrency support

...and simpler syntax and without the quirks. But that's one of the best descriptions of Go I've seen so far.

cies2• 9 years ago

Indeed. Go and PHP are at the complete opposites when it comes to quirkiness. And syntactically PHP is at a very noticeable disadvantage.

oblio5• 9 years ago

I really wish that there was a language which had these features:

- simple (so not Scala, Haskell, Perl 6, etc.; no Rust either, unfortunately)

- clean (nice syntax, preferably Python inspired, but consistent)

- modern (generics, some functional features, string interpolation, etc.)

- decent concurrency/parallelism story

- good IDE, preferably supported by the core dev team

- compilation to a (possibly static) native binary

- decently big ecosystem

- and a big enough community of contributors or commercial backing from at least 1 stable entity

- also preferably didn't have a lot of historical baggage

From what I gather C# might actually be close to this, 2-3 years from now, if the .Net Core transition happens smoothly and the OSS community adopts it... Who knows, maybe even Kotlin 2019 or so?

jhh• 9 years ago

Kotlin comes to mind.

cjsuk2• 9 years ago

A lot of people I know are bailing on .net because it doesn’t live up to the promises. Everyone else gets the cool stuff first, the vendor that owns it is a dick and there has been so much schizophrenia over the last few years that it’s a business risk adopting it in case you have to throw your product under a bus again.

Go is showing itself to be a good compromise especially with the simplified nature of deployment and runtime stuff. It’s like a safer, higher level C, with a decent runtime library. That has so much mileage.

I’ve learned one thing as well over the last 15 years of using .net as well. An IDE is a monolithic brain damager. If you have to do something outside the scope of it, you’re shot. Give me an extensible text editor and shell.

pjc50• 9 years ago

C# actually already has most of that except for "compilation to a (possibly static) native binary", which is really just a convenience for shipping. It's also not really very cross-platform, it's definitely a "Windows First" language.

mfontani• 9 years ago

Other than the compilation to binary, Elixir seems a good fit for what you're looking for

masklinn3• 9 years ago

OCaml seems to match a majority of the criteria, though AFAIK no "good IDE" (tooling like merlin is available, but if you're looking for e.g. a refactoring IDE you're probably SOL), the ecosystem is small, and it does have some historical baggage.

lenkite• 9 years ago

Believe kotlin (especially kotlin-native) should soon relieve all those itches.

oweiler2• 9 years ago

Kotlin covers all those features

- much simpler than Scala while still being much more expressive than Java

- clean syntax, with a higher consistency than e.g. Java

- Generics, Higher Order Functions, Multiline-Strings, Destructuring, ADTs, Pattern Matching

- Coroutines (experimental but production-ready)

- IntelliJ Community Edition (or Ultimate)

- not yet but will be possible with Kotlin Native

- has its own ecosystem but also has great interoperability with Java

- stable for quite some time, big community

- well, a little bit for reasons of Java-interoperability; on the other hand

beagle34• 9 years ago

Nim has everything nailed down except the ecosystem/commercial backing; It does have some of those, but not to the extent that I would blindly tell you they are there.

It's as fun as writing Python (with a similar syntax), but it has essentially all the goodies you want from Lisp when you need them, runs as fast as equally optimized C, produces standalone native binaries, _or_ standalone JavaScript if that's your thing.

nicoburns• 9 years ago

Swift is a potential competitor in this space too.

s_kilk• 9 years ago

Take a look at Elixir, it covers most of what you want.

squeaky-clean• 9 years ago

I think my dream language would be C# but with a slightly more Pythonesque syntax (significant indentation, no semicolons or braces...). I wonder if a transpiler like that exists.

hawski• 9 years ago

C38 (so when the world ends):

- arrays with slices (that could also be used for strings) and Cilk Plus array notation [1]

- maybe actual string type with separate concatenation operator (possibly space as awk uses and C for literal strings) it could probably be folded to improved arrays

- standard SIMD vector types so that most use for operator overload disappears

- cleaned up standard library

- actual macros so the remaining use for operator overload disappears

- module system (so you would generally not need make and could easily compile to static binary)

- already you can build with static musl libc [2]

- something like libdill [3] or libmill [4] in stdlib

- there are multiple good IDEs, debuggers, analyzers

- one of biggest ecosystems and communities

- historical baggage mostly cleaned up

[1] https://en.wikipedia.org/wiki/Cilk#Array_notation

[2] https://www.musl-libc.org

[3] http://libdill.org/

[4] http://libmill.org/

dilap• 9 years ago

seems like Go 2 has a good chance of being this (at least if you consider its syntax clean, which I do).

basically go is missing from your list is generics and string interpolation.

hajile2• 9 years ago

I think F# has the things you're looking for.

didibus6• 9 years ago

By simple, you mean easy?

What about Java? Its easy, modern in the way you describe, has a good concurrency story, multiple great IDEs, can compile to a static binary with commercial JVMs, or experimentally in OpenJDK9. Really big ecosystem. Huge commercial backing. It just maybe fails at historical baggage and syntax.

Kotlin fixes those last two. And Java 10 will also fix a good chunk of the syntax complaints.

In all fairness though, I think you're focused on nitpicks that have no practical benefits. You're asking for a language with a better user experience. Which is great, as it can make programming more pleasant, and I'm all for it.

That said, I'd focus on language changes that provide functional value. Haskell adds a powerful correctness layer at a low expressiveness cost. Go adds a useful CSP concurrency mechanism with green threads, as well as being friendly to containerisation. Erlang and co add a great deal of reliability features and massive parallelism with first class actors. Clojure gives you powerful metaprogramming, state of the art immutability and a similar CSP layer as Go, while maintaining the access to Java's large ecosystem. Kotlin adds null type checking, and better functional primitives. Rust gives you safe memory access without the performance overhead of a GC. Chapel allows for super parallel computing. Scala gives you OOP with the safety layer of Haskell on it. Nim gives you faster python, with type safety. Crystal gives you faster Ruby with type safety. Etc.

Those are the languages that challenge the status quo, and have a chance at bringing real value in terms of the average output they result in when used to program with.

A slightly better UX for Java or Python is a hard sell. Its like the famous Unix quote: "the most dangerous enemy of a better solution is an existing codebase that is just good enough." - Eric S. Raymond

Java, C#, Python, Ruby, C++ are all good enough, even UX wise, to be displaced by something that doesn't offer compeling functional value. At least, in my opinion.

lbruder• 9 years ago

YMMV, but my experiences with SBCL were way better than with Go. Using Hunchentoot and the other Quicklisp libraries made developing fun again. With Go I had to constantly worry about errors, repeat the same code snippets over and over again, had to comment out "unused" variables just to try something... not fun. That said, let's see how Lisp works out once I cross the 10'000 LOC barrier. But by then it won't be a microservice anymore, anyway...

TurboHaskal• 9 years ago

The problem is getting people to actually code in CL or any Lisp. I haven't had that problem with Go so far.

weberc23• 9 years ago

I haven't had a problem with testing in Go, but I also don't mock many things. Most things that should be mocked are already interfaces in my code. My biggest grievances with Go are its limited type system and dependency management. Unlike most[^1], I don't need semantic versions--I just need deterministic, reproducible builds, and for that, vendoring is fine in theory, but Go's tooling around it needs work.

I love that Go is a language that can be mastered in a matter of months; that it has such a simple "get up and running" story; that its tools mostly do the right thing by default; and that almost everything is straightforward. In most other languages, I have to figure out what build system to use, how to script it, how to specify and download dependencies, what testing library to use, how to actually run the tests, how (if it's possible at all) to get a static binary, etc--and then I usually have to learn a complex language on top of all of that. For the most part, Go just works out of the box.

I think folks also underappreciate Go's runtime--it has a fast garbage collector and an awesome goroutine scheduler that all get statically compiled into your application binary. Mostly, I want a functional language that compiles to Go so I can still leverage Go's awesome tooling, runtime, and deployment model, but with things like generics and algebraic data types. I've started prototyping it too, and so far it's coming along smoothly (modulo all the things I'm learning about building a programming language). On that note, there is some effort going on in the Go community to build a VM for Go, which would probably make an even more attractive compilation target, at least from a toolchain perspective. Even if my project doesn't take off (and it likely won't), I think this is a low-effort way to address Go's most significant complaints.

[^1]: Well, "most" judging by the significance many of Go's critics place on semantically versioned dependencies

mrhwick• 9 years ago

Have you used dep? https://github.com/golang/dep

sethammons2• 9 years ago

Chances are, if you are using mocks in Go, you are doing it wrong. Functions should take interfaces (not interface{}, but interfaces that list methods) and return structs. In your tests, you create a fake that meets the interface and pass that to your function under test.

If you have to test file io or DB interaction, you can either do the adapter pattern or, as I prefer, just actually use the filesystem and DB but with some test infrastructure around it.

AnimalMuppet3• 9 years ago

What's the difference here between a "fake" and a "mock"?

pjmlp7• 9 years ago

compiles very fast (for a statically-typed language).

Like Turbo Pascal 4.0 in 1987 or Modula-2 in 1978, as two possible examples. :)

4c2383f5c88e9113• 9 years ago

I can agree with most of the article, but for some it looks like we have not been using the same language ecosystem at all.

Go really feels opinionated around the wrong things, in order to claim "simplicity" as a feature:

- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)

- error handling which is essentially string-based, in 2017? Give me a type system, please

- gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite

- go test is good, but in the end it is found lacking

- dep, well, if you don't mind dumping the code for all your deps in the source tree, it's probably ok, but it solves a deficiency that the language should not even have (see next point)

- stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.

zaarn2• 9 years ago

- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)

Depends on what you do. I've written a good chunk of go code and interface{} is the rare exception rather than the rule, usually employed where a user might supply arbitrary types (ie a unmarshaljson like function)

But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.

- error handling which is essentially string-based, in 2017? Give me a type system, please

You can put anything behind an error:

    type IntErr uint64
    func (i IntErr) Error() string {
        return strvonc.Atoi(i)
    }

The string is only present when you either use the stdlib errors or you output an error.

Additionally, nothing in Go forces you to actually use the error interface (sans stdlib).

You can invent your own Error interface that uses ints. (Won't work with most external libraries but nothing is stopping you)

Panic and Recover can to my knowledge both handle things that aren't errors; recover returns a interface{} that you check for value and type. You can put anything in there.

stability: the language itself is fine on that, but the ecosystem really isn't.

A lot of projects focus on stability. If the API is breaking a lot they usually use Gopherpit or gopkg.in to ensure compatibility in the future.

In my experience, breaking the API is not something well used libraries do lightly (libraries with no or little use do break it sometimes since they lack the usage to refine it)

If you need more, you can use vendoring.

laumars3• 9 years ago

But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.

That's not really true. If you want an SQL database to back your website, then you'll deal with interface{} with Go's sql package. If you want to gzip your output, then you'll use interface{} with your writer.

To be fair, I think interface{} gets an almost unfair amount of hate. While it is fscking annoying when passing around core types (I hate having to use switch clauses just to inspect the type of an interface{}!!!) I do love how interface{} is used for complex structures. In fact there are a few areas of Go's standard library which I wish used interface{} more in that respect (eg Go's `file` struct should be an interface{} so I can create custom methods for os.Stdin/out/err)

drdaeman• 9 years ago

But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.

How do you deal with the lack of sets and trees? I believe those two data structures are frequently almost indispensable - even in the most boring mostly-CRUD web projects. And it's either type assertions on {}interfaces or go-generate based specialized implementations. Both feel odd to me. (Am I missing something?)

cdoxsey• 9 years ago

The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.

Python packages are installed via a separate tool so I'm not sure what you're getting at here.

Including github in the import path is a good idea. It makes it so you always know where the source code for any package is. This consistent approach empowers sites like godoc.org. I don't know how many times I've tried fruitlessly to find documentation for libraries in other languages, but in Go the developer who created the library gets it for free. I wish more languages did this.

Do you remember the left-pad fiasco in npm? Out of the box you'll have similar problems in Go, but at least you'll get them when you work locally, not when you spin up a new node. And to fix the problems you end up having to do the same thing regardless. Either you vendor your dependencies or you setup some sort of caching middle-man proxy.

Go's dependency headaches aren't the fault of the language or the ecosystem - Go actually has a really well crafted module system, specifically designed to fix major performance issues in languages like C++ and Scala. Rather they're a reaction to the problems latent in the "easy" solutions and simply switching is just trading one set of problems for another.

tavert• 9 years ago

It makes it so you always know where the source code for any package is.

Where is a useful question to answer, and nice to have consistent across an ecosystem. But What is also a very important question to answer, and tying What to be mutable and dependent on When is... highly unfortunate.

jussij• 9 years ago

gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features

This could be one reason why that is the case: https://xkcd.com/927/

brudgers• 9 years ago

The opinions expressed in Go are solely those of Google about the productivity of Google programmers. Nothing else is driving design. It is by Google employees for Google employees. Making it widely available and open source is a strategy for extracting value in the form of bug reports and patches and Google friendly upgrades from a world wide community for the price of a few conference talks and some sticker swag.

That Go solves some problems for programmers outside Google is a side effect of being a general purpose programming language. It is not a primary business goal except in so far as it raises Google's bottom line, creates a hiring pipeline, and keeps its employee moral off the floor.

pjmlp7• 9 years ago

The irony is that all the other official Google languages have much better support for generic programming.

mhd2• 9 years ago

I think if you're replacing "Google" with "those Bell Labs guys", you're closer to the truth. A lot of the design decisions are quite familiar if you look at the past work of e.g. Rob Pike.

It's not necessarily all about peak efficiency for specific enterprise applications.

drakonka• 9 years ago

I am new to go and am especially curious about what you find lacking on go test. Could you elaborate on that point? I've found it nice to work with so far and wonder what kinds of issues others have had/what I can watch out for. Thanks!

yla92• 9 years ago

gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite

A Golang noob here. Can you (or anyone else) expand more on this? Are there any links that I can read more about this?

4c2383f5c88e9113• 9 years ago

Actually, the mistake is mine, I meant to write golint (but too late to edit). I have no issue with gofmt. As for golint, https://github.com/golang/lint/pull/100 is the kind of thing I'm thinking about.

lunch• 9 years ago

Context: I've been programming in Go full-time for the last 2 years and come from a Python and C# background.

  > No Generics

The lack of generics was something I was hung up on, too, but TBH I haven't found myself reaching for them for a while now.

  > Error handling

This is not a problem for me, either. I can easily handle different `error` types and if I want to provide more context I pull in github.com/pkg/errors

  > gofmt

gofmt is great

  > go test

go test has been totally sufficient for me and I was quickly writing and running tests. The testing package itself is builtin and writing tests is dead simple.

  > dep / dependency stability

Dependency management has been a huge PITA with Go and is my largest complaint. Personally, I've used everything go GB vendor, godeps, glide, and now dep. dep has been Just Working for me and is decent enough working on a team. As for dumping the dependency code in your source tree-- it lives in a folder called ./vendor so it's easy enough to not look at it.

adamkittelson• 9 years ago

The lack of generics was something I was hung up on, too, but TBH I haven't found myself reaching for them for a while now.

I mean, if you lose an arm you probably stop trying to pick stuff up with your missing hand after a while too, doesn't mean it's not better to have two arms.

zkomp• 9 years ago

I agree completely. Go is opinionated about the wrong things, in the wrong way. Which in the end make it a worse language than it could have been.

One thing you did not mention is very varied quality of the standard library. I’m still baffled about ”standard date” crazyness. There are huge warts in the sql interface etc

My collegues used to joke golang is a great solution to google’s problems (for others, not as good as it arguably could have been to put it mildly)

randomdata2• 9 years ago

- Go's opinion about generics has always been that it should have them, but finding the right expertise to implement them in a non-ridiculous way has been a struggle. They are marked for inclusion in coming versions (2.0).

- error is an interface. If you are returning strings most of the time, you're probably doing it wrong.

- Wasn't gofmt originally executed during compile time? I'm not sure why it changed, but it suggests that such automation fell apart in practice.

Daishiman• 9 years ago

Go's opinion about generics has always been that it should have them, but finding the right expertise to implement them in a non-ridiculous way has been a struggle. They are marked for inclusion in coming versions (2.0).

The fact that people can claim that C#, Java, Scala, Kotlin, Haskell, Rust, Swift have "ridiculous" implementations of generics sounds like a very bad excuse.

olalonde• 9 years ago

Your criticism would have been a lot better if you had pointed to an alternative.

iends• 9 years ago

(Not the OP)

That's the problem, I see the following often suggested instead of Go:

Java - maybe if you use Spring Boot you can get around some of the boilerplate, but there is a ton of baggage in the ecosystem.

Clojure - This is really popular where I work but LISP + Java Ecosystem turn a lot of people off. Also, some people really want static typing (maybe Spec is good enough?)

Node/TypeScript - terrible concurrency, good for throwing something together that works rather quickly, good for cutting corners. Terrible memory, cpu profiling. As a professional node developer in the day job I could go on and on, but at least the type system is better than Go (too bad the concurrency story is so abysmal)

Elixir - not popular, but at least it's on BEAM VM and has lots of libraries. Not statically typed (yes you can use dialyzer). Never done anything with it, but apparently deployment is a pain.

I personally write JavaScript mostly because its ubiquity. It's good enough, until it's not.

4c2383f5c88e9113• 9 years ago

Why? I'm not saying go is a bad language at all, it has a lot of good points, as outlined in the source of this discussion. It is however not a perfect language and as such, it never hurts to criticize it, if only in the hope that it will improve in the future. I personally find the mental model required for go quite abrasive, but that's only me, and I won't hold it as valid criticism.

The alternatives vary according to the issue you want to tackle, so I would be hard pressed to recommend a particular one. And go does fill some vacant spots in language space.

oppositelock• 9 years ago

I've been writing Go for quite a while, and probably used everything it's got to offer as a language. Go is opinionated, yes, however, for some crazy combination of reasons, it's amazing at getting out of your way for the most part.

Using interface{} instead of generics is a bad idea, you lose type safety. It's messy, but code generation is your other alternative, sort of like your own templating system. I've used go's built in templating code to generate type safe Go code from things like SQL table schemas. I'd highly recommend that route over naked interface{}

Error handling is string or type based, that's up to you. I declare specific error types for my functions, like FooError or BarError, the caller can then switch on err.(type) and respond appropriately. Alternately, you can define concrete error objects and return those, and make comparisons like "if err == FooError".

I'm with you that dependency management is terrible, vendoring is a poor workaround.

Twirrim• 9 years ago

- stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.

Google operates on a mono-repo basis, as I understand it (from speaking to ex-googlers.) All libraries etc. etc. all under one big repo. If you think about how that would appear to end tooling consuming it, the approach taken with imports makes sense. It's lousy for the rest of us outside of the google mono-repo ecosystem, but it does make sense.

Dowwie• 9 years ago

What I've noticed, and personally experienced, about these porting projects is that they're generally a premature optimization but are a great way to learn a new language. The gains from a port are largely intrinsic in nature, residing with the programmers. However, systems and operations largely carry on just fine with the original language chosen.

Only one story comes to mind where a programmer had a legitimate problem using a high-level language and solved it with a lower-level language, and that was Armin Roacher, who solved a significant performance issue using Rust: https://blog.sentry.io/2016/10/19/fixing-python-performance-...

Can you think of other stories? Please, share.

I encourage all to be more skeptical about potential gains and realistic about who is actually gaining.

_Codemonkeyism6• 9 years ago

Developers have the gain, company suffers, developers move on for higher salary with more skills, CTO is in it, doesn't fullfil his duty towards the company or investors, CEO has no clue and doesn't manage the CTO.

The story I see over and over again as a startup CEO consultant.

nspattak3• 9 years ago

So if I get your argument correctly:

Developers learn a new language that is faster to develop, safer and faster to run than existing solutions (which work but are of increased technical debt) and successfully & quickly port the existing solution to the new tool.

The company doesn't really benefit from it apparently because they never scale enough or because they lacked a target so they had their developers working on sth small instead on working on the company's products.

This skill is highly valued and the developers leave for an other company while the original one struggles to continue.

Do you think that it may be the case that the problem in all these is not within the developers or tool? Furthermore, shouldn't the CEOs and the CTOs know better and to a better job?

sidlls4• 9 years ago

Hell, I've seen this as a lead/senior engineer. I'm seeing it right now, in fact, as a person on our team wants to write some stuff in language X, which nobody else on the team has experience with, and which isn't used anywhere else in the company, and to solve a problem that is perfectly well solved (in both development time and performance) with the language and tooling we already use.

As far as I can tell this engineer just wants to stick "built stuff in X" on his resume. He doesn't have a good reason to not use our existing stack. If I were the manager I'd demand more technical justification.

fellowmartian3• 9 years ago

Using a compiled language is not a premature optimization if you're not sacrificing expressiveness. With that said, Go is not the best choice. Like, for anything.

hasenj9• 9 years ago

What would you choose instead?

acdha3• 9 years ago

Go isn't my first choice but “like, for anything” reeks of dogma. Can you explain your rationale?

crdoconnor5• 9 years ago

they're generally a premature optimization

This example actually seems to be the opposite of an optimization. There's 50% more code to maintain and there appears to be almost no appreciable benefit to compensate for that.

dx034• 9 years ago

LOC is a bad measure for the complexity of code. Quite the opposite, expressive code can be faster to understand and easier to maintain than the same functions with half the code size.

In any case, switching to a compiled language and static types should bring some benefit, in addition to the gain in speed.

benhoyt• 9 years ago

No, there was no appreciable user benefit, and that was intended. To respond to others: it wasn't a "premature" optimization, because it wasn't any kind of optimization. The sole benefit in this case was for me to learn Go programming -- and I noted that in the article title and intro. :-)

monktastic12• 9 years ago

There's 50% more code to maintain

Given that Go is similarly verbose to Python, probably 100% more code.

hasenj9• 9 years ago

Writing in a fast compiled language is not premature optimization.

Premature optimization would be something that makes the code more complicated and more difficult to follow but produce better performance.

Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.

Writing in a slow interpreted language is more like premature "deoptimization" (if that's a word).

crdoconnor5• 9 years ago

Writing in a fast compiled language is not premature optimization.

If the fast compiled language is equally expressive and equally fast to write then no.

Those things are rarely (if ever) the case, though. It's certainly not true in this case. Go appears to be about 30% less expressive.

monktastic12• 9 years ago

"deoptimization"

pessimization?

But the code doesn't have to become more complicated for it to be premature optimization. It literally just means improving something (generally performance) before there's a clear need.

Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.

I assume you mean "rewriting" here, since we're talking about porting existing code. Sometimes it's the best use of time, sometimes it's premature.

lmm5• 9 years ago

No-one goes out of their way to choose a slower language. People who choose to write something in Python are getting value out of it, or at least believe they are; if writing the program in Go will be more costly (in development time, defect rate, library availability or something else) than writing it in Python then writing it in Go is premature optimization, and if writing it in Go wouldn't be more costly than writing it in Python then why were you thinking of writing it in Python at all?

IME thinking about language performance at all is premature optimization; language performance almost never makes the difference, other between-language differences will swamp any gains from language performance.

sidlls4• 9 years ago

I agree that writing in a fast, compiled, statically typed language is not necessarily premature optimization. It is my first choice for a new project in the absence of any other mitigating factors.

There are often mitigating factors, though.

And switching from an existing language to use a new one that is possibly more optimized carries costs that have to be considered. If the new implementation shaves a few percent off some time or other resource scale, and the sum of that savings over the life of the program is more than the cost to do development and maintenance it's worth it. But that often isn't the case.

didibus6• 9 years ago

I don't follow this logic. Why not use C all the time by default then?

rjammala3• 9 years ago

How about this: once you write code in Go and test it well, you can deploy it and forget about it unless there is a hardware issue? I recently ported some C++ code to Go which processes 8 Billion events per day flawlessly.

joshuamorton3• 9 years ago

I feel like this is true for any language though.

Once you write a well designed, well tested, and feature complete piece of software, you can deploy it and forget about it unless the server it's running on breaks.

Go isn't special in that regard, unless there's something that makes Go easier to write, test, or deploy, which might be the case, but you haven't supported that.

dwhabcdefg• 9 years ago

I ported a Groovy/Grails app to Java/SpringMVC once.

My technical motivations were more about moving away from Hibernate. Arguably there were plenty of reasons for leaving it as it was, though -- if I had known enough to be able to solve certain data access complexities back then, Grails was in many ways a better fit for the problem. The experience in a (more-or-less) functional language wouldn't have hurt any either.

The business motivation was simple: Java/SpringMVC programmers are (or were then) far more plentiful, so adding/replacing tech resources was made much simpler.

oblio5• 9 years ago

I'm not sure about Javascript, but why do you have to have two APIs when you have async? At least in C# you write the async version of the API and that's it.

Then you go asyncStuff.Request on it, and it blocks. There are some quirks but I remember that it worked all right.

crdoconnor5• 9 years ago

However, it was only 1900 lines of Go (about 50% more than the 1300 lines of Python).
However, I’d seriously consider using Go for larger projects (static typing makes refactoring easier)

Assuming a fully tested codebase and a relatively sane runtime type system, I'd much, much rather maintain the shorter codebase than the statically typed one.

Having less code makes refactoring easier.

boomlinde3• 9 years ago

Assuming a fully tested codebase, there is an overhead in writing and maintaining unit tests that has to be considered as well, and using a static type system obviates a whole category of them. You move the type specification out of tests and manual documentation directly into the code where you'll find out at compile time what you did wrong. Not at run-time, or during testing because you diligently wrote a test that considered the case.

I agree that brevity sometimes aids refactoring, but there are many parameters that have to be weighed in, and often I find that the benefit of static typing outweigh whatever ignoring the type or checking it at run-time might offer.

crdoconnor5• 9 years ago

Assuming a fully tested codebase, there is an overhead in writing and maintaining unit tests that has to be considered as well, and using a static type system obviates a whole category of them.

I've not found this to be true, and tests that do type checking are a waste of time in any language.

Bugs that are caught by static type systems are almost always caught by a combination of behavior tests and a sprinkling of sanity/type checks on border code (e.g. a module's publicly facing API).

you'll find out at compile time what you did wrong

In a fully tested code base that is regularly tested the distinction between compile time and runtime does not matter a lot.

By contrast n code bases in dire need of tests (which is more common than generally assumed), the distinction is absolutely vital.

bischofs• 9 years ago

I am doing some python to go conversions for some performance gains in parts of my stack - I really like it so far because of how they brought some of the perks of languages like python into a compiled language. One thing I really miss though is ipdb.

The Delve debugger still doesn't let you run functions and I miss all the autocomplete fun with ipdb. I hope they can make go a little more friendly to debugging in general.

jpcarmona• 9 years ago

I really liked to read the article. You gave your opinion and didn't try to sound that this was the way, it was clear that it was YOUR opinion. Thanks for it.

Learning Go by porting a medium-sized web back end from Python

Related Stories