Tor in a safer language: Network team update from Amsterdam

https://lists.torproject.org/pipermail/tor-dev/2017-March/012088.html

by _vvdf • 9 years ago

343 244 9 years ago

lists.torproject.org

najati83• 9 years ago

This is exciting not only because of the Tor project itself but because this will set an example for other projects to follow.

wklm• 9 years ago

This creates also an opportunity to engage Mozilla into contributing

duneroadrunner2• 9 years ago

If any of the developers are reading this, converting the existing C code to SaferCPlusPlus[1] (a memory safe subset of C++) is probably a more expedient solution (if that's what they're looking for). (And speaking of contributing, an automatic translation (assistance) tool is in early development, and could maybe be functional in short order with a little extra motivated talent... :)

[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus

rebelwebmaster• 9 years ago

Note that Mozilla is already working with Tor developers to get more of their patches upstreamed into Firefox to ease their maintenance burden.

https://blog.torproject.org/blog/tor-heart-firefox

bitexploder2• 9 years ago

I am curious why they were advised not to use Go. Probably not a safety concern.

Edit: cgo != Go. Thanks for the responses. I have done a bit of Go, but just pure Go.

m-j-fox5• 9 years ago

They were advised against cgo. In my not so recent experience, it is a huge PITA.

pjmlp12• 9 years ago

I never understood the idea behind cgo instead of having a proper FFI like Delphi, .NET, Eiffel and so many other languages.

hannob• 9 years ago

Go needs a garbage collector that needs to be set up. Using Go code within C code is possible, but it creates additional hurdles.

Therefore a slow transition of rewriting parts of the code in a safer language and having the core still in C is much less feasible with Go. With Rust you can easier just compile some object files and link them into your application.

masklinn4• 9 years ago

They were not advised against Go but against cgo. Part of what they want is incremental conversion and cgo is at the same time not-go[0], costly[1] and complex[2], and then you still need to manage the Go runtime (GC & al) from within your C system. That makes integrating the two difficult, especially when you want to replace the existing system piecemeal.

A pure-Go rewrite might be an option (in fact Tor seems pretty firmly in Go's use cases), but that's not what the Tor team is trying to do.

[0] https://dave.cheney.net/2016/01/18/cgo-is-not-go

[1] a cgo->c call is ~100 times more expensive than a go->go call, and ~400 times more expensive than a c->c or rust->c call https://www.reddit.com/r/golang/comments/3oztwi/from_python_...

[2] https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...

steveklabnik4• 9 years ago

go -> C calls have gotten way way cheaper in newer versions of Go. There's still overhead but it's not as bad as it used to be.

geofft5• 9 years ago

Go and Rust are very different languages. Rust is, by design, well-suited to Tor's use case, where they have a large C or C++ program and they need to incrementally rewrite parts of it (and maybe never all of it!) in a better language.

It turns out (or so I hear) that Google statically links everything in production, and has been using C++ as a language to implement HTTP endpoints for a long time. So Go is a better C++ for what they want out of a better C++; for the rest of us, it looks more like a compiled language along the lines of Python/Ruby/etc. with a nice deployment story. If you want that out of your better C++, Go is great. If you want to reimplement all of Tor from scratch, Go certainly seems like a reasonable choice.

But as a result of these priorities, Go basically doesn't have interoperability with the platform ABI as a goal. (For some combination of historical reasons and the lack of complicated features in C, the platform ABI on just about every platform these days is a C ABI.) Rust does; it uses a standard compiler toolchain (LLVM) instead of what's basically a custom one (Plan 9), and the standard toolchain knows how to generate calls that follow the C ABI. Rust doesn't have a runtime of its own, and it's safe to directly call into a Rust program from some arbitrary point in a C program. Rust's allocator doesn't care if you do stupid things with pointers it allocates, as long as you give them back eventually. Rust doesn't create threads on its own unless you ask. Rust functions use the normal stack. Rust on UNIX uses the platform libc. And so forth.

It's possible to call C code from Go and vice versa, just as it's possible to call C code from Python and vice versa. But Go is not best tool for this particular job.

socmag10• 9 years ago

Totally agree, and Rust isn't a sane choice either even though the points you make are valid.

Rust has some interesting features, but being C isn't one of them.

JepZ10• 9 years ago

I think Go is a better C, while C++ is the opposite.

+1 for the rest of the explanation.

nickpsecurity12• 9 years ago

Here's what I'd have told them:

https://news.ycombinator.com/item?id=14013617

m-j-fox5• 9 years ago

I don't know much about Tor. But I hope I can route all of my home network traffic through it. That or route everything through VPN. I'll bet you can guess why I'm suddenly interested.

arez2• 9 years ago

you can, but it's not really nice, because TOR is relatively slow, it only makes sense for heavily text based sites and not images/video etc.

searchfaster2• 9 years ago

*This and almost every CDN starts popping up 'Captchas' atleast the last time I tried.. I find it easier to setup a VPN server and use one.

chrisper• 9 years ago

Couple of things. With Tor, you cannot control your exit in a sense. Since some websites are really shoddy (a lot of application portals are) and don't have / support SSL, you would be transmitting your entire application profile through TOR unencrypted. Somehow, I trust my ISP more than some random TOR exit when it comes to this.

Second, many websites (sadly) do not work if you are using a VPN (like Netflix).

yoshuaw• 9 years ago

Netflix's VPN detection seems to work by blacklisting a known list of IPs (e.g. AWS datacenters, etc.). Proxying through a machine that's not on the list works.

repples• 9 years ago

You really don't want to do that. Any exit node could be recording or tampering with your network traffic, e.g. using sslstrip or worse.

seibelj6• 9 years ago

Founder of https://easyvpnrouter.com/ ask me anything or how to build one yourself if you want a project

Tempest19812• 9 years ago

We use OpenDNS for safer kid surfing and logging. Can I get similar functionality? Your FAQ says you use "Private Internet Access’ custom DNS servers".

Also, what about the Netflix issue?

ianopolous3• 9 years ago

Nice! There's definitely a need for this. It would be good to include the actual router specs on your site, like cpu frequency + cores and RAM.

m-j-fox5• 9 years ago

I guess the main question about this product (which is the sort of thing I'd buy -- an hour of saved effort pays for it) is how can I trust you to be less evil than Comcast?

I guess if you're open-source and a lot of people are paying attention, that would do it.

m-j-fox5• 9 years ago

Curious why this is one of my most hated comments in at-least a couple days ...

searchfaster2• 9 years ago

When I read safer language, 'Rust' came into mind automatically.. Not sure if 'Rust' will ever be as popular as golang but certainly see a future with it popping up everywhere mission critical / super safe software is required.

JepZ10• 9 years ago

I am afraid Rust will become much more popular than Go is atm. I think the C++ crowd will embrace it as soon as it will be mature. I am sure it is better than C++, but I still like the concepts and syntax of Go much better.

But as C++ programmers obviously never care about readability and simplicity, I am pretty sure they will take Rust. After all, Rust is a decent language and as users, we will all benefit from the migration.

59nadir• 9 years ago

... but I still like the concepts and syntax of Go much better

The concept of not having features because someone on your team might use them? You subtly say that Rust is a worse language because it doesn't seem like it was designed in the 80's, when clearly as a language (together with the compiler), it's objectively better than Go.

vertex-four4• 9 years ago

I've... never had a problem reading Rust code. It's generally well-typed, and makes good use of "automatic" error handling constructs to ensure errors flow upwards without visually polluting the success case.

Whereas with Go code, I have to filter out all the error handling (which often takes up 2/3rds of lines of code, even when it's just "if there's an error, return the error" which it almost always is), wade through the mass of functions that take interface{} and cast it to something internally, etc etc.

I've tried programming in Go and I find it horrifying - I either get to ignore errors or spend ~80% of my code doing this, repeatedly, a few times per function:

    foo, err := bar()
    if err {
        return nil, err
    }

sidlls• 9 years ago

C and C++ are used in generally very conservative systems programming contexts. Safety is always considered with other items for tradeoff. Adoption just for reasons of safety isn't a given.

As a result, the "maturity" required for adoption differs in different parts of the C and C++ community. If Rust becomes more popular than either of those languages it is unlikely to be on a time scale shorter than a decade or two. That's not to say it can't happen quicker, especially given that communication and adoption can be much quicker today than it was even 15-20 years ago, but it's unlikely.

generic_user3• 9 years ago

I'm disappointed they did not consider COBAL given its vast superiority and flawless track record for April first rewrites.

kilotaras• 9 years ago

Fri Mar 31 21:23:27 UTC 2017

generic_user3• 9 years ago

I believe there is a crate they can use to prevent date bound out of bounds fun leaks. (can't find the link atm)

nickpsecurity12• 9 years ago

Since bitexploder asked, I'll add what I wrote on this on other forums. If it's about secrets or anonymity, make sure you always use a safe language that supports careful control and reasoning about both memory and CPU time. The reason is that this enables covert, channel analysis for vulnerabilities that leak secrets through storage and timing. It's why I wanted Freenet to ditch Java aside from the obvious reasons. It's also why GC languages such as Go are better not used. Although, memory management where programmer controls timing & it's simple to analyze might be used. Reference counting comes to mind.

The other thing you want is proven, successful use in high-assurance systems. That is, systems that either didn't fail or provably couldn't in certain ways. These are almost all written in a subset of C or Ada/SPARK. The advantage of using those is you can combine them with a vast array of proprietary or open-source tooling to catch about any error you can think of if it's implementation. There's also formal specification and protocol analysis tools that combined with expert review can catch the rest. Rust, although a good choice for increased safety/security, doesn't have such tooling yet. That means they will get less correctness overall and over time vs MISRA-C or Ada/SPARK unless similar ecosystem in industry and CompSci emerges for Rust. That's why I recommend against it for high-assurance security for now.

It does seem good for medium-assurance security where you want to knock out low hanging fruit in systems code. It will avoid serious errors in C while providing additional benefits with type system and other features. Ada 2012 + SPARK 2014 are the standard for safe systems since they systematically eliminate all kinds of errors with a consistent design and tooling with decades of field success. I haven't seen a direct comparison with Rust on each protection to see if it matches it already or not. The main advantages Rust has over them are its borrow-checker for temporal safety, more usable method for safe concurrency, and (best for last) highly-active community to provide libraries or help. Go has similar benefits if its GC works with your use case but a lower, learning curve & possibly lower efficiency. Due to ecosystem benefits, these are main two I'm recommending for medium assurance if Ada/SPARK are too much to learn.

wyager• 9 years ago

It's unlikely that the cost of whole-program side channel resistance is worth it for an application like Tor. Side-channel-resistant cryptography is almost certainly sufficient for most reasonable threat/cost models.

nickpsecurity12• 9 years ago

You can limit it to the protocol layer. High-assurance security in military systems just used fixed-speed, fixed-timing transmission with covert channel analysis of setup, crypto operations, and error handling. Any problems at all violating key invariants cause fail-safe. Tor could do something similar.

Still need a language that allows such analysis at least for those components being analyzed.

geofft5• 9 years ago

You can use some analysis tools on Rust code, because it generates C-ABI-compatible objects; for instance, I'd expect that https://github.com/agl/ctgrind would work on Rust.

Looking at the MISRA-C guidelines (or more specifically, a pirated copy - are these available legitimately to the public?), it seems like about half of them aren't problems in Rust, because it's warning you about stupid things in C that can't be fixed for legacy reasons, and half of them are things that could be caught with a linter / static analysis tool like https://github.com/Manishearth/rust-clippy . Do you think building something that implements as many of the MISRA-C checks as possible is useful progress towards the high-assurance Rust goal?

nickpsecurity12• 9 years ago

I said MISRA-C combined with tooling. Many of the best tools using very, diverse methods start from a safer subset of C (eg MISRA-C) or need C itself. Such tools would have to be rewritten for use with Rust. Difficulty of that varies per tool but it's not happening for most right now.

ianopolous3• 9 years ago

G'day Nick!

Java is getting an AOT compiler in July (Graal, http://openjdk.java.net/jeps/295) that will let you AOT compile parts, or all, of your program, including the JVM modules themselves. This would seem to leave the GC as the main source of side channel vulnerabilities. The GC itself will become more pluggable as well, with a pure Java implementation. What requirements would you put on a GC for side channel safety? (Ignoring the obvious just allocate a large heap and never GC anything, eventually just restarting the process when it runs out of heap). What if we had a fully concurrent, pauseless GC (e.g. Azul), would that change things?

What other issues would remain in your opinion?

Sanddancer• 9 years ago

A concurrent GC running on a single processor machine is still going to pause. There are still other ways to get into situations where an attacker can cause the gc to kick in in ways where you can get information -- find a few CPU-heavy functions that nudge the GC to kick in when and where you want it. Harder, but even a background GC can wiggle into the foreground.

nickpsecurity12• 9 years ago

Good to see you, Ian! Yeah, there was a recent post on Lobsters with a pile of progress on tech you sent me a while back. Them getting an AOT can help so long as you can get a good, mental idea of what the assembly is going to be doing for a given piece of Java. Otherwise, you'll be doing a lot of rewriting that might make one reconsider Java in first place. Pluggable GC's is a good think as is ability to turn it off for unsafe native. You told me before they could do the latter, probably were for some of the platform.

Before answering the other question, it helps to understand what covert channels are fundamentally. First, know there's always two parts: Sender w/ access to secrets but inability to do I/O outside the system; Recipient w/ no access to secrets but ability to send its info out. These might be in separate processes, partitions/VM's, or even on a network w/ stuff happening due to protocol interactions. Idea is to find a way to communicate that wasn't intended for communication. Should help confirm the dark things I say about mainstream INFOSEC when I say I couldn't find almost any intros or blog articles on these for you in top results. I found one, though, that describes them well even if not having many examples:

https://arxiv.org/pdf/1306.2252.pdf

Now, back to the GC. The GC might kick in whenever secrets are being processed. This could mask information about them due to unpredictability or leak information about them. The simplest route to dealing with it is not allowing GC while performing any operation that processes secrets. Depending on the app, the impact on memory availability or performance can vary considerably. There's also at least three more channels to look for on even basic app. The keys might leak in memory that's released back out of the system. First needing overwritten is in Java app itself for anything GC releases. Gotta look at assembly since compilers sometimes get rid of that as a "useless" operation. Second, the OS itself might leak by swapping out privileged Java app (Sender) for a Recipient that simply reads the registers before doing anything. Secrets might still be in them. Orange Book & separation kernels required these overwritten every time a process change. Don't know if current OS's do that. The "swap" part of the filesystem itself is a risk and should always be disabled. Finally, a Recipient in another process can receive secrets through the cache activity of the Java app. That's an old one that's hard enough to [confidently] deal with that I just advised running trusted apps on one CPU and untrusted on another CPU. That's physical isolation with them communicating over a pipeline, SMP since caches are separate, or multicore with no shared cache between cores. There's CompSci work such as partitioning caches and potential with embedded CPU's w/ things like locking, real-time caches that might help. Who knows real practicality, though.

So, you prevent or overwrite any storage of secrets that another process could touch. You put a brick wall between two events where a recipient observing timing could possibly learn something. Eliminating non-determinism can go a long way. Reference counting might help since you at least know when you'll deallocate w/ similar checks happening constantly. The deallocation might even be masked. For protocols, the classic response by military systems was fixed-size, fixed-rate transmission with extra attention that error responses didn't leak anything. Every detail you can sealed instead of just data payload since any might be storage channel. So, there's you a start on it.

And this was just running an app + considerations of a GC running in the background. Leads to all those problems. See why high-assurance security invested so much effort into automatically or at least reliably getting these damned things out of our systems? Just imagine how many are in UNIX API's, common protocols, and clouds. The fix is hard and expensive if it's legacy so they're definitely still leaky. :)

bitexploder2• 9 years ago

I get the side channel concerns, but most developers make a mess of languages that let you control these things. A few, very few, people I trust to write in a language like C build tools I trust (Dan B, etc). I also think it is very hard to convince folks about side channel concerns.

We have to live in the world we have too (e.g. Signal on an Android is better than not having it, etc.). When I document side channel issues in customer code there is almost some way higher priority issue they have to fix first, etc. But your advice is good for someone endeavoring to do a "correct" approach from the ground up.

Thanks for the detailed thoughts though. This is the sort of thinking anyone who thinks "I will write a secure chat program" has to do ages before they start writing code.

hckr12922• 9 years ago

What are good resources for learning Ada/SPARK and MISRA-C?

nickpsecurity12• 9 years ago

Those resource change quite a bit over time since it's both a small, legacy ecosystem and a constantly-changing, active ecosystem. I'd have to update my resources when my busy, busy schedules slow down. I plan to, though, on those two tech w/ guides & whatever FOSS tools I find. Just email me at address in my profile about it. I'll just save it in there so I can email it back to you once I'm up-to-date on the learning resources & tools available.

Gaelan2• 9 years ago

This is either a horribly timed announcement or a joke written very seriously. I have no idea which.

danieka2• 9 years ago

How so? I'm honestly wondering why.

steveklabnik4• 9 years ago

Today is April Fool's Day.

masklinn4• 9 years ago

Seems legit, the meeting/sprint where all that stuff was decided/announced and done was last week (22 to 27 of March), it seems to just be the ML update which was unfortunately timed and posted late on the 31st.

Or Sebastian was very dedicated to the joke, but they seem to have posted a comment confirming it's serious on /r/rust: https://www.reddit.com/r/rust/comments/62o9rx/tordev_tor_in_...

Looking at their comment history, both the Tor membership and the interest in Rust are there, and Sebastian just tagged themselves on the relevant Tor issue: https://trac.torproject.org/projects/tor/ticket/11331

Manishearth• 9 years ago

It's legit, Tor devs I know have been talking about it for a while.

lucb1e• 9 years ago

Well not everyone in every culture thinks no serious work can be done on April 1st.

sfilargi3• 9 years ago

As a mere average user of computer languages, every time I play around with Go I start wondering how a language like this became so popular.

It feels like it was invented in a universe where Haskell, OCaml, Erlang, Smalltalk, Lisp and so many more languages and research in languages never happened.

gonehome• 9 years ago

Sometimes I think the syntactical difference of functional languages alone is the reason they're unpopular.

Iterative languages seem to match more closely how people speak/think in verbal language.

moomin• 9 years ago

It's a pity it's hard to be precise in verbal communication without resorting to talking like a functional programming language.

donatj• 9 years ago

I agree completely. I recently went to an Elm workshop and while I found it very interesting, the Haskell syntax just didn't agree with me.

I want more enforced required clarity. I compared it at the time to writing English without any punctuation, you can do it but it makes comprehension much more difficult.

rnhmjoj• 9 years ago

It may match more with the way we speak but not really the way we think, imho. When you ask someone to do something you usually just give him a general description of what the end result would be and then add more details if necessary. You don't tell him exactly what to do step by step from the start to the end, because you don't really care most of the time. Markup and declarative style are closer to the way we think.

ASinclair• 9 years ago

As a mere average user of computer languages, every time I play around with Go I start wondering how a language like this became so popular.

You can pick it up in a weekend. A lower entry bar means more people will try it out.

It feels like it was invented in a universe where Haskell, OCaml, Erlang, Smalltalk, Lisp and so many more languages and research in languages never happened.

It was developed in a large enterprise context not in an academic context. I think it shows.

ue_• 9 years ago

To be honest, you can probably also pick up Lisp in a weekend, experienced programmer or not. The syntax is also simpler.

im_down_w_otp4• 9 years ago

So was Erlang. The language, the runtime, and the standard library were all purpose-built together to solve a real world problem... to build fault-tolerant, distributed network applications.

Which makes Go's rapid popularity as a language for solving the same problem even more peculiar. Especially since Go's surrounding tracing, debugging, and online code swapping facilities are so much worse.

I'm sure some component of that is Ericsson's complete lack of interest in evangelism and Erlang Solution's apparent lack of capability to effectively evangelize, but it still seems like Go isn't so much filling a void as it is being better at attracting an ecosystem that delights in reinventing a particular set of wheels.

Java was very much the same way for a very long time.

lomnakkus2• 9 years ago

[I apologize for the absurdly snarky and ranty post, but I'm going to do it anyway:]

You can pick it up in a weekend. A lower entry bar means more people will try it out.

... and then you get the JavaScript situation where everyone thinks they're an actual programmer at $TEAM_LEAD level of sophistication when it comes to modeling, design, architecture, testing and implementation... but they're really not. It's not their fault per se, it's just that they do not have the necessary experience to realize the areas that they're lacking in. (This is a well-known cognitive bias: More than 50% of people think that they're an above-average driver. There are a precious few who realize that they're not -- they are the exception.)

Programming is hard[1] and if there was an instant-humbling device, I'd buy it in spades.

[1] Most people focus on the "oh, that's Undefined Behavior" bits of it, but it's not really about that. It's about recognizing the types of mistakes that you make and working to avoid them or creating a system that does, whether that's by testing or proof or whatever. Obviously it still has to be _practical_, but AGAIN... it's about tradeoffs. If you don't care about correctness, I'm pretty sure I can whip up a "solution" to any problem that's fast as hell... and not correct. Summa summarum: I think we need to be thinking a lot more in terms of tradeoffs and not so much in absolutes.

agumonkey3• 9 years ago

IIUC Pike spent most of his career in research right ?

fjdlwlv• 9 years ago

Go is a genersl purpose industry language. The ones you mention are niche or academic research languages.

sfilargi3• 9 years ago

I just put some names randomly. You can add C++, Python, Ruby, and pretty much any other language after C.

TheDong• 9 years ago

There are companies built on Haskell and Erlang.

Just because a language is founded on good programming theory doesn't make it unviable in the industry.

Ideally, the reverse would be true.

akerro• 9 years ago

Wow, have you seen JavaSCript?

sfilargi3• 9 years ago

Yes, but I think know how JS became popular; syntax similar to Java/C++ and pretty much the only language available on the browser.

chrismealy• 9 years ago

Go is designed for average programmers working in a huge organization. That's why its dullness is a virtue.

TazeTSchnitzel• 9 years ago

So, Java?

bwanab• 9 years ago

Seems weird that from what I hear about the intense interview process that Google programmers go through that they'd feel a need to dumb down the language of choice.

nickpsecurity12• 9 years ago

It was intended to be a simple language. Each of the three authors had to agree on every feature (lowest, common denominator). One made C, one worked on Java, and one did an innovative language for Plan 9. Two of those languages ignored as much of the kind of work you described as possible. Something like Go is a natural result in terms of feature choices for the language itself.

d0vs• 9 years ago

If you're used to dynamic languages, it gives you type safety and performance for little effort.

I love Python but always wished for a simple, type-safe language; Go gives me that. It's not worse than Python IMHO.

pjmlp12• 9 years ago

Without support for meta-classes, annotations, generators, iterators and list comprehensions it surely is worse.

solidsnack90003• 9 years ago

Colleagues I've spoken with who use both still say, that Python is "20 times" as productive as Go. For some applications this multiplier likely goes down considerably; but Python holds an edge in a lot of areas.

zem• 9 years ago

it was also invented in a universe where haskell, ocaml, erlang, smalltalk, lisp and so many more languages never achieved mainstream adoption. go will never be all things to all people, but it's clearly popular in the niche it's targeting.

JepZ10• 9 years ago

When you talk about Go I think you should start with C.

One of my professors used to tell us, that C was built by people who wanted to use it and didn't care about academic style.

In many ways Go is just the next step of C. C did not have object orientation and even passing functions around was kinda hard. While C++ tried to bring object orientation to C (total failure) Go decided to keep the core values of C and instead improved the rough features (e.g. easier binding of functions to structures, faster build times).

By making it easier to pass around functions Go enables functional programming styles, but at its core, it is still just an improved C. The only revolution within Go (as a language) is the concurrency and channel concept and I think that was taken from some functional language (not sure).

I really like Go, because it just feels right. It might not be as clean as Smalltalk or Lisp, but it has data structures and functions, teaches you how important interfaces are and lets you build highly concurrent applications with ease. In addition, it brings a nice set of tools which integrate well into a shell driven workflow.

After all, the whole thing should not surprise anybody as Ken Thompson[1] was part of the Team which invented Go.

[1] https://en.wikipedia.org/wiki/Ken_Thompson

kem• 9 years ago

I wish people understood this better.

I'm not a Go programmer, but I have a lot of respect for it.

If you're wondering "why Go"? Think of it as a modern version of C, at a slightly higher level, developed by the same people for slightly higher-level tasks. They made C for low-level stuff, and then picked up with Go for higher-level stuff. It's like C+.

C has been very successful in part because it's so simple in certain respects (although not in others) and I think Go will be successful for many of the same reasons. Go does what it does very well.

I think Rust is actually a great choice for something like Tor, but I wouldn't use Rust for some of the things I'd use Go for.

pcwalton• 9 years ago

Go doesn't have faster build times than C. Rather, it has a de facto single implementation that's fast because it doesn't do the same level of optimization that modern C compilers do.

The one improvement over C related to build time in Go is that you don't need include files. Granted, that's a big one. But modern C compilers make header parsing extremely fast.

psiclops2• 9 years ago

Go's concurrency model is based on CSP [0] and the previous languages Rob Pike worked on that were also heavily influenced by CSP.

[0] https://en.wikipedia.org/wiki/Communicating_sequential_proce...

solidsnack90003• 9 years ago

I really like Go, because it just feels right.

I think we've seen a few posts like that on HN and now I wonder, what that means exactly. What else falls under the "feels right" umbrella for you?

giancarlostoro• 9 years ago

The success for Go was in the out of the box libraries, and by this I mean you didn't have to "pip install" , "npm install", "nuget install" - no idea what the command is for nuget, always used the GUI, but you get the idea... For example when I was trying to learn Go for the first time I saw how easy it is to dive into Web Development. I can just import a package that came with Go itself, and I would be good. I think also the way it pulls in packages is part of it's success as well. It's not a traditional package manager (yet) but it worked. You put in a repository from the web and it pulls it, it made sharing and reusing code really easy. I never had issues just running Go code either if it was a codebase that was full of mostly Go code.

Btw I don't consider myself a Go developer, at work I use C# and Python. I would love to learn Rust, used to be the other way around, but I've lost hope in Go and have taken a second look at Rust, I love the direction Rust is going overall, but I understand why Go got so popular so quickly, it came at the right time with the right amount of working parts.

ilovecaching• 9 years ago

Don't frame the question as language X vs language Y. Think about the ecosystems instead. Golang provides a batteries included API that allows developers to rely less on third party libraries with shaky SLAs, a promise of version stability, and great tooling that has increased consistency of coding styles across projects. It's a simple language with a a low barrier to entry, and the Google brand encouraged earlier adopters to take risks and build products that luckily became very popular (Docker, Kubernetes, etc). It looks a lot like the languages that are already popular in Industry, without providing any real groundbreaking concepts. Building software is an economic value proposition, and it turns out that there are many ways in which a language's ecosystem can provide good value for the investment beyond the pros/cons of the language itself, and often these advantages are easier to measure from a managerial standpoint.

loup-vaillant• 9 years ago

From the perspective of a user, you make some sense. From the perspective of a language designer however, great ecosystems are no excuse for crappy languages.

The Go language doesn't have to be crappy to provide batteries included API, version stability, and great tooling. No generics and no sum types? Those are no longer groundbreaking, they're the bare minimum.

Old languages like C at least had the excuse of being created a long time ago, when we possibly didn't know better (and computers were much slower). Go's creators however don't have that excuse. They screwed up, plain and simple.

eternalban3• 9 years ago

Not sure why Ada is not taking off these days. It is mature and has all the safety and concurrency bells and whistles:

http://www.adacore.com/uploads_gems/07_safe_secure_ada_2005_...

https://en.wikibooks.org/wiki/Ada_Programming/Tasking

http://courses.cs.vt.edu/cs5204/sp99/Overheads/6UP/6UPCSPand...

bitwize2• 9 years ago

+1 for Ada. Rust is basically hipster-compatible Ada, which explains why Rust will be adopted and Ada will not.

the_mitsuhiko• 9 years ago

Repeating this nonsense won't make it any more correct.

SomeStupidPoint• 9 years ago

I mean, isn't SPARK basically meant for high assurance things (like Tor should be)?

eternalban3• 9 years ago

& considering that Naval Research Laboratory gifted us with Tor it is surprising it wasn't written in Ada to begin with.

nickpsecurity12• 9 years ago

Case in point:

http://ironsides.martincarlisle.com/

http://www.adacore.com/uploads/technical-papers/SPARKSkein_S...

http://www.anthonyhall.org/c_by_c_secure_system.pdf

bluejekyll6• 9 years ago

1) Two words: "begin" and "end";

2) Unix, C is so fundamental to building software, that I think any language that doesn't share syntax with it is doomed. Having a common syntax helps in learning new languages, IMO, and can also be a launching point for differing semantics...

RandomOpinion• 9 years ago

"BEGIN" and "END" aren't all that bad. Some of us got our start in software using Turbo Pascal long, long ago.

scott_karana• 9 years ago

"End" didn't stop Ruby from taking off... admittedly a different crowd.

nickpsecurity12• 9 years ago

I'll give you it's overly verbose even though quite a bit of it was justifiable. The theory of Ada's designers was that people read software more than write it. So, the syntax should be designed to facilitate catching errors in maintenance mode, during extensions, or during integrations. It's done phenomenal at that per industrial, case studies despite having been invented in the 80's when lots of language decisions were still being debated.

EDIT: I keep thinking a different language that acts as a front end w/ a better syntax might be a good idea. It outputs Ada that integrates with the tooling ecosystem. Also, a seemless FFI for C libraries like Julia's.

kbenson2• 9 years ago

Regarding #2, there's definitely a penalty a language pays for not following C-like syntax and semantics, as people consider it "harder" to learn, because people generally discount the time they've already put into learning similar languages when considering how easy a language is to learn. I suspect someone that knows Lisp but not C might find Clojure easier to learn than Rust, or even Python. The less software in use written in C, the less fundamental it is.

There's also a penalty a language pays for following a C-like syntax and semantics but diverging in specific but significant ways. In this category, I present Perl, which much of whole generation of users decided to treat like C, and got very confused and upset when it didn't always behave like they expected (which I maintain is because they didn't actually understand the language as well as they thought). There's a penalty for being very like something else to the point that people can mostly ignore the differences, but occasionally those differences come out to bite them if they haven't actually learned what they are.

twic2• 9 years ago

I looked into Ada last year. Getting a toolchain working sanely on a Mac seemed quite a lot of work; it's not in Homebrew, the MacPorts version has some weird bootstrap process, and the various random versions available for download had a murky mix of license and implementation issues that i don't remember in detail.

So i read documentation instead. Ada mostly seems like a pretty sensible language. Its story on memory safety, though, seems to be "don't use dynamically allocated memory", or at least, if you do, you're on your own. It doesn't have anything like Rust's safety guarantees, or those that fall out of having a garbage collector.

If it was 2007, and Ada was a bit more easily available, I'd be agitating for it. But i think its shot at the open source big time has passed.

twic2• 9 years ago

Someone left a comment mentioning that i could download a compiler from AdaCore, but seems to have deleted it. I was going to thank them! AdaCore's website is not very helpful, but googling turned up:

http://libre.adacore.com/download/

You can download the 'GPL edition' of GNAT [1]. This is a complete toolchain for Ada 2005, with the same actual compiler as the commercial version, i think. The libraries are GPL'd, so if you distribute a binary built with it, it has to be GPL'd. So, no Ada 2012, and no way to distribute binaries willy-nilly, but certainly enough to explore the language.

[1] http://libre.adacore.com/tools/gnat-gpl-edition/faq/

_0w8t2• 9 years ago

Ada allows to return from a function a stack-allocated array of runtime-dependent size. This alone removes wast number of cases when one needs to allocate memory dynamicalky in C/C++.

I always puzzled why even C++ does not provide any facilities like that out of the box.

jeffdavis2• 9 years ago

(1) Better explanations about how to live without heap allocations or how to use them effectively in Ada. It feels like unchecked_deallocate is wrong, but then I don't really see explanations about what programmers should do instead.

(2) Compiler and runtime licenses. I always feel like the version of Ada I get is either somehow not the "best" one or has some lingering license issue with the runtime library. I don't think this is necessarily true, but it's easy to get that impression.

(3) Packaging, libraries, and documentation. Many developers expect a tool like cargo to be available and have access to a wide variety of open-source libraries. They also expect lots of tutorial documentation and blogs.

pjmlp12• 9 years ago

I advise you to watch this presentation.

"Memory Management with Ada 2012"

https://archive.fosdem.org/2016/schedule/event/ada_memory/

z3t4• 9 years ago

we all want to be safe. no one can say no to safety ...

socmag10• 9 years ago

I live and love in Amsterdam and Golang seems to be the quintessential hipster language for this quintessentially hipster city.

Booking.com's soup du jour if you will.

Basically, if you aren't using JavaScript in a web shop, and you claim to be a "full-stack" "ninja", then probably you are using Go around here as a jobbing programmer.

I know that sounds terribly cynical and obviously a massive generalization but that is my personal experience.

I don't have a problem with Go per-se, but I do have a problem that a lot of people seem to go to extreme lengths to defend what someone else mentioned is frankly a pretty "mundane" language, citing memory safety, but more strangely portability and performance as its wonderful virtues.

When I meet with the zealots of the Go community around here I often have to quickly excuse myself with good grace.

Seriously, if that's what you want... go (pardon the pun) use C#, it is a saner and more expressive language, has a higher performance runtime and is way more portable with less vendor / career lock-in.

Go... I just don't get it.

Positives... Has a fairly nice package manager like npm.

Isn't made by Microsoft if that's your thing.

Oh and heaven forbid you don't have to think much... until you do because it is slow.

For a project like Tor though, which is damn slow as it is, I think it is totally the wrong choice.

Developers, please suck it up, get over it and learn C/C++ and some variation of Lisp.

Fine, go play with different languages for fun stuff. Use python for ML, try serious meta-programming in D, or jump into Haskell for kicks.

On the other hand modern C++ really can be a very safe language to work in if you can be bothered, and you are only deceiving yourself and your project going with something less sympathetic to the machine itself, especially if you are working on infrastructure level systems.

Sorry if this offends anyone, and of course it is just one opinion, but I think I'm being fairly nice as compared to what Linus might have said in comparison.

Basically stop it with the hobbyist shit. You are horrifying me and probably many others.

You aren't writing a web page here, so please treat the project seriously.

IMHO

FiloSottile• 9 years ago

You do realize they didn't pick Go, yes?

socmag10• 9 years ago

Yes, but we got into a heated discussion about Go earlier in the thread. And it got my blood pressure up. :)

hamburglar2• 9 years ago

Developers, please suck it up, get over it and learn C/C++ and some variation of Lisp.

I've got over 20 years of writing production C/C++ under my belt and I know Lisp. So I've "sucked it up." Am I allowed to like Go now?

It never ceases to amaze me how many people are bothered about other people's taste in something so mundane. If you don't enjoy programming in Go, don't do it. I personally think it feels light and easy like a scripting language, but with more C-like performance.

I like the fact that its stdlib is very complete, so I can sit down and do pretty much any kind of small project with zero external dependencies, I like the fact that the core libraries are well-designed so the interfaces are consistent and easy to learn, I like the fact that it produces statically-linked binaries so deployment issues are minimal, I like the simplicity of the CSP approach for a lot of concurrency problems. I could also come up with a list of things I don't like about Go, but I'm not going to bother, because I've decided that on the whole, I really like it as a tool.

Btw: > Positives... Has a fairly nice package manager like npm

One of my biggest complaints about golang is that it really doesn't have nice dependency management. It's terrible and probably the one issue that has made me seriously consider walking away. :)

socmag10• 9 years ago

Sure, what you say is true but equally so or better for .Net or Java as well, plus they are generally faster and in the case of C# a better language imho. Why not them?

You are right, the standard library is a pleasure to work with in Golang and for the most part I trust it, but hey anything you can do in that library is easily reproducible in more or less any other mainstream language.

I really don't have an axe to grind with the language itself, it's a fun and productive thing to use for sure.

My main problem is with people who spend most of their time writing customer service applications thinking that the same language they are ultra successful with in that space is suddenly appropriate for writing systems software, because, ughh "safety"?

There is a heck of a lot more to safety than buffer overruns, and if you can't formalize abstractions for dealing with memory usage patterns, one might argue you have no business writing Tor in the first place.

Hah well compared to C/C++ dependency management, Golang does pretty darn well!

eropple2• 9 years ago

If you don't enjoy programming in Go, don't do it.

Ordinarily, I would agree; I don't really care what people do when writing application code. The way it's filtered into devops tooling makes the choices of Go peoplea problem for me, though. If I'm going to be stuck with a language with bad error handling and inexpressive typing, I'd rather it be Python or Ruby so at least I can leverage dynamic typing instead of bad static typing.

kodr• 9 years ago

Go doesn't really have a package manager (and that's a problem), it pulls the sources of the dependencies from github for example. There are multiple tools [1] to manage versioning, but Google only released recently (end of january) an official tool, still in beta.

[1] https://github.com/golang/go/wiki/PackageManagementTools

omidraha• 9 years ago

Concurrent thread for this safer language: https://news.ycombinator.com/item?id=14013444

poland2• 9 years ago

I hope Linus can develop a new language.

viraptor• 9 years ago

I like their approach of chipping away pieces instead of porting the whole system. It will result in better modularity and some generic libraries that can be used by other people. Seems like a win-win.

muraiki• 9 years ago

How has this community reached the point where the vast majority of comments on this decision are arguing about Go, which isn't even the language that they picked?

concede_pluto• 9 years ago

pg's early essays like http://www.paulgraham.com/avg.html and http://www.paulgraham.com/power.html drew a crowd who want to see language design move in the polar opposite direction. From that viewpoint, Tor only barely dodged a bullet.

dguido2• 9 years ago

Literally none of this matters. None of the flaws exploited in Tor are memory corruption flaws. They are entirely architectural and design flaws. No attacker cares if they move to Rust. _NO ONE_.

steveklabnik4• 9 years ago

None of the flaws exploited in Tor are memory corruption flaws.

http://www.cvedetails.com/product/5516/TOR-TOR.html?vendor_i...

This list is full of memory safety issues.

dguido2• 9 years ago

None of the flaws exploited in Tor are memory corruption flaws

_exploited_ is the operative word here.

Show me the list where people wrote exploits for the bugs you point out, or where someone abused them to de-anonymize a Tor user? There aren't any.

imron• 9 years ago

Rust is almost always going to win for any projects that take a 'gradually replace' approach, simply because it can interact so easily with C code in both directions (i.e. C calling Rust and Rust calling C) in a way that other languages can't, or can't without significant efforts/hassle.

The Rust team showed great foresight for bringing about those changes before the initial 1.0 release and it will pay off in spades.

It's a key step in removing barriers to entry:

https://www.joelonsoftware.com/2000/06/03/strategy-letter-ii...

Tor in a safer language: Network team update from Amsterdam

Related Stories