agentS

I am sorry that you are sick of the HTTPS everywhere movement.

But forcing your sites to use HTTPS will also prevent your users from unwittingly participating in DDOS attacks on other sites (e.g. https://en.wikipedia.org/wiki/Great_Cannon). Consider it herd immunity.

Also, to respond to some of your other anti-HTTPS comments:

regarding overhead: people are also working hard to minimize the amount of overhead inherent to TLS. For instance, TLS 1.3 will establish an encrypted connection in a single roundtrip, and is capable of resuming encrypted connections in zero roundtrips with application opt-in (see https://blog.cloudflare.com/tls-1-3-overview-and-q-and-a/). The encryption itself has fairly ubiquitous support in hardware, making most of them ridiculously fast.

regarding CAs: with HTTP you are implicitly relying on the honesty of people in the network path. With HTTPS you are implicitly relying on the honesty of the intersection of a) people in the network path, b) people who control a CA. This is strictly fewer people than with HTTP. People are also working hard to solidify our faith in the set (b), by requiring Certificate Transparency for all new certificates, thereby ensuring that misbehaving CAs can be detected, and drastically raising the cost of mounting a CA-based attack.

You say "What if I for what ever reason don't want to use HTTPs", I you'll have to layout some of those reasons explicitly. You'll probably find that people are working on all of them.

In general, the default expectation on the web should be encrypted and authenticated (i.e. only both endpoints can read/write the data). Once we live in that future, asking for the ability to allow plaintext network traffic will seem a lot like asking modern programming languages to explicitly allow buffer overflows. The language designer would be justified in saying "No", and ignoring you. The considerate language designer might ask "why would you want that", and try to address your real need. But they would still never actually give you what you ask for. This may be "taking away choice" in the same sense that mandating airbags is "taking away choice", but people shrug and accept it because the baseline has moved.

Which leads to the question, why is google doing this? They, you, could easily promote AMP pages while not masking the real URL!

Perhaps to allow the content to be served from a CDN (over HTTPS), without requiring the site to CNAME over their domain to Google.

If webmasters are willing to CNAME over their domain to a caching proxy, then a less intrusive design is possible[0], such as the one recently announced by Cloudflare[1].

[0] https://github.com/ampproject/amphtml/blob/master/spec/amp-c... [1]: https://blog.cloudflare.com/accelerated-mobile/

Monomorphization is not a reasonable option, in my opinion.

It forces the compiler to accept some truly awful running times for pathological cases. Atleast quadratic, probably exponential.

For languages that have reflection or pointer maps for GC or debug information for types, it can force large blowups in space as well. Go has all three of these.

The implementation would likely require runtime code-generation (or accept warts like Rust's "object safety").

Indeed, all of Ian's proposed implementations are polymorphic and seem to avoid each of these issues at first glance. The only advantage of a monomorphic implementation is performance, and considering the downsides, this'd be premature optimization forced by a language spec.

If its actually performance critical, I imagine it'd be easy to write a program that monomorphized a particular instantiation of the generic types. Indeed, the compiler would be free to do that itself, if it felt it would be worth it. Small, guaranteed non-pathological scenarios for instance.

Where if you guarantee monomorphization in a language spec, the compiler and all users are forced to accept the downsides in all instances, in exchange for often meaningless performance gains (example: any program that does computation then IO).

But you cannot write a non-blocking Go function in the same way.

The caller can make function "non-blocking" by wrapping the call in a goroutine themselves. (There's some subtle differences, but they are mostly irrelevant here). For this reason, I'd say there is (almost) no reason to introduce asynchrony in your API in the way you suggest. The rest of your post built on this example seems shaky to me, since it seems built on an example API that doesn't need to exist.

I'd say that "you don't have to care whether your code is async or not" is a overstating the case. I would append the qualifier "unless you're introducing concurrency". Considering that almost no low-level APIs are asynchronous, this usually happens rarely (or happens in low-level code like the HTTP server). Examples that have come up for me: making N parallel RPCs, writing a TCP server. In those situations, you care about async vs not.

In event-loop based systems, it seems like async is in my face all the time, even when doing things that are entirely sequential.

... Reactor pattern ...

I don't understand, and the link seems unclear. Perhaps a more direct question: I get a request X, and I need to consult a backend service to answer the request. Do I write synchronous code calling that backend? Or do I have some callback mechanism?

... generators or async/await

Ah. This perhaps answers my question. Both of these are essentially compiler-written callbacks.

If this is going to be like C#, then I presume there will be a thread-pool where user code will execute. It seems like a non-ideal story for concurrency. Users will have to take inordinate care not to call any blocking code; otherwise they will prevent one of the threads in the pool from doing useful work.

For servers that primarily speak RPC or HTTP, do you foresee Rust going thread-per-request or something more callback-y?

Without SSL, there is a trivial way to upgrade a temporary MITM capability to a permanent MITM capability using ServiceWorker.

A binary protocol's parsing is usually something like read 2 bytes from the wire, decode N = uint16(b[0] << 8) | uint16(b[1]), then read N bytes from the wire. A text-based protocol's parsing almost always involves a streaming parser, which is tricky to get correct, and always more inefficient.

Besides, I think this is a moot point, because chances are that less than 100 people's HTTP2 implementations will serve 99.9999% of traffic. It's not like you or I spend much of our time deep in nginx's code debugging some HTTP parsing; I think its just as unlikely we'll be doing that for HTTP2 parsing.

Also, HTTP2 will always (pretty much) be wrapped in TLS. So its not like you're going to be looking at a plain-text dump of that. You'll be using a tool and that tool author will implement a way to convert the binary framing to human-readable text.

Another way to put it is that the vast majority of HTTP streams are not examined by humans and only examined by computers. Choosing a text-based protocol just seems a way to adversely impact the performance of every single user's web-browsing.

Another another way to put it is that there is a reason that Thrift, Protocol Buffers, and other common RPC mechanisms do not use a text-based protocol. Nor do IP, TCP, or UDP, for that matter. And there's a reason that Memcached was updated with a binary protocol even though it had a perfectly serviceable text-based protocol.

If your GC is triggered after the allocated memory increases by X% (which is fairly common), then this technique is effective, since it lowers allocation rate.

Also, Go doesn't scan arrays that are marked as containing no pointers, so representing an index as a massive array of values has proven quite effective for me.

Out of curiosity, why do shells need to use fork?

Note: I am not familiar with the implementation techniques behind shells.

Out of curiosity, what form of "spying" do you think is enabled by Javascript that would not be possible without Javascript?

Edit: And since I replied to a small part of your comment, I should say that I disagree completely with your "few people truly care about their craft" statement. At least, I think that writing code that handles a lack of Javascript is only valuable if you have enough users to justify it. i.e. if you spend 20% of your time working on features for 0.1% of users, then you are doing a disservice to the rest of your users. Even more so if you have to compromise the experience for everyone else such that degrading is an option.

In some cases, you go out of your way to accommodate small fractions of your audience. ARIA and catering to those with disabilities is a good example. But turning off JS is a choice; one I respect, but feel no obligation to cater to. I think pages should show a noscript warning, but other than that, its a matter of engineering tradeoffs.

Note on encoding/json: you only need to mention the fields you care about. If there are extra fields, the parser will just scan past them.

For the record, Raft does mandate what happens in that split-brain scenario. Neither side will be able to elect a leader, and writes will halt. Electing a leader requires a quorum.

The attacker will merely reply on your behalf. Does not improve the situation.

Keeping average response time low is only part of the point of a circuit breaker. Another goal might be to reduce load on the back end service to give it a chance to recover.

Ah, I understand. I interpreted that comment as a list of 3 separate examples where centralization helps.

A nitpick:

Auto-correction for a user's contacts could probably be done on-device, although I'd guess that machine learning across all users will probably massively reduce your success rate. Consider an ambiguous correction; you accidentally type "Gob", but have contacts of "Rob" and "Bob". I imagine that ranking the suggestions can be improved using a globally trained model.

One of the reasons that spelling corrections are so good on Google is (probably) that they are machine learning models trained on query logs(1). i.e. if you search for "hacer news", and then without clicking on any results, issue another query for "hacker news" in a very short timeframe, then it will learn that "hacker" is a good suggestion for "hacer"(2).

Similarly for "Mark as Spam", Priority Inbox, Recommended Videos on Youtube, Voice Recognition on Android, etc.

Note 1: Yes, you could also do a pretty good job by having a model of your problem. i.e. computing a weighted levenstein distance where the weights are the probabilities of making that error. However, I'd argue that this would still be better with centralized data; you can compute much better probability vectors. And regardless, the best solutions in the field will be with the combination of both.

Note 2: All of the above is speculation. While I help write some of the tools that these guys use, I have no knowledge of how they write their software. This is just how I'd do it.

Note that gofmt sorts within groups. So if you leave spaces between groups of imports, it'll sort them all separately. Presumably, this sort of grouping is the reason it does it this way.

I vaguely remember reading somewhere that they figured out a way to run their collector on stock Linux; I think the price they paid in exchange was (even) lower throughput (trying to track down reference so I know I wasn't daydreaming) (yes, I daydream about GCs).

A while ago, I read through the kernel patch they provided, and it was basically adding batch APIs for mremap so you don't have to have a TLB flush for every call. Also because all mutator threads needed to be paused at a safepoint while the remaps happen, so the batch API has much shorter pause time.

I've been following along on golang-dev, and they seem to be making progress on precise stacks.

See https://groups.google.com/forum/?fromgroups#!topic/golang-de... for example.

That post both advocates for TLS-everywhere (which I support) and thinks it would be beneficial to drop HTTP Keep-Alive... Aren't you concerned about the latency hit? TCP has 1 RTT to setup, TLS has 1+ RTT to setup.

Also, TCP's congestion window grows over time; with your proposed model, you'd continuously open connections with tiny congestion windows, rather than a few connections with growing congestion windows.

I think all it'd take to change your mind is to load Facebook or Twitter with SPDY and Keep Alives turned off...

You never know when you have enough coverage to rule out NPEs or any other bug. And to get confidence about lack of NPEs you want to have coverage of all lines involving dereferences, which means you need near 100% test coverage to have a reasonable level of confidence.

This is not true. In the example I spoke about above, if I take a CacheClient, and a ServiceXClient when my type is being constructed, assign them to local fields and then never modify that field again, then I don't need to exercise every dereference of these fields, just one. And again, I don't test that my code handles NPEs, I test that my code does what it is supposed to, and in the process of doing that, NPEs get flushed out.

Generally types are far more concise and guarantee more than tests. I find 5 lines of types more readable than dozens or hundreds of lines of tests.

I think you are viewing this through red-black-tree colored glasses. Specifically, you believe that a lot of code has mathematical constraints the way that example did. To me, this is an extremely remote possibility. I think if you tried to encode even the smallest real-world example of this, say a service implementing a URL shortener, you would run into a wall.

Your code is much safer simply by construction, so I am not sure what you mean here.

I should have said more reliable.

I do think non-nullability aids reliability, but that having sum types, proper pattern matching and parametric polymorphism aids it even more. And Go lacks all of these.

Truly, it baffles me that people still harp on the reliability aspect. It is quite likely that every piece of software you use day-to-day is written in a language with nullability, without pattern matching, and no sum types. Most of that software probably doesn't even have memory-safety (gasp!). Probably every website you visit is in the same sorry state. I'm sorry, but your arguments would be far more convincing if the world written in these languages were a buggy, constantly crashing hell. It's not.

Testing is a cost. It is more code to write, more code to maintain.

I'm not saying you write tests to protect against NPEs. I'm saying that you write tests to ensure correctness of your code, and as a side-effect NPEs are flushed out of your code. This is my theory explaining why NPEs are not a timesink for me.

They guarantee the correctness of the invariants of the Red Black Tree, and they easily replace hundreds of lines of test code which give no guarantee.

Code with mathematical invariants seem like such a niche area, though. The average type of test that I write is "ensure your service calls service X first checking for values in a cache; ensure that it can handle cache unavailability, service X unavailability, cache timeout, service X timeout". Maybe you could figure out a way to encode that in a typesystem, but I'd wager that it wouldn't be as readable as the equivalent written as a test.

Scala shuns null - and only has it for Java interop. All Scala developers I've discussed this with program as if null did not exist, and never use it to signify lack of a value.

That's exactly right. Languages with nulls, and mutable shared state are perfectly reasonable to use if programmers do the right thing by convention.

When "arguing with a compiler"...

I misspoke. I was thinking of the learning process, not the process of writing code.

Although my current focus is backend systems, I have worked across the spectrum in the past. I've worked (not including languages I dabble at home) in JS (for browser UIs), Java (for Android UIs and servers), Obj-C (for iOS UIs), Python (for servers and scripts), Scala (for servers), Ruby (for servers), and C++ (for servers). These languages represent a wide spectrum on the dynamic to static spectrum. I don't find myself writing much safer code when I go from JS to C++. The same for a shift from Ruby to Scala. The same for Python to Java. These shifts are relatively large on the type-system spectrum, as they go from dynamically typed languages to statically typed ones. You claim that a significantly smaller shift, removing nullability from a language would be a big deal for reliability. That doesn't seem likely to me.

I think people tend to underestimate the amount of time they spend on problems...

Again, the implication is that people who use languages with looser type-systems than Haskell spend lots of time dealing with the problems that you mention. In my experience, that is not the case. You can claim that I'm underestimating the impact of such bugs if you'd like.

When you implement a red-black-tree, do you not spend any time testing your invariants?

Testing is, as far as I can tell, the reason that these problems don't come up. In the process of solving all of those problems I listed above, you tend to write a bunch of tests that exercise the same code that is run in production.

It's weird that you bring Twitter as an example, as they canned a dynamic language solution for a static language that is in many ways very similar to Haskell.

I assure you that this transition is far less complete than you might think, and even when complete, will have more Java than Scala. Note that both of these languages allow shared mutable data, both allow null pointers.

They all fail to scale well, both performance-wise and maintenance-wise. Statically-typed systems scale far better along both of these axis.

Oh, I agree. Go is statically-typed. However, you're advocating going even further along the spectrum, and I'm saying that going further brings diminishing returns, and starts costing you in terms of available engineers, and your productivity in writing code. I think Go occupies a good point along this spectrum, where I can write robust code without arguing with a compiler.

I never said the Go runtime will crash, but your programs will, because of unsafe nullability, mutability in the concurrently shared state, and various other problems in Go.

Personally, these problems are not what are a time-sink for me. The problems that I spend a vast majority of my time on tend to fall into two categories.

First, there are design problems. i.e. how do you model your data to be queryable, how do you architect services to make them resilient to machine failure, how do you monitor services, how do you route logs, etc. With these problems, Haskell doesn't help here. Go doesn't help here either.

Second, there are operational problems. i.e. your FS might randomly corrupt some data you stored previously (remember to checksum), your caches might get out of sync (fun problem if you ever go multi-DC), your service has shoddy backoff and DDOS's a failing downstream service, etc. With these problems, Haskell doesn't help here. Go doesn't help here either.

Am I saying that there's no value to the fact that Haskell solves these at compile time? Of course not. Just that the relative amount of time that would save for me is not my deciding factor when picking a programming language.

Another way to think about it is that people have (I include myself in this category) written large-scale projects in dynamic languages which have all the problems you mentioned plus a few more. And yet, the developers at Twitter, or Facebook, or Reddit don't spend the vast majority of their time face-palming at type-errors or NPEs, or ConcurrentModificationExceptions. They have other concerns.

No, its more like this. You download a series of truncated hashes; you generate a bunch of permutations of your URL (strip the query params, strip components of the path/domain), you hash those, check them against your local list. If you get any matches, you request an expanded list from Google, giving them the truncated hashes that matched. This gives you a cacheable list of full hashes; you check your matched hashes against those full hashes, and if any match, then its a match.

tl;dr No, its not that easy.

FYI, you should replace DocsHandler with net/http's FileServer.

Reasons: 1) the implementation is not vulnerable to path traversal attacks (i.e. if url.Path == "/../../../../../etc/passwd"). 2) It automatically handles index.html 3) It uses "/etc/mime.types" and other system files to automatically set mime types 4) It streams the file back instead of reading it all into memory.

> The JVM engineers have been free to do whatever they want in terms of optimizations at runtime, like code inlining or escape analysis.

Both of these optimizations are present in this release of Go. Granted, there is no doubt more inlining opportunities exist, and better escape analysis is possible, but I think you'd want to chose different optimizations to highlight your point.

What specifically makes you think that there'll never be a precise, generational GC for Go? What about Go makes it too low-level for such an implementation. Unless I'm missing some key issue, I definitely think its possible...

> Go on the other hand feels like a hack on multiple accounts.

Again, what specifically about the language gives you this impression?

> If Go will indeed get popular, it will join a long line of languages that people hate, because they have to maintain code-bases written in it, with no easy way out.

Forget about evidence, you're making claims with absolutely no reasoning behind them. What is different about Go as opposed to say Python, or Erlang, or Node, or C++ in this situation? A code-base written in any of these languages needs maintenance... I don't see how Go is any different in this situation.

I assume you're calling the go command a few times in parallel, hence the improvement with -j4. I wonder if it would be faster to aggregate all the packages you need to build in a variable, and call the go command with all of them.

i.e. instead of issuing "go build foo/mylib" and "go build bar/myotherlib" in parallel, just issuing "go build foo/mylib bar/myotherlib".

I say this because I remember there being logic inside the go command to parallelize builds already, see the "-p" flag in "go help build". In particular, the go command will be able to avoid statting all the files, comparing mod times twice, recomputing the DAG, etc. Although, considering how fast it runs anyway, this speedup might be negligible...

I guess you could roll your own... or use the one from the standard library: http://golang.org/pkg/net/http/#ListenAndServe