HTTP 2.0

http://tools.ietf.org/html/draft-ietf-httpbis-http2-04

by abhshkdz • 13 years ago

434 322 13 years ago

tools.ietf.org

abhshkdz• 13 years ago

Source code & issues tracker: https://github.com/http2/http2-spec

Wiki: http://tools.ietf.org/wg/httpbis/

samwillis5• 13 years ago

I understand the motivation to going to a binary protocol for a SPDY inspired HTTP 2.0 but I would have liked to see an ascii based protocol similar to what jgrahamc proposed last year [0]. I thought it was a much cleaner protocol to read and understand, and much more in tune with what the web is supposed to be. Why not keep the clever binary stuff separate in SPDY, endorse it though the IETF and keep HTTP ascii?

[0] http://blog.jgc.org/2012/12/speeding-up-http-with-minimal-pr...

steveklabnik16• 13 years ago

There were so many people just calling for rubber-stamping SPDY as HTTP 2.0 without any changes that frankly, I feel lucky that we're getting revisions at all. The editor of the draft is a good guy, and I trust him to make good changes.

I think it's pretty ludicrous that so many people are willing to let an advertising company basically re-write the way that every computer on the web talks to each other with very little questioning, but this is the place that we're at nowadays. For better or worse, the big vendors guide the standards process. I'd like to see more involvement from the little guys, but that has its own set of challenges.

aroch• 13 years ago

I think it's pretty ludicrous that so many people are willing to let an advertising company basically re-write the way that every computer on the web talks to each other with very little questioning

I don't think there's very little questioning...in anycase, Google's goal is to deliver ads to you in the most efficient and fastest way possible. The more you browse the web the more Google makes; it's in their interest to develop SPDY/HTTP2.0. So what is wrong with them doing it? IEFT spec drafts are public and they're audited (as SPDY has been).

Also, most of the specs that we love and rely on today came from "big vendors", its nice and all to say you want the little guy to be a part (and they should be) but it takes quite a bit of man power to develop, draft and finally get ratified a spec.

tptacek15• 13 years ago

What does Google's involvement in advertising have to do with the design of the SPDY protocol? Can you make a substantive criticism of SPDY based on Google's advertising incentives, or is this just innuendo?

9h1d9j809s• 13 years ago

I think it's pretty ludicrous that so many people are willing to let an advertising company basically re-write the way that every computer on the web

First, to call Google an advertising company makes no sense. It's a tech company. You don't call a newspaper an advertising company either.

Second, there were people involved in designing this, not just an anonymous corporation. You can actually see their names in the proposals. It's a good design, that's why it has been adopted.

smutticus2• 13 years ago

For better or worse, the big vendors guide the standards process.

It costs money to have people on staff who write IETF drafts and haggle them up to RFCs. Hopefully the standard isn't too degraded by the needs of Google in this instance and everyone benefits.

youngtaff• 13 years ago

Oh come on don't let paranoia encourage you to throw the baby out with the bath water…

The way Mark Nottingham ran the original CFP for HTTP 2 and the eventual adoption of SPDY as a starting point was very fair - it's all there in the IETF archives for anyone to see. From memory there were only two other proposals (from Microsoft and someone else)

The reason Google were able to get a new protocol up and running is because they have both heavily used web properties and a browser. They're also willing to carry out experiments in public.

As it stands HTTP 2.0 will be good for the little guys too, based on the testing I've done little guys will see an improvement in performance without needing to do all the merging that destroys cache lifetimes.

3rd party-content is the fly in the ointment to the performance improvements so we'll need to be much more careful about the performance of the 3rd party sites we include.

N.B. Apart from using their products I have no affiliation with Google

nodata• 13 years ago

Point him towards the github repo!

nileshtrivedi• 13 years ago

Genuine question: Is it really that big of an issue? We still have to use tools (telnet/curl) to inspect text-based protocols. We don't examine raw bytes by hand, do we? In other words, isn't every protocol ultimately a binary one and the issue here is only a matter of degree?

voidlogic10• 13 years ago

It probably already exists, but you could write a telnet that simply displays ASCII and hex side-by-side. I used something like this when I used to reverse engineer laboratory device control protocols (but it was a hardware implementation: http://www.atecorp.com/ATECorp/media/ProductImages/R/HP-Agil...)

Not to mention wireshark...

tptacek15• 13 years ago

An argument that it's a fake issue: the fact that best-practices compliant HTTP applications already tend to run under HTTPS, and that understanding what's happening at an HTTPS level in operationally-relevant ways (namely: are we performant enough? and are we secure enough?) already requires parsing of that binary protocol.

nmcfarl• 13 years ago

I think it is. We have a large amount of tooling in existence to inspect text based formats. Binary format inspection requires new tooling to be written and deployed. This is a major barrier.

I may be a bit old-school, but I learned HTTP via telnet, a tool definitely not written to inspect HTTP, and I still use it when I’m trying to debug things. Not having to install tooling is still something I take advantage of.

Obviously binary formats can and do succeed, and with sufficient backing tools will be written and deployed. But if HTTP hadn’t been so easily inspect-able I don’t think it would have been nearly as successful the 1st time around, when the benefits of the protocol where less well known.

And I do think some of the "culture of the protocol" will be lost moving to binary°. It just isn’t as hacker-friendly or at least newbie-hacker-friendly and that sends a message.

° Of course this is already happening with HTTPS, so this is probably not a winnable fight. And the benefits of binary formats are significant, so it might not even be a fight I want to win. Still something is being lost here.

josteink7• 13 years ago

Genuine question: Is it really that big of an issue?

Yes.

Text-based means everything is open for inspection and self-evident if well designed. This goes away with binary.

Not to mention this protocol will need to be implemented everywhere and used by everyone. This means everyone will need to understand it as well. Open text-based protocols support these requirements simply by being open and text-based.

As if there's not enough bad code around already, in any language not assembly/C/C++, the reduction in code quality and clarity, and amount of additional issues involved the second you change the word "text" to "binary" is staggering. Let's not go there if we don't have to.

Most debuggers today can dump strings fine. Dumping binary, while not impossible, brings extra hurdles. Putting hurdles into debugging, pretty much means you will end up with more buggy code.

As if that's not enough to make think "hmmm", there's the issue of future-proofing and extending the protocol. That is much easier to do cleanly in a text-based format. A binary HTTP 2.0 will be brittle and short-lived.

We don't examine raw bytes by hand

Because that is very, very impractical. And thus making that a de-facto necessity for working with HTTP 2.0 doesn't make much sense.

I say we should look at it the other way around: Considering everything a binary-format will cost us (just some of those mentioned above), what benefits does it bring us to justify this huge cost?

I say the answer to that is near none, and in an ideal world that would be the end of the discussion.

voidlogic10• 13 years ago

I would have liked to see an ascii based protocol

I disagree I think HTTP should have been a simple binary protocol from the start and HTML should have required compilation into a binary format.

How much work would have it really been? htmlcmp foo.html providing foo.bhtml. No whitespace, No end tags, one or two byte tags, etc. Strictness in the reference HTML compiler implementation could have saved the web from all the stuff outside the actual standard that browsers (and other tooling) now have to support (so they don't "break" the web).

I'm not suggesting anything as crazy as the flash binary format (I wrote a Java flash player once...), but when I started to write things like proxy servers and HTML minifers I was blown away by the extreme inefficiency of HTTP/HTML.

This is a step in the "right" direction IMHO.

samwillis5• 13 years ago

I believe that the "world wide web" took off for two reasons. It was completely free and it was incredibly easy for ANYONE to make a website. If you needed to "compile" all HTML it would have discouraged a lot of casual experimentation. Not everyone understands what a compiler is.

Just as an example I started playing around with HTML when I was about 12 (in 1998), it was easy and I got instant results. A year or so later I tried to learn Perl and quickly gave up because I couldn't get my first script to run. It was another year before I tried to "program" again and became hooked.

HTML and HTTP were successful because they were incredibly easy to understand. I wish we could keep it that way.

dangoor• 13 years ago

Despite the "extreme inefficiency", HTTP/HTML have managed to work successfully for a very long period of time (and even worked decently well on very slow hardware in the '90s).

There is actually a binary format that much of the web content is compiled into: gzip. It's remarkably effective.

I will grant that following Postel's Law means that browsers have more work to do to ensure that all kinds of "busted" stuff on the web continues to work, but I'd guess that, at this stage, that work is pretty small compared to everything else browsers are trying to do.

specialist• 13 years ago

HTTP should have been a simple binary protocol from the start

What are the advantages of binary HTML vs plaintext HTML + gzip?

I've observed many binary format efforts. Various binary XML, BSON, binary VRML.

The only efforts that made any sense (penciled out) were binary geometry representations eg enabling mesh compression.

(PS- I hate tagged formats like XML and HTML with the passion of a billion burning suns.)

jakejake2• 13 years ago

Compiled binary HTML seems to me would be the end to the open nature of the web. The fact that anybody can view the HTML source is probably irrelevant from a network point of view. But I consider it a fundamental aspect of the web.

schrodinger• 13 years ago

Doesnt GZipping HTML fix most of what you're complaining about?

rjsw• 13 years ago

The "standard" way to have done this at the time would have been to use ASN.1, either on top of an OSI stack or used in a similar way to SNMP.

josteink7• 13 years ago

understand the motivation to going to a binary protocol for a SPDY inspired HTTP 2.0 but I would have liked to see an ascii based protocol

You're not alone. I've opened an issue on the subject [0]. Feel free to add weight to it.

[0] https://github.com/http2/http2-spec/issues/169

klodolph• 13 years ago

That's a childish way of asking for what you want: (1) opening an issue in the wrong forum and (2) failing to familiarize yourself with the previous discussion threads. You know this is a controversial topic, please act more maturely in the future.

Any time a bug report is linked in Hacker News the community ends up spamming the tracker with our enlightened opinions. Show some restraint.

peterwwillis18• 13 years ago

This idea is bad for a couple reasons.

First, ASCII is inefficient. People don't interpret HTTP, computers do. Web servers and browsers. People only look at HTTP when they want to troubleshoot without any tools. With real tools, you can find out what's broken much quicker. And there's plenty of things you can miss without a real HTTP interpreter. Most hackers prefer to think of themselves as wizards that can spy 0's and 1's and tell you what the weather is. It doesn't make for a better protocol, though.

Second, we can already break HTTP responses up in multiple parts, using a novel idea called "multipart". It sucks and nobody has used it since HTML/JS found new ways of providing content. http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html

Third, it's a hack. If you want to improve the protocol, improve the protocol, don't just hack onto it to make it do what you want. I could make a horse and buggy go 60mph, but would it be a good idea? How about just designing a better buggy that is intended to go 60mph?

Fourth, fixed-length records are the wave of the future! It solves crazy problems like header injection and request integrity checking. Moreover, it makes for simpler, more efficient parsing of requests.

Fifth, redundancies introduced from the beginning of time need to go away, like terminating every record with "\r\n", or passing the same headers on every single damn request when once should be just fine for a stream of requests. Little inefficiencies like this don't go away if you just hack onto the same old protocol forever.

Sixth, the flow control improvements can make different applications more efficient by both not having to hold state of where and when traffic is coming and improving flow across disparate network hops.

Seventh, as to a general question "why use binary when we can compress stuff", what is smaller: a string of bits, or a compressed string of digits? Add to this that every header could have a 32-bit identifier (4 bytes) and you've got more efficient compression than gzip. Of course, most people would argue that you don't save enough time or bandwidth or cpu cycles to make a difference, but it would make for easier-to-write parsers, which would make working with the protocol in general more attractive. But then you have your binary-detractor-wizard-hackers and the whole conversation becomes an infinite loop.

romaniv3• 13 years ago

The same arguments can be applied to HTML, CSS, JSON, RSS and so on. I fail to see the crucial difference between those and HTTP. Or would you say the web as a whole should be binary?

Fifth, redundancies introduced from the beginning of time need to go away I wholeheartedly agree with this, but it doesn't automatically warrant binary encoding.

Human readability is a huge bonus in any protocol or format. Not because normally people read those protocols, but because people read ASCII and therefore they have good tools to work with ASCII.

bajsejohannes2• 13 years ago

First, ASCII is inefficient. People don't interpret HTTP, computers do.

I can't tell you how many times I've manually read HTTP. To be sure, it's insignificant compared to how many HTTP headers have passed through my computer unseen by me.

ASCII may be inefficient, but computers are really fast, people are not. I don't have any measurements, but in making a browser, HTTP header parsing/writing was never near a performance issue. Bandwidth wise it also tiny compared to that image file you'd inevitably download for every page visit.

And sometimes you don't have tools. Sometimes you don't want tools. Sometimes you want to use tools that work with text to analyse your problem.

(You other arguments might still stand, though :) )

antitrust2• 13 years ago

First, ASCII is inefficient.

Why aren't we using binary file formats anymore?

XML is more easily debugged when the machine tools don't work.

It's also more accessible, more easily created and modified, and thus, more available to a wider range of people than just web design professionals.

smsm424• 13 years ago

ASCII is inefficient, but nobody cares. For vast majority of Web users, which aren't working on scales of Google and Amazon, difference in performance couldn't even be measured. And for them easiness of use - low barrier of entry, basically all you need is a basic ASCII tool and you're ready to go - is vastly more important than completely immeasurable performance gains from using opaque protocols.

Of course, you can say TCP/IP is still binary, and it is true. But TCP/IP tools are built in every OS in existence now, so they do not form a real entrance barrier. Would HTTP tools be in the same position? I'm not sure - most HTTP tools right now are not standard and do not cover even HTTP/1.1 completely, what reason is there to expect they'd cover the whole 2.0 protocol properly and be widely standard and available on the level tcp/ip tools are? Which means much higher barrier of entry.

brokenparser6• 13 years ago

A binary protocol is bad for a couple of reasons, too.

First, OSI layer 6 called and it wants its old job back. It sat around connecting layers 5 and 7 peacefully since the dawn of the ARPANET, all while the HTTPbis guys were passing around messages back and forth trying to obsolete it.

Second, you can't type a binary protocol. Yet, you have to somehow make the server work without a client or vice versa (for the initial implementation). That's going to be a lot more difficult. With HTTP, you could literally hook up a teletype to the Internet, let it print incoming requests and type the response back to the user agent. I've done this occasionally on a terminal emulator for debugging purposes.

Third, the Internet is big-endian while most common processors in use today are little-endian. This is going to haunt peoples lives forever because you have to continuously convert between the two and although the conversions are orthogonal, the methods aren't idempotent (as opposed to converting a string to ASCII, or a text buffer to DOS style line endings).

You mention 32-bit identifiers as opposed to a string of digits. This is more error-prone than you think, two's complement isn't the only integer representation out there. Implementations written in C would have to deal with their underlying architecture as the standard allows for 3 different representations (so the compiler wouldn't help you out). Then there's signed and unsigned, either of which might not be available in the implementers programming language. You end up unpacking the identifier by hand, which may end up being slower than just looping through a string. ASCII is hardly an inefficient serialisation format.

Fourth, any fixed-length records are going to be useless at some point in the future. Several versions later the fixed length records are going to either point to an extra set of records tailing the 2.0 records or will simply have a (designated) backwards compatible value for consumption by older peers. With HTTP, we can add a header anywhere in the request or response except for the very top. We can even shuffle them at will without adverse effects on peers.

Fifth, it doesn't make sense to optimise a tiny fraction of the entire HTTP session. Any benefits are too small to be worthwhile and would therefore result in a net-negative to most implementers.

Sixth, you can still make improvements to HTTP without moving to a binary protocol. Not sending the same headers on every single request isn't one of them. HTTP is essentially a stateless protocol and every request could be handled by a different server. You can architect clusters of servers routing incoming requests however you please and satisfy every one of them correctly and efficiently. For starters, you can replace any underlying protocol in the stack with a more cluster-friendly protocol in transit.

Seventh, just because no one is using a given content type in HTTP (I think you were referring to multipart/related) doesn't mean the protocol used to transfer that content is bad. Heck, it's not even part of any HTTP standard.

guelo• 13 years ago

Plain-text formats have always been slower for things that are not plain text. But even 30 years ago, when computers were even slower, Unix designers decided plain text was still the way to go, because it was easier to debug and easier for humans to work with. No specialized tools required, no poring over hex dumps. HTML won over other document formats. JSON and XML won over other binary formats. Any coder can look at JSON and see what is being transferred, without the aid of anything but a text editor. Plain-text marshalling formats for binary data (e.g., base64) are still useful for pasting data into an email or adding ssh keys to authorized_keys with "cat >>". Tool support is not going to make SPDY any nicer.

kragen3• 13 years ago

Fourth, fixed-length records are the wave of the future!

1960s-style arbitrary field size limitations: the wave of the future! No doubt any day now we'll reorganize the internet around shipping punched-card images around, too. We could call the project the "Because It's Time Network", or BITNET.

Of course, most people would argue that you don't save enough time or bandwidth or cpu cycles to make a difference, but it would make for easier-to-write parsers.

Most people would argue that, yes; famously, Kernighan and Plauger did argue that in The Elements of Programming Style in 1974. By "easier-to-write parsers", you mean easier than dict(((key.lower(), value.lstrip()) for key, value in (line.split(':', 1) for line in header.split('\r\n'))))? Because I think that's going to be a pretty tough bar to fit under. (Yeah, I know you need another couple of lines of code if you're going to handle indented continuation lines, but you can get rid of those without returning your protocol design to the Summer of Love. You could also get rid of the .lower() and the .lstrip() while you're at it.)

Add to this that every header could have a 32-bit identifier (4 bytes)

Padding out sub-byte-sized values to fill out fixed-width fields: the Intelligent Man's Approach to Saving Bandwidth! Or you could just use one-letter names in an ASCII protocol.

[Fixed-length fields] solves crazy problems like header injection and request integrity checking.

Clearly we've never had parsing bugs in binary protocols full of fixed-width fields, now have we? Surely not bugs that produced security holes? Except maybe TCP, IP, and DNS. And X.400, and X.500, and X.509, and some of those were the fault of ASN.1 BER and DER, which are hardly fixed-width formats. And surely silently truncating a value to put it into a fixed-width field would never change its semantics, right?

as to a general question "why use binary when we can compress stuff", what is smaller: a string of bits, or a compressed string of digits?

Well, let's see. How many files do I have here?

    $ find | wc
       5946   17074  330147

Each one has a ctime, an mtime, an atime, and an inode number. The ctime is generally going to be the same as the mtime in this case, so we'll leave it out. I think they're technically 64-bit values in the current inode structure, but let's count them as 32 bits instead, since none of my files are from after 2038. The inode number is also 32 bits. So if we take these three 32-bit values per file, we have 5946×12 bytes, or 71352 bytes. And if we print them out as digits and compress them?

    $ find -printf %.10A@%.10T@%i | gzip -9c | wc -c
    46278

But wait, that string of digits isn't parsable; it's just digits. So let's add delimiters.

    $ find -printf %.10A@\ %.10T@\ %i\\n | head -3
    1373396230 1373338685 2359306
    1372967497 1372967489 5246218
    1365458166 1365458157 5248264
    $ find -printf %.10A@\ %.10T@\ %i\\n | gzip -9c | wc -c
    47506

So a compressed string of digits is a lot smaller in this case. But you could argue that that's just because my data is highly redundant, since most of the timestamps are going to be within the current few years, which is true. But then, most data is highly redundant. How bad can it get, in the worst case of representing uniformly distributed random 32-bit values as compressed strings of digits, with spaces between? It adds about 44% overhead:

    $ dd </dev/urandom bs=1024 count=1 | 
      od -w1024 -l | tr -s ' ' ' ' |
      gzip -9c | wc -c
    1+0 records in
    1+0 records out
    1024 bytes (1.0 kB) copied, 0.000622635 s, 1.6 MB/s
    1460

> Little inefficiencies like this don't go away if you just hack onto the same old protocol forever.

But you know what? They do go away if you tunnel over a deflate-compressed VPN, or if you increase the available bandwidth by a few percent. Maybe instead of trying to take us back to bug-prone 1960s designs you should be working on that.

stiff• 13 years ago

what is smaller: a string of bits, or a compressed string of digits?

What are you talking about? You could have had something on your mind, but this is a terrible terrible juxtaposition of words...

rfugger• 13 years ago

The Art of Unix Programming by Eric Raymond makes an excellent argument for textual protocols in Chapter 5:

http://www.catb.org/~esr/writings/taoup/html/textualitychapt...

neuroscr4• 13 years ago

Real tools? I would argue that there are way more real tools for debugging ASCII-based protocols than there will ever be for binary-based one. ASCII-based protocol is highly compose-able by allows us to separate intent (contents of the message) from ever-evolving encodings (rebasing, encryption, compression, etc). In this world of fighting complexity, why are we not favoring the simple?

zobzu2• 13 years ago

I strongly disagree with "protocols are observed with tools we don't need ASCII".

That's pretty annoying to see this kind of thinking. The reason why everyone codes in JS and uses HTML, CSS is because its ASCII. Its easy to understand, hack, etc. Same reason python is so popular. Even Go, is pretty simple like that. Sure its languages vs protocols, but the reasoning is exactly the same.

And in fact, the comparison works with protocols as well: SMTP, IMAP, HTTP, IRC are EXTREMELY easy to understand and code for. Binary protocols are a huge PITA to code for. The argument that you're going to use a lib or whatever tool just doesn't hold any water. You want to understand what exactly happens.

Thats how everyone learns, etc. I could write my own SMTP, IRC clients when i was 10. I could understand it. It works. No way I could understand fully the documented binary protocols. I tried, and it was just too painful and not fun at all (hey, I was 10).

I'm not certain the added performance of using a binary format and some of the other advantages are really good enough to make the world unable to understand whats going on anymore by just looking at it.

Sure purely technically speaking, it sounds like "binary is the way to go" for pure performance.

But if you think about it, making hacking around that stuff a niche thing is perhaps a much greater loss. Even the reliability of a binary protocol is VERY arguable.

In fact I'll put a last comparison. Shell pipes and ASCII. Many tried to replace them with smart binary protocols, objects, etc. Its cool. Its more powerful. More efficient. At the end of the day tho, a quick hack with regular pipes transferring ascii is just easier to understand and we all use those - not the fancy binary objects.

H3g3m0n• 13 years ago

There is no reason why someone couldn't build a tool to send/receive binary responses and print them out as text. We have wireshark anyway. Just need a cli translator or a program that lists all the possible valid inputs for a given connection state and lets you select one (and allows for raw input just in case you want to try and break something).

For most cases you will probably be able to get the old HTTP protocol anyway. The only potential issue would be for people who are implementing the HTTP 2.0 protocol itself, just about everyone else in the world will be using a webserver or at least a library.

In fact raising the bar to prevent people developing their own HTTP 2.0 implementations would probably be a good thing, it would cut down on the number bug riddled implementations.

In any case I don't think it makes sense to slow down the whole internet just for a few developers.

Binary could also see the possibility of hardware implementations. Think webserver on a chip.

abionic• 13 years ago

no matter what all says, it's all because Google researched on SPDY

charonn0• 13 years ago

I agree. HTTP, like many core internet protocols was designed to work with any client setup, even someone sitting at a telnet terminal sending hand-rolled HTTP requests. FTP is just the same, and SMTP, etc. While very few people these days will actually wind up using telnet to access network resources, it is a very successful design and one which ought not be abandoned without considerable thought.

bitwize7• 13 years ago

HTTP 1.x was fine for the early 1990s. It's 2013 and Windows has won. Unix is dead.

Maybe you have a nice Ubuntu box and that's fine. I'm talking philosophically. Modern developers favor the performance and functionality advantages that large applications, deep APIs, and binary communication protocols provide over the transparency advantages that small, highly focused tools, orthogonal APIs, and human-readable communication protocols provide.

olentangy• 13 years ago

Unix is dead?

Aren't Mac OS X, iOS, and Android basically unix OSes?

I'd say Unix won.

antirez2• 13 years ago

Congrats to Google for ruining the internet standards in an official way, without strong proofs of big significant improvements over the simpler past protocol.

tptacek15• 13 years ago

Don't confuse the cart and the horse. The IETF standardization process has never been less relevant than it is today. The reality of deployment of a "next-generation HTTP" involves two things, neither of which the IETF has any say in: (a) whether anyone can improve HTTP as it runs between a mainstream browser (or some as-yet unforeseen browser replacement) and a content serving backend in a way meaningful enough to drive adoption, and (b) buy-in from the major browser owners.

That's the way it's supposed to work; the horse is meant to drag the cart. The RFC database is littered with cart-led insurgencies that went nowhere. If binary HTTP is one of those, it'll join them as a historical curiosity.

antirez2• 13 years ago

My fear is that now that Google controls a significant part of HTTP traffic, plus the client side of the story (Chrome + Android), it is in a condition to force things in one way or the other...

jyrki• 13 years ago

Could you elaborate on the part of no strong proofs? I was under impression that SPDY fixes lot of flaws with HTTP/1.{0,1}

marcioaguiar• 13 years ago

I think it's Google interests to make a simple and faster protocol.

RyanZAG• 13 years ago

  Only implementations of the final, published RFC can identify
  themselves as "HTTP/2.0".  Until such an RFC exists, implementations
  MUST NOT identify themselves using "HTTP/2.0".

Here we go again - as with -webkit in css, expect 'HTTP/2.0 DRAFT' or similar to be the new http 2.0.

dragonwriter2• 13 years ago

I think in the real world its not "HTTP/2.0 DRAFT" but "SPDY".

sktrdie2• 13 years ago

Can anybody summarize the main differences between this and the older versions of HTTP?

jeffasinger• 13 years ago

Binary protocol, every connection is TLS, and multiple requests can be put over the same TCP connection and answered in any order.

I didn't read this super carefully, but this is basically Google's SPDY protocol.

ctz• 13 years ago

I don't think every connection is TLS.

This is obviously a critical mistake, given HTTPS stripping is possibly the biggest weakness in current web transport security.

joosters4• 13 years ago

Binary protocol. Header compression. Multiple concurrent 'streams' (or requests) over the same TCP connection.

sktrdie2• 13 years ago

  Binary protocol.

I don't get where the "binary" part is. Does "GET /docs/index.html HTTP/1.1" get converted into binary? What are the benefits of this, given that it's really a limited amount of information compared to the body of the request/response which can of course be binary - or any other format.

xcnm35• 13 years ago

Current version of HTTP was devised by engineers. The new version is devised by an advertising company/NSA puppets.

jimgumbley• 13 years ago

i thought google and nsa employed engineers?

frou_dh• 13 years ago

If you're a proponent of engineering then why are all your comments on the matter vague and/or emotional?

joosters4• 13 years ago

At first glance, the spec appears to be re-implementing a TCP stack inside a TCP connection...

wmf3• 13 years ago

Yes, the reason is that starting a new stream multiplexed inside an already-open TCP connection is faster than starting a new TCP connection. Attempts to improve TCP itself are slowed down by slow uptake of new Windows versions.

joosters4• 13 years ago

That's not true. HTTP/2.0 (or at least the successor to HTTP/1.1) has been discussed for over a decade, there have been many OS releases since then!

Inventing a whole new protocol to overcome a TCP deficiency is IMO a terrible motive...

bcoates• 13 years ago

What efforts are these? That sounds much more interesting than mucking around with HTTP micro-optimizations.

lgg• 13 years ago

While there have been some tweaks around the edges in terms of congestion control and the like, the TCP wire format has not changed in over 25 years and it has very little to do with Windows. It has to do with the all the middle boxes and routers in the ossified backbone of the internet that choke if they see anything over IP that is not exactly what they expect.

joosters4• 13 years ago

HTTP/2.0: For all those people who thought 'the web' was 'the internet'. Well now it is! We've implemented TCP inside of HTTP. There's even a PING message there so you can do ICMP over HTTP as well :)

donpdonp• 13 years ago

I'd like to see increased adoption of SCTP to get multiplexing benefits for all protocols including HTTP.

_delirium2• 13 years ago

Google mentions SCTP briefly here, along with some other alternatives: http://www.chromium.org/spdy/spdy-whitepaper

Their main reason for going with SPDY seems to be that rolling out an application-level protocol is easier, while rolling out a transport-level protocol would require changing routers and such. That is almost certainly true. Fixing the multiplexing problem in an HTTP-specific way rather than generally does seem like an unfortunate hack, but it's probably the pragmatic approach.

ddalex2• 13 years ago

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.

Guess what the designers chose.

drawkbox3• 13 years ago

Being open and readable is also a check on this. Compare for instance the HTTP spec or JSON or XML files to binary file formats of applications. If it is hidden from sight things get ugly fast. Case in point the PSD file format and any Office file format (http://www.joelonsoftware.com/items/2008/02/19.html). Binary formats are usually only a good idea on the onset, with change it becomes a pile. Binary formats only really work well when you control both endpoints (even then they get crufty), for exchanging information being unreadable leads to many bad things and it a great step backwards.

I work in game development and application development and the former really loves binary formats (slowly changing away from that with server and editable needs). In many bugs or crashes the root of the problem is some offset in a binary file or some incorrect custom binary file format node that breaks everything after it. Readable, keyed and debuggable formats are so much better at their root (they can be binary, base64'd, compressed after but at the root they should be standard in some way and able to change without breaking the whole thing).

joeyh• 13 years ago

"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"

Ah, so every HTTP 2.0 connection has PRISM in it.

bbanyc• 13 years ago

It started out as "FOO ... BAR" but that made the header an odd number of bytes long. In the previous draft it was "FOO ... BA". It was briefly "STA ... RT" which made too much sense to use.

Trufa• 13 years ago

I'm worried about the amount of reddit like comments in HN.

xcnm35• 13 years ago

Given the close relationship between Google and the NSA, I suggest they are removed from the IETF until further notice. I would trust Microsoft more than the snake oil Google produces.

No surprise they've opted for a binary protocol rather than ASCII, anything to reduce transparency

tootie• 13 years ago

Microsoft has been just as cooperative with the NSA as Google. They have access to Bing and Skype.

cromwellian6• 13 years ago

What close relationship between Google and the NSA are you referring to?

xcnm35• 13 years ago

Google providing direct access to their servers and infrastructure. Every Google service was open to the NSA; GMail, Google Drive, Google Talk, you name it.

josteink7• 13 years ago

No surprise they've opted for a binary protocol

Independent of the reasons behind doing so, I too have a problem with that. I you're interested, I've opened an issue on it [0].

[0] https://github.com/http2/http2-spec/issues/169

jjmanton• 13 years ago

A Microsoft employee is the editor on this document.

Also, how is binary protocol used to "reduce transparency" ? Its simply a different way of encoding, and once this takes off there will be a vast set of tools for analyzing http/2.0. The purpose of binary is to reduce data transmitted.

ge0rg2• 13 years ago

I really disliked the following part: A client MAY immediately send HTTP/2.0 frames to a server that is known to support HTTP/2.0, after the connection header. [..] Prior support for HTTP/2.0 is not a strong signal that a given server will support HTTP/2.0 for future connections. (Section 3.4)

That means, whenever you open a new TCP connection to a http2 server, you must start with HTTP/1.1 and perform the Upgrade negotiation anew.

Making the spec require HTTP2 support on all servers of a cluster if at least one server supports it would have been much better, performance-wise. The only reason I see why they kept it is transparent proxies (if your browser is using an explicit proxy, it should be able to figure out if the proxy supports HTTP2).

peterwwillis18• 13 years ago

Browsers already send HTTP/1.1 by default, which would be rejected by HTTP/1.0 servers. HTTP/2.0 would be rejected by HTTP/1.1 servers and the browser could retry, or it can try for the Upgrade.

Either way, with time, we can change the browser to behave differently based on the number of servers supporting that protocol, the same as with HTTP/1.1.

ge0rg2• 13 years ago

Yes, but storing "supports HTTP2" for a given server for a given time after connecting to that server and negotiating the upgrade would also do the job.

One of the problems with HTTP 1.x is that it is stateless - every time you want something from a server, you treat it as if it is the first time you talk to it. I understand this is by design, but as a consequence we are transmitting all the same headers to the server all the time... which can be mitigated with HTTP2, except that we still need to do that (send all headers) and the HTTP2 upgrade, every time we open a new TCP socket to that server.

Sephr2• 13 years ago

You still can't self-sign certificates and rely on certificate pinning for transport security without domain verification. With all of the security concerns people have with governments logging all unencrypted internet traffic, I was hoping HTTP 2.0 would require or at least optionally allow self-signed cert TLS for every http: connection (with no UI indication to the user that they are secure) and require CA-signed cert TLS for https:

We're also still stuck with DEFLATE, which is vastly inferior to modern compression formats like LZMA.

samwillis5• 13 years ago

Encryption is only half the problem, authentication is the other. Self signed certificates don't authenticate the web server and allowing them would open the web up to MITM attacks. What we need is some form or decentralised certificate authority, i have no idea how that could work though.

HTTP allready supports LZMA, its just the browser and web servers that dont. If a browser sent the header:

  Accept-Encoding: compress, gzip, lzma

instead of:

  Accept-Encoding: compress, gzip

and the server supported it, it would "just work". Unfortunately none of the major browsers or server have implemented it.

Sephr2• 13 years ago

Unencrypted HTTP is already open to MITM attacks. What I proposed was using self-signed certificates in conjunction with certificate pinning, which means that as long as your first connection to a website isn't MITMed, later connections cannot be MITMed or snooped on. As for the compression, I was referring to LZMA for header compression. I am well aware of implementation efforts of LZMA in Firefox and Chrome.

skrause• 13 years ago

What we need is some form or decentralised certificate authority, i have no idea how that could work though.

DNSSEC could be used to distribute certificates: https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Na...

justanother• 13 years ago

This was my main objection as well, after reading this. Make SPDY a standard, people say it's overengineered, but Goog and MSFT say it'll get us 10% performance benefit. Whatever, I trust them. And hey, think of all the next-generation tcpdump-style tools we'll get to write to decode the multiplexed connections (assuming any existing ones for SPDY won't already suffice).

But while you're going to the trouble to push a new version of HTTP, why not put some sort of mandatory encryption in there? Make the browser generate its own RSA key when it's installed? Require pinned self-signed certificates at the absolute minimum? Hell, I'm not an expert, I just want the NSA to have to work harder.

ancarda• 13 years ago

Hmm... I have a few scripts that do something like:

    header($_SERVER['SERVER_PROTOCOL'] . ' 400 Bad Request');

Since I'm using SERVER_PROTOCOL, if I upgraded Nginx and it started to send HTTP/2.0 traffic to the server, would I need to rewrite this line? How would I send a specific status code using the new binary protocol?

iso-8859-1• 13 years ago

Just use

    header("Status: 400 Bad Request");

lnanek2• 13 years ago

Well this is clever:

" HTTP/2.0 enables a server to pre-emptively send (or "push") multiple associated resources to a client in response to a single request. This feature becomes particularly helpful when the server knows the client will need to have those resources available in order to fully process the originally requested resource. "

If someone asks for your HTML, might as well send them the CSS, JS, and images it is going to use. Would cut down on requests. Only thing more efficient than compression, is not having to ask in the first place.

jimktrains23• 13 years ago

Couldn't we already do this with multipart requests that no one uses?

duaneb• 13 years ago

It also complicates caching: do you cache the response by request, or cache the resources individually by update time (or whatever)?

cientifico• 13 years ago

Hope http 2.0 is the new ipv6 !

popee• 13 years ago

HTTP/2.0 should be SIP because of similarities and also integrated session management. Not some application layer binary protocol to spy on people even faster. Yeah baby :-)

antitrust2• 13 years ago

This makes it easier for web application designers, but is less optimal for those publishing static pages, which are actually the mainstay of the web.

Further, it's less user-explorable, which further distances the web from access by the everyday tinkerer as opposed to the professional with lots of time to explore.

ddalex2• 13 years ago

Is this a proposal to reimplement (badly) TCP inside of HTTP ? So we'll have a TCP stack running on top of a TCP stack ?

Those who don't learn history are doomed to repeat it.

I predict that this will lead to unexpected performance problems due to collisions in Tx window size between TCP windows and HTTP 2.0 frames.

Aloisius2• 13 years ago

Oh wonderful, it appears to use SPDY-like header compression. Why don't we just create a new compression algorithm and predefine common HTML tags and words to improve compression of the response body too! But we shouldn't base it on any known compression scheme and only use it for the case where the content length is less than 220,498 bytes but more than 8,494 bytes to optimize the behavior of today's more common MTU settings for PPPoE in Scandinavian countries minus of course the most common size of compressed headers today. It will be particularly optimized for the kind of responses Google sends back to minimize their own server load and common adwords will of course be included in the predefined list of compressed tokens.

/sarcasm

What happened to simple protocols? Seriously, per-hop flow control (which works best with which of the dozen version of TCPv4 or TCPv6 flow control?)? TCPv4-like framing with weird limits (16383 bytes)? Keepalives/Ping? Truly ridiculous specialized compression for headers which ignores the role of HTTP proxies? QoS?

Why not just implement the whole protocol over a raw IP connection and stop pretending like we're operating in layer 4+? I get that multiplexing is difficult without flow control, but good lord does this thing look overdesigned for what few benefits it offers over HTTP/1.1.

tptacek15• 13 years ago

Recapitulating a comment from downthread, but, look at DNS for an example of how the IETF botches compression in its "simple" protocols. Not that compression isn't fraught (look at TLS), but I see its use as a sign of maturity.

angersock4• 13 years ago

Well, we had a nice thing for a while, and now it's time to fuck that all up. The wheel turns.

Look on the bright side--if everyone decides the public Internet is too insecure, maybe we can convince them to keep using the old standards on darknets?

nly6• 13 years ago

I can't help thinking some of the pain could be resolved if we had a reliable datagram protocol between UDP and TCP. Delimiting a TCP stream to create a messaging protocol is already suboptimal and error prone, and it's the root cause of the head-of-line blocking problem experienced by HTTP ('fixed' in SPDY).

ldng• 13 years ago

You mean like SCTP ? It's struggling to get traction.

jahaja• 13 years ago

Something like QUIC? I reckon a few of the authors of this draft is also involved in QUIC.

http://blog.chromium.org/2013/06/experimenting-with-quic.htm...

Camillo• 13 years ago

There can be no new layer 4 protocols due to NAT. You have to use either TCP or UDP.

angersock4• 13 years ago

Erm, reliable UDP?

You can get a startling amount of the way there with sequence numbers and a few other things--it's a fun exercise.

osth• 13 years ago

"What happened to simple protocols?"

Answer: The Internet is still running on them, 30 years later.

Whenever I read something like "simplicity is hard", it makes me cringe. I hear that a lot, and I see evidence of gratuitous complexity everywhere I look these days. I'd hazard a guess the engineers behind SDPY would find simplicity (and reliability) boring.

Debugging binary protocols is either great job security for over eager engineers like the SPDY team or a great waste of our collective time. I'll let you all decide which.

stormbrew3• 13 years ago

I still feel like this is just being rammed through far far too fast. SPDY might well be the right way to go, but it's only been deployed for a small number of very large sites for a couple of years at this point. It was a foregone conclusion when the schedule was announced for http2.0 that it would be spdy, because only spdy has demonstrable value in production. There was no time for anyone to come up with or evaluate an alternative. But on the other hand, there hasn't been near enough time to demonstrate that spdy is good for the whole internet, which is what blessing it with the http protocol name implies.

And I just don't see why it should be rammed through. SPDY has a connection upgrade path, that afaik is largely unchanged for http2.0 anyways.

drivebuy2• 13 years ago

because only spdy has demonstrable value in production.

What demonstrable value? Google has never compared it to pipelining and when Microsoft did so they found it about the same and sometimes slower.

There is next to zero value in SPDY over pipelining across several connections. It's just complicated for no good reason.

stormbrew3• 13 years ago

I think you're seeing a defence of spdy in my post that isn't really there. By value I mean simply that it is at least functional. Any alternative proposal, given the short proposal cycle, would not even have that.

yalogin• 13 years ago

Totally superficial but why do I have to type http every time? Why can't it be h and hs (for https). Quicker to type.

neuroscr4• 13 years ago

Most of the complaints are around the ascii vs binary. Can we just make it accept both and move on?

jlarocco• 13 years ago

Going to binary seems kind of silly at this point.

HTML is text. Javascript is text. XML is text. JSON is text. SVG is text. CSS is text.

Is it really worth the complication to save a few bytes on HTTP headers when you're just going to shovel out inefficient text data in most cases anyway?

I don't understand why they're trying to shove this down everybody's throat as a new version of HTTP.

There are a ton of ports besides 80. The better thing to do, IMO, would be to make a new protocol on some other port and add it to their browsers.

josteink7• 13 years ago

According to the shills employed by or rooting for Google, here on HN and elsewhere on the internet, it's not silly: It is paramount.

See, it allows the request for www.google.com, including all of Google's tracking cookies, to fit inside one TCP-frame, causing a 5ms improvement in load time, and Google has evidence that this means they make more money.

Never mind the open, exploratory nature of the internet and how text-based protocols was what made the internet into what it is today. We're going to throw away all that which the internet has taught us about that because Google says we should.

This thing is fucked up beyond belief.

Edit: found a link to the IETF discussion about this:

http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/...

It admits in plain sight that the only thing they care about with HTTP 2.0 is solving Google's massive-scale issues at the cost of everyone else:

I finally admitted it was a dead end. At the moment the challenges consist in feeding requests as fast as possible over high latency connections and processing them as fast as possible on load balancers

A good, open, self-documenting protocol didn't suit Google, so let's throw it away. It's a "dead end", after all.

Jesus christ these people need a bitch-slapping.

More emails on the subject can be found here:

http://www.w3.org/Search/Mail/Public/search?keywords=&hdr-1-...

HTTP 2.0

Related Stories