HTTP/1.1 just got a major update

by treve • 12 years ago

248 73 12 years ago

purephase• 12 years ago

Forwarded and a permanent re-direct are nice changes. This is a welcome change.

It will likely be awhile before widespread adoption, but to see a standard move forward in such a seemingly small, but considerable way is great.

Kudos to those involved. I can't imagine it was an easy feat.

gamegoblin• 12 years ago

And I just finished implementing 2616 in a toy server of mine. Sigh. C'est la vie.

I'm glad this new spec apparently resolves a lot of ambiguities. I hated reading 2616 and some specs it depended on (email, URI, etc).

treve4• 12 years ago

These specs clarify 2616. 'Major' may have been a poor choice of words, but if you were 2616 compliant, you should be largely compliant with these specs as well.

profil• 12 years ago

Is your toy server open source? I would love to see the source.

perbu• 12 years ago

Are you lacking OSS HTTP/1.1 servers to study? There are tons.

BorisMelnik• 12 years ago

I wonder what Googlebot will make of this. There is a lot of debate already amongst search engine developers on which type of redirect to use.

treve4• 12 years ago

A crawler like that should typically only do GET requests. The 308 is really mainly useful for HTTP clients doing for example a PUT or POST request on some url, and the server wants the client to repeat that exact request on for example a different server.

dsl• 12 years ago

Search engines largely handle all redirects the same, because they know nobody uses them correctly. If you aren't seeing the behavior you want, you can use Webmaster Tools or metadata to fix it.

__david__• 12 years ago

Finally, the spec is crystal clear that message bodies on GET requests are not illegal.

colanderman5• 12 years ago

Pardon my ignorance but how are GET payloads useful? Does that not violate REST principles?

wtetzner• 12 years ago

I'm not sure how. As long as a GET request doesn't change your resources, it should be fine.

The reason I can think of for sending a payload in a GET is if the data you wanted to send in a query string is too large.

argonaut• 12 years ago

Is there something wrong with violating a REST principle?

saryant• 12 years ago

Elasticsearch: it accepts JSON bodies in GET requests to define the parameters of a search. These can get quite large so it's preferable to encoding everything in the query string. The operation is read-only, so an argument can be made that a GET makes more sense than a POST.

That said, I POST my search params to Elasticsearch anyways.

Cakez0r• 12 years ago

Having the parameters of your GET request in the query string or the request body doesn't have any semantic difference. You're just fetching a resource given some parameters, which doesn't violate REST principles.

colanderman5• 12 years ago

From the horse's mouth; they are not useful: https://groups.yahoo.com/neo/groups/rest-discuss/conversatio...

dragonwriter5• 12 years ago

They aren't. HTTP has a layered design, and the structure of requests is a lower level feature than the semantics of specific request type -- a request can always have a body, but that doesn't mean that there aren't request types (e.g., GET) for which the body is meaningless.

Does that not violate REST principles?

Most attempts to impose semantics on GET bodies would violate REST principles as well as breaking HTTP (given that GET responses are cacheable and the body is treated as not relevant to the response.)

lttlrck• 12 years ago

Is it really so different to URI encoded parameters?

grk• 12 years ago

Phusion passenger throws 500 on get requests with body, I guess now they might change that.

FooBarWidget• 12 years ago

Actually we've already changed that. It's slated for 4.0.45, due next week.

derengel• 12 years ago

As someone new to HTTP, what would be the most pragmatic way to read through this RFCs in the context of building web applications or HTTP APIs but not to the level of wanting to implement a http server or http client? for example: order of reading, what can be avoided, what is not widely use or implemented, the basics of the protocol, etc...

treve4• 12 years ago

On a high level:

You can certainly skip most of "Message Syntax and Routing". That's the stuff that concerns server and client implementers that just have tcp sockets to work with.

I would absolutely read "Semantics and Content". It's a really good idea to be aware of "Conditional Requests", and you only really have to read "Caching", "Range requests" and "Authentication" if you need to know about those features.

muaddirac• 12 years ago

HTTP methods, status codes, and headers are all you need to understand for developing at the level of HTTP APIs.

jalfresi• 12 years ago

And media-types and rel types if you're developing HTTP APIs

masklinn6• 12 years ago

The problem being that it's been spread all over the specs.

The essentials would be:

* 7231 which covers core methods, statuses and basic headers. It looks like the spec authors have also added security considerations sections

* 7232 is probably a good read as it covers conditional requests (304 and 412 statuses)

* 7234 covers caching and cache controls, don't skip it. Even if you don't want your response to be cached, you need to know how caching works, which actors are involved and how to disable it

* 7238 is the 308 redirection, understanding it and the background for its introduction is a good idea and will help with understanding other redirection statuses (301, 302 and 307)

"Various others" would be

* 7233 is Range requests, can probably be skipped unless you have big media payloads. On one hand it's underused, on the other hand it has limited general applicability

* 7235 is Authentication, can be useful for API (the user experience being terrible for browsers) but can probably be skipped unless absolutely necessary

* 7239 is forwarding, to understand what happens when your HTTP endpoint is behind a proxy. Although I'd guess proxies don't implement it yet the ideas already existed as non-standard extensions and reading this is a good idea for "real-world" concerns. Not completely necessary, but useful

* 7240 is the Prefer header. It's a fairly recent and quite advanced addition, probably useful but not utterly necessary

You can ignore

* 7230 is about req/resp format. The only interesting parts are the URI and Host parts which your HTTP library probably handles for you

* 7236 complements 7235 with auth scheme registration for standard auth types. Only read it if you've read 7235

* 7237 is a registry of additional (wrt 7231) methods, mostly from WebDAV

userbinator• 12 years ago

I don't think splitting it up like that is such a good idea; now, instead of searching through one file, I have to remember that there are several and look through them all, just for one conceptual protocol. (TCP has a similar issue, although most of it is still in 793.)

As for the extra verbosity, I'm not sure what to think; while some things may be specified more precisely, standards should also attempt to be concise and to-the-point. Some of the sentences in the new RFCs seem almost parenthetical (e.g. look at the description of GET.)

masklinn6• 12 years ago

OTOH, that means I don't have to dive through the minutiae of response message format when I'm just looking for the basic header stuff. All the important concerns (core, caching, conditional requests, auth and forwarding) get their own RFC and are thus easier to skim and search through. Although 308 and Range (and Prefer) also getting their RFCs is a bit weird. Likewise, syntax and routing get RFC 3270 so if you're implementing a client or server the reading experience should be much tighter,

colanderman5• 12 years ago

Copy+paste them into one text file?

lloeki• 12 years ago

The clarifications are very welcome but I wish it included embedded unit-less progress information on chunk encoding without having to rely on a side channel [0] (shameless plug, but any progress — ha ha — on this front would be fine)

[0] https://github.com/lloeki/http-chunked-progress/blob/master/...

fafner• 12 years ago

RFC7238 is still marked as experimental.

cyphunk• 12 years ago

Great writeup. One issue...

   It's now suggested to use the about:blank uri in the
   Referer header when no referer exists, to distinguish
   between "there was no referrer" and "I don't want to 
   send a referrer".

For the sake of privacy would it not be better if there was no such distinction. Basically now any privacy conscious services need to add 'about:blank' as the referrer when users do not want to have their behaviour categorised and fingerprinted?

robryk2• 12 years ago

If a user doesn't want to send the referrer when there is no referrer, no referrer should be sent. This then allows sites to distinguish between direct traffic from users that don't block referrers and traffic with blocked referrers. I wouldn't expect this to be a significant concern, because the volume of actual direct traffic is not very large.

gorhill2• 12 years ago

This then allows sites to distinguish between direct traffic from users that don't block referrers and traffic with blocked referrers

Any example of benefits for servers to distinguish direct traffic vs. blocked referrers?

TheLoneWolfling• 12 years ago

Yep.

Yet another way to fingerprint.

kccqzy• 12 years ago

The standard doesn't seem to suggest that. Quote from http://tools.ietf.org/html/rfc7231#section-5.5.2

    If the target URI was obtained from a source that does not have its
    own URI (e.g., input from the user keyboard, or an entry within the
    user's bookmarks/favorites), the user agent MUST either exclude the
    Referer field or send it with a value of "about:blank".

Am I missing anything?

0x04• 12 years ago

Why is it not called HTTP/1.2? Or, how will clients (or servers) tell the difference between a peer implementing RFC 2616 (HTTP/1.1 old version) and RFC 723x (HTTP/1.1 new version)?

Could it be that there is so much software hardcoded to look for "HTTP/1.1" that a "HTTP/1.2" string would break them all?

Lukasa• 12 years ago

The argument is that the two should be able to transparently interop together, and specifically that RFC 723[0-5] simply codifies the way HTTP/1.1 already works in the real world.

We'll see how well that goes.

0x04• 12 years ago

Shouldn't minor revisions of HTTP interop anyways? I.e. a HTTP/1.2 server should be able to talk with a HTTP/1.1 client even if they use different minor versions in their protocol string? So, it "SHOULDN'T" hurt to bump the version to 1.2, plus it would make it easier to identify up to date spec compliance?

masklinn6• 12 years ago

Because it does not really change the protocol. It clarifies details and implements spec fixes (e.g. matches the spec better to actual real-world use).

zurn• 12 years ago

There are significant changes of semantics according to TFA, for example:

Default charset of ISO-8859-1 has been removed
The 204, 404, 405, 414 and 501 status codes are now cachable.
The Location header can now contain relative uri's as well as fragment identifiers

0x04• 12 years ago

Can you send a 308 permanent redirect to existing software and get the expected behavior?

mjs• 12 years ago

The thing I'm most surprised by is the change in the default cacheability of 404 responses from uncacheable to cacheable. Though I guess since defaulting to cacheable doesn't mean that responses must be cached, so you can still be compliant with RFC7231 by never caching 404s.

RFC2616:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13...

"A response received with [a status code other than 200, 203, 206, 300, 301 or 410] MUST NOT be returned in a reply to a subsequent request unless there are cache-control directives or another header(s) that explicitly allow it."

RFC7231:

http://tools.ietf.org/html/rfc7231#section-6.5.4

"A 404 response is cacheable by default"

mrottenkolber• 12 years ago

I am still happily speaking HTTP/1.0 [1] :)

[1] http://mr.gy/software/httpd0/

HTTP/1.1 just got a major update

Related Stories