HATEOAS: Hypermedia as the Engine of Application State

by carride • 3 years ago

87 91 3 years ago

anotherhue• 3 years ago

A REST client needs little to no prior knowledge about how to interact with an application or server beyond a generic understanding of hypermedia.

This language always bothered me, 'client' in this case surely means a advanced general intelligence, or 'human', who can futz around with the URIs to try and get what they need. Maybe that's better than CORBA, but it's not the Universal Translator of interfaces by a long shot.

sgeisenh• 3 years ago

Client means browser in this context.

ivan_gammel9• 3 years ago

Only if hypermedia API is based on HTML/HTTP, which is not necessarily the case.

a_cardboard_box• 3 years ago

The client is the web browser. Firefox can be built entirely without any knowledge of what hacker news is, and you can still browse hacker news on it.

kbenson• 3 years ago

I think you're providing evidence for their point? HN isn't exactly useful without a person to browse it. "A generic understanding of hypermedia" is easy to say, but may be much more complex in reality to encode into a set of rules that a program knows to follow.

And if we're going to compare "a REST client" as purely the requestor and not the interpreter of the data to something that does JSON RPC calls or graphql, which is often basically REST+, I'm not sure there's any real difference.

spankalee• 3 years ago

Your comment is being misunderstood, maybe because of the way you phrased it, but you have a point: a generic client might be able to traverse links of arbitrary data, but it won't know what they mean or why to traverse the links.

Besides a structured data browser for humans, or some kind of transforming proxy, it's hard to see what use cases this is clearly and practically better for.

yellowapple3• 3 years ago

My impression is that the lack of knowledge of what/why is kinda the point: it's the server's job to make the intent clear (via hypermedia), and it's the client's job to faithfully represent that intent to the user. The use-case is therefore related to that of the "semantic web": provide as much expressivity for that intent as possible, and make it as easy as possible for the client to faithfully convey it.

ivan_gammel9• 3 years ago

Hypermedia can be domain-specific, which already includes a lot of knowledge about data and operations. In case of HTML client needs to know the semantics of all HTML tags and understand how forms work. JSON-based RMM level 3 API can require knowledge of the schema of domain model and logical names of operations, leaving a lot of business logic to server, e.g. availability of certain operations in current security context. Client can render a button based on whether an operation with given name is mentioned in response or not instead of checking user roles etc.

dsego• 3 years ago

You are supposed to define custom hypermedia types (instead of application/json which is generic, you have application/foobar+xml).

whstl11• 3 years ago

That's probably the most important post here. Whenever this discussion comes up, people believe that HATEOAS is about magic clients consuming arbitrary JSON APIs. It's not. In HATEOAS the URLs can be arbitrary, but the format cannot.

e12e3• 3 years ago

a generic understanding of hypermedia

Is doing a lot of work here. Firefox can help me search Google and eBay (and so can lynx and w3m) with "only" a "generic understanding of hypermedia".

Groxx• 3 years ago

While attempts have been made to impose more elaborate hypermedia controls on JSON APIs, broadly the industry has rejected this approach in favor of simpler RPC-style APIs that forego HATEOAS and other elements of the REST-ful architecture.

Because HATEOAS is basically useless for machines, and industry runs on machines.

It's fine when you've got a human clicking things that they're looking at, but there's no reliable way to reinterpret it for a different use because it's so wildly context sensitive in ways that only humans understand. Meaning of things in HATEOAS may be / is intentionally allowed to be communicated by non-structured text that accompanies the actions, as literally every example in this page shows.

If you're doing anything but rendering it for humans to interpret, that flexibility makes it basically impossible. Or binds you to a single output structure, with no way to structurally verify if changes are relevant or not.

taberiand• 3 years ago

I think it's also basically useless for user interfaces.

The purpose of HATEOAS seems to be to provide web crawling clients (human or otherwise) visibility of the arbitrary actions/links that a particular endpoint provides in a particular state - but the links are disconnected from their context and intended interaction with the page. HATEOAS doesn't seem to acknowledge that a UI needs to provide context for user actions, or that API integrations are designed up front with full knowledge of the API contract .

It seems to me the only thing HATEOAS is useful for is browsing black and white pages with a bunch of blue and purple links. Or maybe SEO

ivan_gammel9• 3 years ago

HATEOAS is perfect for SPAs with server-side persistence because it allows to reduce complexity of the frontend and to avoid implementing things like security checks twice.

_glass• 3 years ago

One point is that you should develop versioning with HATEOAS, instead of putting an ugly /v1/ into the URL.

thom• 3 years ago

It’s possible that in the age of LLMs this stops being a silly idea, but let’s not pretend it hasn’t up until now been a very silly idea.

phpnode2• 3 years ago

It’s a brilliant and beautiful idea that has approximately zero success stories in the real world

schnable2• 3 years ago

The World Wide Web is a smashing success of HATEOS and REST, properly understood.

Sohcahtoa822• 3 years ago

From what I understand, Web 1.0 was essentially HATEOAS.

Apreche• 3 years ago

I agree with the fundamental principle that "true" HATEOAS is desirable, and that HTML provides it while JSON doesn't.

That said, I don't quite agree with the criticism of out-of-band information. Is the HTML spec not out-of-band information?

Likewise, if they had used an SGML api as the example instead of HTML, would not all the same criticisms of the JSON, for not being HATEOAS, be valid?

There are already many JSON-based APIs on the right path like ActivityPub, JSON-LD, and others. Having to know the specs of these things is no more out-of-band than knowing HTML.

TL;DR: SGML:HTML::JSON:XXXX. If we can all agree on a good XXXX instead of freeforming from just JSON every time, we can get closer to HATEOAS.

schnable2• 3 years ago

JSON is fine too, if it carries the same semantic information.

whstl11• 3 years ago

"Is the HTML spec not out-of-band information?"

It isn't. The HTML spec isn't transferred every time a webpage is accessed, so it's not exactly information. Also not really helpful, it's like saying "english" is "out-of-band information" when reading a website written in english.

"Out-of-band" has a very precise meaning here, it comes from the telecom term "out-of-band signaling". There's a wikipedia page: https://en.wikipedia.org/wiki/Out-of-band_data

ivan_gammel9• 3 years ago

This article makes a mistake calling RMM0-2 APIs „JSON APIs“. JSON is data format just like HTML, XML or protocol buffers and it can be used both by true REST or RPC APIs. Focus on JSON is indeed what makes this article ineligible for Wikipedia, it’s just the wrong angle to look at the subject.

bertylicious• 3 years ago

What I don't get about these examples is how the HTML from the responses should actually be rendered. I assume no frontend would want to render the HTML as is. Hence they would need IDs on those DIVs to style and place them. But wouldn't that require out of band information too?

yellowapple3• 3 years ago

I assume no frontend would want to render the HTML as is.

Well yeah, that'd be pretty ugly on most browsers without some kind of stylesheet, but...

Hence they would need IDs on those DIVs to style and place them.

...this doesn't really follow from that. Like yeah, that's probably a good idea (even better would be some more specific tags than just divs), but the client has everything it needs to present the data to the user even without that.

The point, I think, is that styling is a separate concern; HATEOAS and REST have more to do with structuring information and interactions with it, and the examples therefore emphasize that instead of a possibly-accidental prescription of some specific styling for that information.

Sohcahtoa822• 3 years ago

Articles talking about "REST" and "HATEOAS" really need to define "client". Do they mean a person? Do they mean a web browser? Do they mean an application performing actions autonomously?

A REST client needs little to no prior knowledge about how to interact with an application or server beyond a generic understanding of hypermedia.

If in this instance, "REST client" means an application performing actions autonomously, I don't see how this statement could possibly be true. Someone writing a client will need to know how the server is going to return responses to requests. It needs to know what operations the server supports.

It really seems like REST and HATEOAS, using the strictest definitions, really just mean server-side HTML rendering, which is nothing new. I figured we swapped to client-side rendering (And therefore, servers sending XML or JSON, and the client handles putting it into the page) to reduce server-side loads and the cost of increased CPU usage client-side.

throwing_away• 3 years ago

Articles talking about "REST" and "HATEOAS" really need to define "client". Do they mean a person? Do they mean a web browser? Do they mean an application performing actions autonomously?

My understanding is yes, those are all clients.

If in this instance, "REST client" means an application performing actions autonomously, I don't see how this statement could possibly be true. Someone writing a client will need to know how the server is going to return responses to requests. It needs to know what operations the server supports.

I think the "generic understanding of hypermedia" is knowing how to follow links, use the HTTP verbs, and interpret the HTTP status codes.

I think the idea is that if you know to query your account balance with "GET /accounts/12345" and it offers something in its response that looks like an HTML web form or json containing a link to "/accounts/12345/deposits", you could easily infer that a "POST /accounts/12345/deposits" would be used to make a deposit.

How would you know what to POST? Look at the web form and emulate that, or just try it and get a specific error about what fields are missing, ideally with a link to docs.

It really seems like REST and HATEOAS, using the strictest definitions, really just mean server-side HTML rendering, which is nothing new.

That's actually htmx, which makes sense you got that vibe, because it is the htmx blog and they are highlighting a way to self-document the API with web forms, but REST is connected to the HTTP verbs and HATEOAS is connected to the payload to make REST discovery more intuitive.

As far as I know, nobody really uses this stuff, or at least I haven't seen it in production, but it's been a good idea floating out there for at least 10 years now.

If someone made a HATEOAS linter and a Postman extension, in addition to the obvious htmx endorsement, maybe it would gain some traction.

3cats-in-a-coat• 3 years ago

The theory behind HATEOAS was bullshit until LLMs made smart clients plausible. But LLMs don’t need HATEOAS to discover an API. So turns out HATEOAS is still bullshit.

aatd862• 3 years ago

Is it me who does not understand what HATEOAS is?

Always thought that it was basically each server generated page providing the list of links/actions that was allowed from then on.

Basically a lazy, incremental, and fine grained API.

The issue being that it might not be very useful since links have to persist in general on the web. (maybe I misunderstand and it's a non issue?) whereas HATEOAS would be useful if links were never shared: then they could be dynamically generated or updated?

mkl95• 3 years ago

HATEOS consists mostly of linking responses ("resources") to each other, as opposed to nesting data. So instead of looking at some nested data, you will follow a link to that data, and so on. You probably won't see it in the wild too often, because RPC rules the world.

zamalek• 3 years ago

Wasn't that the premise of OData? It was Atom(RSS)+hyperlinks.

chromanoid• 3 years ago

I think the idea is to provide your API as some part of the consumer internet. The problem is that a bunch of websites tends to be hard to be navigated meaningfully by a machine, since the machine must be able to process the moving parts in way they do not loose meaning. The idea is that you can simply change a link in a result to be able to split your API in an API that is served by multiple endpoints (e.g. microservices) etc.

aatd862• 3 years ago

Oh makes sense... It's like these books in which you are the hero.

Depending on the use agent and what not, the routes through a given server-rendered website can be different, tailored, per request.

benzible• 3 years ago

Worth reading this: https://twobithistory.org/2020/06/28/rest.html

REST purists often complain, for example, that so-called REST APIs aren’t actually REST APIs because they do not use Hypermedia as The Engine of Application State (HATEOAS). Fielding himself has made this criticism. According to him, a real REST API is supposed to allow you to navigate all its endpoints from a base endpoint by following links. If you think that people are actually out there trying to build REST APIs, then this is a glaring omission—HATEOAS really is fundamental to Fielding’s original conception of REST, especially considering that the “state transfer” in “Representational State Transfer” refers to navigating a state machine using hyperlinks between resources (and not, as many people seem to believe, to transferring resource state over the wire).8 But if you imagine that everyone is just building FIOH [author's acronym for "Fuck It, Overload HTTP"] APIs and advertising them, with a nudge and a wink, as REST APIs, or slightly more honestly as “RESTful” APIs, then of course HATEOAS is unimportant.

draw_down• 3 years ago

I’m not sure a “real” REST API has ever been built.

recursivedoubts2• 3 years ago

you best be believing in REST APIs... you're using one.

danbruc6• 3 years ago

Seriously, has anyone ever made a non-toy implementation conforming to the idea of HATEOAS?

A REST client needs little to no prior knowledge about how to interact with an application or server beyond a generic understanding of hypermedia.

Nothing but a generic understanding of hypermedia? You need some substantial understanding of language and the world to figure out where this link will take you.

<a href="/accounts/12345/close-requests">close-requests</a>

Sure, English is not my first language and financial terms are probably a real weak spot, but I have at best a vague idea what this will do. Can I request to close my account? But why would I need requests for that, plural? So maybe it is not about closing my account but instead a close request is some financial term? If it is, I still have no idea what it is good for.

For consumption by humans, developers poking around in a new API, HATEOAS might be nice. But how on earth are you going to write software that can consume such an API, dynamically discover, understand and use available operations? Maybe you could use ChatGPT to do this. And how are you going to build an UI for this that is any better than what naked objects [1] can deliver if you do not even know which operations the API has to offer?

[1] https://de.wikipedia.org/wiki/Naked_Objects

ZeroGravitas• 3 years ago

I think their response would be something like: If you want that kind of data API then look at GraphQL which was designed to do that, not some misuse of REST which was designed to do the things they are talking about.

https://htmx.org/essays/hypermedia-apis-vs-data-apis/

edit to add key part:

The crux point of this short essay is this: API churn is fine in a hypermedia system because the messages in a hypermedia system are self-describing. We can thrash the API around and the application doesn’t break: human users simply see the new hypermedia (HTML) and select what actions they want to do.
Humans, compared with computers, are good at deciding what to do and are reasonably OK with change.
This is in contrast with data APIs. Data APIs cannot be modified without breaking client code and thus must be much more disciplined in their changes. Data APIs also face pressure to provide higher levels of expressiveness so that they can satisfy more client needs without modification.
This latter situation is especially dangerous when these data APIs are consumed in a browser, because any data-api expressiveness available to a front-end developer is also available to a potentially hostile user, who can fire up a console and begin hammering away at the API. Apparently, facebook uses a whitelist to deal with this.
Do you?

whstl11• 3 years ago

"Seriously, has anyone ever made a non-toy implementation conforming to the idea of HATEOAS?"

Yep, your web browser uses hypermedia. But if you want something that is an API, then there is OData, GData and AtomPub all follow HATEOAS.

danbruc6• 3 years ago

My browser has no idea how to interact with a proper HATEOAS conforming REST API, I would be doing that, the browser would just show me responses. And if one claims that this is all that is necessary, because the API response already contains some HTML form for me to interact with, then you have done nothing but relabeled web applications as REST APIs and API responses are just a web pages.

How do you consume an OData API without baking the structure of it into your code in one way or another? And just rendering generic list and detail views does not count, I think, that will most likely not provide any good user experience.

natural219• 3 years ago

I mean, you kinda answered the question there yourself. The whole REST paradigm (spearheaded by the Ruby on Rails community) was strongly influenced and modeled after the HATEOAS paper and ideas. There's some talks and papers about it but I'd have to dig them up, but basically the working reference implementation that shaped the web for years is here: https://guides.rubyonrails.org/routing.html

napoleongl• 3 years ago

We are using ShareASpace for some things at work and they claim to use HATEOAS. Not sure where to link to prove it though… https://docs.shareaspace.com/v1-8/developer/external-extensi...

lolinder2• 3 years ago

If you read the article, it's pretty clear that they're not advocating using HATEOAS for a computer-consumable API—it's all HTML, meant to be rendered by a web browser and clicked on by the user. You could code a computer agent against this as well, but a hypermedia system is designed for users first.

danbruc6• 3 years ago

In that case your REST API is just a new name for web application and your API responses are just web pages.

JustLurking2022• 3 years ago

Humans can just as easily read an OpenApi spec.

didntcheck2• 3 years ago

In that case what's the point of any of it? The user doesn't care about how your URLs are structured. They just read pages and click links, and anything below that is up to the programmer's preference

39Wc• 3 years ago

Apache Causeway [1] generates a RESTful API that is HATEOAS compliant, and exists in parallel to the built-in viewer that is based on Wicket.

There are a few HATEOAS client projects out there, such as [2], that will generate a UI from such a (Restful) API. As far as I know it works quite well, but of course it's a bit rough on the edges.

One interesting observation is that the Wicket UI generated by Causeway does not itself use the RESTful API, but it uses the same meta model that is also used to generate the API.

[1] https://github.com/apache/causeway [2] https://github.com/lagoshny/ngx-hateoas-client

ivan_gammel9• 3 years ago

I have been developing HATEOAS clients and servers for a while. The key to understanding of HATEOAS is that it is not about browsers or arbitrary clients, it is about abstraction. Client and server share the knowledge of the protocol, the data formats, logical names of links and semantics of operations with those links. Client does not need to maintain the complete state and complex business logic for that state, because it can derive the complete state and possible transitions from each server response. If some link is present in response, operation is possible, otherwise it is not. If link with given name exists and semantics imply GET, PATCH or POST, then make this request to this link. No constraint checks, no access management on client, just follow what server tells you. If server allows to cache resource, do it, but don’t forget to do HEAD.

dragonwriter• 3 years ago

Seriously, has anyone ever made a non-toy implementation conforming to the idea of HATEOAS?

Yes, the Web.

Nothing but a generic understanding of hypermedia?

To understand basic structure and how to determine if you have the ability to do more; actually deep interaction requires knowledge of specific media types as well.

But how on earth are you going to write software that can consume such an API, dynamically discover, understand and use available operations?

The basic software handles generic hypermedia, and there are modules to handle particular media types and interact with them. The meat solving a problem with REST is defining resource representations (on the service side), and then building handlers that do what you need with those resource types (on the client side).

lucideer• 3 years ago

You need some substantial understanding of language and the world to figure out where this link will take you.
<a href="/accounts/12345/close-requests">close-requests</a>

There's two types of understanding here: a technical understanding of hypermedia behaviours, and a contextual understanding of application domain intents. A hypermedia client doesn't need to understand the latter at all: it only needs the former to be able to present a user with direct context (whether the user can then understand what to do is mostly up to the application designers, but it's outside of the responsibility of the hypermedia client - it's just the messenger here).

For example: this is a HTML anchor element (presumably the client has been served a Content-Type: text/html header here to inform it of that) - HTML anchors initiate GET requests to an associated href. In this case the href starts with a / indicating that it is relative to the serving origin. That's the beginning and end of so the hypermedia client needs to know: all self contained, no out-of-band knowledge required.

Any further confusion suffered by the user related to financial or English language knowledge is up to domain knowledge of the application and not within the bounds of what HATEOAS does.

pharaohgeek• 3 years ago

I believe that Apple's AppStore Connect API uses HATEOAS. That's pretty non-trivial.

rdtsc• 3 years ago

I've done the HATEOAS thing as much as possible. Links were rendered in json like the site claims not to do. There is json-ld, of course, but that was just too much, so I didn't bother.

Can also do a nice trick of rendering nicely formatted json objects in html with proper links if a browser is requesting html/text. For an extra bonus split the page and render the docs right next to it, too. If application/json is requested, return just json to be consumed by programmatic clients.

That said, while it was really nice to learn and navigate the API from the browser that way, none of the programmatic clients ever used the "auto-discovery" feature. They jumped right to the resource the needed and didn't bother with the links.

At this point REST is just whatever most popular companies designed and called "REST-ful" is what REST-ful became. As much as Roy wants to claim it's not "pure" enough, the ship has sailed so speak.

gremlinsinc• 3 years ago

I consider it a fad, that we all go through when we're mid level devs wanting to use every shiny new thing. It's perfectly fine to use rest almost like JSON over grpc, or however you want the only thing is having good documentation for

jmull• 3 years ago

I think this article is focusing a lot of attention on a rather arbitrary aspect of web apps, and suggesting some bad conclusions.

Taking the example from the article... if you want to present bank account information to a user and allow them to take actions on it, then focusing on HTML (and CSS) is a great idea.

However, the data is going to be stored somewhere, and you're going to want to retrieve it and transform it into the desired presentation HTML for your app. In the case of the article, you could say describe these two operations abstractly like this:

    retrieveAccountInfo(dataSource) -> (accNo, balance, status)
    presentAccount(accNo, balance, status) -> HTML

so the page HTML is determined like this:

    presentAccount(retrieveAccountInfo(dataSource))

In the "hypermedia" option from the article these two operations run server side in an unspecified way, while for the "rest" option retrieveAccountInfo is implemented as at HTTP GET returning JSON and presentAccount runs in the browser.

presentAccount is just as logically coupled to retrieveAccountInfo in the "hypermedia" case as it is in the "rest", and possibly somewhat more coupled in terms of infrastructure, depending on what's going on at the application server. The application state is determined in exactly the same way.

Of course, there are tradeoffs between running application code server side vs client side. But it really has nothing to do with hateoas and decoupling, and the article doesn't touch on it. (One of the big advantages to running the app server-side is that there's no need for client-side javascript -- this makes the htmx approach -- where the app runs server side but you also need client-side javascript -- a bit questionable, IMO.)

chrsjxn• 3 years ago

For me, this feels worse than wikipedia using JSON as an example.

JSON isn't a great HTMl representation, and the wikipedia examples are definitely missing details about how the underlying API is used. But extremely simple HTML just feels worse.

Both pages use a bank account as the example, but my actual bank account has dozens of actions available and is certainly modeled in a way that wouldn't be intuitive for people. Which means we've got an example that isn't easily parseable by computers, because we have mixed labels and data in the HTML, and isn't useful for human beings due to how complex real world entities are.

I genuinely can't tell if this is like monads (which are useful, but often poorly explained) or if this is just a bad idea.

gjvc2• 3 years ago

The REST paper was not worth a PhD

gjvc2• 3 years ago

for downvoters: show me where in the paper he says "this is a REST call" and why, and "this is the same call, but not REST" and why.

recursivedoubts2• 3 years ago

I am the author of this article.

I see a lot of people saying "HATEOAS doesn't work/has anyone ever used it".

The answer is YES: the world wide web and any traditional web 1.0 application uses it quite successfully. The important thing to realize is that an HTML-based web server is providing an API: a hypermedia-based API. It is providing it to a hypermedia client, namely the browser.

The concept has not mapped well to JSON-based data APIs because those APIs are not typically consumed by REST-ful clients. Or, if they are as in an SPA, the hypermedia functionality of the client is ignored in favor of a thick-client approach.

We have written a book on the idea of a hypermedia system that explains this in far more detail:

https://hypermedia.systems

which stresses the systemic nature of hypermedia when building a distributed application.

HATEOAS: Hypermedia as the Engine of Application State

Related Stories