Show HN: Hasura – Fast, instant GraphQL APIs on Postgres

https://github.com/hasura/graphql-engine

by tango12 • 8 years ago

382 103 8 years ago

tango1220• 8 years ago

Hi HN! Super happy to announce the first open-source release of the Hasura GraphQL engine.

This is a standalone service that gives you GraphQL on Postgres. It comes with fine-grained access control that can integrate with any auth provider. It can also work with any existing postgres database as is, and allows you to expose select tables/views across your postgres schemas over GraphQL.

Looking forward to your feedback!

minxomat• 8 years ago

How does this project relate to prisma?

tirumaraiselvan2• 8 years ago

Prisma doesn't work with existing postgres or access-controls.

vsurabhi5• 8 years ago

Hi, I'm one of the core contributors to the project.

Hasura and Prisma have very different approaches and end-goals at the moment.

TL;DR: Very simply put, Hasura is postgres first and adds an access-control layer to make it possible to query from the front end directly.

The longer version:

Prisma is intended to be a GraphQL ORM to Postgres (and several other databases with a common API) whereas Hasura is more database first and intended to let you leverage Postgres features over GraphQL.

Hasura comes with and lets you use any postgres migration system and then gives you GraphQL on top of it. This is what allows Hasura to give you GraphQL over any existing Postgres database and application even.

Hasura also comes with an optional access-control layer that makes it feasible to be used by frontend clients directly. This access control is inspired from postgres's RLS but does not depend on or use postgres RLS and can be coupled with application logic/auth.

bsaul• 8 years ago

Thanks for releasing such a project. Your website is really well done, it immediately made me want to dig in the tech, congrats !

PS: as a sidenote and since i'm sure you'll get the question a lot : when i saw it was coded in haskell i was a bit nervous, but at the same time a bit more confident this wasn't another junior-level spaghetti code base.

tango1220• 8 years ago

Haha, yes we do this get this a lot!

We're also planning to release a lot of cool GraphQL OSS stuff for the haskell community because of our work over the last year.

vetinari• 8 years ago

Any comparison with PostGraphile?

vsurabhi5• 8 years ago

Very similar projects ! The emphasis of Hasura is to make it seamless to work with existing databases.

The primary difference is how authorization is handled. In Postgraphile, you map your application roles to Postgres users and define policies using Postgres's RLS. With Hasura, you define rules on roles (similar to Postgres's RLS) but it integrates into your existing auth system using webhooks.

ASTP001• 8 years ago

This looks great! Is there any interface for extracting encrypted columns through this? For example, decrypting fernet encrypted values, given the key?

kilburn2• 8 years ago

This is very impressive. Congratulations!

What I'm not seeing is how do you envision enforcement of complex business rules when using this API. Examples (silly, but you get the idea):

- The price is only visible for projects of type "whatever".

- The price may only be updated when it is not closed (closed_date is null)

- The project is only visible on mondays

The other point I don't see is how do you expect to trigger side-actions when changes are made. For instance:

- Send an e-mail when a project is created

- Call a webhook when a post is modified

Finally, and this may be from my lack of expertise with GraphQL, how do you deal with recursive data structures? For instance, how would you store and fetch a tree of categories?

marksomnian• 8 years ago

For the latter, GraphQL itself explicitly forbids recursivity [0], as it's impossible to guarantee that in all cases the data isn't infinitely recursive, especially with cyclic queries, which could lead to a denial-of-service. Postgres has its own support for recursive queries [1], however you'd need to strap a pagination layer on top in order to make it work with GraphQL.

[0] https://github.com/facebook/graphql/issues/91#issuecomment-2...

[1] https://www.postgresql.org/docs/9.1/static/queries-with.html

dustingetz2• 8 years ago

What a cop-out. Datomic Pull, which predates Graphql by two years and does not have an impedance mismatch problem, handles recursion perfectly fine. https://docs.datomic.com/on-prem/pull.html#recursive-specifi...

vsurabhi5• 8 years ago

What I'm not seeing is how do you ..

You can write fairly complex rules with the access control layer ! The first two conditions are possible (except for the project is only visible on Mondays).

  allow select on rows which satisfy this property: 
  { "project_type" : { "_eq" : "whatever" } }

  allow update on rows which satisfy this property: 
  { "closed_data" : { "_is_null" : true } }

The idea is simple, we provide a boolean expression based access control layer which can use variables from headers. When you can't specify it using the access control layer, you can write this in your own graphql layer and schema stitch with hasura. So you only have to write what is not possible with the access control layer.

The other point I don't see is how do you ...

You can use the tooling around Postgres ! You can create an `ON INSERT` trigger on the table and listen to these events.

We're launching something very cool in the subscriptions/event-source to solve these kinds of problems. We also have 2 nice projects that work independent of the graphql-engine to help with change subscription on postgres (skor[1], pgdeltastream[2])

[1] https://github.com/hasura/skor [2] https://github.com/hasura/pgdeltastream

dustingetz2• 8 years ago

For "access control layer" to work in the general case we need to be able to code it in a real programming language, no? Or are you going for something one-size-fits-many like Sharepoint?

kilburn2• 8 years ago

u can write fairly complex rules with the access control layer

Oh, I see. As a suggestion then, many business rules are not tied to roles. You may want to look into some "always enforced rules" or something like that, to avoid having to repeat these rules for each role (and the maintenance nightmare this entails).

your own graphql layer and schema stitch with hasura

Again, I'm not versed in GraphQL in general, so this is from my ignorance. Maybe you could provide some example/tutorial on "stitching your own GraphQL apis with hasura" (your other examples, posts and tutorials are awesome btw!).

You can use the tooling around Postgres

Fair enough. I've always found a bit hard to properly manage and test these native features (much harder than managing/testing pure code), but it is a sound solution nonetheless. Maybe you want to hint to these other projects somewhere in hasura's docs (I've checked them and couldn't find it, but maybe I'm just blind/didn't look hard enough).

Again, awesome project and good answers. I can't stop going through possible pet-projects to try this out soon! :)

eganist• 8 years ago

The price is only visible for projects of type "whatever".

Tangentially, this one's shockingly common and not something I'd consider silly. Many manufacturers obligate their distribution partners to market only certain prices in order to keep pricing expectations for a certain product at a certain level.

evadne• 8 years ago

I suppose this can be done with row-based security if you are willing to mirror/maintain this in Postgres and/or fork the query engine to issue them under the right roles. There was a talk at PG Day UK 2018 about this but there are no performance benchmarking results available AFAIK.

As to triggers, you can totally do this with event sourcing in triggers, so when something is modified you write a row to the audit table, and you use another process to read from it and do the right thing, etc.

mikl3• 8 years ago

First glance reaction: Awesome, just the thing I need.

Then I saw the fine print: AGPL. Haskell.

The thing might be awesome, but the combination of the legal risk from AGPL (very little legal precedent for AGPL, especially outside the US, so the real legal implications of the license are very murky) and maintenance risk from Haskell (finding good Haskell developers is hard, and the learning curve is brutal), makes it a lot harder to choose.

tango1220• 8 years ago

Hm...Haskell was an important initial choice for us because of our team's comfort with the language and also because it's neat to implement parsers and compilers. The Hasura engine is essentially a GraphQL to SQL compiler :)

I'm not sure I understand your point about AGPL? Do you have the same issue with the MongoDB license and would hesitate to use Mongo because of their AGPL? Why is this different? Just curious, would love to understand where you're coming from!

seddona• 8 years ago

Haskell seems like the perfect fit for this particular problem.

mikl3• 8 years ago

I’m sure Haskell is great, just saying its a bit of a risk to add to your stack if you don’t already have Haskell developers in house.

That, and the AGPL issue are both more important in this case (compared to MongoDB), because of the position in the stack that Hasura GraphQL Engine (HGE) would have. In the MongoDB case, we would almost always have an application server between Mongo and the GraphQL client, where custom business logic could be implemented, so Mongo can be treated as an external system whose source code we do not need to modify (like PostgreSQL in the case of HGE).

In case of HGE, the GraphQL client would talk directly to HGE, and thus any customisation would most likely need to happen inside HGE, whether by plugins or patches or whatnot, making it tricky to avoid having to open source your custom business logic (as well as probably having to write it in Haskell).

UncleEntity• 8 years ago

The Hasura engine is essentially a GraphQL to SQL compiler

How much of the code base (and complexity) is doing that vs the auto-schema mapping and whatnot?

I would just "read the source, Luke" but haskell isn't one of those languages I can idly read.

simonw• 8 years ago

I like that they include a page with further details of their intentions with regards to the AGPL: https://github.com/hasura/graphql-engine/wiki/License-Explai...

HugoDaniel• 8 years ago

Really cool! Congrats.

It would be awesome if it could be made compatible with Postgraphile (https://www.graphile.org/postgraphile/introduction/)

sheeshkebab• 8 years ago

GraphQL is interesting although it doesn’t appear to be answer to designing better APIs (poor modularaization, monitoring, and tooling come to mind). “Mutations” look to be an afterthought, and schema/ui are not impressive either.

Above plus companies tripping hard to monetize tooling around something that you get completely for free with REST...

Even as UI engineer I don’t get the appeal of it - state, caching and all the “performance” tuning make it too complicated.

jarym• 8 years ago

Great to see another project at this problem! We've been internally building something very similar but it uses Kotlin (Java) instead of Haskell.

zapita• 8 years ago

This is exactly what I was looking for. I will give it a try on my next project.

alberteinstein2• 8 years ago

Please do reach out to us on discord [1] if you have any questions whatsoever :)

[1] https://discord.gg/vBPpJkS

michaelmcmillan4• 8 years ago

I am very hesitant to couple the business logic of any application to the transport layer (GraphQL) or the persistence layer (Postgres) for that matter.

M_Bakhtiari3• 8 years ago

If you're not going to couple any business logic to the persistence layer, then what the hell is the point of using a RDBMS?

Anchor• 8 years ago

Data consistency with concurrency (transactions).

michaelmcmillan4• 8 years ago

Your reply suggests you either couple everything or nothing. It’s more nuanced than that. Everything is a trade off. With that said, it looks like this library will position itself in the center of your architecture — and I would be very hesitant to do that.

tensor• 8 years ago

Databases aren't simply persistence layers, they also encompass querying which implicitly couples your application to them. The simplest query model is the key value store, which itself is a form of coupling, but it's quite limited in the operations it can support with high performance.

GraphQL is similarly not a transport layer. The transport layer for GraphQL is HTTP. GraphQL is a query language. To use it with a particular database you need to transpile GraphQL to the query language supported by your database. Usually this is done in an unprincipled manner with ad-hoc code. In other cases it's done more globally and formally, as I assume Hasura is trying to do.

michaelmcmillan4• 8 years ago

I agree – but that doesn't actually challenge my argument. I would be reluctant to use this due to the very tight coupling to GraphQL and Postgres.

Unless – of course – you simply want something working quickly that does not need to be maintained or kept alive for a long time.

Alex39172• 8 years ago

GraphQL is basically the new Mongo, so of course someone will try doing it in Postgres.

dberg• 8 years ago

Heh i kind of agree here. Reasonable docs , easy to bootstrap and completely unaware of the underlying performance probs you are about to create bc it “just works”. Curious to your reasons though ?

Alex39172• 8 years ago

It’s seemingly a complete nightmare from a security and architecture perspective. And front end devs and bootcamps love it.

I’ve never used it so my opinion isn’t fully baked, but every time I look at the docs I’m just like wtf.

castis• 8 years ago

but GraphQL isn't a storage engine. or are you saying that in a few years we'll realize we've been using to solve a whole lot of things that really didnt need solving?

no-such-address• 8 years ago

This is a good trend. Will we see more work aimed at separating the data model and human interface (to make programming with attributed graphs safe and convenient) from back end issues (how to store data and process queries efficiently, where traditional databases are already very advanced.)

victor106• 8 years ago

Great job!!! Looks awesome

I see that one of the ways you want to monetize this is by offering support. Providing support is a good approach to monetize an open source project but it’s not very scalable. Have you thought of any other ways you can monetize this?

If others in the HN community can pitch in with any ideas they have on how to monetize an open source project that would be great as well.

faitswulff• 8 years ago

How does it handle batch execution of queries?

vsurabhi5• 8 years ago

We do not batch queries. We take a slightly different approach. We convert the GraphQL query AST into a single SQL query (no matter how nested the GraphQL query is).

reilly3000• 8 years ago

This is going to get tested today for a project. Thanks!

tango1220• 8 years ago

So cool! Please do reach out to us on discord [1] if you have any questions whatsoever :)

1: https://discord.gg/vBPpJkS

miga• 8 years ago

Why does hasura.io block residential network providers in Poland? I need to use VPN to just check the docs. That does not bode well for users.

mydigitalself• 8 years ago

I wouldn’t imagine that to be intentional, possibly upstream infrastructure is blocking somewhere.

antigirl• 8 years ago

Any subscription support?

tango1220• 8 years ago

It's in an early preview right now, and we should have something in a week or two! Just going through some performance and security checks over the next few days.

Do feel free reach out to us via discord if you'd like to take it for a spin.

mmanfrin• 8 years ago

I have a bit of a dumb question: where does GraphQL fit in a stack? I've seen in used primarily as a means of aggregating a bunch of disparate microservices, but then I see things like this that put it in the role of app server, almost. What advantages does this give over, for example, a Rails API (also on Postgres)? If it is supposed to fill the role of app server, how do you handle things more involved than simple CRUD operations? Like, if I sign up a user, how -- in a GraphQL world -- would I send a welcome email, typically?

FooBarWidget• 8 years ago

I see GraphQL as a protocol replacement for REST. With REST you have the following problems:

* Responses either contain more data than you need, or not enough data (e.g. you need data for related entities as well).

* With REST, if you need data for related entities then you need to perform additional REST calls, resulting in more network roundtrips.

With GraphQL you can specify exactly what you want, and the server responds with just that, in a single call.

yashap• 8 years ago

We’re experimenting with it on a new product at my company, and this is how we use it. Specifically, we use it as an API gateway in front of a bunch of restful services.

Another benefit is a more robust type system than you get with JSON. GraphQL supports union types, interfaces, etc., allows you to provide more strongly typed contracts between the backend and frontend.

One downside though - while the responses can become smaller, with fewer round trips, the requests get A LOT bigger (than with REST), as you have to specify every field, subfield, etc. This is especially true if you make heavy use of union types - then you have to specify all the fields, sub fields, etc. of all the various concrete members you might get back. Like requests with payloads over 1KB is not uncommon. If you’re polling, and the result of the polling is generally an empty response, you may end up sending significantly MORE data over the wire, not less.

Cthulhu_• 8 years ago

I believe it's intended as an API gateway, a means of creating a single front-facing web API for one or (usually) more back-end services. That way, clients can pick and mix apis and the data they want from it (e.g. streaming posts for mobile, or 20 posts at a time for desktop), without a developer having to create a REST / Rails API endpoint for each possible use case / client.

y4mi2• 8 years ago

GraphQL is the equivalent to REST or even SOAP.

Just like there are frameworks for these protocols, this is one that automatically creates a service for a postgres database.

postgrest[0] would probably be it's REST equivalent.

https://github.com/PostgREST/postgrest

dnomad• 8 years ago

GraphQL is pretty different from REST and SOAP.

In practice it is like exposing a very limited interpreter directly to the public internet. Instead of working with resources and limited verbs and hypermedia or SOAP (which is literally just documents) you allow clients to submit what are essentially programs with are then evaluated. In other words its mobile code all over again. [1]

[1] https://en.wikipedia.org/wiki/Code_mobility

tango1220• 8 years ago

There could be 2 ways to solve that specific problem of sending a welcome email:

1) Write a GraphQL schema stitching gateway, that can write a custom mutation that delegates to another API (or GraphQL API) to write to the database, or writes to the database directly. This is becoming popular as more backend services expose their functionalities via GraphQL.

2) React asynchronously after the database write is done. Put the database event in a queue, for example, that runs the email send async-ly.

adamkl• 8 years ago

The most prevalent use case for GraphQL right now is to act as a thin API layer over a bunch of microservices (which is what Facebook originally developed it for) -- but that is not the only use case.

GraphQL is actually much less than a replacement for a REST API. Its really just a type system with an execution engine that can map queries against that type system to resolvers (functions). That's it. This means that you can actually use it all over the place:

- Because it provides a strongly typed interface, you could use GraphQL as the basis of an RPC protocol (infact, GraphQL doesn't specify a transport mechanism, since its only job is to map queries to functions, so you could send GraphQL queries over RabbitMQ if you wanted to.. or encode them with protobuf and send them via HTTP/2).

- You could use a GraphQL schema to define the internal object model of an application and use GraphQL internally as a data access layer. You don't even have to expose it outside your codebase.

- You can use the tooling that comes with GraphQL (at least in the Node.js implementation) to parse incoming queries into their abstract syntax trees and to all kinds of fun stuff with them (automagically stitch multiple GraphQL schema's together into a combined API).

- I'm actually working on a proxy layer that can actually sit on top of an existing GraphQL API and apply additional logic to the incoming request and outgoing responses (it would actually be a good fit as a business logic layer for a Hasura based GraphQL api)

I think that in the future we'll see lots of very interesting uses of GraphQL that go beyond is current status as "REST API replacement"

*To answer your question "if I sign up a user, how -- in a GraphQL world -- would I send a welcome email, typically?" -- Define a mutation (update method) in your GraphQL schema to create a new user. Inside the resolver (function that is executed) for that mutation, write code to create a user (API call? Database update?) and send an email (drop a message in a queue to your email sending serivce?). It works just like it would for any other application codebase, you just initiate process with a GraphQL query, instead of, say an HTTP Post to a REST API.

sixdimensional• 8 years ago

I always thought it was in the name- a web API standard inspired by SQL over a graph data structure. This way, rather than having to predefine API endpoints that are super customized, you define fewer that follow the standard and act as a standard interface. Like if HTTP client = ODBC and GraphQL = SQL.

benoror• 8 years ago

Impressive! How's it different from postgraphql (https://postgraphql.github.io/) or postgraphile (https://www.graphile.org/postgraphile/)?

tango1220• 8 years ago

Broadly similar projects! The emphasis of Hasura is to make it seamless to work with existing databases.

Also, peripherally, we have quite a bit of tooling with an admin UI and a rails-like migration system to make the dev workflows easier :)

mathnmusic2• 8 years ago

The emphasis of Hasura is to make it seamless to work with existing databases.

Yet, I spent 15 minutes searching for example of using Hasura with an existing schema. Your "Getting Started" page has two examples: Deploy on Heroku and Run with Docker - in both cases, using a fresh database, not an existing one.

Sonicrida• 8 years ago

It's worth mentioning that postgraphql was renamed to postgraphile. They are the same project.

rcdmd• 8 years ago

Nice work. There are several competitors in the automagic Postgres REST/GraphQL space, and you're nailing documentation and ergonomics here. Your site and the Hansura Console sets your project apart from the rest.

spocklivelong• 8 years ago

I work with a Django app on Postgres, and I'm not sure how I would migrate that to GraphQL. How things play out in terms of django specific tables? What about relationships between tables? We mostly use the Django app to expose REST APIs, which deal with making complex queries using QuerySets to Postgres. Moving this to GraphQL mean that all the business logic that sits on the REST APIs need to be moved to the frontend?

tango1220• 8 years ago

Well it depends on your app. Technically, you have to rebuild your app to speak GraphQL to the frontend instead of REST. Internally, your GraphQL resolvers (like URL handlers) would then do similar work with using QuerySets as you're doing now.

If you'd like to leverage Hasura, it would be a good fit to potentially replace the "read" portions of your app with GraphQL quite easily. And then eventually you would have to move to writing a GraphQL gateway to handle some of the transactional logic and delegate the other stuff to Hasura via schema-stitching.

machbio• 8 years ago

there is mention of GraphQL proxy - any idea about any available proxies for GraphQL at this moment ?

garindra• 8 years ago

We at Sale Stock (fashion e-commerce in Indonesia) built an in-house system similar to this but more automatic and works with CockroachDB. Not open source yet, but might be interesting to you guys: https://speakerdeck.com/salestock/laskar-high-velocity-graph... (skip to around mid-way, first half is intro to GraphQL)

uptime2• 8 years ago

Interesting. Does this mean you plan to release it?

tango1220• 8 years ago

Oops! GitHub is down :(

We're on https://hasura.io too.

SOLAR_FIELDS2• 8 years ago

Huh, I just tried it and Github is up (making this post 2 minutes after the parent). Located in EU if that makes a difference.

SOLAR_FIELDS2• 8 years ago

As a newbie to GraphQL who is having some growing pains adopting a UI for a data warehouse that was basically bolted onto a REST API designed for backend queries, can someone help me understand what the pros and cons are for something like this vs hand rolling all of the backend GraphQL stuff yourself? We have been evaluating switching/offering a GraphQL backend for our UI to consume for our product since it seems well suited to our use case but so far the effort required has outweighed the gain we would get from it. Because of this a somewhat turnkey solution like above might be more appealing.

tango1220• 8 years ago

I think it depends on how much business logic you have in your REST API layer. If you can get away with using a turnkey solution like hasura with the right access control rules or by adding schema stitching then it would save a lot of effort.

I think the best thing would be to try a turnkey solution for your appliaction's database and then evaluating the effort saved. Turnkey solutions also might end up partial effort if not all. For example, Hasura is a great fit for incrementally adding read-only GraphQL queries which is where a lot of benefit of GraphQL for the UI lies.

jxub• 8 years ago

Almost a third of the codebase is written in Haskell, that's really nice to hear ;)

alberteinstein2• 8 years ago

The GraphQL Engine is written in Haskell, the Console in JavaScript and the CLI in Go. The repo contains all the three components. :) (I am one of the maintainers)

uptime2• 8 years ago

Thank you for releasing this. I am impressed with Hasura! No knock on postgraphile, I am trying that out too as well as prisma, but this looks very well done and the boundaries are clear (like postgraphile, views are important to make things work). Your UI is going to make people happy.

I have two questions: 1. can I point the hasura cli at my local install? and 2. Can we have UUID fields? I'd like to use them as primary keys.

tango1220• 8 years ago

Yes and yes. 1. https://docs.hasura.io/1.0/graphql/manual/migrations/enable-...

2. For UUID just do what you would normally do. We don’t have it on the column drop down but you can create the table via the run_sql window with the right type and default value, if you’re generating the UUID server-side

jamhan• 8 years ago

"Fast, instant"? That's, err, just dumb. Fast perhaps. Instant, never.

wjonas2• 8 years ago

I have been playing around with it. It's great.

Better support for ENUMS and MANY TO MANY relationships would be very needed though.

tango1220• 8 years ago

Thanks :)

We're adding docs for enums as we speak. What did you find lacking around the many-to-many?

wjonas2• 8 years ago

For 2 reasons inserting is tricky and querying is not super intuitive.

Example: I have 3 tables pages, page_blocks, and blocks.

- Inserting blocks into a page is tough.

- To query it's not very intuitive I would like to do.

pages {

  blocks {

    id

  }

}

& not

pages {

  page_blocks {

    blocks {

      id
    }

  }

}

Show HN: Hasura – Fast, instant GraphQL APIs on Postgres

Related Stories