Introducing OpenCypher, the open graph query language project

http://neo4j.com/blog/open-cypher-sql-for-graphs/

by ryguyrg • 11 years ago

76 61 11 years ago

baconner• 11 years ago

Wait so... It's not about crypto? That's a little confusing name wise.

No, not crypto. Cypher is a graph query language, popularized in the Neo4j graph database. Its name may have been inspired by the Matrix movie...

phpnode• 11 years ago

This is great:

GraphQL has nothing to do with graphs.

OpenCypher has nothing to do with ciphers.

jonpaine3• 11 years ago

The database is called NEO(for java). All the examples are graphs of relationships in The Matrix, or of movies in general.

Cypher is named for the character in the movies. :)

mk32• 11 years ago

Neo4j has this thing on ignoring other graph databases. One example is their O'Reily book Graph databases covering you guessed it one graph database. I know it's good for marketing purposes. I like the way Cypher works, though not sure whether I would like to see Neo4j controlling how graph query language evolves.

rdrey• 11 years ago

At least it _sounds like_ they've been talking to other graph DB vendors about adopting (Open)Cypher. Who knows if it's really going to happen, but it makes me slightly more inclined to invest time in Neo4J's Cypher when I finally start playing with GraphDBs.

mk32• 11 years ago

In all article they mention only one another graph db which is not publicly available :) if you call this talking to other DB vendors :)

jakewins6• 11 years ago

So, it's to a large extent to address that last concern that we are doing this, right. We are moving the ownership of the language out into its own project, and inviting others to design the future of graph query languages together with us. We think that, while that means we loose a lot of control of our golden goose, so to speak, it is exactly the right choice if we want, as we do, for cypher to be the go to standard for querying graphs across the industry.

grandalf3• 11 years ago

Thanks for all the work you guys are doing. I have used neo4j on a few projects, one in node and the other in python, and have really loved using cypher. The projects end up so simple with so many fewer lines of code it's amazing.

marknadal• 11 years ago

To add to your list (disclosure: I am the author) is http://gunDB.io/ which is an Open Source graph database written in javascript. Although we currently have not implemented any query languages (we'll be adding SQL soon), so you can only do graph traversal as of now.

I'm curious to see what would be the popular vote of what query language the industry wants to standardize (I definitely hope not GraphQL, I'm personally think RQL and LINQ are interesting). Rather than some company attempting to standardize it, I want to see the users vote. What would you choose?

klapinat0r• 11 years ago

Why We Need a Common Graph Query Language
I believe that graph query language is Cypher.

I, and others, believe gremlin is that common graph query language.

My first meet with graph databases was in fact Neo4j, but quickly pivoted to Titan, and now using Cayley for smaller projects (although its Cassandra model will make an interesting future).

All except Neo4j supported gremlin, which to me is expressive, formal and actual-human readable.

Cypher looks cool, and has intuitive method of writing the edge query parts, but while it looks hip, I didn't find it obvious nor human parsable. Human readable yes. You recognize the terms, but translating it into an AST is simpler and easier in gremlin than in Cypher (in my opinion)

optimuspaul4• 11 years ago

Gremlin is certainly more readable in my opinion. Cypher has always felt like it was trying to hard to be different.

okram9• 11 years ago

http://www.slideshare.net/slidarko/the-gremlin-traversal-lan...

okram9• 11 years ago

Gremlin3 is now apart of Apache and nearly every graph vendor supports TinkerPop3 (or in the process of migrating to TinkerPop3). http://tinkerpop.incubator.apache.org/ Next, with TinkerPop/Gremlin you not only get a language but a virtual machine that can execute other languages (e.g. SPARQL) http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin... . The virtual machine executes over OLTP graph databases (Titan, Neo4j, OrientDB, etc.) and over OLAP graph processors (Spark, Giraph, etc.).

iamtherhino• 11 years ago

Neo4j does support Gremlin: https://github.com/neo4j-contrib/gremlin-plugin

Gremlin was originally developed by Neo4j co-founder Peter Neubaurer and Marko Rodriguez (Titan founder) while they were both working at Neo4j.

th0ma5• 11 years ago

I was thinking / hoping that language was SPARQL.

okram9• 11 years ago

https://github.com/dkuppitz/sparql-gremlin

jzd• 11 years ago

https://xkcd.com/927/

jakewins6• 11 years ago

Except, we're not introducing something new :) We're taking what's already used by 70-80% of graph database deployments and opening it up for other vendors to collaborate with us on the design and implementation of it.

optimuspaul4• 11 years ago

citation needed.

tsturge2• 11 years ago

As the designer of MQL (Metaweb/Freebase's query language) which is a GraphQL like language on top of a graph database, I'm always surprised by other graph query languages. Since JSON represents a tree easily, it a natural model for non-cyclic queries.

Using JSON for the template also leads to a clear correspondence between the query and the result which is a very nice property few query languages have.

http://mql.freebaseapps.com/ch03.html

to take trip back to 2007.

owyn• 11 years ago

Now that freebase is shutting down, is there ever going to be an open source version of the server side implementation of MQL?

tsturge2• 11 years ago

I don't think so. Freebase became Google Knowledge Graph and I suspect the technology underneath is basically abandoned inside Google at this point. A pity, I think the 2008 version would still be the fastest and most compact graph database available today.

barakm• 11 years ago

I implemented a "light" version of MQL atop Cayley (https://github.com/google/cayley) that I'd be happy to extend :)

tsturge did a prescient job with the original (yeah, GraphQL is 2007 all over again) and I wanted to keep the flame alive.

interdrift• 11 years ago

I work with graphs at daily basis and I can tell you that I was really suprised anyone hasn't come up with a solution to this.

dmoreno• 11 years ago

Why is similar to SQL but different, wouldn't be better to add the MATCH parameter to SELECT on good old SQL?

From https://en.wikipedia.org/wiki/Cypher_Query_Language:

    MATCH (charlie:Person { name:'Charlie Sheen' })-[:ACTED_IN]-(movie:Movie)
    RETURN movie

  SELECT movie
  FROM Movie movie, Person person
  WHERE person.name = 'Charlie Sheen'
  MATCH person-[:ACTED_IN]->movie;

jakewins6• 11 years ago

While Cypher borrows a lot of (good) ideas from SQL, the two languages are fundamentally different in their underlying model. SQL deals with Relational Algebra, while Cypher deals with Graph Theory.

For that pattern to work in SQL, your SQL engine would need to be able to view its relations (tables) as a graph. I'm not smart enough to sort this out in my head, but my instinct tells me that it's better to have the two models separate, mixing them in one query seems like a recipe for confusion.

okram9• 11 years ago

This is an exact replica in Gremlin3:

  g.V().has("name","Charlie Sheen").
    as("person").out("actedIn").as("movie").
      select("movie")

However, this can be expressed in a much simpler form as you don't need all the variables. Simply do:

  g.V().has("name","Charlie Sheen").out("actedIn")

cheerfulstoic• 11 years ago

Sometimes you do need the labels, depending on what you're trying to query for. If, for example, a person could have acted in both plays and movies, sometimes you might want one or the other specifically.

jerven4• 11 years ago

I always wonder how readable those queries really are. Its a nice claim but has anyone done any really serious research about this at all? For example the same in SPARQL 1.1

  SELECT ?cypher_attributes
  WHERE {
    ?cypher a <QueryLanguage> ;
            <queries> ?graphs ;
            <attributes> ?cypher_attributes .
    ?user <USES> ?cypher .
    FILTER (?user IN ‘Oracle’, ‘Apache Spark’, ‘Tableau’, ‘Structr’)
    ?opencypher <MAKES_AVAILBLE> ?cypher .        
  }

Instead of

  MATCH (cypher:QueryLanguage)-[:QUERIES]->(graphs)
  MATCH (cypher)<-[:USES]-(u:User) WHERE u.name IN [‘Oracle’, ‘Apache Spark’, ‘Tableau’, ‘Structr’]
  MATCH (openCypher)-[:MAKES_AVAILBLE]->(cypher)
  RETURN cypher.attributes

In the SPARQL case the graph flow does not revert on the edge with <USES> (it can using ?cypher ^<USES> ?user, but it would be weird and in the larger queries very confusing). The SPARQL case also tends to group related concepts together.

This assumes a DEFAULT BASE URI is selected for the SPARQL version that contains all the modeled relations. Which in a straight comparison to Cypher is a fair comparison.

I find Gremlin a lot nicer than Cypher, and a lot more powerful as well. Also up to today Neo4J just has not scaled all that well. I am awaiting the LDBC Benchmark results of Neo4J to see if I am wrong.

What Neo4J has been great at is making a nice solid product that aims at solving developer problems. I believe as a database it has not been that great at solving enterprise or life science community problems. Its still a single database instance without federation on demand.

jakewins6• 11 years ago

There's nothing stopping you from flipping that relationship order though, or making that pattern more compact. In fact, I'd prefer an overall reversed order, something like:

  MATCH (openCypher)-[:MAKES_AVAILBLE]->(cypher:QueryLanguage)-[:QUERIES]->(graphs),
        (u:User)-[:USES]->(cypher)
  WHERE u.name IN [‘Oracle’, ‘Apache Spark’, ‘Tableau’, ‘Structr’]
  RETURN cypher.attributes

I guess, in this particular case, it's subjective preference which language you feel expresses the query pattern most legibly. I certainly prefer the visual approach of cypher.

okram9• 11 years ago

Ah, thats better as you don't need all the unbound variables. However, you still don't need "graphs" unless for the English reading of the promotion.

In Gremlin3, the above is:

  g.V().match(
    as("openCypher").out("makesAvailable").hasLabel("QueryLanguage").as("cypher").out("queries").as("graphs"),
    as("user").out("uses").as("cypher"),
      where("user", within(["Oracle","Apache Spark", "Tableau", "Structr"])),
        select("cypher").by("attributes")

jerven4• 11 years ago

I like cypher, then gremlin for small queries. The problem is the queries I see are much, much larger. And at about 10 lines in the use of white space in SPARQL starts to make a real difference in readability in my opinion.

That can of course also be affected by my slight reading disability where the shape of the words is important. This shape could be disturbed by the connecting sigils in both gremlin and Cypher. So I understand that my preference might not hold for the whole population :)

okram9• 11 years ago

In Apache TinkerPop's Gremlin3.

  g.V().match(
    as("cypher").hasLabel("QueryLanguage").out("queries").as("graphs"),
    as("user").out("uses").as("cypher"),
      where("user", within(["Oracle","Apache Spark", "Tableau", "Structr"])),
    as("openCypher").out('makesAvailable").as("cypher")).
      select("cypher").by("attributes")

The original query is sort of an odd query as you don't don't need all the unbound variables...

And if you want to use SPARQL over TinkerPop, just use a SPARQL->Gremlin Virtual Machine compiler. https://github.com/dkuppitz/sparql-gremlin

johnymontana2• 11 years ago

Yeah, I actually wrote that query just as an example to show what Cypher looks like and some fun around the openCypher announcement. Wasn't expecting it to end up on HN, let alone have Marko convert it to Gremlin...

grandalf3• 11 years ago

I find Gremlin a lot nicer than Cypher, and a lot more powerful as well.

Do you have an example of a query that you think is better in Gremlin? I've yet to see one but haven't spent much time with Gremlin.

taylorbuley• 11 years ago

A major benefit of Gremlin is portability to e.g. Google's Cayley. If I was specializing in Neo4J I would specialize in Cypher too.

timwilliate• 11 years ago

I disagree with you on the statement that Neo4j does not work well in life sciences. I am a data scientist building large scale systems for mining genomic data, and we built a fairly critical piece of that infrastructure around Neo4j. I actually presented an overview of that work at GraphConnect this week:

http://speakerdeck.com/timwilliate/graphs-are-feeding-the-wo...

Many meaningful lineages in life sciences can be hundreds to thousands of levels deep (our datasets are great examples). Neo4j is the only graph database I have evaluated that handles traversals across lineages of this depth while still achieving the performance scalability promised by maintaining index-free adjacency across which ever node in the cluster a traversal is sent to.

a_bonobo• 11 years ago

The recent "huge open tree of life" paper uses a Neo4j database as well: http://www.pnas.org/content/112/41/12764.full

rymohr• 11 years ago

What can cypher do that GraphQL can't?

jakewins6• 11 years ago

GraphQL ends up being capable of whatever the backend exposes in the GraphQL schema, right - so in a way, they are isomorphic. However, in reality I'd argue GraphQL is a brilliant compliment to Cypher.

Here are some use cases I think Cypher expresses nicely that I (as a GraphQL noob) don't know how to do in GraphQL:

Simple recommendation engine - suggest people with lots of friends in common that I don't already know;

  MATCH (me:User)-[:KNOWS]->(friend)-[:KNOWS]->(fof)
  WHERE NOT (me)-[:KNOWS]->(fof) 
  AND id(me) = blah
  RETURN fof.name, count(friend) AS friendsInCommon
  ORDER BY friendsInCommon DESC

Basic routing - what's the shortest way for me to get to work?

  MATCH p = shortestPath( (home:Address)-[:ROAD*]-(work) )
  WHERE home.street = .. AND work.street = ..
  RETURN p

okram9• 11 years ago

In Gremlin3:

  g.V(blah).out("knows").aggregate("friends").
    out("knows").where(not(within("friends"))).
    select().
      by("name").
      by(count("friends")).
    order().by(valueDecr)

lobster_johnson2• 11 years ago

GraphQL, surprisingly, is neither a graph query language nor a query language. The comparison isn't really valid.

It's better to understand GraphQL as a protocol that competes with REST. It's only a language in the sense that JSON is a language; i.e., it has a syntax that can be parsed.

For example, GraphQL supports queries like this:

  query movie {
    whereYear(max: 1985)
    actors {
      hasName(like: "goldblum")
    }
  }

But this is something the particular schema and implementation would need to implement. If you want to filter by arbitrary attributes, you're out of luck because the spec is just a syntax. I suppose you do something like:

  where(what: "year", max: 1985)

but you still have to invent a standard set of parameters here: min, max, eq, notEq, lessThan, lessThanOrEq, like, etc. Again, totally ad hoc.

GraphQL, not being a language, also doesn't support variable bindings. So you cannot do self-referencing queries like "find all movies with a director who also acted in it", because that would require some kind of variable support.

(This is not a criticism of GraphQL, by the way. It's great at what it's defined for.)

optimuspaul4• 11 years ago

I believe that is coming. The latest spec has variables for some use cases.

jonpaine3• 11 years ago

I think the 'Graph' in GraphQL confuses people. GraphQL is no more relevant to a graph database than it is to a relational database. It is an intermediary that facilitates efficient communication by standardizing req/res structure.

So, for people more comfortable with SQL, your question could just as well be "what can SQL do that GraphQL can't"?

The answer is, of course, that they occupy different domains and have different functions. Both can do lots of things that the other can't.

optimuspaul4• 11 years ago

GraphQL is still very undefined I've found. It can do basic queries and very limited traversals. The biggest problem is that there isn't really any real implementations for it, the spec is still a working draft, and it hasn't evolved much past defining schemas and doing basic queries.

I find what they have defined so far far more approachable than Cypher or Gremlin. As they have been adding features it is starting to sprawl and look just as nutty as the others. But I do like how it is defining the whole ecosystem around how graphs can be defined and interacted with, much like Gremlin has, but with a much more focused and disciplined approach.

sireat• 11 years ago

I tried Cypher and Gremlin when working on smallish graph of about 3M nodes and 20M edges a few years ago

Back then both were pretty rough, Gremlin looked more polished but wasn't really (most promised features in the documentation were not working just yet).

What has changed since?

okram9• 11 years ago

Apache TinkerPop3 has been in development for 2 years. It is a complete rewrite since TinkerPop2. It was just released in July 2015. Its light years ahead of Gremlin2.

http://tinkerpop.incubator.apache.org/

The two big things:

1. Gremlin language is much cleaner and easier to use.

2. It supports OLTP graph databases (e.g. Titan/Neo4j/Stardog) and OLAP graph processors (e.g. Hadoop/Spark/Giraph).

Introducing OpenCypher, the open graph query language project

Related Stories