HN user

graphviz

456 karma
Posts0
Comments90
View on HN
No posts found.
Monosketch 5 months ago

Is it easy to write a renderer in another program? Do people still say lazyweb?

Various tools use tool-specific graph attributes. For example, "rank" and "minlen" mean something to the hierarchical or layered graph layout tool (dot) but not to other layout tools. "size" and "label" are the same in all the layout tools. They all use the same underlying graph representation library with a parser generated by yacc or bison.

The documentation includes a big table of attributes that graphviz tools recognize.

With the availability of LLMs, there is better automated support now to find features that are needed. Just imagining here, but "make the layout fill the available space" or "make all the nodes look like points with associated text labels" (not sure if that even works but it should).

Don't most companies seek a supported product, like yWorks or Tom Sawyer Software in the case of graph layout?

Isn't the EPL "weak copyleft", so LGPL-ish? Would companies raise similar issues about glibc and GNU libstdc++? Just curious.

One of the nice things about this work is that by assuming the environment is a web client, it supports some basic interactive exploration, and offloads a lot of bothersome rendering problems.

Also, by focusing on control flow graphs, the proposed method does a better job with domain-specific layout. Apparently CFG visualization and exploration is a current topic; e.g. CFGExplorer. Probably Graphviz some users would benefit if it incorporated CFG-friendly level assignment as an option.

There's already machinery in Graphviz to support polylines instead of splines, and to control edge ordering, but it is not well tested or documented. It seems tempting to incorporate an edge routing algorithm of Brandes and Kopf, based on long vertical runs with at most 2 bends per edge. This seems close to a master's degree worth of work to understand and implement.

Graphviz started almost 40 years ago, is only supported by a few (one or two?) 2nd-generation volunteers with no 3rd generation on the scene yet. Over the years we've had plenty of our own disdainful "What is all this junk" moments, about our own code and other people's (c.f. various xkcd comics), but sometimes a better perspective is asking "What is being optimized that led to some team choosing or ending up at this point in the design space". Generally, the market is addicted to features.

It is a little dismaying to see the relatively slow progress in the broad field of declarative 2d diagramming. Given the way the pendulum has swung so hard back toward language based methods and away from using interaction to do everything, you'd think there would be a bigger payoff now for doing the work. Unfortuantely tool-making has always been a tough market. The customers are generally smart, demanding, and work in cost centers so don't have generous budgets.

Interesting, glad to know it's been useful for some specific contributions. (Not questioning that interesting-looking, appealing displays as overviews for general awareness are also worthwhile.)

What do people learn from visualizations like this?

What is the most important problem anyone has solved this way?

Speaking as somewhat of a co-defendant.

We learned the hard way, for some of us it's all too easy to make careless design errors that become baked-in and can't be fixed in a backward-compatible way (either at the DSL or API level). An example in Graphviz is its handling of backslash in string literals: to escape special characters (like quotes \"), to map special characters (like several flavors of newline with optional justification \n \l \r) and to indicate variables (like node names in labels \N) along with magic code that knows that if the -default- node name is the empty string that actually means \N but if a particular node name is the empty string, then it stays.

There was a published study, Wrangling Messy CSV Files by Detecting Row and Type Patterns by Gerrit J. J. van den Burg, Alfredo Nazábal, and Charles Sutton (Data Mining and Knowledge Discovery, 2019) that showed many pitfalls with parsing CSV files found on GitHub. They achieved 97%. It's easy to write code that slings out some text fields separated by commas, with the objective of using a human-readable portable format.

You can learn even more by allowing autofuzz to test your nice simple code to parse human readable files.

Random thoughts:

Sketching backed by automated cleanup can be good for entering small diagrams. There used to be an iOS app based on graphviz: http://instaviz.com

Constraint-based interactive layout may be underinvested, as a consequence of too many disappointments and false starts in the 1980s.

LLMs seem ill-suited to solving the optimization of combinatorial and geometric constraints and objectives required for good diagram layout. Overall, one has to admire the directness and simplicity of mermaid. Also, it would be great to someday see a practical tool with the quality and generality of the ultra-compact grid layout prototype from the Monash group, https://ialab.it.monash.edu/~dwyer/papers/gridlayout2015.pdf (2015!!)

Modularity is an excellent way of attacking complex problems. We can all play with algorithms that can carry on realistic conversations and create synthetic 3D movies, because people worked on problems like making transistors the size of 10 atoms, figuring out how processors can predict branches with 99% accuracy, giving neural nets self-attention, deploying inexpensive and ridiculously fast networks all over the planet, and a lot of other stuff.

For many of us, curing cancer may someday become more important than almost anything else a computer can help us to do. It's just there are so many building blocks to solving truly complex problems; we must respect all that.

This seems like it could be done in a little script in the graphviz gvpr processor.

Someone could implement this as a feature in graphviz. It might take a day to implement.

There are probably other "solutions" that already handle this more gracefully, but otherwise don't have a very complete set of features relative to graphviz.

It's reasonable (and constructive) for you to complain, but your "issue" is representative of a wide set of "issues" we've seen, and the real problem is we're just a little unimportant niche service and all this work is being done by volunteers and they mainly work on the things they know how to do.

It would be great to see new practical software for declarative specification of general 2D diagrams. With increasing emphasis recently on language-based tools, this is an excellent opportunity.

It would be good to find ways to make the internal algorithms and their parameters more "open" or more easily parameterized and recombined in novel ways. It would be good to find ways to rely on more generic optimization methods, though software like this is probably skating on the edge of computational intractability (not to mention usability or interpretability issues) so it's a very challenging problem.

Kudos.

The Graphviz project often wished to find a convincing formal definition of what it means for curves (spline sequences) that connect two points and are routed around obstacles to be "natural". Maybe machine learning will help with this before long. Something similar could be said for other geometric properties of diagrams. Many aesthetics have been proposed and studied, but do we know what blend is preferred by human subjects? How much do individuals vary in this?

A lot could be said. If there's a mistake, it was the economy not providing sufficient resources to solve this problem well. There was certainly awareness of this topic and some attempts to address it (for example North, Stephen C. "Incremental layout in DynaDAG." International Symposium on Graph Drawing, Berlin Heidelberg, 1995, or North, Stephen C., and Gordon Woodhull. "Online hierarchical graph drawing." Graph Drawing: 9th International Symposium, GD 2001 Vienna, Austria, Springer Berlin Heidelberg, 2002.

Successful systems like Tom Sawyer Software or yWorks have dynamic or incremental layout features. The market of course strongly rewarded work on interactive systems like Visio that were well integrated into larger platforms.

Not sure who remembers but high performance incremental constraint-based layout was one of the computer science lights that failed in the 1980s and 90s but with so many fundamental advances in technology (processors, algorithms, solvers, software platforms) and new emphasis on language-based approaches (in part due to LLMs) there might be a better opportunity now.

Anyway it's a hard problem, it's not as if nobody was aware of it.

It's well understood, graphs can be conveniently represented as matrices/tables/relations, and they are equivalent to edge lists.

It might be interesting to discuss to what extent did "graph databases" (you know who you are!) get a foothold because relational database platforms were slow to develop convenient notations and algebras (libraries) for working with abstract graphs. As Hou points out, there is some justifiable skepticism about the argument that graph databases are somehow intrinsically "more efficient" than relational databases for working with graphs. This would be surprising, given the obsession with optimization and performance that dominated the database community for many years, while issues like usability were a bit neglected (leaving the door open for other communities to innovate graph databases and data visualization platforms.) (Another point, don't I sometimes need both relational and graph algebras?)

Because I'm looking mainly for expressive convenience (with good in-memory runtime performance) it's not enough to know that Datalog can represent any abstract graph. If I find textbook pseudocode for, say, maximum matching in graphs, or transitive closure or connected components, how hard will it be to program in the target graph programming system? I'm confident that Datalog, Recursive SQL, and Cymbal or Gremlin can all get the job done, but at what expressive cost (assuming my algorithm is not already a graph language primitive)? Will anyone still even recognize the algorithm.

Graphviz has its own foundation graph library, that's not used by any other project. It has its good and bad points.

Based on that experience, we had our very own second-system-syndrome experience.

We decided our graph library should be modular, type safe, and efficient. (These properties came up in the comments here, too.) This is probably just a variation of "good, fast, cheap - pick any two."

By modular, want to write collections of graph algorithm libraries that are developed and even compiled independently.

By type safe, we mean we want to detect programming errors during compilation, or link time at the latest. We don't want programs to throw runtime errors like "your node does not have a color attribute".

By efficient, we mean that accessing an attribute of a graph is as cheap as a field in a C struct. (So, we don't want to carry around external hash table, or do a lot of string conversions, for instance.)

You can argue about whether these things are worth the price or even make sense, but that's what we wanted. We had some famous C++ creators in our lab, so we figured we could get help, and we were willing to give C++ another chance.

Gordon Woodhull, who had been an intern and kept working for us, is a brilliant programmer, and wrote an implementation of this kind of graph library working in templated C++. It's even published at https://www.dynagraph.org/ via sourceforge. The rest of us were not really sure we could ever understand how the code worked, so we had a code review with said famous C++ inventors. There were a lot of screens of code, and chinstroking, and greybeards pronounced "That would probably work." We knew we might have gone over the complexity cliff. (Let's not even talk about compile-time template errors, where one error fills an entire screen with details that only a... C++ inventor could love.) It was our fault, not anyone else's, and Gordon kept plugging away and even made all the dynamic graph layout stuff work, in Microsoft OLE, too. In hindsight it was probably our own Project Xanadu. While we got lost in this, a lot of things like Gephi (Java) and NetworkX and NetworKit (python) happened.

Also, John Ellson, a very talented software engineer who had written parts of Graphviz, revitalized the main effort.

This is nice work on graph visualization, but we learned years ago that readable network visualization does not necessarily mean good software architecture. For example, a good drawing of a tree may be easy to read and even beautiful, but may reflect an underlying design with no re-use or modularity. A graph of relationships between functions in an abstract machine may look very complicated, but that doesn't mean the design is poor.

Graphs are wonderful abstractions for the structures that arise in many kinds of engineering, but you need to focus on understanding those abstractions, not just pictures rendered by heuristics. Visualization can be wonderful, but has its limitations, especially when used out of the box.

This is interesting and a little bit entertaining.

There has been excellent work on taxonomies for data visualization. Tamara Munzner's book "Visualization Analysis and Design" is a good starting point.

There has been excellent work on identifying "dimensions" or properties of data visualization. Lee Wilkinson's "Grammar of Graphics" is a landmark. You could search backward to find Bertin's Semiology of Graphics, or forward to find ongoing work like UW's Vega https://idl.cs.washington.edu/papers/vega-lite/

Some of this work might somewhat capture the properties of a periodic table to organize a catalog and even support further discovery.

In practice, often people don't know if graphs will be planar or if they should expect it. However this is great work for specialists. In the graph drawing community, work that focused on planar graphs has been quite elegant but applications are not that widespread.

Vim Boss 3 years ago

What? Project founders and software authors might not always be around?