Clay (short for C Layout) is a high performance 2D UI layout library

Author here - yes you caught me. I'm building an asm IDE in C (which is why I ended up building this layout library in the first place) and the screenshot is from that application, specifically while building a NES game for Pikuma's NES/6502 course :)

teruakohatu• 2 years ago

Great work Nick. My email is in my profile, could you send me an email I would like to get in touch.

cyco1303• 2 years ago

Both Clay and the IDE look awesome!

michelreij• 2 years ago

Would this work on a microcontroller like an ESP32?

b3orn• 2 years ago

Probably, but you'd have to write a custom renderer for it.

torlok• 2 years ago

Has anybody made an attempt at making a library like this, but with cross platform user input, and support for accessibility? From personal experience, if you can output triangles and text, writing a UI library like this is maybe 2-3 days of work. The fun starts when you consider that younger people are touch-first.

kevingadd• 2 years ago

The accessibility is the hardest part. My custom library (in C# atop a weird rendering stack) has partial narration support and full touch/gamepad/mouse/keyboard navigation, but getting all the way to integrating with native screen readers is basically impossible at this point - from investigating it, it'd probably take me at least 3 months to get it working at all, and it wouldn't be portable.

One thing you have to do for reasonable accessibility is maintain a retained model behind the scenes even if you have an immediate mode API, so that's what I did. The immediate mode API does a bunch of caching in order to construct and maintain an appropriate retained mode tree across frames, which makes it possible to cleanly handle things like focus, selection and narration for invisible controls, etc. You also have to bake accessibility into the API from the start, for example by making certain every single widget has a description or a reasonable approximation of one, and by making sure there is an approximation of roles for every widget too.

A simple 'read a text description of the focused/clicked control' also doesn't get you far enough for narration - for example if there's a slider or textarea, you don't want to read the description and then the new value every time it changes, your narration has to be 'smart' and know to only read the description initially.

I'm hoping eventually AccessKit (https://github.com/AccessKit/accesskit) will be mature enough to use though.

0x0203• 2 years ago

Have you seen libAgar (https://libagar.org)? Cross platform support is certainly there, covering everything from windows XP (and earlier) to *bsd and SGI IRIX. I'm not sure what all having support for accessibility requires as I've never had to worry about it, but am curious if 1, agar has what's needed, and 2, what exactly is required of a GUI library for accessibility. Screen reader support? (Are there SR standards for desktop applications)? Dynamic scaling? High contrast?

(For embedded and/or touch first UI, LVGL is pretty nice, but probably lacking any semblance of accessibility features apart from keyboard navigation, but you could hook that yourself).

Teever• 2 years ago

I've never seen this before, very cool.

The style in the screenshots reminds me of KDX[0] from Haxial.

[0] https://wiki.preterhuman.net/Haxial_KDX_(software)

agentcooper2• 2 years ago

I wonder if you can get better performance than the built-in browser engine for certain complex layouts by first calculating the layout using Clay and then absolutely positioning the elements with HTML/CSS.

There was some news feed web app that used <canvas /> for better scrolling performance.

chii2• 2 years ago

if you can guarantee that layouts don't change during interactions, i guess it _might_ save some time for the browser (and thus battery, for low power devices).

If layouts change during interaction (e.g., orientation swap), then you will have a roundtrip to the server to recalculate. I assume this would cost more time than letting the browser css engine do their thing.

agentcooper2• 2 years ago

In my comment I assumed that Clay layout is running in the browser as well, but your idea of running it before serving the HTML is quite interesting!

sevnin• 2 years ago

Quite a lot of work went into those docs. I won't use it (because I don't have a need for it) but the examples look quite pleasing, nice work!

pistoleer4• 2 years ago

A big improvement over CSS, but still seems pretty manual and finnicky, I wonder if a constraint solver based system and syntax would be ideal for laying out UI.

For instance, being able to set the constraint "element.[x,y] = other.[x,y]+other.[width,height]/2;", instead of working with "attachment" objects.

loa_in_• 2 years ago

I think there already are existing mature solutions like you described. Isn't QT constraint based?

1oooqooq2• 2 years ago

qt lost it's golden momentum taking way too long to relax licensing in the 90s. had they done it before the gtk shift the world would be very different.

btw, gtk is based on tcl/tk which i believe is the, or one of the, original auto layout engines

speps• 2 years ago

Look up "cassowary" layout algorithm and all of its implementations.

airstrike• 2 years ago

Thank you for this! If you or any one else has any other literature to recommend, I'm all ears!

pavlov• 2 years ago

Apple adopted such a constraint-based system about 10 years ago in iOS and macOS. It's called Auto Layout.

It's powerful but not trivial to adopt. In particular, the design experience in Interface Builder has gone backwards in usability. The old system of resizing rules visualized as "springs and struts" was easier to understand in a visual design tool.

One might argue that's the cost of progress, and that designers using Interface Builder become better UI engineers when they have to figure out how to express themselves in constraints. But it seems to me that the reality is that a lot of people just stopped using IB.

IB used to be a crown jewel of NeXT's development suite in the 1990s. It was simple and focused, and allowed you to build surprisingly powerful UIs that connected to high-performance native code (unlike its mainstream competitor Visual Basic).

I don't think a lot of people have such fond feelings about Apple's current IB. Something was lost along the way.

pistoleer4• 2 years ago

I sense a parallel between that and the dropoff of SQL over lossy and impedance mismatched but "easy" ORMs and document stores. People don't realize they're trying to have their cake and eat it too maybe?

I wonder how we can stimulate "expert" tools and systems for those who don't want to take the greedy path of least resistance, and are tired of painting themselves into corners that way.

WillAdams• 2 years ago

That is really sad.

I always thought that Apple should have really pushed the whole "Applescript Studio" thing so as to create something even better and more approachable than Visual Basic --- it could have been a true HyperCard replacement.

gwking• 2 years ago

This is a topic about which I have long simmering opinions. I suspect that several things happened.

First, expectations and requirements went up. Laying stuff out with a mouse and snap-to alignment guides is ok for simple UI, but the more complex the design, the more I would end up fighting the vector-art style design interface. I remember staying up late fixing pixel alignment errors in nib files. Often you would need to move objects around to inspect another underneath and then play undo games to get things back exactly where they were.

The interesting parallel here is that the designers were doing all the designs in vector art apps, and were also frequently missing dynamic aspects of the design requirements.

It was one thing to design complex dialog boxes for desktop; think of a photoshop filter control pane. You target a minimum size screen, and then work with a fixed pane and lay things out. When the iPhone came out, IB worked ok for early versions and stock components. Screens were crowded but fixed width.

Once bigger screens came out, more designs started needing variable width layout computation. The APIs for dynamic layout were (as I recall now) more subtle than they sounded in the docs, and I recall joining several teams that misused them. “sizeToFit” and “sizeThatFits” were two culprits. Perhaps it made more sense in the simpler NeXT days, and perhaps the docs degraded. If you didn’t read Apple’s “Programming Guide” docs, it was hard to know how these things were supposed to work together, and those were like mini books. The guides got increasingly ignored as the iPhone’s UIKit took center stage. I was always a fan of NSAutoresizingMaskOptions but rarely saw others use them.

Second, modern version control made the binary nib files unmanageable, and merging the xml xib files was also awful.

Third, and to the parent comment’s point, there were quality problems with Interface Builder. New stuff got heaped on every year. The data model was proprietary but looking at the xml clearly just got more complicated and version-encumbered over time.

In summary, it is tempting to glorify the original NeXT tools, but I never used them. I started toying with IB in 2004. It never felt like a brilliant system because I couldn’t express the underlying logic of designs in the visual box dragging paradigm. Ironically auto layout pushed me further towards UI in code. But to each their own.

HexDecOctBin3• 2 years ago

"springs and struts"

This seems to have originated in something called Visix Galaxy, and supposedly done better there than in Interface Builder. See here: https://wiki.c2.com/?SpringsAndStruts

I tried finding any documentation on this tool/SDK, but no luck. Any one else has any more information on what this Galaxy looked like?

jacobp1004• 2 years ago

Auto Layout was just such a terrible system - as evidenced by it not being present in SwiftUI. It was slow, unintuitive, and not easy to handle UI updates

danielvaughn4• 2 years ago

I'd been working on a similar syntax for a while, mostly was inspired by some work I did in iOS back in 2015/2016. They used a constraint system and it's honestly pretty great, though some of the more complex features can be daunting.

I'd love something like this:

  red-box {
    width: 100
    height: 100
    top: 10
    left: 10
  }

  blue-box {
    width: 50
    height: 50
    top: $red-box.bottom * 2
    left: $red-box.right + 100
  }

pistoleer4• 2 years ago

Yes, that's exactly what I'm talking about!

graypegg• 2 years ago

I think CSS's own internal logic prevents this sort of syntax from working, since you could be talking about any element matching `red-box`. There's no good way to refer to a specific match of a selector, unless you enforce that the selector only selects 1 thing, even then, that would be conceptually hard to deal with.

I think flexbox and grid handle a good chunk of what you'd want here, but having to handle constraints at the "group" level. Either flexbox, so the browser finds the best way to place elements into a line based on your rules, or grid for 2 dimensions:

  boxes {
    display: grid;

    /* there are only 2 columns, 100px each */
    grid-template-columns: 100px 100px;

    /* all rows are 100px tall */
    grid-auto-rows: 100px;

    > * {
       /* All children take up 1 col, and 1 row */
       grid-column-end: span 1;
       grid-row-end: span 1;
    }
  }

  blue-box {
    /* blue-box must start at col #2 and row #2 */
    grid-column-start: 2;
    grid-row-start: 2;
  }

doodpants• 2 years ago

It seems that the Clay website (https://www.nicbarker.com/clay) can't be scrolled via keyboard.

shakna3• 2 years ago

It's worse than that. The site isn't accessible at all. I can't get my screenreader to make heads or tails of it.

lelandfe2• 2 years ago

That's because all the text is inside <div> elements, so there is no hierarchy for a screenreader to latch on to: https://imgur.com/a/D78Tbgk

...And the three nav links are made via empty, transparent <a> containers absolutely positioned over <div> text. Focusing, therefore, has nothing to read.

N.b both left- and right-clicking activates these anchors, because navigation is implemented as a delegated `mousedown` event on the document.

daemonologist• 2 years ago

Yeah the scrolling is handled by Clay rather than being native (everything has position: absolute and there's almost no nesting). It's very cool but I don't know if I'd want to ever use it for a website. Native apps though, might be worth trying out.

SkiFire13• 2 years ago

As if native apps do not need to be accessible

TinkersW• 2 years ago

Lots of other issues also such as page up/page down on keyboard not working and mouse scroll wheel is very slow(it behaves as if acceleration is turned off).

avhon1• 2 years ago

It doesn't render anything for me in Firefox, just a blank white page. After a moment, a message shows up: "This page is slowing down Firefox. To speed up your browser, stop this page."

kragen8• 2 years ago

yeah, i have the same problem. it's been hanging for almost an hour for me now. probably an infinite-loop bug rather than a working but slow algorithm?

codethief4• 2 years ago

Same thing (blank page) for me in Chrome/Vanadium on a Pixel 8.

EDIT: It does work in Firefox for Android!

kombine• 2 years ago

yeah, for me it renders on Firefox but is sluggish compared to Chrome.

nicbarkeragain5• 2 years ago

Author here, thanks for pointing that out - I'll implement it now.

nicbarkeragain5• 2 years ago

I've fixed it up :)

cowboylowrez• 2 years ago

still getting a blank page, browser is "links".

shipp02• 2 years ago

Arrow keys work for me.

MacOS 11 Firefox 129

jb19913• 2 years ago

The website says:

Fast enough to recompute your entire UI every frame

Yet, when I scroll the front page, made with Clay, it stutters and feels like it can barely handle smooth scrolling, even on a modern Apple Silicon laptop.

xyst• 2 years ago

On mobile, iPhone 15 PM, feels janky as well. It’s subtle but it’s there. Thought it was just me, but checked the comments to find similar experiences.

Also, poor accessibility as well.

kragen8• 2 years ago

i'm guessing that this is because safari doesn't allow the layout code (or anything else) to run while it's scrolling, which with normal websites (which don't use fixed positioning for everything) results in less janky experiences, and was famously critically important to get reasonable scrolling on the original iphone. it doesn't matter how fast clay's algorithms are if they're not allowed to run

(otoh when i try to load the web page it doesn't work at all, not even jankily, if we're talking about https://www.nicbarker.com/clay)

nicbarkeragain5• 2 years ago

Author here. I'm sorry that it performs poorly on your machine - if it makes any difference, it's the rendering that is slow, not the layout. The HTML examples are more meant as a demo than anything else, as the library actually doesn't do any rendering itself at all, it's exclusively a layout tool. I'm honestly not sure why the performance differs significantly between machines - I'm on an M1 mbp / firefox and it scrolls at 120 fps for me.

jb19913• 2 years ago

Hi, thanks for the response. It’s not actually rendering using native browser features, is it? I think that’s the reason. For example, on mobile, in which it also has problems for me in addition to the laptop, if I do a pinch zoom on the webpage, it all messes up and is not usable. I frequently do this on mobile to zoom in on text that I have trouble reading or images, etc.

jacobp1004• 2 years ago

On the same machine but in Safari, it is unusably slow

zoogeny• 2 years ago

I literally thought of building this kind of thing recently, although I have some differing ideas on how to implement it. But the general idea of a single header C-like file that compiles to wasm and outputs primitive drawing commands is exactly what I was thinking.

What I really want is something like the old Flash/ActionScript display list. Just a 2d scene graph with the option to output draw commands, text or sprites. Things like containers (with things like border/backgrounds/etc.) and layouts can be built on top of that, so you could have two separate header files, one for the display list and another for a layout library.

aarongeisler• 2 years ago

This looks great! I am a big fan of the single header format. I've linked Clay from my list of game resources for C developers. Cheers!

https://github.com/aaron9000/c-game-resources

mottosso• 2 years ago

I was looking for exactly this about two weeks ago for a 3D editor UI and ended up with Facebook's Yoga library. So far things are running smooth.

I would look closer at this library too, if it wasn't for:

"Clay UI hierarchies are built using C macros"

Yikes :S

jacobp1004• 2 years ago

Yoga is pretty good. Used to have a few bugs that stayed around way too long when it was abandoned for a few years. Development has picked up again though

jasonjmcghee• 2 years ago

For what it's worth, my experience with the linked website was:

- Text selection isn't possible, except on the final slide when I change to HTML Renderer and then it works very strangely (randomly selects all texts sometimes)

- The page crashed: "Error code: STATUS_ACCESS_VIOLATION"

- Also rounded corners look very strange

nicbarkeragain5• 2 years ago

Author here - apologies that the site didn't work correctly for you. Just OOI did it default to the Canvas renderer when you first opened it?

codethief4• 2 years ago

2000 loc, no dependencies, well-documented — this looks very nice! Kudos to the author!

zelcon• 2 years ago

Nice work. Love the arena allocators. Show those Rust weenies that memory management doesn't need all that ceremony.

Clay (short for C Layout) is a high performance 2D UI layout library

Related Stories