Ratty – A terminal emulator with inline 3D graphics

https://ratty-term.org/

by orhunp_ • 2 months ago

678 236 2 months ago

ratty-term.org

pelagicAustral• 2 months ago

Can I really render a 3D rat on my terminal? If I can then I'm sold.

sevenzero• 2 months ago

This is exactly what I thought as well.

mghackerlady7• 2 months ago

I saw it this morning on reddit, the I beam was replaced with a spinning rat for the demonstration. It was very cool B)

voidUpdate• 2 months ago

I was going to comment how it reminded me of TempleOS and the author should look into that, but the accompanying blog post explains how it was inspired by it https://blog.orhun.dev/introducing-ratty/

2ndorderthought2• 2 months ago

I actually see some use cases for this. It's one of those should be nonsense projects that somehow isn't.

panzi• 2 months ago

What use cases do you see?

drob518• 2 months ago

Yea, gotta be honest here; I’m struggling to see many use cases here other than 3d graphs. I really don’t need a spinning 3d rat cursor.

avaer• 2 months ago

Game development.

2ndorderthought2• 2 months ago

Checking 3d models in a directory inside my terminal to see what's what without opening an application and clicking 100 times.

jayGlow• 2 months ago

we could bring back the 3d file browser and render it in the terminal now.

https://youtu.be/dFUlAQZB9Ng?si=3fE-vE8xF5rSVhRR

herrj• 2 months ago

pranking your co-workers

iugtmkbdfil8343• 2 months ago

Rip Terry. May you never be forgotten.

edit: But your spirit lives on ( based on the project:D )

arusahni• 2 months ago

As do his sprites.

basilikum• 2 months ago

This looks a lot like it'd qualify for a ShowHN. Add "ShowHN: " to the beginning of the title and it should show up in /show

sigseg1v• 2 months ago

Seems... really good?!

Questions:

- rendering capabilities of this seem like it should also be able to handle 2d well, or am I mistaken? every solution I see for getting high quality 2d images or rasterization in terminal is all pretty bad. Could this do better than other solutions or is there a fundamental limit being hit somewhere?

- What happens with ssh given that this is gpu accelerated?

berkes2• 2 months ago

There are quite a few GPU powered terminal emulators around already.

Is that what you're looking for?

jetbalsa• 2 months ago

I think they are looking for full 2d graphics, bitmaps, sprites and the likes.

the_gipsy• 2 months ago

The kitty graphics protocol is pretty good. Ghostty implements it fully.

Lucasoato• 2 months ago

Imagine this with VR dev environments!

neomantra2• 2 months ago

Really fun project! Dude, I spent the last week implementing Kitty Graphics and Clipboard protocols in ghostty-web in the Canvas render.

Then I added WebGL and WebGPU renderers [1], including support for Kitty.

Then I see this this project on a Monday morning... so now I have to implement Ratty Graphics Protocol?!?! [2].

ETA: I looked into this; Ghostty would need patched to support Ratty since Ghostty-Web now defers APC handling there. It would also require pulling in a 3D engine like three.js or otherwise implementing file parsing, lighting, etc. Finally, since local filenames are part of the protocol, a browser would need some file resolver helper, either to get the data over the APC channel or via a URL.

[1] https://github.com/NimbleMarkets/ghostty-web/tree/nm-webgpu

[2] https://github.com/orhun/ratty/blob/main/protocols/graphics....

kokey• 2 months ago

I am a bit surprised that I had to look hard for someone to mention Ghostty in the comments.

tikimcfee12• 2 months ago

Your wish is my project.

Glyph rendering in three.js, fully instanced and addressable and positionable instances. Handles tens of millions. Sample app loads up full GitHub repositories in the web in a few seconds.

https://github.com/tikimcfee/glyph3d-js https://ivanlugo.dev/ide

neomantra2• 2 months ago

That's pretty handy, thanks for the links. IDE is slick! Given the structure, I think one could make a threejs backend on for ghostty-web. Makes sense if one will pull in more of three.js anyway. I'm adding it to my backlog to explore.

iugtmkbdfil8343• 2 months ago

Dude. Congrats. You actually made a compelling argument to put rust on my machine:P

silon42• 2 months ago

IMO, next crazy step is for terminal to just have wayland or X11 protocol ? (/s or not?)

mghackerlady7• 2 months ago

someone made an x server that renders to sixel[0]

[0]https://news.ycombinator.com/item?id=45341683

MayeulC• 2 months ago

I have been thinking about this for a while. It's not as crazy as it may sound, especially in light of the other comments making a parallel between terminals and notebooks.

A few thoughts:

1. Linux VTs kind of have this feature already: there is the normal buffer, the alternate buffer (that something like htop would draw on), and an IOctl can change them to/from graphics mode.

2. It makes sense for interactivity. Kitty's graphics protocol is quite useful for static shapes, can be abused for animations, but doesn't really cut it for interactivity (say, pan a graph around). Wayland is designed for this.

3. Wayland would be a good fit: isolate each command from another, let them request buffers, but keep control of where to display them, do not update them when off screen, etc.

4. One downside is that terminals excel for one-shot tasks. What's the purpose of the display when you are done with it? Should you kill the process driving it? Due to this, it may make more sense to delegate more features to the terminal emulator (displaying the 3D model, etc). Or maybe just allow the app to temporarily take over the window.

5. Once you have it up and running, have it talk directly to the direct rendering manager. Your "kmscon" is now your compositor / desktop environment. That's a fun thought! Add some basic terminal features like tabs and tiling, and you've inverted the usual setup.

6. One downside is accessibility. I really like that I can copy-paste any part of the interface for reference, "screenshots", etc. It's good for screen readers, too. You lose these advantages by going to Wayland.

7. Another current terminal limitation is fonts. Power line, yazi & other make use of custom fonts for drawing part of the interface, logos, etc. AFAIK there is no good way to query their availability (which is also an issue for color emoji). Custom fonts or a new protocol could be useful, but client apps could draw it themselves if given a surface (that can already do that with the kitty graphics protocol, mind you)

Obviously I am not seriously considering to make such a terminal emulator, but it would be an interesting experiment (heck, maybe something I should try this "vibe coding" with, since I wouldn't want to spend too much time on it).

pjmlp8• 2 months ago

Outside UNIX world, there is no much to think about, see:

- Atari ST GEM OS

- AmigaDOS

- Smalltalk

- Interlisp-D

- Genera

- Oberon

- MS-DOS (mode 13h and VESA)

- TempleOS

- Mac OS classic

pjmlp8• 2 months ago

That is how terminals work in OSes that were born with framebuffers instead of teletypes.

It is just another graphical application window on the OS.

amelius2• 2 months ago

Terminal is slowly becoming a full featured web browser.

iugtmkbdfil8343• 2 months ago

Always has been meme incoming. Also, more seriously, the purpose of a tool is to do a job. The question becomes whether this tool can be made useful. I.. honestly don't know, but I will be finding out soon:D

shevy-java2• 2 months ago

And rightfully so! \o/

microflash• 2 months ago

Or Terminal is already a full featured web browser?

https://hyper.is/

tekla• 2 months ago

Isn't there a Terminal that renders everything with React?

Super slow, but I guess thats what web devs want.

EMM_386• 2 months ago

There are terminal libraries that do this:

https://github.com/vadimdemedes/ink

Which is what Claude Code CLI uses (or was using?) and it caused many issues such as flickering, thrashing, and latency.

tikimcfee12• 2 months ago

Do you one better.

Rendered as instanced quads in 3d space. Tens of millions at >60fps, and an entire class of "grid based text rendering" evaporates.

https://github.com/tikimcfee/glyph3d-js

yagizdagabak• 2 months ago

i have been waiting for this.

rexthonyy2• 2 months ago

It's inevitable

corysama• 2 months ago

You asked for it…

https://github.com/fathyb/carbonyl

amelius2• 2 months ago

Heh that's ... truly the worst of both worlds.

Levitating2• 2 months ago

Or firefox flavoured https://www.brow.sh/

shmerl• 2 months ago

It does, yeah.

l-albertovich• 2 months ago

You know, Fox turned into a hardcore sex channel so gradually, I didn't even notice.

shevy-java2• 2 months ago

This is a great idea. I always wanted KDE konsole to e. g. show images inlined as is. This is possible via magick six:-, but I wanted this to be natively. I want the terminal to be able to work with any data and display it in any way. No need to simulate the 1980s era anymore (except for backwards/legacy support). So great idea here really.

berkes2• 2 months ago

Kitty and several other terminal emulators, have built in graphics display already. IIRC, this is called the kitty protocol, but I might be mistaken.

anthk2• 2 months ago

I did that with Sixels, no Rust needed, no 3D crap, no ad-hoc addons, just old vt340 support in XTerm.

That's how I read images under a remote pubnix with tut using a Mastodon account over plain SSH.

Chafa and XTerm. It works.

kjs33• 2 months ago

I would like to subscribe to your newsletter.

pjmlp8• 2 months ago

UNIX still trying to catch up with Xerox workstations in the REPL experience, or general Lisp machines for that matter.

Inline graphics from 1981,

https://youtu.be/o4-YnLpLgtk?t=376

pocksuppet2• 2 months ago

Or TempleOS.

Onavo• 2 months ago

That was a work of art. Also Oberon.

mghackerlady7• 2 months ago

People joke about templeos a lot, but it had some really neat ideas (holy-c is a pretty nice language)

jszymborski• 2 months ago

I came here to mention how it reminded me of the sick 3D icons TempleOS had in its terminal

steezeburger5• 2 months ago

That's not 3d

pjmlp8• 2 months ago

The example on the linked video it isn't, correct.

Here is another video, this time with S-PACKAGE used to develop Nintendo 64.

https://www.youtube.com/watch?v=gV5obrYaogU

Which given the REPL capabilities, you can easily embedd them on it, just like the other video.

SpaceNoodled2• 2 months ago

It's also from 2013

incanus77• 2 months ago

It is when you use a CRT instead of a flat panel.

convolvatron• 2 months ago

that's a poorly chosen counter-complaint. before SGI, symbolics owned the market for 3d graphics. this was a world where you could also just do (create-window), see the window, and get back a handle you could use to draw in it. starting with X10 afterwards for me was like drowning in mud.

hans-l• 2 months ago

Sometimes I really feel we're chasing UX that were solved in the Mother of all Demos (in 1968).

For those who haven't watched it: https://www.youtube.com/watch?v=yJDv-zdhzMY

pjmlp8• 2 months ago

Agreed.

nickcageinacage• 2 months ago

so cool. well done

mohamedkoubaa• 2 months ago

Emojis in a terminal are a step too far for me. This is just... Indulgent.

noelwelsh2• 2 months ago

I like this. No reason the terminal should only support text. Data science notebooks show one way the terminal can evolve. Lots of interesting stuff happening in this space, with Kitty probably being the most aggressive innovator here [1]. I'm not sure there is an overall vision, though.

[1] https://sw.kovidgoyal.net/kitty/protocol-extensions/

bcjdjsndon2• 2 months ago

Terry A Davis already did this. It was as crazy then as it is now

Wololooo• 2 months ago

Obligatory Temple OS unhinged video.

https://www.youtube.com/watch?v=o48KzPa42_o

Joking apart, the whole thing was both an exercise in madness and genius. Sometimes I wonder what he would have done if he had not gone crazy. We will never know...

ch4s3• 2 months ago

The person who built this directly cites Terry as the inspiration.

alias_neo• 2 months ago

I wonder if something like this could work for thumbnails in the terminal; I prefer to browse my filesystem from a terminal rather than the point and click file manager typically, and it would be really useful if I could have a grid-style `ls` with terminal based renders of the 3d models (thinking STL/STEP, 3D printing) in that directory. Bonus points if I could preview/rotate the model to inspect it.

calvinmorrison• 2 months ago

You can do this with thumbnails using sixels already

noelwelsh2• 2 months ago

eza [1] is a step in that direction. It lacks the interactivity, however.

[1]: github.com/eza-community/eza

em-bee8• 2 months ago

as a compromise i started using nemo/n̶a̶u̶t̶i̶l̶u̶s̶ with a plugin that puts a terminal at the bottom of each tab. so i have a graphical view of the terminal but a commandline in the same folder right next to it. the two don't interact other than being able drag and drop filenames from the filemanager into the terminal, so it is far from what we really want, but it's a small start.

Imustaskforhelp2• 2 months ago

yazi[0] seems to be able to display images iirc. Does it work for your use-case, hope this helps ya!

[0] https://github.com/sxyazi/yazi

the_other• 2 months ago

Mix this 3d graphics, with data science notebooks, with local LLMs, and perhaps an integrated coding harness, with visibility over your personal data and you’d have something absurdly good.

This might overtake “a haiku+macOS mashup” as my idealised computing future.

miah_• 2 months ago

At that point you've re-invented emacs.

sublinear• 2 months ago

push the state of terminal emulators forward

What's overlooked here are the insane political and economic forces that were required to get anywhere close to the (sort of!) consistent implementation of plain text we have today. These projects try to piggyback off that success yet only contribute back harm. We have standards for a reason.

I'm not saying people can't have fun, but don't try to start a cyberpunk-inspired revolution and then blame the side effects of groupthink and software rot on everyone else when it goes sideways.

krautburglar• 2 months ago

Exactly this. They are slowly turning the terminal into a web browser, just for attention. We already have web browsers. If you want something at the midpoint, make it, but please don’t call it a terminal & destroy one of the few non-trojan-horse standards that we have left.

joouha7• 2 months ago

No evolution necessary! With my project, euporie [1], you can have use your data science notebooks with graphical image outputs, HTML, LaTeX, etc, all in the terminal.

[1] https://github.com/joouha/euporie

Imustaskforhelp2• 2 months ago

This is such an amazing project. I find it so awesome that I can bump on such projects (and their creators, Hi!) on hackernews.

I wish to ask a question if I may (and as such pardon my ignorance on jupyter kernel, I don't know much about it and I hope you can tell me more about it :-D)

but my question is, is there a way to swap the jupyter kernel within euphorie to something else more minimalist?

And when you run a project with ssh, there are ways to give access to other users with user:password if I may ask?

I didn't know that there were ways to run jupyter kernels in terminal, I don't know when I might need it but I am prepared with this information now, this feels so nice to me, thanks for making it!!

This is like a checklist of a thing I didn't know that I needed/existed but the second I know that it has existed, it feels like my mind has checked it off and just a satisfaction from knowing projects like these existing.

(I think in some sense this is a bit of same reaction to me on Ratty too), Its just so good seeing projects in these spaces :-D

Edit: just remembered the one time I think I was using some websites which gave me jupyter and then I tried to use browsh to run jupyter to run jupyter in terminal so that it can be controlled by terminal but it had some issues and I wasn't able to run it.

I also wish to ask if there is a way to sign in to jupyter instance like that itself perhaps? (IIRC it was a jupyterhub instance)

gcr3• 2 months ago

incredible work!!

You mention using this over ssh. Is there any way to get this working in tmux or anything similar by any chance? Or is the idea that euporie itself is acting like a multiplexer?

fittingopposite• 2 months ago

Awesome. Can this work with Julia/Pluto.jl?

em-bee8• 2 months ago

that looks awesome (btw: a demo video would be nice, i could not find any), but it's not solving the problem terminals are generally used for.

i want something like jupyter but for unix shells, not for programming languages. and, i don't want it in the terminal, i want it to be the terminal, that is, i want to get rid of terminal escape codes that you currently need to make this work.

think about it like this: in jupyter you have pieces of code and their output. you change the code, it changes the output.

a unix shell version of this would be a commandline, and its output which would be text or an image or whatever the commandline produced. every output box would be itself a terminal if the output is text. but that's only necessary to support programs that produce terminal output. new programs could produce structured data that this jupyter for shells could interpret and display directly.

panki27• 2 months ago

I managed to get `pyvista` to render arbitrary 3D shapes directly to the terminal using kitty graphics. It's a giant hack, only way to make it performant is using shm.

https://git.theresno.cloud/panki/kglobe

dTal• 2 months ago

ipython-qtconsole seems very underappreciated to me.

pjmlp8• 2 months ago

It doesn't, that is an issue with how UNIX terminals came to be, and the whole backwards compatibility pretending that an HiDPi screen is a VT 100.

Terminals on other operating systems that grew up with a framebuffer don't have this limitation.

jdougan• 2 months ago

Or for that matter, the magic that was a Tektronix storage scope terminal (and compatibles. At school there was a vt10x that had been modified to act like a Tek 4014 by some third party).

darkwater• 2 months ago

What would happen when you use `cat` in Ratty then?

arkwin• 2 months ago

We are one step closer to the terminal in the movie Hackers, and I am all for it.

CTDOCodebases• 2 months ago

"Don't worry, all of these dependencies are worth it."

That had me in stitches.

whywhywhywhy3• 2 months ago

I was gonna comment here "real TempleOS vibes" then the TempleOS logo appeared a moment later in the demo video.

rullelito• 2 months ago

Can anyone explain why this is novel? It seems pretty basic?

dude250711• 2 months ago

1. Not another LLM: +100 points.

2. 3D rat: +100 points.

3. Outdated 80s UI paradigm: +100 points.

4. Uses Rust: +100 points.

lioeters• 2 months ago

inserted 3D objects in the demo above are actually from the TempleOS codebase itself

Brilliant. The dream lives on! This is the best form of paying respects.

It's walking a fine line between madness and genius, and who knows if it'll ever be practical, but more important is the sense of wonder and "fuck yeah" as King Terry expressed so eloquently.

sgt• 2 months ago

How do I enter zoom mode or pan mode?

whywhywhywhy3• 2 months ago

ctrl + alt + enter

randusername• 2 months ago

Here's the bit from the blog post about it:

When I first got introduced to [TempleOS], I was shocked and impressed by the flashy colors, graphical sprites and uncomprehensible UI. There are so many things that makes it so unique, weird and fascinating at the same time, somehow.... Basically, the command line becomes the direct interface for everything. You can write code, interact with the system and render graphics all in the same place, which is why TempleOS feels so unusual compared to conventional operating systems.

I think this could be a really cool approach. I enjoy tools like Chafa, imgcat, etc but something always feels a little clunky about the separation between text and images. Paradoxically having text and non-text all jumbled up like this feels better somehow.

liamwire• 2 months ago

You had me at spinning rat cursor

lackoftactics• 2 months ago

Hantavirus inspired?

xrd• 2 months ago

You should know that using a TERMINAL instead of a BROWSER ON THE DANGEROUS INTERNET is the ONLY WAY to *avoid* viruses!

BaardFigur• 2 months ago

Reminds me of TempleOS

mobeigi• 2 months ago

So TempleOS was ahead of its time!

olivierestsage• 2 months ago

In a world of slop, one truly noble project emerges

namar0x0309• 2 months ago

More of this please! Outside the box thinking! Yes and yes!

wolvoleo• 2 months ago

This would be nice in VR

joouha7• 2 months ago

It's very interesting to learn about the newly proposed glyph protocol [1] in the linked blog post. I was bemoaning the lack of exactly this here about 6 months ago [2]!

[1] https://rapha.land/introducing-glyph-protocol-for-terminals/

[2] https://news.ycombinator.com/item?id=45805072

CobrastanJorji• 2 months ago

Oh hey, that's a nice idea! Unlike some of the terminal projects I've seen recently, it addresses a problem without entirely reinventing the idea of what terminals can do.

tikimcfee12• 2 months ago

Agreed, glyphs are the key

https://github.com/tikimcfee/glyph3d-js

codedokode2• 2 months ago

The idea that every application should ship their own glyphs because some proprietary systems do not have normal fonts, is not great. Fix the terminal instead.

Also, if you want advanced GUI with icons, maybe you should just write a GUI app.

As for shipping custom icons, this is not very bright idea as well. If you switch between several applications on one terminal, then one application can redefine glyphs from another application. Also, when the application terminates, nobody cleans up its glyphs. Also, this increases attack surface because font standards are pretty complicated and one will be able to attack the system by just providing a glyph. We already have programs that can break the terminal, which should never happen.

Also, as for icons, I find emoji characters too distracting (and too large). They stand out too much and take away user's attention, breaking any visual hierarchy. The icons in terminal should be monochrome, and with thin lines, so that they do not distract you from the text and its structure.

joouha7• 2 months ago

I'm specifically interested in querying for support of particular glyphs (e.g. the symbols for legacy computing block), so applications can use a different fallback if it is know that a particular glyph cannot be rendered and will break the interface.

I agree that the addition of sending custom glyphs to the terminal is potentially problematic.

gosub1002• 2 months ago

Has anyone tried to create 3D fonts? It sounds like a ton of work but might look cool if done correctly.

You could also do really cool text highlights by working with light sources and shader effects

Another feature I'm looking for is smooth scrolling when you hit enter. I've had debates before where they claim it's not possible, that the text must jump one line. But I think it's possible, by shifting the frame buffer up.

tikimcfee12• 2 months ago

https://github.com/tikimcfee/glyph3d-js

If you render your text like a polygon, you get full 3d animations for free.

gorgoiler• 2 months ago

Hah, reminds me of the Quantel broadcast equipment on the 1990s. Why fade to black when you can fade to 3d butterfly!?

rs545837• 2 months ago

Damn this was really fun to use.

drakythe4• 2 months ago

My first reaction: "But why?"

My second reaction: "Oh wait is that TempleOS being cited? This is either awesome or terrible."

chakintosh• 2 months ago

People complain about token limits

Then spend their tokens on abominations like this

Make it make sense

kspacewalk2• 2 months ago

I mean... Why not?

torben-friis2• 2 months ago

I much prefer seeing tokens used for silly fun stuff, rather than sad get-rich-quick attempts like filling YouTube and Spotify with LLM crap.

koiueo• 2 months ago

Filling GitHub with LLM crap isn't on your list... I wonder why

gosub1002• 2 months ago

Did this developer complain about token limits?

Gracana• 2 months ago

Make it make sense

It's not hypocrisy when different people do different things.

gavmor2• 2 months ago

People use their tokens, and then complain of limits. Where's the incongruity?

Panzerschrek• 2 months ago

The question is - why do we still need the terminal abstraction at all?

dkersten2• 2 months ago

That’s what I came here to ask. Their demo looks like Compiz back in the day: ok, cool, you have 3d effects, but… why? What does it do for me?

Compiz 3d effects were ultimately a useless gimmick and I predict this is too.

reaperducer2• 2 months ago

Lack of imagination doesn't mean this isn't innovation.

It's the ability to convey more information in less space.

Top-of-my-head notion: The cursor spins (or changes in another way) to reflect CPU use, or bandwidth use, instead of taking up space elsewhere on the screen.

pjmlp8• 2 months ago

I would argue that a proper REPL is much better.

gavmor2• 2 months ago

The terminal is keystroke-driven. It's character-selectable. It's reliable in a way that the GUI is not. When I drop frames, I can still enter the commands to rescue myself with some assurance they'll be interpreted, eventually.

I agree, a REPL isn't Unixy in the streams of text kind of way... or is it?

mr-wendel• 2 months ago

Let me add more! PIPES.

It's a bit more abstract and useful than "character-selectable" when viewed at the byte-level abstraction.

The ability to chain together utilities with no complicated data structures is extremely flexible. One of my favorite current use-cases is using FFmpeg to process RTSP streams that send output (e.g. high quality stream for recording, low quality low FPS for processing, max quality low FPS for stills, etc) to separate file descriptors. FFmpeg doesn't care whats on the other end (e.g. redirect to file, read via Python, etc) due to these lovely abstractions.

Reliability translates directly to scriptability. Yes, you can create monsters, but through the use of sub-shells and pipes I think it's the fastest, cheapest, most concise way to pull off some really cool multiprocessing tricks.

bsder• 2 months ago

The question is - why do we still need the terminal abstraction at all?

Because nobody is willing to put in the work to create a GUI toolkit that doesn't suck ass.

It's not that people want the "terminal abstraction". What people want is "Put <thing> on screen without me needing a PhD in graphics programming." That's why the dominant desktop interface paradigms have become TUIs and a Browser-In-A-Trenchcoat.

tootie• 2 months ago

Make me think of the infamous Unix scene in Jurassic Park.

mghackerlady7• 2 months ago

That was an actual file manager if you didn't know, it was called FSN. Theres a FOSS clone called FSV[0]

[0]https://en.wikipedia.org/wiki/File_System_Visualizer

brunoborges• 2 months ago

Cool... why?

kandros• 2 months ago

Expect to see Orhun in here before clicking, not disappointed

HumblyTossed• 2 months ago

How long until we have a web browser in a terminal (not just Lynx, but a full on web browser)?

PUSH_AX• 2 months ago

Soon it will just be browsers all the way down.

mghackerlady7• 2 months ago

there was a project that rendered firefox to the terminal through box drawing characters. When libweb is more complete I kinda want to do something similar

austinrm• 2 months ago

Excited to see others equally inspired by TempleOS’ 3D feature :)

I tried something similar a few months ago that acts more as a library to ratatui than a separate terminal emulator [0].

Was surprised how far one can get using some off the shelf characters like half-block when rasterizing.

The Glyph protocol mentioned in the blog post is interesting … perhaps custom glyphs could help smooth some of the (literal) rough edges from the low effective resolution of a terminals character grid.

[0] https://github.com/limlabs/ratatui-3d

hartjer• 2 months ago

"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." - Jeff Goldblum (OG Jurassic Park)

injidup• 2 months ago

Anybody remember "wobbly windows"? It never sticks.

mghackerlady7• 2 months ago

I have wobbly windows on whenever I use KDE. I like how it gives the movements more momentum, though I have it turned down by a lot so it isn't distracting

semiinfinitely• 2 months ago

temple OS?

Bluescreenbuddy• 2 months ago

Another terminal to murder your battery life

JSR_FDED• 2 months ago

This is pure Hollywood OS - hackers feverishly entering obscure incantations like “upload virus”…but now with the terminal twisted into a Moebius strip!

d--b• 2 months ago

https://m.youtube.com/watch?v=kSdvNH1BYQI

Still giving me goosebumps

nialv7• 2 months ago

Terry A. Davis will be proud

dstnn• 2 months ago

Friggin waste of resources

ceayo• 2 months ago

why would you want this?

antran22• 2 months ago

why wouldn't you want to see your htop output on a moebius strip

rexthonyy2• 2 months ago

That's quite cool, visually pleasing to the eye and high on data usage.

HackerThemAll• 2 months ago

What a twist, having textual window manager within a graphical user interface, and that textual window manager implementing bits of graphics.

You'll soon may be able to implement overlapping graphics windows in TUI within GUI.

This is stupid af.

quotemstr• 2 months ago

Cool.

Seriously, though, when are we going to see the convergence of terminals and GUI remoting protocols? People have already departed far from Unix pipeline utilities. "TUI" programs are already GUIs in disguise. Why keep pretending that the terminal (as used by TUI programs) is a different kind of thing?

tikimcfee12• 2 months ago

Agreed. Terminals are just addressable glyph tools with programming tools wrapped around it.

https://github.com/tikimcfee/glyph3d-js

sscarduzio• 2 months ago

Well deserved HN #1

xantronix• 2 months ago

Was this aided with LLMs or purely for the love of the game? I don't see an AGENTS.md or anything similar in the repo.

alexprengere• 2 months ago

If this turns out to exfiltrate all my keys, I will have some explaining to do to my security department.

zackoverflow• 2 months ago

This is kinda possible already today with the Kitty graphics protocol, I made a demo here of rendering 3D graphics[1] with kitty. The actual important missing thing (and which ratty seems to also not include) is vsync.

If rendering is not aligned then it's possible for the terminal emulator to read the framebuffer while the application is writing to it, causing visual artifacts.

[1] https://x.com/zack_overflow/status/2035921425341763756?s=20

ruler88• 2 months ago

y tho?

ghostoftiber• 2 months ago

This reminds me of when compiz came out and everyone was like MY WINDOWS ARE ON A CUBE and I NEED WOBBLY WINDOWS.

So anyway, being that guy, I immediately installed it.

dpacmittal• 2 months ago

I remember those good old days. I had virtualbox running win xp on its dedicated face on the compiz cube. It felt magical to switch between windows and Ubuntu specially with all the compiz animation goodies

krs_• 2 months ago

You can still have that in KDE Plasma to this day. I have the wobbly windows enabled because they're neat.

mnorris2• 2 months ago

I built DeepSteve (https://github.com/deepsteve/deepsteve) with a similar itch but went the other way. Instead of adding graphics to the terminal, I put the terminal in a place that already has graphics.

I kept trying to optimize my terminal layout and realized I could just run my terminals inside of the browser, and let Claude Code write JavaScript in the same browser tab to customize the experience however I want. It's kind of a terrible idea, but it's my terrible idea, and I love it.

wonger_• 2 months ago

Have you ran into any issues with handling inputs? Like how vim Ctrl+P would maybe be intercepted by the browser Ctrl+P shortcut for print.

And have you run into any other issues, maybe like performance?

I feel like web-ified terminals get nerfed pretty hard and I'm not sure if/how people overcome that.

I like the idea of customizing multiplexed terminals with on-the-fly JavaScript, tho.

mnorris2• 2 months ago

I haven't seen any performance issues for Claude Code, even when I'm running like 20 in one browser tab and looking at them all at the same time (rendered with xterm.js), but Gemini and OpenCode flicker a lot even if you have one open.

muyanapar• 2 months ago

Amazing project and the ratatui community is really wholesome :,)

pawelduda• 2 months ago

This is so cool I'm sad I can't think of any use case for me

gariti• 2 months ago

last night I was pondering if there was a ghostty plugin that can make my terminal like the opening scroll from a Star Wars movie. Can we make that happen?

mncharity2• 2 months ago

A couple of comments here mention using this in VR. Fwiw, years back I played a bit with shallow-3D UIs for software dev. Shallow like within a few cm of a laptop display, to minimize VAC eye strain for all-day use. Think more being able to layer and draw in color, but in 3D, rather than waving arms in a room.

The 3D can be wiggle 3D, or perspective from webcam head/eye tracking, or stereo from shutter glasses, or XR HMDs. Wiggle is easiest - just move the object orientation back and forth. Cute but distracting. Well, cross/parallel-eye gaze is easier, but limited - ok for little UI test swatches. Perspective is more subtle, less intrusive. Can be simple with a head tracker driving a single orientation, or go all in with eye pose (for distance) and window locations, to do an accurate 3D render. App stereo pairs can be "I give you two windows Left/Right-eye", or "alternating L/R view, labeled/synced/polled". Other possibilities. Many of these need window system/manager/desktop support. I found a lot of leverage in using a stack of electron and X.

It's fun to displace text in 3D. Like colorization, but more so. And if you don't mind a cluttered appearance, you can add secondary information layers segregated by depth. And... etc. Emacs with characters-have-a-depth finally gets you something LispMs didn't have. Fun aside, to explore possibilities with code text, with anything not inherently 3D, far easier to prototype UX with fg/bg colors, fonts, unicode, and animation. Or in browser, overlaid divs and transparent 2D/3D canvases.

tikimcfee12• 2 months ago

I have a working fully 3D glyph based text rendering system I can't seem to get people to look at.

It's this. Every character is a 3d placed quad, instanced rendered, so you get tens of millions and then some. They are individually addressable and mutable like any polygon. I use it to render entire GitHub repos in one go. I have two versions, native Apple and web. Web has the basics of an ide setup. Would love insight or thoughts.

https://ivanlugo.dev/ide

em-bee8• 2 months ago

it opens source files in an unreadable small size, presumably to fir the whole file into the window. i can zoom in, but i can't properly scroll around or select text. and i don't see the benefit of using 3d here. it doesn't seem useful.

pitched• 2 months ago

I opened and it said to use a repo tab. There are no tabs so I pushed a button. It had a list with Linux so I clicked that. It redirected to a 404 page on github and I gave up.

I don’t know what this is supposed to do, let alone how to use it. But I looked at it for you!

chaidhat• 2 months ago

looked at it. I don't think it works, as the other person said.

wavemode• 2 months ago

Is there a reliable way to pan around? Middle click + drag doesn't pan if my screen is covered by an object, it just moves the object itself. Scrollwheel pans up and down but I can't figure out how to go left and right. The minimap is too coarse - when I'm zoomed in close enough to read any text, very tiny movements of my mouse on the minimap pan around massively, too massively to be useful.

mncharity2• 2 months ago

I use it to render entire GitHub repos in one go

Nifty.

Some scattered thoughts...

First-time visitors easily bounce away if there's friction. My experience was: After reloading the page to reread the "load a repo" message, it took a search to find it. Two-fingers unexpectedly panned not zoomed. I consulted the keyboard shortcuts found earlier. My scroll panned. Keys zoomed, but speed was limited by key autorepeat. No response when clicking on shortcuts, suggesting they can't be changed. I eventually noticed shift-two-finger zoomed.

So perhaps add an initial popup, as with browser games, giving basic orientation/keys? And perhaps have some default repo already loaded?

Is antialiasing turned off? The text is less readable than I'd expect, and there's flicker during zoom.

I didn't find the usual github-icon link to the repo. Add it?

Perhaps try google/bing maps-like zooming towards cursor location, rather than always towards center?

Perhaps clamp zoom-out and scroll so one can't reach no-text-visible blank black screen? Perhaps clamp zoom-in earlier, so "slam inward" stops short of the current one-line-high screen?

Clicking README.md brings up a 3 column render, which is too small to read on my screen without further zooming. I'm unclear on what I'd like instead.

A big picture observation. Early phone apps did skeuomorphic UIs - a calendar app might render a wire binder spine, and animate paper page turning, with sound effects. It helped onboard a user population unfamiliar with phones. We don't do that anymore. It'd be silly. Most VR UIs seem to me skeuomorphic transients. Why hobble ourselves by unnecessarily importing the severe constraints of a Euclidean physical world?

So for example, file presentation might combine content-aware and location based scaling. A README might show up with the header lines enlarged, as in some md editors, and perhaps with fisheye-like distortion (when at the top, the top is rendered normally, but with increasing shrink below). Or mix in outliner-like expand/collapse. And so on. Whatever dynamic geometries are task helpful for the human. Once text colorization is working, it might be fun to work little exercises (make videos?) - with files X and task Y, how might they be most helpfully presented? When skimming agentic markdown as it scrolls by, I keep wishing for a couple of cloud-fast small models that watch it with me, and adjust the text size/color/annotation/etc to guide my attention to notable bits.

In the vein of making functionality easy to explore, perhaps give diff a default pr?

It's neat to see a repo slurped into something like this. Thanks for sharing.

PunchyHamster• 2 months ago

It's funny when terminal emulators add shit like that but search in windows names/content ? nah who needs that

slopinthebag• 2 months ago

Remember when the Claude Code devs claimed that CC is a game engine?

This is a game engine.

Levitating2• 2 months ago

I have been waiting for a proper implementation of this since forever. Thanks orhun.

commieneko• 2 months ago

It's like someone crossed an 80s Silicon Graphics workstation with a Vic-20.

I, Beldar, approve.

zhxiaoliang• 2 months ago

I’m not sure why I’d use it, but I enjoyed the visual and loved the brutalist design of the website -- it brought back some fond memories of the good old days.

cr545l• 2 months ago

You can also use the kitty protocol or sixel in modern terminals.

LennyHenrysNuts• 2 months ago

This is gloriously bonkers.

alexvkw• 2 months ago

Thanks. I hate it

holg• 2 months ago

it is the fantastic tool!

lupodevelop• 2 months ago

I don't know. mixed feelings... but fine

Ratty – A terminal emulator with inline 3D graphics

Related Stories