Show HN: FrameOS – operating system for single function smart frames

by mariusandra • 3 years ago

474 112 3 years ago

whycome• 3 years ago

Is there a good equivalent to keep old Amazon fire tablets from being e-waste? They are Android based.

The bootloaders have been locked; and the 2015 models (iirc) were the last ones where a bug allowed unlocking them by shorting some pins. Without an unlocked bootloader, you might be able to root Android, but you can't replace Android or the ROM.

Not that it matters anyway - it's MediaTek. Their partition layout and driver model from those eras is generally a disaster. There's no way you'd be able to make FrameOS in a reasonably elegant way work on that hardware.

womod• 3 years ago

You can unlock the bootloader of the 2018 Fire HD 8 if you force it into download mode by opening it up and shorting the CLK test point to ground. Older tablets don't need to be opened up, it's all just software. Everything after that though has been unable to have the bootloader unlocked.

paxys• 3 years ago

What's the problem there? I believe you can just install and display any Android app on it right?

daredoes• 3 years ago

Use Fire Toolbox to disable all the bloatware. Buy a license for Fully Kiosk to convert it into many different practical things.

Personally, I use Amazon Fire Tablets as audio drivers around my house for Snapcast. Each tablet has USB, wifi, 3.5mm Aux, and a built in battery, which make them great for this task, especially on sale at $45 a tablet

m463• 3 years ago

pretty cool

Seems to require a pi. Wonder if it can work with this: https://amazon.com/dp/B0C4TS1NMS

Gys• 3 years ago

The frame you referenced has a Pi Pico build in. FrameOS needs at least a Pi Zero.

traverseda2• 3 years ago

Although NIM has a pretty good embedded story, it's not entirely out of the question to port it.

mariusandra19• 3 years ago

I'd love it to work on the RP2040, and perhaps even an ESP32, but we're not there yet. Just yesterday I converted FrameOS from being an interpreted python app to a 3MB compiled binary, so we're half of the way there. I'm not sure what changes I must do to get it working (e.g. integrate a wifi driver, remove GC and threads, etc), but it's in the realm of the possible.

mariusandra19• 3 years ago

Just found this project that could help: https://github.com/daniel-j/picostdlib

djazzz4• 3 years ago

I have that device, it has no wireless connectivity (built on rp2040). The only IO is usb and microsd card (hijack for spi?). I have Nim drivers for a similar rp2040 based product (Inky Frame) which works great and has wifi/bluetooth (Pico W). As of now FrameOS doesn't run on rp2040.

smith7018• 3 years ago

Fun little project! I worked on a 7 color eInk Arduino-based picture frame two years ago for a gift. My biggest issue was that I had to manually crop, dither, and color index the images to get them to look _okay_ on the display. If FrameOS could handle all of that for me then it would have saved hours of manual editing and testing.

I ended up getting that same person an Aura frame this year because remotely adding photos ended up miserable

mariusandra19• 3 years ago

I definitely do not want anyone to manually crop, dither and color index images. This should be done by the FrameOS software. It's already working if you use e.g. Inky Impression 7 color frames, but mostly because I just defer to their software/drivers, which already handle this...

gadgetoid3• 3 years ago

Our crude (actually Pillow’s IIRC) dither has a certain nostalgic appeal to it, but can definitely be bested by other algorithms. I’m still not totally convinced we have a great perceptual match for the colours either. The approach to get them - take a photo of the screen, use a colour dropper and a bit of eye balling - was crude.

(Hi, I wrote most of the Inky driver and your project is the kind of awesome I wish the Pico/RP2040-based Inky Frame devices could pull off.)

kevin_thibedeau• 3 years ago

Imagemagick can batch process images with restricted palettes. No reason you had to do it all manually.

eightysixfour• 3 years ago

This is awesome! There are a lot of projects trying to duplicate this but very few have the polish, especially on the creation side.

Selfishly, I’d love support for the lower power eINK devices, like the InkPlate series which has a built in ESP32, but this is making me consider swapping them to a Pi.

mariusandra19• 3 years ago

I'd love for FrameOS to support ESP32. Two days ago FrameOS was a slow and interpreted Python app. Today it's a 3MB statically linked binary. The margins are tight, and I'll likely need to cut out some features (e.g. threading), but theoretically it should be possible to get it to run. However I'm not planning on taking this on any time soon, as there's still a lot of work left to make the Raspberry version great. :)

squarefoot2• 3 years ago

The ESP32 port of LVGL might be a different approach to what you're looking for. There are also ready made boards with included display:

https://github.com/lvgl/lv_port_esp32

https://www.youtube.com/watch?v=d2OXlVcRYrU

fudged71• 3 years ago

for dashboard software like this are there any reliable techniques to prevent burn-in? I'd like to set up a large screen with a dashboard but I don't want to wreck the screen

linuxdude314• 3 years ago

This is specifically about eInk displays. They do not suffer from the problem you are describing.

Generally this is solved by displaying more than one image and rotating between images on a time interval.

_flux• 3 years ago

This is for both:

It's designed for both screens that update 60 seconds per frame, and for screens that update 60 frames per second.

Or that's what at least I infer from that, because I don't know of 60 fps eInk displays :).

samstave3• 3 years ago

If your site's always flapping, don't worry about burn in?

BHSPitMonkey• 3 years ago

They will generally transpose everything on the screen by a small randomized difference every minute or so.

mariusandra19• 3 years ago

There's nothing built in now, but a common technique would be to make the image e.g. ~10px larger in each direction, and then choose a random crop every other second.

It should be really easy to build a FrameOS app that does this. Add one node at the start of the render loop to increase the image accordingly, and one at the end to crop it back down. I was planning to use a similar technique to display smooth scrolling animations/slideshows, but it can definitely be used to prevent burnin.

justusthane• 3 years ago

Looks cool! I've been wanting to set up an e-ink display with my HomeAssistant, so I'll be checking this out.

In your "Why FrameOS" post, you have a photo that looks like an e-ink display displaying a HomeAssistant dashboard, and under it you say "However the software side of things was rubbish." What exactly were the issues you had before writing FrameOS? It looks like it's working great in the picture at least!

I'm also curious what this actually means:

GPT4 Support: Ask your favourite LLM to write and debug FrameOS apps for you.

H8crilA• 3 years ago

RE ChatGPT: I think it's a clever joke highlighting that it is a product that often needs no further integration (to make blockchain powered self driving social AI clouds, or whatever sells best on private equity markets these days).

bbor2• 3 years ago

Idk I think it’s real… hopefully a sobering moment for people who think of LLMs as the successors to blockchain, OOP, beanie babies, etc. There is real value here and you don’t need think pieces to see it: it’s already delivering it.

I look forward to seeing this project cited as one of the first frameworks/OSs to offer first party developer assistants.

airspresso2• 3 years ago

The author integrated GPT-4 into the "app builder" that FrameOS has for building custom views, so you can ask GPT to make improvements to the underlying code. See https://frameos.net/blog/gpt4-support

My first thought seeing this was that it validates that new apps from now on will be LLM-integrated from the start. Why not, right? LLMs used right are a great UX-improvement for almost any app.

JeffSnazz2• 3 years ago

My first thought seeing this was that it validates that new apps from now on will be LLM-integrated from the start. Why not, right? LLMs used right are a great UX-improvement for almost any app.

Well, any apps that you can actually build yourself. This excludes the vast majority of them.

eddieroger• 3 years ago

This makes me miss the Chumby, which had such untapped potential and was given up on so quickly. I have many uses for a smart, controllable and low-code screen in my house, and barely the time to figure it out myself. Very glad this project came across my page today!

konschubert6• 3 years ago

I make an off-the shelf eink calendar. You can also develop your own content and then serve it to the screen as a .png or .jpeg on a web url.

https://shop.invisible-computers.com

Yujf3• 3 years ago

That privacy policy... at least it is honest and upfront about it

AdamTReineke• 3 years ago

I don't see the screen dimension listed anywhere. Am I overlooking it or is that detail missing?

selcuka2• 3 years ago

Looks great. Small typo on your web site: "iCalendar setup is only possibly through the iPhone app".

peterhajas• 3 years ago

Looks really cool! Can I use it totally locally on a non-internet connected VLAN? Or do I need to use an app / cloud service?

djazzz4• 3 years ago

Looks great! Something I have planned to code for Pimoroni's Inky Frame (7 color) is a calendar firmware that syncs with Google Calendar etc.

vosper• 3 years ago

I have many uses for a smart, controllable and low-code screen in my house, and barely the time to figure it out myself.

I also don’t have time to figure it out, so I’m probably just going to buy a used iPad and put it on the wall. I’ll get a nice bright screen and anything I want to see just needs to be viewable in a web browser.

mariusandra19• 3 years ago

I used to have an iPad (3rd gen) on the wall, then it died. I bought the cheapest new one to replace it, but the old Android tablet I had around had better bluetooth reach, so it went on the wall instead.

Old android tablets are notoriously unpatched against all kinds of things. It's easy to be paranoid that some passing foot traffic will infect it with a bluetooth worm, and you'll be none the wiser.

I'm accepting the risk for now, as I do enjoy configuring Home Assistant on it, and looking at the rainmap, but there are times when when a full tablet is overkill. FrameOS is for those cases.

samstave3• 3 years ago

This would go a long way in Medical Kiosks, Digital Art and Signage.

The digital art thing is big in hospitals (Source: I was the technology designer for SFGH, UCSF, and the technology implementation manager for El Camino Hospital - and director of OPs for Vizio (TV))

Ive evaluated, purchased and installed systems such as this, and wayfinding in hospitals - paying hundreds of thousands for digital artwork to be looped in elevator lobbies, room screens etc - they, like you said - had rubbish software.

For the lobbies in El Camino Hospital, I had to replace some units with Mac Mini's playing an MP$ on loop via VLC because the digital signage system was so crappy.

So, having this with a piZero would be great.

However - wayfinding would also be elegant with this - so long as when you say "interactive" surely it could display a webapp/app via the PI, and "screen-saver" to your digital art loop when idle...

But - seriously, the digital art/signage/wayfinding market needs a fiesty upstart like you to ruffle their muffins - I dont have any PIs currently else, I'd be using this....

I wonder if I could install it on my shitty android TCL tablet from T-Mobile...?

(Also look at controlling Feit Electric bulbs)

EDIT:

I think the most lucrative option for a device/OS like this would be travel signage at BART, any 'regional transit' -- find the dumbest person at that organization and train them to install and configure and display something in a fool-proof manner...

Look at Sacramento (california) regional transit - they have so many screens with zero usable information...

Have you been in a train in APAC? THey have LCD screens with status maps on the upper walls...

Have you been in a train in the US? THey have pee and poo on the upper walls, and zero map status anyplace.

lmz• 3 years ago

Have you been in a train in APAC? THey have LCD screens with status maps on the upper walls... Have you been in a train in the US? THey have pee and poo on the upper walls, and zero map status anyplace.

It sounds like installing screens on US trains would just end up with pee and poo on the screens...

shiroiuma• 3 years ago

Yep, that's not a technology problem, it's a culture problem.

nerdbaggy• 3 years ago

What is your opinion on brighysigns? While the default software is kinda crap the programability of them has been great for us. But we aren’t very far in.

niutech4• 3 years ago

Installing Mac Mini just to play MP3 files? That's a gross waste of money. You can use a dirt-cheap portable MP3 player for $10!

It's crazy how much money is wasted in the US, where other people are begging for food on the streets.

mariusandra19• 3 years ago

I believe the OP here is not playing 1999s music formats on a TV, but referring to MK4 and similar _video_ formats that a portable music player definitely can't play.

jauntywundrkind• 3 years ago

The first picture on the page is the editor, and dammmnnnm wowwww. From it's bullet point:

Diagram Editor: A drag-and-drop interface to combine Nim apps into scenes. Fork and edit existing apps like "OpenAI image", and "Text overlay" to suit your needs. Overwrite all fields with inline code snippets.

This looks really slick. It feels like it must be a huge effort! I wonder what they used to build this. Is the editor written in Nim?

What do you run once you write a scene: a runtime similar-ish to the editor but with a presentation mode, or does it spit out a new Nim program to run the other nim programs? Do apps get once-off invoked or is there persistent communication for apps to keep running? What triggers a frame to start, how does composting happen?

I feel like they've built a very cool generic flow-based programming system here, that happens to be used for smart frames.

dgellow• 3 years ago

Based on screenshots that looks like dearimgui. You can see similar widgets here: https://github.com/TimoSalomaki/awesome-dear-imgui

Edit: I’m wrong, it’s actually React Flow!

https://github.com/FrameOS/frameos/blob/037af53e94a3b47ce5df...

https://reactflow.dev/

mariusandra19• 3 years ago

Hey, thanks for the feedback!

The editor/controller itself is just a React+Kea app using React Flow for the diagramming part, and Flask for the backend.

For now to test that the app you built works, you must deploy it on a frame. I have a fast RPi 5 for testing, and the feedback loop is typically ~30sec. It's about a minute with a Zero W2, and about 2min with a Zero W. This is not ideal, and I'd like to improve that.

Ideally I'd like to use Nim's JS backend to build a "frame" that can be run directly in the browser for testing.

As for the other questions, the FrameOS system runs at boot, and all the apps are compiled in to that one binary and run as the scene dictates. New frames get rendered when a render event is dispatched, which can happen automatically on an interval, or as requested. Frames send logs back to their controller, but can otherwise operate completely independently.

cryptozeus• 3 years ago

Worked on a similar product 10 years ago at Pandigital. They used some custom os from china and the way you send photos to the frame is by giving guid email id. It was doing good in those days until they got ddos on holidays season and lost all the business.

zubairq• 3 years ago

Cool OS! Could it work for shop window displays?

mariusandra19• 3 years ago

Absolutely! Just hook up your display with a HDMI port to a raspberry, and you're good to go.

If you're fine with just displaying an image from an URL, it'll work today. If you want nicer slideshows or rotating images, you'd have to write some code to make it work, but it's all possible today.

A nicer interface tailored specifically for shop windows slideshows is something that could come at a later date.

NoboruWataya• 3 years ago

Looks really interesting. Am I right in thinking it's not actually an OS, but rather software that runs on top of an OS on the Pi?

surajrmal• 3 years ago

Yeah it seems more like a full screen application which can be remotely controlled.

squarefoot2• 3 years ago

It appears so, which would mean it could probably work with only minor changes on other boards.

mariusandra19• 3 years ago

You're technically right (the best kind?), but there's some nuance to the question.

FrameOS is a compiled binary that runs on top of Linux. At present you need to install the "Raspberry Pi OS Lite" (without the desktop environment) on a SD card before you can install FrameOS itself on top of that. So it's "just an application".

However, this is mostly because it's an early project, and this was the fastest way to get going. The next step is to see how much can be removed. Since FrameOS is a statically compiled single binary, it shouldn't need that much more than the Linux kernel? Will FrameOS be a true OS then? Android seems to be considered its own OS despite using the Linux kernel.

Once activated, FrameOS takes over your entire system, has its own `apps/` and `drivers/` folders, and in many ways already behaves like an Operating System. I'm also pondering about getting it running on an ESP32 (will need to add `drivers/wifi/`?). Without any OS below it, will it then be worthy of being called an OS? You all tell me. :shrug:

Thus is it an Operating System? It is to me, but you can choose to disagree.

At my actual job (PostHog), we sometimes call ourselves the "Product OS". This annoys me more, as a suite of webapps definitely isn't an "Operating System" in its traditional sense. I've seen -OS at the end of other things as well that aren't pure Operating Systems. It slightly bothers me, but mostly because I was around when "what OS you're using" was a defining question. Language is evolving, and "operating system" seems to describe more things to different people, than it does to you and me.

pharrington• 3 years ago

You're already aware of the word "kiosk," so why not market it explicitly as a kiosk?

traverseda2• 3 years ago

What is an OS? Genuine question, not being sarcastic here. "OS" is a pretty loosely defined term at the moment. Clearly Android and Ubuntu are very different operating systems, despite having roughly the same kernel.

FrameOs uses Nim, which means it can be ported to run on bare-metal with no kernel. The async dispatch means that Nim also provides some light-weight co-operative multitasking.

I'd say that FrameOs is an OS (or userpsace if you prefer) that currently only runs on top of the linux kernel, but there's no reason it couldn't be ported to run directly on a microcontroller.

ycuser2• 3 years ago

Does this work only with HDMI displays? I have a waveshare e-ink display which connects only via GPIO interface.

alexchamberlain• 3 years ago

The docs seem to indicate it works with both

bitwrangler2• 3 years ago

This listed HDMI displays.

https://frameos.net/#supported-platforms

rhussmann• 3 years ago

Check the "Devices" section: https://frameos.net/devices/ Several Waveshare devices that connect via SPI for example are listed as compatible.

mariusandra19• 3 years ago

I have gotten the new Nim-based FrameOS working with two different Waveshare displays. I have two other ones waiting to be tested. Check this folder [1] for the currently enabled drivers.

One of the next items on my todo list [2] is to add back support for all the different waveshare drivers out there. They all follow a similar pattern, so I should be able to generate "best guess" drivers for most of them.

Which display do you have?

[1] https://github.com/FrameOS/frameos/tree/main/frameos/src/dri... [2] https://github.com/FrameOS/frameos/issues/1

bitwrangler2• 3 years ago

I was imagining something like MotionEYE OS https://github.com/motioneye-project/motioneyeos/wiki

Where I would download the sdram binary image and flash with Balena then boot on the Pi.

Great work anyway! It was a nice opportunity to read up on Nim language too.

mariusandra19• 3 years ago

Very interesting. As replied here [1], I do eventually want to investigate "slimming down" the Linux distro below FrameOS.

However for now, the RPi imager, with its built in "enter your wifi credentials and SSH key" dialog was the best and easiest way to get started.

[1] https://news.ycombinator.com/item?id=38859572

isoprophlex• 3 years ago

Those circular color displays look incredible! I had no idea these things were available, but outlets like adafruit have them in stock. Amazing.

mariusandra19• 3 years ago

I was quite shocked at the amount of tiny and large display options out there. I might even make a YouTube video titled "Home dashboards have gotten cheap and easy" with everything I've learned so far.

Take a $15 Pi Zero W 2, and a screen costing anywhere between $50 to $200. 3d-print a case around them both, and for a third of the price of a new iPad, you'll have a simple and secure single function screen in your house...

gotwalt• 3 years ago

Would be amazing if this could be adapted to replace the Tidbyt OS. Such a promising device with such terrible software.

proee• 3 years ago

Can a Pi go into sleep mode running FrameOS so the device can be battery powered?

mariusandra19• 3 years ago

Theoretically yes, but I have not yet investigated it.

malcolmosh4• 3 years ago

Very cool ! I built a series of scripts to exchange e-ink images through Gmail last year which I baptized DispatchPi (https://github.com/malcolmosh/dispatchPi/blob/main/README.md) and a program like this would definitely have accelerated development. It was a bit fussy to always have to use a terminal to SSH into the Pi, then update the scripts through FileZilla (ftp client) + Vscode.

I've just completed a dashboard project which I'm about to release and I was also thinking of switching my frame drivers to ESP32 to run them on batteries, since they consume far less energy than even a Pi Zero. So +1 for ESP32 compatibility if time allows!

omtinez• 3 years ago

What hardware are you using? I recently finished a project myself and I'd like to do more with e-ink but the hardware "driver" that plugs into the pi costs as much as the e-ink display.

malcolmosh4• 3 years ago

Yes, it can get pricey. I'm using a Raspberry Pi Zero W with a Waveshare 7.5 e-ink panel, which came with its own raspberry Pi driver. I also added a PiSugar2 battery. Waveshare seemed to be the most affordable option out there.

I find the costs to be reasonable, but I just bought a Waveshare e-ink ESP32 controller to try boosting battery life, since I don't need a fully fledged OS. All I'm doing on the frame is pulling an image from an URL or displaying a local image if the connection fails.

brk• 3 years ago

How deep are you in the ESP32 dev? I've found image rendering (the display is LCD, not eInk) to be less straightforward than anticipated.

In my case, massaging images into a specific size and format prior to sending them into the ESP32/Display is easier and more reliable than trying to have the ESP libraries display any arbitrary image.

malcolmosh4• 3 years ago

I haven’t actually started coding yet with the ESP chip. In my case the aim is to pick up a preformatted image from an URL, avoiding all of the finicky formatting you describe. I like the idea of the screen controller as a single function machine just for display, with the backend being a simple GitHub repo connected to Cloud Run via CI/CD.

12th• 3 years ago

This looks really good. I can see it's not listed in your list of devices but I'm wondering if there is anyway to get this running with an old kindle.

bluepuma77• 3 years ago

Would be cool to run this on old kindle ebook hardware.

sydbarrett74• 3 years ago

Reaaallly cool project. These are the innovative OS stories I like to see, not the latest useless marketing department driven 'feature' that some Microsoft PM thought was a good idea.

293984j29384• 3 years ago

I use MagicMirror to do this. It's a great piece of software. I'm a bit shocked nobody has mentioned this yet. https://magicmirror.builders/

snvzz• 3 years ago

Note this is not an OS, but something that runs on top of Linux.

greendude292• 3 years ago

Thanks for posting, this is a fun space to see more software in.

Indirectly related, but I'm in the market for a 32inch+ screen which can be used for a photo frame. With the exception of Samsung's Frame, I haven't been able to find other products which is surprising to me because it seems like such a simple use case. Here's what I've found so far:

- TVs and monitors: downside is keeping these on 24-hours isn't great for energy consumption

- color e-ink: these are of course usually a maximum of 13 or so inches

- Samsung the frame - alas, has good features and a good look, but downsides are expensive(!) and Samsung(!!)

Do you have any recommendations?

mikepurvis• 3 years ago

How does a dedicated frame device save power over a generic TV display? Doesn't it boil down to the same underlying parts?

greendude292• 3 years ago

A TV will likely have multiple inputs like HDMI, USB, Cable, etc and hence presumably more parts, more complex board and higher power needs.

I might be wrong, but that's an assumption.

A 'feature' of the Samsung TV that copes with this is that it has a motion sensor and turns off when no one is around. Would love an open hardware version of this.

niutech4• 3 years ago

Why do we need full-blown RPi to run FrameOS? There are many cheap ESP32-based e-paper displays like LILYGO T5-4.7", so please add support for ESP32.

mariusandra19• 3 years ago

I apologize for the confusion, but you do not need to run FrameOS at all. You can just go about your merry day, and either just not use FrameOS, not post this comment, or, if you prefer, not read any of the comments addressing this question before posting yours.

On a serious note, yes, it's planned. The ESP is a very limited chip. If you want to display a BMP from a HTTP source, or a quick calendar, it's great. Try doing HTTPS, image downscaling or any serious processing though. Forget about things like browser screenshots, which you can't even do on 32bit linux anymore. Forget the 60fps mode over HDMI we can now do.

I'll try to get it working regardless, but it's but not a personal priority today over the other items in the todo list. You're free to lend a helping hand of course.

mariusandra19• 3 years ago

Re-reading, my tone in the first sentence sounds rather hostile. I apologise for that. I have no ill intent, the joke was likely funnier in my head.

FrameOS is also targeting 60+FPS LCD and other screens, which will be hard for ESP to run. Thus it's for the raspberry. I'd love to scale it down though, and deliberately chose Nim so that it would at least be theoretically possible.

I also need to get doom running on it...

Show HN: FrameOS – operating system for single function smart frames

Related Stories