Node-RED: Low-code programming for event-driven applications

by omarfarooq • 5 years ago

283 89 5 years ago

julius• 5 years ago

I remember trying to use it as a web-server few years ago.

You can see an example graph to get a user from the DB by id here: https://github.com/julius/node-red-contrib-sqlstring#example

64cde• 5 years ago

Because of Node-RED i learned a LOT about APIs, protocols and stuff in a playful way. Especially:

-REST

-GET, POST, PUT

-JSON

-Javascript

-MQTT

And all that helped me a lot in my career, just because i wanted to automate a few things. And now Node-RED is just irreplaceable in my private life.

DanWritesCode• 5 years ago

My friend uses Node-RED to automate and run a lot of devices/things in his van (which he lives in). I'm not sure if it's better to not have a programming background, because it's one of the more difficult environments I've helped someone with and the debugging when something wasn't working was really poor, it all felt very un-intuitive.

kingo552• 5 years ago

Even when you know how to script up automations, I still find Node Red invaluable for visualising and rapidly prototyping complex logic before you write a single line of code.

Plus, it is the perfect accompaniment to Home Assistant.

salzig• 5 years ago

Complex: yeah, especially if you find a twitch streamer that uses node-red to automate his stream. (for the curious: JvPeek [ger])

a254613e• 5 years ago

I use Node-RED in combination with home assistant, for automations.

It's sometimes overly complicated to follow the flows, or do something relatively simple, but otherwise it's a nice middle ground between built in YAML based automations, and something like AppDaemon which requires writing python scripts.

kingo552• 5 years ago

I find myself prototyping automations in Node Red before "productionising" the code in YAML. It's great for visualising flows but nothing beats automations running directly within HA. One fewer service to crash.

Inversechi• 5 years ago

They changed the engine a lot so you can script the automations from the UI now.

See: https://www.home-assistant.io/docs/automation/editor/

ashtonkem• 5 years ago

The new UI editor is a better experience than yaml, but still has limitations that make node-red valuable for more complex automations. In particular juggling timers, state, and Boolean conditions with HA automations can be kind of awkward as they require you to coordinate across multiple automations, but very easy in node-red.

All in all I use both. I use the native ones for simpler automations (exterior lights on at dusk, blinds up at sunrise, etc.j and node red for the complex stuff (bathroom fan on for 5 minutes unless if the door is closed or the humidity is above 65%, turn off two minutes after the door is opened and the humidity is below 60%j.

quinn_jh• 5 years ago

Yeah node-red really rocks. Maybe I am biased as a particularly visually oriented person but i really enjoy the ease of prototyping with nodes and flows. I cannot help but sense that they may become more commonplace- there are applications for embedded sensing and data pipelines where it helps to think less in packets and more in streams or currents... also great for mocking up a synthesizer which I did with Paul Stoffregen's super rad audio tool for teensy/ARM : - ) https://www.pjrc.com/teensy/gui/

nsomaru• 5 years ago

How does this compare to n8n? I read somewhere that node red was more focused on IoT applications, is that correct?

sneak• 5 years ago

Node-RED is free software, n8n is not.

rcarmo7• 5 years ago

It originated in that space, but the ecosystem has taken it in very different directions. It has lower-level abstractions than n8n (which focuses on automating third-party SaaS), but can arguably do the same.

I've used it for an insane amount of things, starting with home automation and telemetry but also fairly sophisticated uses, including:

- Sniffing for multicast streams on a LAN and exposing IPTV set top boxes as HomeKit devices

- Automating image conversion and video transcoding

- Building an RSS aggregation and transformation pipeline that does translations, scraping, etc.

- Building a simple GUI for managing backup snapshots and batch file copies

- Monitoring Azure deployments (and exposing the data as simple dashboards).

- Doing App store price checks and sending Pushover notifications when an interesting app goes on sale (backed by SQLite).

- An embarrassing number of Slack and Telegram bots for various user groups/events/etc.

As a general rule of thumb, Node-RED can do pretty much any kind of automation you can get a REST API for, and quite often someone out there has already built a module for it.

I think of n8n more along the lines of a graphical workflow/BPM tool for SaaS, but with much less flexibility/ability to go "low level".

beardicus• 5 years ago

thanks for listing all of these examples... i really like node-RED, and wrote up a tutorial on it a few years back, but have never actually reached for it to solve a real problem. you've given me some ideas!

slightwinder2• 5 years ago

- Building an RSS aggregation and transformation pipeline that does translations, scraping, etc.

About how many feeds are we talking here? One per hour? Ten? Some hundred? I've thought at some point doing similar, but was not sure whether node-red or similar tools would perform well enough with this kinds of jobs.

clearcarbon• 5 years ago

I use nodered daily in my home automations, has been absolutely rock solid for my use case.

adontz• 5 years ago

Cannot find if it supports any version control? Does it?

hwillis2• 5 years ago

It does (everything is saved as json), but it's a bit of a hassle. Scripts are saved to a single line, so you really need a plugin that breaks them up. It's really pretty silly that isn't the default.

The other huge PITA is that touching a node in the UI causes the json to update even if you didn't do anything, and changes will often cause nodes in the json to reorder. That makes git commits very cluttered.

Merging is also very hit-or-miss and duplicate nodes are common. You can mostly fix this with another plugin that saves flows to individual files instead of one big file (default). As long as you split your flows and subflows up enough, team members will usually not step on each others toes.

I used this at work. Not a fan. If you only touched the system once every 4 months I can see the usefulness- It's easier to see the overall structure than wading through code, particularly if you're a visual person. It makes it relatively easy to pick something up that you have mostly forgotten.

It is a huge PITA if you are trying to build API features quickly with a team. Especially if there's nobody to tell you to use the plugins.

samlents• 5 years ago

I've run into these issues as well but haven't come across those packages. What plugins are you all using?

rcarmo7• 5 years ago

It does. The projects feature has a visual diff editor that makes it very easy to see what is going on, and uses git as a back-end.

pierregillesl• 5 years ago

Also, you can use Node-RED with Gladys Assistant (open-source home automation software) to have a clean dashboard on top of Node-RED:

https://gladysassistant.com/blog/integrate-node-red-with-gla...

RedShift1• 5 years ago

I love Node-RED. Great tool to make otherwise invisible workflows visible!

noduerme4• 5 years ago

I have just been going through a HELL of a time building a web-based app ("PWA" in progressive lingo) that needs to auto-discover certain devices on the user's LAN. This is a rebuild of an existing AIR-based app that's been in use in a corporate setting for 10 years and needs to be replaced because "death of flash". The AIR app I wrote back then handled discovering the local devices fantastically well. The problem now? Mixed-origin blockades. I can't get into all those devices and routers in each location to set up local DNS servers and maintain certificates. So the connection from the ultimate app to the local node has to be unencrypted. But this means the only answer to a web-based discovery app is to have it be served over HTTP as well.

Interesting thing google missed when trying to prevent people from doing this exact thing with PWAs: If you have a valid manifest and service worker, Chrome won't run the service worker if it's not served over SSL, but it WILL include the app name and icons and let you save it to the home screen on Android; it will even cache a good deal of what you intended.

For auto-discovery, I used a fuckton of hidden iframes loading asynchronously and waiting for any window.postMessage that matches what I want from a local device.

Seriously, Google has made my life fucking miserable as a solo dev. They don't want anyone to be able to write an IoT app without going through them.

tonyedgecombe• 5 years ago

Actually I'm kind of glad it's difficult to do those sort of things in Chrome.

noduerme4• 5 years ago

There are fantastic reasons why it should not be allowed on random local devices, unless the local device is configured with some sort of authentication... but the problem now is that total ban on accessing local devices from a secure web app is a serious choke point for anyone trying to develop something to control appliances outside the walled garden.

Fascinating discussion here with a Plex dev expressing similar frustrations with the general direction of these context barriers... which in their case affects millions of users and threatens to pull down their entire platform. Their solution currently is bizarre; they actually have to hijack DNS resolution and issue SSL certs on the fly for each of their users' servers to allow their web app to connect to a local node. Not possible in my smaller corporate sphere.

https://github.com/WICG/private-network-access/issues/23

_red2• 5 years ago

We're a industrial-control company whose traditional UI has been standard win32 applications - we've slowing converting our stack over to a web interface but as our UI needs to talk to lots of local devices, we've hit the problems you described.

Our solution has been to basically mandate that the user install either a small embedded device (ie. RPI or equiv) or manually launch a container. These essentially just serve as a local websocket / api router so Chrome connects to some local "device/container" and then that connects to all the local devices.

Although this is a logistical hurdle, its actually simplified other aspects of development since now this device/container has a bit of intelligence and can act as real middleware to simplify what the UI needs to contemplate. Basically MVC-style development using hardware bits as Controller.

noduerme4• 5 years ago

That's cool. I guess our problem could be described as allowing non-technical users to control the in-shop devices from an app on their phone (both android and iOS) without knowing the local IP at all. Like, how do we determine the shifting dynamic IP of our in local routing system from an app served from the cloud ...and the answer unfortunately is we can't. We have to serve the app locally in each facility.

handrous2• 5 years ago

React Native and (maybe) Electron are nice if you need access to real networking but want to write webtech, and can afford to impose somewhat on your end user. Browsers (very much rightly, IMO) lack a lot of handy stuff for local device discovery, including, notably, all of UDP.

knolleary6• 5 years ago

Node-RED co-creator and project lead here. Happy to answer any questions you have.

pabbasian• 5 years ago

IOT guy here, I like node-red but never dare to run it on edge device to perform ops on critical infrastructure. what is the state of the project? prototype only or business ready?

knolleary6• 5 years ago

(Node-RED project lead here) It is not at all limited to prototyping. There are an ever-growing number of companies incorporating Node-RED in their own products and services - you can see a list of some of them on the project homepage.

They span a wide range of industries and technology areas. Quite a few are industrial controller companies who include Node-RED on their commercially available edge devices.

freen• 5 years ago

Would love a swagger api importer!

knolleary6• 5 years ago

(Node-RED project lead here) There are a few nodes for dealing with swagger in Node-RED - either generating Swagger doc for your HTTP end points, or on the client side to make it easy to interact with a Swagger documented API.

For example: https://flows.nodered.org/node/openapi-red

If you're looking for something that can take a Swagger/OpenAPI doc and generate a stubbed-out set of flows - that would be a neat idea. Sounds like a cool project to create as a pluing.

ish• 5 years ago

How do you feel Node-RED stacks up against products like Boomi, MuleSoft, SnapLogic, etc., and is competing with those iPaaS vendors something you want to do?

knolleary6• 5 years ago

(Node-RED project lead here) Node-RED is an open-source project and part of the OpenJS Foundation. We'd certainly aspire Node-RED to be comparable at a technical level with other offerings.

But as an OSS project, we aren't looking to create a hosted SaaS offering - the core project isn't trying to compete commercially with those vendors.

But there are companies looking to create commercial services around Node-RED that certainly want to be competitive. (My own company, FlowForge Inc, is one of them)

peter_retief• 5 years ago

Node-RED is a great prototyping tool. Thank you, I have been a fan since the beginning.

slightwinder2• 5 years ago

What are the flaws of Node-RED? The areas that you plan (or hope) to give many, many necessary love somewhere in the future?

What is something people use the software for, where you would say they should not do this?

Node-RED seems so far optimized for IoT and online services. Is there any ability (or plans for them) to use it locally for desktop-automation, similiar to AppleScript & Automator on MacOS, shell & batch-scripts, Selenium for websites, Tasker on android, etc.?

How would you see the state of the project generally and feature-wise ? Is it still growing for the next decade? Or has it reached it's peak and will move to settle down at its optimum somewhere in the next years? Should it reinvent itself every some years and stay fresh and adapting?

knolleary6• 5 years ago

(Node-RED project lead here) Some great questions, and not necessarily short answers.

What are the flaws of Node-RED? The areas that you plan (or hope) to give many, many necessary love somewhere in the future?

It works really well as a single user system. One of the areas I really want to improve is the multi-user/collaborative features available in the editor.

You can certainly have >1 users editing the same flows in parallel, but you do end up having to merge back in each other's changes. That user experience isn't ideal.

I'm also keenly aware the documentation and getting started experience needs attention.

What is something people use the software for, where you would say they should not do this?

I wouldn't tell someone not to use the project if they had found something that worked for them. As with any solution, you need to make sure it satisfies your needs and requirements.

I would say you need to acknowledge the limits of the system - partly born of the single-threaded nature of the underlying Node.js runtime.

Is there any ability (or plans for them) to use it locally for desktop-automation, similiar to AppleScript & Automator on MacOS, shell & batch-scripts, Selenium for websites, Tasker on android, etc.?

It isn't something we're thinking about in the core of the project. But the project is designed so that this could all be done via 3rd party nodes created and shared by the community to integrate with those tools. In other words, this sort of integration could be done by anyone - it doesn't have to be the core project that does it.

How would you see the state of the project generally and feature-wise ?

There's always more to be done.

Should it reinvent itself every some years and stay fresh and adapting

There's a risk for all software projects that continually reinvent themselves that they sacrifice their existing community in search of something else.

A core principle we have is stability. That doesn't mean stagnation, but it means evolving in a considered way that brings the community with us and doesn't leave users stranded because they depend on something that has since been 'reinvented'.

zeroflow• 5 years ago

I can on reply from the perspective of a multi-year user:

It is primarily functional programming. This means, you have an input, that is processed through blocks (functions) and does some things.

This is GREAT for things similar to IFTTT (If this then that) applications. E.g. you receive a datagramm via MQTT, and then format it, add some info, and upload it into influxdb.

You can also easily add a small dashboard with some buttons and labels for easy home automation.

What needs some working around with context store are side effects, or persistance. I have two examples that can be worked around, but are rather hacky:

1) Reading a list from a homepage, and only sending updates for new items. This is the prime example for context - you save a list of seen items in the context of the function node, so the next invocation can filter out the existing ones

2) Using a switch in the dashboard for selecting between automatic and manual mode. E.g. I want to have my rack fans controlled by the temperature, but with an override to set it manually. This needs a bit more hacking with context where you have two flows into one node, and you have to set the internal state of the context depending on the override button.

unixhero• 5 years ago

Just a massive fan, that's all. You've changed the world.

chucky_z• 5 years ago

I’ve been looking at building an auto remediation system for server alerts, is Node-RED a decent system for this? It will probably start with just triggering some kind of ssh command to run a script.

natenatenate3• 5 years ago

Thank you for this wonderful piece of software.

my use of it for hobbyist projects helped me get my foot in the door for my current employment in automation and controls. I have multiple instances of it running to track various things, run my chicken coop door, send alerts, aggregate data... It's uses are endless and I enjoy the puzzle of using the new updates to the software to make my old flows better.

verifex• 5 years ago

I'm a big Node-Red fan, been using it for years to automate and do lots of neat things. One problem I kept coming back to was trying to create dynamic urls for websockets. I'm curious if that might be on a future roadmap for Node-Red?

IOT_Apprentice• 5 years ago

I want to thank you and the team for developing and improving this project. I used it early on to prototype a system based on Raspberry Pi's and AWS. It is very easy to build on and the library of integrations is great.

mrandish• 5 years ago

Thank you for your service.

I'm not an NR user yet but I've implemented my basic Home Assistant set up and plan to begin using NR extensively as I evolve my home automation beyond "Hello World".

RNCTX• 5 years ago

I love it, have used it for lots of little things over the years.

Suggestion: when I last used it, loops was a third party add on, you should look at getting that into the core, maybe hire the dev of the add on?

akavel2• 5 years ago

<enso-fanboi mode="on">

If you're interested in something more generic, check out if https://enso.org could be worth a look! Thanks to GraalVM, you might be able to import JS packages too, among others (Java, Python, & more...); OTOH with the caveat that the project is still in ways in-dev...

goodmachine2• 5 years ago

I've been following Enso since it was Luna. Can you be a bit more explicit about 'in-dev'? Not to be used yet in production? Thanks.

akavel2• 5 years ago

Hm, honestly, this was kinda precautiously overly-defensive statement from my side. I mean, I'm also following Luna/Enso with huge interest, but it's kinda on-and-off for me; it seemed to me to still have some rough edges in the GUI last time I checked, but also it was quite long ago already - somewhere right around their 2.0 public announcement, in January 2021. No slightest idea at what state the GUI is now, or the libraries; haven't had time to approach it again since. But I expect to do that quite soon.

robk• 5 years ago

Obligatory comment that Huginn is also useful for similar applications albeit less home automation than web automation

bluehatbrit• 5 years ago

Huge fan of huginn, I've started running n8n and huginn on the same box recently. Some of the huginn modules are a little more work to use with common SaaS offerings. I'm hoping together they can cover basically everything I want, although I don't do any home automation.

sokoloff• 5 years ago

Is anyone using the node-RED ecosystem to deliver end-user customization of their not-home-automation product/service?

I’m looking for ways to offer practical/accessible low-code (or not considered scary code) composition for our internal users who are running light industry factory production lines, but open to learning about commercial applications in any field to draw inspiration from.

jdcaron• 5 years ago

We use it as a flow designer, we export the graph data it produces to handle real procution code. It's open source, highly customizable & well made.

Jedd• 5 years ago

We're (extremely early days into) looking to use NodeRed to provide customised configurations for a fleet of telegraf agents.

Telegraf has the ability to phone home and ask for its telegraf.conf as it starts up, but the server-side of that equation is frustratingly left as an exercise for the reader.

There's not many ways to skin that particular cat, which is odd and unsettling in its own way.

Graffur• 5 years ago

I'd like to hear from some people that don't like Node-RED.

knolleary6• 5 years ago

(Node-RED project lead here) So would we! The only way we improve is by listening to the community and taking on the feedback we get.

deepspace2• 5 years ago

I am personally not a huge fan because my brain is not wired to do visual programming.

I have Node-RED installed and running alongside HomeAssistant, but I never use it because automations that are easy to do in NR are just as simple to create directly in HA. Automations that are complex enough to need NR are easier (for me) to code and debug in Python.

I think NR is great for non-programmers and maybe for quick prototyping, so I am not disparaging it in any way. The functionality and implementation is excellent; it's just not for everyone.

finder83• 5 years ago

I didn't dislike node red, but it didn't really fit my use case well. I wanted to use it as a REST service to write APIs for clients to use to generate reports from our various databases. Basically a low-code data pipeline from cassandra, mongo, sql, etc. It worked...but things like switching git branches while keeping things running, updating the service, aggregating results, etc, were difficult. More difficult than just writing the code.

I wrote one then never used it again opting for node instances inside of our Kubernetes cluster instead.

darkteflon2• 5 years ago

I run a Homebridge / Homekit ecosystem at home but have always been curious about leveraging Node-RED to handle more complex automations.

Does anyone have any recommendations and/or recommended reading on putting these two together?

rcarmo7• 5 years ago

I have a setup with homebridge, zigbee2mqtt, z2m (for better glue between both) and Node-RED, and a few write-ups here:

https://taoofmac.com/space/blog/2019/01/13/1900

Essentially you can use Node-RED to either act as another HomeKit bridge or to handle MQTT events from homebridge itself (I chose the latter).

You can easily "fake" and inject new accessories into homebridge via the homebridge-mqtt plugin (which allows you to dynamically add accessories and "see" them in MQTT).

darkteflon2• 5 years ago

Excellent, thank you very much for taking the time - much appreciated.

SEJeff2• 5 years ago

This software is great on the esp8266 and esp32 series of microcontrollers for those who like to tinker.

rcarmo7• 5 years ago

Erm. You can't run Node-RED on those, the NodeJS runtime alone would eat three times the flash storage. You can _talk_ to them, though, from a more powerful system.

SEJeff2• 5 years ago

Sorry I meant in concert with. I run node red on a raspberry pi and then send it stuff from the esps. My brain was thinking the right thing and my fingers didn't type it. You're absolutely correct.

I have them send MQTT to Node Red.

Havoc• 5 years ago

Recently discovered that you can build simple api style end points with it.

Eg built one that tries resolving an ad domain to check if ad blocking is functioning correctly and publishes that as a end point that is suitable for an uptime dashboard

d21d3q• 5 years ago

I use node-red for Escape Room game high level automation. Gathering data from all the puzzles (over CANbus and mqtt), displaying game state for game masters, fallback control, audio+light sequences, etc

eBombzor• 5 years ago

What do people automate at home? I genuinely have never thought to have the need to buy a home assistant but apparently lots of people in this thread do

deepspace2• 5 years ago

Lights are the most obvious use case. Done right, you may almost never have to touch a light switch again.

In my house, the central lights turn themselves on when it becomes too dark (except after bedtime), and off when it becomes too bright. Lights turn on automatically when someone enters a room (when it is dark) and turn off when they leave.

When the last person leaves the house, all lights go out, except when I am on vacation, in which case the lights turn themselves on and off in a pattern that looks like I am at home.

After lights come extractor fans (keeping the humidity constant) and heat/air conditioning.

It is all about maintaining a comfortable home environment without having to manually turn things on and off all the time.

recursive• 5 years ago

I wish I could figure out how to de-automate my bathroom light. It tends to wrongly notice that I've left the room leaving my in total darkness. My workaround is to leave the light on in the adjacent closet, which, for some reason has no such amenity. I doubt a lifetime of saved button pushes would ever make up for the inconvenience of troubleshooting this not-quite-smart-enough light system. But other people have different utility functions.

rcarmo7• 5 years ago

I automate heating and lighting, essentially. Haven't had the time to hack into the AC (although I already know how to do it).

thrashh• 5 years ago

Doing something when we forget to close the garage door lol

IshKebab• 5 years ago

I am working on having my phone alarm slowly open my curtains in the morning.

Unfortunately automating normal curtain opening is pretty tricky mechanically. It was easy to get as far as "Ok Google, open the curtains" -> stepper motor moves. But connecting the stepper motor to the curtains is tricky. I think that's why smart blinds are way more common.

theshrike79• 5 years ago

Lights and notifications about events in the house are the biggest ones.

Lights come up automatically based on time of day and motion in rooms.

Our family channel in Telegram gets notifications if the house is too hot/cold dry/humid, maybe a humidifier gets turned on automatically.

supermatt• 5 years ago

Had a quick look at this recently, but was finding it hard to grok where the value gets "returned". Any pointers from the knowledgeable?

yewenjie• 5 years ago

I really like Pipedream - https://pipedream.com

rcarmo7• 5 years ago

Since I've been asked a couple of times in the threads for Node-RED examples, here's my home automation write up from a couple of years ago:

https://taoofmac.com/space/blog/2019/01/13/1900

These days I've changed the setup a bit and now take advantage of Node-RED's debugger plugin and Projects/git features, but the rest still applies.

dang• 5 years ago

Some past threads. Others?

Node-RED 2.0 - https://news.ycombinator.com/item?id=27892226 - July 2021 (62 comments)

How-to normalize home volume levels with Node-RED - https://news.ycombinator.com/item?id=23349902 - May 2020 (28 comments)

Node-Red – Flow-Based Programming for the Internet of Things - https://news.ycombinator.com/item?id=18859019 - Jan 2019 (77 comments)

Replacing a Pool and Spa Controller with Raspberry Pi, Arduino, Node Red - https://news.ycombinator.com/item?id=18142335 - Oct 2018 (3 comments)

Node-RED: A visual tool for wiring the Internet of Things - https://news.ycombinator.com/item?id=13495156 - Jan 2017 (6 comments)

Node-Red, a tool for wiring together hardware devices, APIs and online services - https://news.ycombinator.com/item?id=8419984 - Oct 2014 (8 comments)

Node-RED - https://news.ycombinator.com/item?id=8310536 - Sept 2014 (1 comment)

Node-red, a visual tool for wiring the Internet of Things - https://news.ycombinator.com/item?id=8217664 - Aug 2014 (3 comments)

Programming the Missing Links in the Internet of Things - https://news.ycombinator.com/item?id=6459564 - Sept 2013 (1 comment)

Node-RED: Low-code programming for event-driven applications

Related Stories