lbutler

I'm building epanet-js[0], a local-first web application for simulating water network.

The goal is to replace clunky, expensive enterprise desktop tools (which often charge $10k+ for a wrapped version of the open-source US EPA engine). We want our version to be frictionless: login optional, runs entirely in the browser, and saves files locally to your device.

We are using the Functional Source License (FSL) to keep it sustainable while remaining open.

[0] https://epanetjs.com/

The GIS system is generally the base for a hydraulic model. You use that data to build a connected graph structure that the hydraulic engine, EPANET, uses to run calculations to figure out the pressure at the nodes and also flow rates in pipes.

There is also a water quality component where you can calculate the age of water in the system or chemical, such as chlorine, or other by-products you may or may not want in the system.

The US EPA site goes into technical details on what the engine can do [0], but the vast majority of modeling is done as part of a water master plan for a water utility.

A water utility will build a hydraulic model of their network and calculate and model the growth of their city over a 30-year period. The model will highlight areas of concern, generally low pressure, and the water utility can propose new infrastructure like larger pipes, tanks, or pumps, and will schedule future capital works to keep service levels acceptable.

They generally repeat this process every 3–5 years, rebuilding the model and rewriting their master plans. Here is an example of a master plan by the City of Kyle [1].

Generally, a water utility is proposing tens of millions of capital works, if not more. So traditionally, the high price tag has just been accepted. But obviously, this doesn't scale down to smaller utilities, and normally consultants will do the work on their behalf, including holding the right software license.

[0] https://www.epa.gov/water-research/epanet

[1] https://www.cityofkyle.com/media/69766

Currently the two largest vendors of hydraulic modelling software are Autodesk and Bentley. Both have taken the EPANET engine and created private forks in the 90s/2000s and never contributed back.

The commercial tools have made it easier for engineers at consultancies and utilities to build hydraulic models by integrating GIS and providing support for scenarios to compare different states of the model or future developments of a city.

Though as Tom points out, this comes at a huge price.

The US EPA does offer a simple GUI which can be used for smaller systems but without a connection to GIS, its usage has been limited.

These commercial versions have become enterprise monsters, they are very complex and expensive.

We wanted to create the right balance between what the US EPA already gives away for free and what the big vendors offer. We believe that releasing the software as FSL which transitions to MIT gives us the right head start and for the advanced features we're charging about 10% of what Autodesk and Bentley do - and for those that think that's too much, they of course can download and host their own private version too.

For those that are still curious, here are some extra links and context.

https://app.epanetjs.com/ – Try the app, it's local first and registration optional

https://github.com/epanet-js/epanet-js – Here is all the source code

https://github.com/epanet-js/epanet-js-toolkit – See how we converted the C engine to WASM

https://epanetjs.com/ – Read a landing page to see what we're doing and why, also our pricing

https://www.autodesk.com/products/infowater-pro/overview – Autodesk's product $10k/yr/user

https://en.virtuosity.com/openflows-water – Bentley's product $16k/yr/user

One of the two authors of epanet-js here - you can check out the source code for the app here:

https://github.com/epanet-js/epanet-js

We created this from a fork of Placemark[0] to give water utilities an alternative choice to the big players, Autodesk and Bentley.

The majority of commercial modeling software uses the U.S. EPA's engine and adds a nicer interface. They all fork the engine to add enhancements but don't contribute back.

This is our attempt to create an open-source alternative for water engineers and contribute back to the core engine at the same time.

You can check out the source code on GitHub[1].

[0] https://github.com/placemark/placemark

[1] https://github.com/epanet-js/epanet-js

I’m building a local-first web app, and SQLite works well for my case since a single project can be contained in one database file, just like users are used to with existing desktop applications.

What I’d really like is an easy way to sync the SQLite database state to a cloud service. Most existing options expect you to query against a remotely hosted database and charge per read/write.

Since the database will have around 100,000 rows and you're typically working with all the data at once, streaming parts of it doesn’t make sense for my use case.

The closest I’ve found is Turso, which has offline writes in private beta, and SQLite Cloud, which lists local-first and offline sync as "coming soon."

The simplest approach might be letting users push to S3 storage with versioning. Ideally, it would also support point-in-time restores, tracking incremental updates alongside full snapshots.

Even better, I’d manage minimal server-side infrastructure and just pull the SQLite database from a service that handles syncing and management.

At the start up I work at [0], we use an open source library I developed to run hydraulic models of water networks in JavaScript [1].

A hydraulic model may be between 1-10MB and the simulation results can end up being 100+MB of time series data.

Other vendors with proprietary engines have to scale up servers to run their simulation engineers and will store and serve up results from a database.

Having everything done locally means we only have to store a static file and offload the simulation to the client.

Because we've architected it this way our hosting costs are low and users generally have faster access to results (assuming they're running a moderately decent machine)

[0] https://qatium.com/

[1] https://github.com/modelcreate/epanet-js

For those that haven't seen them, the two Technology Connection videos on dishwashers are must watch

1 - https://www.youtube.com/watch?v=_rBO8neWw04

2 - https://www.youtube.com/watch?v=Ll6-eGDpimU

I've found decent success sharing short product features videos, always less than 15 seconds, on LinkedIn.

Making the videos run much faster than normal, at least 1.5x and up to 3x in parts means you can provide a lot more density of information.

It does require the first pass to be visually interesting, and if you don't do something to stand out in the first three seconds you risk being scrolled past.

I also use subtitles to fill in the gaps and assume it will auto replay and if they make it through one replay, that further viewings will be easier to understand what is going on.

I have a relatively small following on LinkedIn ~2.5k but can get between 200-400 likes.

I then use these short clips on my GitHub pages for my work as well.

Some examples below:

https://www.linkedin.com/posts/lukepbutler_water-epanet-qati...

https://www.linkedin.com/posts/lukepbutler_water-epanet-inno...

I would also recommend the Egghead instructor videos, while much longer, they still try to push the idea of short videos to get the point across

I'm a civil engineer and I wrote an open source library that compiled a C library to javascript for my own personal projects - epanet-js [1]

A water utility in Spain spun off a start up called Qatium [2] and they used my library as the engine of their simulations and asked me to join.

[1] https://github.com/modelcreate/epanet-js

[2] https://qatium.com/

I created a handful of application around water engineering/modelling [1], plus an open source library to run the simulations in javascript [2].

A water utility in Spain spun off a start up to create a similar web based water modelling application and they used my open source library.

They approached me and I joined them and have been able to maintain the open source library as part of my role.

[1] https://github.com/modelcreate/epanet-js#featured-apps

[2] https://github.com/modelcreate/epanet-js

I work as a water engineer, specializing in building hydraulic models so water utilities can simulate their network.

A big part of that is calibrating them which can be time consuming, you look through hundreds of options. I create a few web based apps to help grind through these tasks but ultimately they were for my own use as a consultant to close projects quickly.

I did pull out the engine as its own open source library for other to use, and that ended up helping me get my current role where I can now maintain it and be paid at the same time.

https://github.com/modelcreate/epanet-js

Many thanks to tsholmes for finding a solution and sharing it with me.

He has requested the full amount be donated to Nine Lives Foundation, proof of donation can be seen here: https://ninelivesfoundation.networkforgood.com/

I'll be posting the script and the format documentation shortly

I've had confirmation from a couple of sources now that this is indeed a SQLite file, or at least a derivative of it, with some key information missing.

If you email me how you know it is a SQLite file and what is missing, I can also send you the data that is contained within the files if that will be helpful to you.

SEEKING FREELANCER | $2,000USD bounty - Work complete, and bounty donation at users request

I'm working on an open-source library in a niche civil engineering field. I have a proprietary file format that I would like to decode and support.

I am almost certain the file is an SQLite database with the header changed, though my attempts to reconstruct the header and find any other changes have failed.

If you think you are up to the challenge, I've linked to some example files below, the first person who can share the procedure to update the file that it will open in a native SQLite reader I will pay a $2,000USD bounty or donate to a charity (your choice).

Email in profile, I'll post below if an answer has been accepted.

edit: This has been decoded and the bounty will be donated! Waiting for confirmation from the person who provided the script to decode the file and I will share results and proof of donation

EPANET (https://github.com/OpenWaterAnalytics/EPANET & https://www.epa.gov/water-research/epanet )

Chances are the water network that's connect to your home was modelled using this FOSS by the US EPA.

Unfortunately, it was probably done using a commercial tool by Bentley or Innovyze/Autodesk but the underlying modeling engine was from the EPA.

I think the vast majority of this thread agree with you that what we saw in the article was incredibly dangerous to the employee.

In my example these sites are unmanned 99% of the time but if there was an operator on site and someone back in the head office went on the loud speaker to tell off some trespassers I can't imagine OH&S or the on-site operator would be the slightest bit impressed.

When I worked at a water utility we would occasionally have people breaking into our remote sites like water towers either to vandalise them or climb the assets for the view.

When you just want people off your site ASAP we found providing the operators with remote loudspeakers and allowing them to say "Hey we can see you, we've called the police" was a very effective way to convince anyone that was trespassing to leave.

Though we did lose a few cameras when kids decided to throw rocks at the CCTV system before they left...

A webapp to help me calibrate hydraulic models of water networks.

The original concept was in excel and VBA.

I ended up pulling out the engine as a separate open source library which I've used on a few other projects.

https://github.com/modelcreate/epanet-js#model-calibrate

I'm a Civil Engineer who spends roughly half my time programming.

While I'm not the best engineer or programmer out there, blending both skills has been a massive advantage for my career.

My boring advice is to just start solving your own problems with code in your existing field.

In the video comments he includes a link to a tutorial but essentially he is dumping all the frames and then running a script to content aware scale down the frames 50% and then merges it all back together.

You can also apply this to each frame in a video for a rather interesting effect.

My brother has a YouTube channel full of content-aware scaling videos:

https://youtu.be/a8k3b-QNbhs

Simulating water distribution networks in JavaScript.

I maintain a JS conversion of the EPA's simulation engine EPANET. https://epanetjs.com/ https://github.com/modelcreate/epanet-js

I've built a few open source apps and few other little projects to help automate my workflow.

There are only a handful of providers of modelling software, most are commercial and one recently sold to Autodesk for $1B.

Not sure I'll convince the industry to change but I'm enjoying tinkering around and making my own small difference.

I finished the website for my open source library to simulate water distribution networks, epanet-js.

Can't imagine there are too many civil engineers here specializing in water modelling, it's a pretty niche field!

https://epanetjs.com/

Thanks! If you're interested I wrote an article listing all 7 FOSS apps I created this year on LinkedIn [1], I also have a site for my main library [2].

I'll eventually convert part of one of these into a B2B app to keep what I do sustainable but I want to keep as much free and open source.

[1] https://www.linkedin.com/pulse/seven-water-modelling-apps-on...

[2] https://epanetjs.com

This particular software is for water utilities to model and simulate their water and waste water networks.

It's mostly used so utilites can forecast growth in their areas for the next 25+ years and see the impact on their networks and feed into their capital work projects.

A decently sized utility may spend up to $200M/yr on capital works so $40k isn't even a line item!

There is completion in the market but consultants are forced to use what their clients pick and most utilites aren't that price sensitive.

There are also open source alternatives by the EPA[1][2], and most commercial operators are just wrappers around this public domain software.

I'm trying to create FOSS to help view and run these models.

[1] https://en.m.wikipedia.org/wiki/EPANET

[2] https://en.m.wikipedia.org/wiki/Storm_Water_Management_Model

The version is stored in byte 96 of the header of the sqlite file[1].

I haven't looked but there will be some sqlite command to query it and I'm sure some viewer tools will display it as well.

[1] https://www.sqlite.org/fileformat.html

I can understand why they do it, it's enterprise software so the more open you make it, then the less licences they can sell.

They do have a module you can purchase to run API calls and access their files/software but as you probably guessed that's another $40k license!

Most of my apps I build use this API, but for me to provide to other companies they need them to also buy the API extension.

I'd love to cut out the middle man and I'll do it eventually when I reverse engineer the header!

I have an application that uses sqlite as their file format but they've cleared the header so you can't open it directly.

I'd love to be able to make secondary applications like you've described but being enterprise software they don't want to make it too easy.

They obviously want to keep people locked in with their $40k per seat application!

I guess the first step is figuring out the page size and other bits the other meta data you set in the header [1].

I know I just have to sit down and understand the format better and I will eventually figure it out...

[1] https://www.sqlite.org/fileformat.html

If you publish content for a professional niche, then LinkedIn may be your best choice to reach your audience.

I'm a hydraulic modelling engineer, and people in my field don't hang out on the internet together like software developers.

So when I write articles or develop some FOSS, I'll go to LinkedIn because I can reach thousands of other engineers doing the same type of work.

I don't believe I could get the same reach or engagement on any other platform.