A little fail-safe filesystem designed for microcontrollers (2017)

by opless • 7 years ago

189 30 7 years ago

roland35• 7 years ago

I am curious - does anyone have some feedback from using the mbed ecosystem? I have generally shied away from using too much vendor middleware but it does look like there are a lot of useful libraries.

LeifCarrotson• 7 years ago

I've uzed the original mbed NXP LPC1768 several years ago, and didn't do too much with the vendor ecosystem, but it was very easy to just use the regular ARM-GCC compiler and extract the functionality I needed from their libraries into my own projects.

hak8or• 7 years ago

I would also love to hear replies to this. I used mbed in the past and have been fairly dissatisfied with it for two reasons:

It uses c++ for the sake of using c++ instead of leveraging the extreme bonuses c++ provides over using c for embedded. For example, it tends to use dynamic memory allocation heavily, doesn't make use of unique/shared pointer, doesn't use templates properly (if at all) in places it would be ideal (hence increasing code size and slowing things down).

Last I checked, it doesn't support cmake at all, and instead used an extremely complicated and conveluted build system/process.

Both of those combined made me run away and never look back. Another side thing as of late is it seems to be officially supported by arm, which means it will never run on other cores like risc-v. I am not interested in getting stuck with one platform when riscv (in my opinion) has incredible potential for microcontrollers and is seemingly right around the corner.

notacoward4• 7 years ago

Your criticism seems a bit idiosyncratic, to put it kindly. So it doesn't use the flavor of C++ you prefer? Sure, but there's a lot of room for argument about whether shared_ptr or template-instantiation bloat would be good choices for an IoT-level embedded system. Similarly, the fact that it doesn't use cmake seems like more a matter of preference than of actual suitability. As for being supported by ARM, well hey, at least it's open source. It could theoretically be forked for RISC-V. Lacking any mention of alternatives, that criticism also seems rather hollow. Beggars can't be choosers, y'know.

BTW, you maybe should have mentioned that you work on a competing project.

kevin_thibedeau2• 7 years ago

There is a difference between mbedOS and the support libraries which can all be used independently of their monolithic system. Modern C++ is not supported by some commercial compilers so you get lowest common denominator in that case but that is irrelevant to the core libs all implemented in C.

ross-squires• 7 years ago

I'm a huge fan of mbed OS. The build system adapts to any environment, and you can "eject" if you want to tweak a specific build style. I personally use VS Code with the "mbed compile" workflow.

The official libraries are well-documented and easy to use. I've had to fight a couple vendor-specific oddities when adapting third-party components from their online community, but everything else seem to work as expected.

tom_• 7 years ago

I worked with mbedTLS on a desktop/mobile project a couple of years ago. I'd previously used OpenSSL, and I wasn't very impressed, but mbedTLS seemed OK: easy to get it building with cmake, easy to get it building on Windows/OS X/iOS/Linux, no actual I/O in the library.

I remember finding integration with libuv a mite annoying, as mbedTLS is rather pull-minded, and I didn't find the documentation for the I/O callbacks super-clear. If your I/O callbacks are making blocking calls of a pipe or socket, though, I'm sure it's much simpler.

Would use again.

(Would not use OpenSSL again.)

Gibbon12• 7 years ago

(Would not use OpenSSL again.)

Reminds me of someone using this gif to describe working with OpenSSL.

https://imgur.com/gallery/ikBZF82

kevin_thibedeau2• 7 years ago

mbedTLS is mostly nice but it does suffer from some poor coding practices like large buffers allocated on the stack.

I would warn anyone to stay away from Cypress/Broadcom's WICED OS which has a modified version of mbedTLS with foolish changes to the API that makes porting code extremely challenging.

cozzyd• 7 years ago

I have a tiny bit of experience with it. I found it hard to keep track of which libraries worked with each other or with my particular microcontroller (particularly anything involving networking), but it was probably just my own ignorance.

Gibbon12• 7 years ago

My experience with it was all the packages for the platform I wanted to use were broken.

catherd• 7 years ago

mbed is great for prototyping. It's basically the Arduino system you would have gotten if Arduino was developed by people who mostly already know what they are doing instead of learning as they go.

Lots of libraries for various sensors and other applications, so if you do early stage development on hardware you can quickly throw prototypes together and not need to deal with sorting through a lot of mediocre Arduino stuff.

YMMV on keeping it for production use. Some parts of it should be fine for some applications. Other things not. I've run into issues with things like threads and timers not always working right for every library, but really that's par for the course on a system of libraries that diverse, and supporting so many different chips.

diydsp2• 7 years ago

yes. i found a bug in the gcc export along with a dozen others. mbed devs hemmed and hawed about it for 6 months until they convinced themselves it's not a bug. I dropped mbed after that, switched that project over to Arduino.

notacoward4• 7 years ago

I didn't read all of the design, but it looks like the author had an unusually clear understanding of the tradeoffs involved and how they relate to the specific target environment. There are some clever solutions there, all explained quite well. Definitely worth a look.

SlowRobotAhead• 7 years ago

I think you can be sure this is not his first attempt at a file system.

Documentation does look really good. I’ll hang in to this.

howard9412• 7 years ago

Nice, perhaps a more robust fatfs replacement?

If you've used this (or are the author) how is the "storage on disk ... always kept in a valid state" feature implemented? Does it write two copies in case a write's interrupted so one can always roll back to the earlier valid copy?

notacoward4• 7 years ago

https://github.com/ARMmbed/littlefs/blob/master/DESIGN.md answers those questions in admirable detail.

howard9412• 7 years ago

Thank you. Makes me wish everything else in the world was so thoughtfully documented.

woodrowbarlow• 7 years ago

i am very impressed that, although it's designed for the mbed ecosystem, this code is obviously written so that it can be used in any embedded project (especially those that use a single, statically-compiled binary for the entire firmware) with virtually no changes. i could have this working on my project's bespoke hardware in a couple hours.

* zero dependencies, no assumptions about libc, doesn't use timers or heap, fully synchronous, and it's passive when you're not interacting with it.

* variables, constants, and functions are well-prefixed (without being too verbose) to avoid collisions.

* BSD license is friendly for single-binary firmwares.

* the Makefile doesn't assume anything, it just outputs a .a file which you can link however you want.

* the Makefile is simple and doesn't require cmake or autotools. it's easy to point it at your cross-compile toolchain.

it has other hallmarks of a well-planned and well-maintained project too:

* the readme shows exactly why i might want this (or might not), and how to use it. the separate design doc shows exactly how it works.

* functionality is cleanly separated by file.

* consistent style (could specify which style guide they're using, though).

* the tests serve their usual purpose but also serve as examples. they're well encapsulated and the code was obviously written to be tested.

these authors have absolutely nailed everything i strive for at my embedded firmware development job.

scoutt• 7 years ago

I had the same reaction. Really impressive and well documented. I will surely do some tests.

variables, constants, and functions are well-prefixed (without being too verbose) to avoid collisions.

Unfortunately it has the same name (Little File System, LFS) and the same function prototypes (lfs_*) of a filesystem for NOR/NAND based systems I did like 10 years ago and still in use, so it will be at least confusing.

As a matter of fact, I guess every embedded technology company using NAND/NOR/whatever memories have their own version of a LFS.

loeg• 7 years ago

You're referring to LFS as in LogFS?

pkaye• 7 years ago

One problem I see is when they write the pairs of metadata blocks for redundancy, in many FTL implementations they might fall on the same physical page of flash so there is not much redundancy. When FTL write metadata for their own needs, they are congnizant of the flash layout and make sure to write metadata to separate die/block/page.

TickleSteve2• 7 years ago

Seems well thought through, but the concept of putting a general-purpose file system on an embedded microcontroller is a common mistake.

These are single-purpose systems, you don't need to use filenames and have POSIX semantics, you typically just need to put your data into a circular buffer on FLASH. The circular buffer (with checksummed elements) will take care of bad-block management and wear levelling for you.

I've seen this far too often, filesystems are not ideal for this class of system.

diydsp2• 7 years ago

i have the exact opposite opinion. such circular buffers only suit limited file system activities, such as logs. if you want editable config files of varying lengths, you need a real fs. i'll concede that features like directory structures and timestamps can be omitted though.

TickleSteve2• 7 years ago

No, you can do log-structured BLOBs which essentially given you version control in a simple manner with circular buffers.

As for variable-length configs, yes, but you should always be allocating the worst-case (i.e. defined maximum), hence no need for variable length.

A little fail-safe filesystem designed for microcontrollers (2017)

Related Stories