Sane C++ Libraries

https://github.com/Pagghiu/SaneCppLibraries

by cozis • 2 years ago

73 145 2 years ago

Profan3• 2 years ago

It's still funny to me how something titled "Sane C++ libraries" has zero interop with the STL; guaranteeing an even more fragmented C++ ecosystem.

C++ has its place, but something new could not possibly displace it soon enough, even with C it feels like libraries fit together more easily.

chornos• 2 years ago

Its the very reason C++ is still alive. Its unopinionated on how u code and coding enviroment. Plenty of other language are far more restricted in their ecosystem

Profan3• 2 years ago

I just think things could be a little more aligned as I use C++ daily and often end up interacting with many different libraries and each has their own slightly idiosynchratic String/Vector/etc type, which quickly makes life.. interesting

But I see the point in it helping C++'s unusual longevity as well

pagghiu21• 2 years ago

Yes the library is trying to model an alternative C++ world where the standard library tries to be more like the standard libraries of other languages (Python, nodeJS for example) providing actual functionality out of the box rather than just "containers and algorithms".

Profan3• 2 years ago

Huh right I see, I do like the idea of that and I wish that is what the actual C++ stdlib was more like, it would make my life of using C++ a lot more pleasant than it currently is :)

(also sorry my initial comment came off like ragging on your library, it wasn't meant that way, it was more of a commentary on the overall state of the C++ ecosystem, so I appreciate people with a slightly broader view like yours!)

shortrounddev22• 2 years ago

The lack of STL containers or even a fully implemented set of custom containers, to me, makes this kind of a non starter. I use C++, in part, because I don't want to have to implement Data structures and algorithms myself

pagghiu21• 2 years ago

That's a fair observation.

What containers, beside Vector<T> (and Map<K,V> made with Vector) + variants would you like to see the most?

shortrounddev22• 2 years ago

Set, Stack, Queue, and their various implementations (HashSet, PriorityQueue, etc)

smallstepforman2• 2 years ago

My projects use stack and deque often.

sublinear• 2 years ago

https://github.com/nlohmann/json

I used this for JSON last time I wrote any C++ a few years ago and it still seems popular. It seemed sane enough to me.

nly• 2 years ago

Boost.JSON is arguably the better choice these days. It's roughly as usable as nlohmann but faster than RapidJSON

Overview: https://www.boost.org/doc/libs/1_84_0/libs/json/doc/html/jso...

Benchmarks: https://www.boost.org/doc/libs/1_84_0/libs/json/doc/html/jso...

Parsing Options for non-standard JSON: https://www.boost.org/doc/libs/1_84_0/libs/json/doc/html/jso...

gumby• 2 years ago

Looks like the author likes C but would like a few of the features of C++ and wants to have some fun. There’s noting wrong with that, but it ain’t for me.

The stuff he considers complex is mostly complex because it handles a lot of corner cases, or else it has back compatibility constraints he will be dealing with as well.

Good luck to him though. Doing stuff for fun is the best!

pagghiu21• 2 years ago

I honestly like well written C a lot :)

And yes, complex stuff sometimes tries to handle "everyone's use case" but if you can limit yourself to 95% of use cases, your code suddenly become a lot simpler. The backward compatibility consideration holds true as well.

For example, I have been creating an Async Library (plus a few other things like the FileSystemWatcher etc.) that cover a good portion of what is done in libuv. Of course libuv code handles A TON more of edge cases and has a lot of compatibility constraints, but with a lot less code I can provide enough functionality to satisfy a lot of use cases. Not all use cases, but a lot of use cases.

Thanks for the good luck! I am definitively doing it just for fun :)

colejohnson66• 2 years ago

As someone who doesn't write C++, why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Both C++ and C# take a "bags included" approach to the standard library, but no one is writing their own `List<T>` for C#.[a] Yet, everyone seems to have their own opinion about `std::vector<T>`.

[a]: Obviously, people still write their own collections/containers in C#, but they tend to only do so for very specific/performance-sensitive circumstances.

blovescoffee• 2 years ago

Because people tend to write c++ for performance reasons and the perf profiles of std::vector are not always sufficient

jeffbee6• 2 years ago

The only thing you can really quibble about with std::vector is whether your library has made an optimal choice of growth strategy, which you can often hack around by reserving. Aside from that, access via `operator[]` and growth via `emplace_back` will compile down to optimal code that is going to be close to impossible to beat. After the compiler gets done with it, it looks the same as if you had hand-coded it with arrays in a C-style, but without the plethora of bugs that often results from that approach.

CyberDildonics3• 2 years ago

What specifically is wrong with vector? There have been a lot of hash maps done with flat memory to minimize allocations and pointer hopping over the STL but vector doesn't have those problems.

sfpotter4• 2 years ago

There are so many reasons. I'm sure others will provide their own favorites.

One that I find particularly annoying lately: if you work on a project that makes heavy use of the STL (or, really, any heavily templated library written in a similar style, with a focus on "ergonomics" :-( ), you'll quickly find that backtraces for debug builds can easily be 50-100 levels deep with most stack frames just consisting of incomprehensible layers of abstraction which get optimized away. So, debug builds are totally useless, and build times are typically long. Contrast that with something written in "C style" or simply making far fewer use of C++ features, written in a more straightline style, with fewer levels of abstraction: builds will be fast, stack traces will be small and can easily map directly onto the concepts relevant to the program or library, and debug builds are useful once more. Night and day difference.

jeffbee6• 2 years ago

In decades of professional C++ experience I have never noted depth of backtrace as a usability issue. Give us an example of how a 100-deep backtrace arises as a consequence of the STL.

SleepyMyroslav2• 2 years ago

It does not have to be night and day. In gamedev I saw over years that giving up debug builds for a set of your own abstractions has benefits. You do not have to use STL or exceptions so you can have a balance in your build times. Having abstractions in the code has a lot of value. People overestimate how hard is to do deoptimize on checkout or on demand which solves most of debugging needs.

jsheard3• 2 years ago

Take your pick: undercooked functionality, lack of behavior/performance consistency between different toolchains, bloated implementations causing long compile times, inscrutable template vomit error messages and poor performance in debug builds (a dealbreaker for realtime apps like games), and design mistakes which can't easily be fixed, such as std::vector<bool> being a footgun or the way that the standard map types are defined making them impossible to implement efficiently.

neonsunset2• 2 years ago

Because unlike CoreLib, the standard library in C++ tends to have a lot of insanity backed in. My knowledge on it is somewhat limited but looking into its APIs for working with strings (in large inherited from C) was so traumatizing that I only ever wonder why there are still self-respecting projects that don't reimplement parts of STL.

Example: want to measure the length of a UTF-8 code point in a string and did not synchronize the call? Well, too bad, now you might have corrupted the global mutable state it relies on! (the fact that you can make such a trivial piece of code have two!!! points of thread-safety failure one is C locale and another is transcoder still refuses to leave my mind)

neonsunset2• 2 years ago

Can't believe someone actually went out of their way to downvote this haha

lenkite• 2 years ago

Because C#, at several points in its history, sensibly took the decision to explicitly break backward compatibility to satisfy legitimate requirements, adapting and evolving to the tenets of modern software practices. (Generics and new collections)

Whereas due to C++'s "never break compatibility" decision, the standard library has progressively decayed over time. It has become a bloated, rotting dinosaur where even the slowest of interpreted languages can comfortably beat several of its aspects. (Ex: std::regex is pathetic and pitiful, vector<bool> triggers laughter, substandard maps, etc). Considering that C++ thumps its chest and loudly proclaims its superb performance, this has now become a sad joke.

In the natural world, a species that cannot adapt to new circumstances and never discards undesirable characteristics simply perishes.

The C++ Standard Committee has firmly and unequivocally decided that the C++ language should mirror the same approach and limp down the road, carrying the full-weight of its sins for all its journey, until it falls into oblivion.

pjmlp9• 2 years ago

Note that as documented in the history of F# HOPL paper, .NET already had a generics prototype before 1.0 release, Microsoft decided to go ahead without generics to avoid delaying it further until they were stable.

Also breaking everything a couple of times is why most corporations are nowadays stuck in a Python 2 / 3 parallel world in .NET Framework / .NET Core , or in Xamarin.Forms / MAUI, UWP / WinUI,...

jandrewrogers3• 2 years ago

The containers in the STL have issues due to the limitations of C++ when they were designed, catering to the lowest common denominator user, and simplifying things for implementors. It is not uncommon to find cases where the STL is a poor fit or annoying to use due to its implementation and design details. In the case of modern C++, you also have the issue that the STL containers aren't designed to be used in an advanced metaprogramming context, because they pre-date that being an intentional part of C++. Some of this is unfixable because backward compatibility. Also, C++ is commonly used in contexts where performance is critical, so the optimality of the STL for purpose matters more.

Some of the common issues: static allocation or lack thereof, requiring default constructible classes, initializing memory you are going to overwrite anyway, inability to be used in some metaprogramming contexts, suboptimal allocation behavior, etc. The STL is opinionated but unfortunately that opinion dates to a time when C++ was primarily used for ordinary app development, not high-performance code.

Like many, I maintain my own C++ "standard library" that is much better designed for the kinds of software I tend to work on (database kernels and data infrastructure, mostly).

MichaelMoser123• 2 years ago

Doesn't it contradicts the 'bloat free' requirement, if every library needs to have its own version of std::list std::vector, std::unordered_map etc ?

I mean: i remember the days, when each and every project had its own string class.

MathMonkeyMan• 2 years ago

The implementations of vector, list, and unordered_map in every standard library I've used have been sufficient for every C++ library I've worked on.

There are special cases, and there are engineering politics, but it's all basically fine.

pagghiu21• 2 years ago

The bloat free is more referring to executable size, build complexity, compile time and in general to hidden complexity.

Every library having its own version of common data structure is unfortunately something that C++ programmers can't really seem to agree on :)

summerlight3• 2 years ago

why does almost everyone seem to insist on ignoring the STL and writing everything themselves?

Not really. Unless you have a very specific set of performance requirements, STL containers are usually more than enough. And we have third party libraries such as Abseil/Boost to cover the major gaps in the rest. I do see some legitimate cases to write own container libraries. But for many cases people don't really measure their primary workload before writing such libraries, instead they just write it because they can write (and it's fun).

comex• 2 years ago

Many of the principles here align with my tastes, such as focusing on fast compile times and supporting allocation failure. On the other hand:

Unplanned Features:
SharedPtr
UniquePtr
In Principles there is a rule that discourages allocations of large number of tiny objects and also creating systems with unclear or shared memory ownership. For this reason this library is missing Smart Pointers.

I don’t like that at all. I take the common view that all heap objects should at least be allocated via smart pointers. Doing so is safer and easier and usually zero-overhead. After allocation, it may be necessary to pass those objects via raw pointers/references, but smart pointers should be used where appropriate.

So while I agree that it’s undesirable to allocate “large numbers of tiny objects”, I would want smart pointers as long as there’s any dynamic allocation at all.

markisus3• 2 years ago

I usually like to place all dynamically heap objects of type T into an std::vector<T>, if possible. There is sometimes an obvious point where the entire batch should be discarded and a new batch should be built. At this point, you can call vector.clear() which avoids deallocation/allocation cost for the new batch, as long as it is not bigger than the old batch. This is a sort of quick and dirty arena allocation.

This style is also more cache friendly if you are going to be looping through the elements.

neeeeees2• 2 years ago

Interesting - is there a type safe way to do this? vector<variant<>>? and/or a custom “vector allocator” to hide the details?

dkersten• 2 years ago

I do this too, when I can either use a handle/index instead of a pointer, or when I can guarantee that the vectors size is constant (so that pointers/iterators are stable). I’ve also written my own vector that stores its elements in pages so that if its capacity needs to increase, the elements don’t need relocation.

I only really use C++ for a toy game engine right now and in that codebase I don’t use any smart pointers and most objects/functions get passed references to their object dependencies. I classify objects into groups where each groups ownership is very clear. So its owner is responsible for maintaining the memory and any raw pointers can always be assumed to be borrowed references. I use handles then the underlying objects lifetime might differ from whatever is holding a handle to it. Short lived objects are kept trivial and allocated from stack/bump allocators or pools and reset at well defined times (every frame, end of level, etc)

I’m much happier this way than when I used smart pointers or when I had less well defined memory ownership.

LoganDark• 2 years ago

Yeah, I prefer RAII over manual memory management. Sorry, strawman.

slaymaker19072• 2 years ago

Shared pointers are most definitely not zero overhead. The trouble is that they use atomic operations which can cause tons of problems for highly concurrent systems. You can use non-thread safe smart pointers, but at some point you have to ask whether that is really lower risk than just not using shared pointers at all (Rust gets around this with type checking).

o11c2• 2 years ago

Adding a bunch of lines like `#pragma GCC poison new` to the last-included header of every source file is very useful. It doesn't fully stop manual memory management from sneaking into headers (since it absolutely will break system headers, though maybe if modules work that would be avoided).

For the rare case of porting software with unclear ownership, I use a `dumb_ptr` template with allocation and deallocation methods. Since this is header-only it naturally avoids the poisoning.

In particular, the `vector` method mentioned elsewhere is completely broken since objects move and thus you can't keep weak/borrowed references to them. If you use indices you give up on all ownership and are probably using global variables, ick. Please just write a proper pool allocator if that's what you want (possibly using generational references to implement weak).

markisus3• 2 years ago

Everything is a tradeoff. Even things like goto, and global variables will occasionally be the right choice.

Regarding the std::vector method, you may have a very loosely coupled system where a bunch of T1's enter into a pipeline and come out as T2's. For this use case, std::vector<T1> and std::vector<T2> are great. On the other hand, if you need to create an object and hand it off to someone else with no knowledge of how long they will need to hold onto it, then std::shared_ptr could be a good option.

In the in-between you have entity component systems that do the type of index tracking you mention so that identities are decoupled from memory location, allowing objects to move. I didn't understand your point about global variables and why they are necessary to implement this type of system. I also didn't understand how this gives up on all ownership. The owner would be the system that maintains the index to memory location mapping.

pagghiu21• 2 years ago

For me personally SharedPtr is very rarely needed as it encourages building difficult to untangle ownership hierarchies. I did use a lot of Shared Ptr in the past when creating a node.js like library in C++ but breaking the ref cycles everywhere was needed has always been a pain. That's why I am currently against its use, unless there is a very special case.

Regarding UniquePtr<T> I used to have one but I later on decided to remove it.

https://github.com/Pagghiu/SaneCppLibraries/commit/9149e28

However, that being said the library is lean enough so that you can still use it with smart pointers provided by any other library (including the standard one) if that's your preference.

wokwokwok• 2 years ago

you can still use it with smart pointers provided by any other library

Is the point of having a kitchen-sink library like this not that you dont have to reach for a 3rdparty library for things that you need 'all the time'?

Certainly, not everyone needs it.

...but, not everyone needs threads either. Not everyone needs an http server; and yet, if you have an application framework that provides them, when you do need them, it saves you reaching for yet-another-dependency.

Was that not the point from the beginning?

unique_ptr is a fundamental primitive for many, as you see from some other frameworks (1), and implementation is not always either a) trivial, or b) as simple as 'just use std::unique_ptr'.

This does seem like a very opinionated decision with reasonably unclear justification; perfectly fair, you're certainly not beholden to anyone to implement features just because they want you to, but I think it's difficult to argue there's not concrete use for something like this, in a way that aligns with the project principals.

I would go so far as to argue that:

Do not allocate many tiny objects with their own lifetime (and *probably unclear or shared ownership*)

Is hostile to not having a unique pointer.

[1] - eg. https://github.com/EpicGames/UnrealEngine/blob/release/Engin..., https://github.com/electronicarts/EASTL/blob/master/include/...

arccy• 2 years ago

how does it compare to https://abseil.io/

jsheard3• 2 years ago

Abseil is more of a complement to the STL rather than a complete replacement like the OP is, for example the only containers that Abseil provides are maps and sets since the STL ones are particularly bad, and the rest of the STL containers are good enough (for Google at least). Whether that's the right approach depends on what your specific qualms with the STL are.

j16sdiz• 2 years ago

The first library I checked was "Reflection". This is where I can see if their principle holds up

pagghiu21• 2 years ago

Of course I would have been doing the same :)

The documentation here states:

https://pagghiu.github.io/SaneCppLibraries/library_reflectio...

Note Reflection uses more complex C++ constructs compared to other libraries in this repository. To limit the issue, effort has been spent trying not to use obscure C++ meta-programming techniques. The library uses only template partial specialization and constexpr.

dzogchen• 2 years ago

'Sane' C++ is apparently still using macros all over the place. :) Not really to be taken seriously.

pagghiu21• 2 years ago

The majority of the macros are SC_PLATFORM_XXXX to inject platform specific code here and there. There are macros in the Reflection library but you have the option not to use them.

What macros are bothering you the most?

bsdpufferfish• 2 years ago

Just use C and guarantee your sanity. The whole reason to adopt C++ is to support huge projects and libraries all of which are going to use the “bad” stuff.

pagghiu21• 2 years ago

That's actually an excellent advice! :D

I love well written C libraries, like the sokol or stb libraries.

nemetroid• 2 years ago

Looked at the first library in the list, "Algorithms". The first item in that library is bubbleSort(). I'll pass.

pagghiu21• 2 years ago

Yes, the Algorithms library is just a placeholder, as specified in the docs.

https://pagghiu.github.io/SaneCppLibraries/library_algorithm...

Hopefully it will get expanded with more useful algorithms, it has not been a priority in the first releases cycle.

slaymaker19072• 2 years ago

Why would you implement your own atomics? It seems very similar to std::atomic, except you need to do a bunch of platform specific hacks to get it to work. The only possible reason I could think of why you would do this is if you aren't using at least C++11. If you aren't using C++11, then you probably shouldn't be using threads in the first place.

qwery2• 2 years ago

I can't speak for the author, but using `std::atomic` would presumably break one of the project's listed principles:

No C++ Standard Library / Exceptions / RTTI

reactordev6• 2 years ago

Which is like making cookies without butter. It can be done, but why would you want to? The standard library exists for this reason. To provide these abstractions. And I laugh at the idea that C++ could ever be exception free.

pagghiu21• 2 years ago

Exactly!

This is all in an effort to bring down compile times, avoiding including anything from the standard because you can never know if including <atomic> is bringing 10K lines of code in your header.

I will probably provide an optional USE_STANDARD_HEADERS flag someday to allow including a few standard things, including atomics, to avoid doing things wrong on compilerS that are not tested enough (as I clearly can't test every compiler).

jbandela1• 2 years ago

Also, std::atomic is one of those special types.

It is deeply integrated into the C++11 memory model. The compiler has to know about the semantics of the type to make sure it doesn’t reorder operations around it.

csjh• 2 years ago

What is the point of `algorithms::bubbleSort`?

greysphere• 2 years ago

Last I checked bubble sort was the fastest sort for small counts of small objects. That was 10 years ago though so thing might have changed potentially something that might take better advantage of vector ops.

It's also often the correct choice for something like a collision partition for a physics sim where elements are most likely in the same sort position each frame.

usefulcat• 2 years ago

Around half as much code is generated for bubble sort compared to std::sort, e.g.:

https://godbolt.org/z/KbaTeno3j

countWSS• 2 years ago

Actually this sounds useful as alternative to C++ stdlib: i've often compiled C++ code via GCC for simple stuff where C++ stdlib isn't included by default and all 'nice' C++ things are not linked, giving less overhead per file, but forced you to rely on ancient C functions. This would be a middle ground solution between unsafe C and bloated stdc++.

pagghiu21• 2 years ago

I don't like to market it as an alternative to C++ stdlib, also because it doesn't cover all the things done by the C++ stdlib (in particular regarding Containers and Algorithms, as noted in other threads on this discussion).

I like to market it as an "alternative world" where the C++ stdlib is more a platform abstraction library focused on carrying practical tasks like networking, Async I/O, HTTP etc.

It's also definitively placing itself in the middle between unsafe C and bloated C++.

pjmlp9• 2 years ago

No C++ Standard Library / Exceptions / RTTI

This alone already rules them out as sane on my book.

rockwotj3• 2 years ago

No stdlib or no exceptions/RAII. Because everyone at Google is on the no exceptions/RAII train (according to the Google Style Guide), and I happen to like exceptionless C++

pjmlp9• 2 years ago

Just because it is Google, doesn't mean their style guides have any quality value.

Any company that worships that style guide is one I am happily never going to work on.

lpribis• 2 years ago

You mean RTTI instead of RAII, right? Google definitely uses RAII.

pagghiu21• 2 years ago

Author here! Feel free to ask me any question, here or on discord/X/Mastodon I just saw this posted here, wow :)

deterministic• 2 years ago

POCO is another solid collection of C++ libraries with similar objectives. It looks as if POCO is more mature.

spacechild1• 2 years ago

It looks as if POCO is more mature.

POCO started 20 years ago, so it has a slight advantage :)

hermitcrab2• 2 years ago

Interesting. But I think the vast majority of this functionality is already available in the Qt framework, which I use for pretty much all my projects.

pagghiu21• 2 years ago

Of course Sane C++ Libraries targets a much much smaller functionality subset than Qt (that is a good library in many ways) and of course it has orders of magnitude less complexity. You can use Sane C++ Libraries adding a single file to your project for example. Also, Qt used to have an LGPL + Commercial licensing scheme (not sure how this has recently evolved), while this project just MIT.

hermitcrab2• 2 years ago

There probably isn't a compelling use case for Qt-philes like me. But I wish your project the best of luck!

(the community version of Qt is LGPL, I have a small business licence)

eterevsky• 2 years ago

I don't think describing the build in C++ (https://pagghiu.github.io/SaneCppLibraries/library_build.htm...) can exactly be called a "sane" idea.

Declarative build definitions are generally much easier to work with and scale than using an imperative language.

pagghiu21• 2 years ago

I could bring multiple examples, but just to make one CMake, what I think today is the most popular way of describing builds in c++, describes builds in an imperative language.

https://en.wikipedia.org/wiki/CMake#CMakeLists.txt

Most "declarative" build systems are not actually what they "declare" to be. I've seen too many DSLs introducing half backed imperative concepts here and there to do _if_ and _for_ constructs or function calls, redoing the same as imperative languages but poorly.

jokoon• 2 years ago

Still no 3D renderer

TinyEngine is good, but doesn't build on windows yet

pagghiu21• 2 years ago

Well that would be slightly increasing the project scope

integricho• 2 years ago

I've come to despise how tightly integrated the C++ standard library is now with the c++ language itself, it is clearly evolving in a direction to make the two inseparable, but it is also making it less desireable. One cannot even compile a c++ executable with the /NODEFAULTLIB flag and not break core language features, e.g. static global object construction, dynamic_cast, ... For the purists that would like to separate the standard library from the language, and make tiny executables not depending on the runtime library, C is the only choice now it seems.

trws• 2 years ago

This is what “freestanding” is for, FWIW. There are a variety of papers and implementations around this goal, which is generally to have a set of features of c++ the language and the standard library that can work entirely unhosted.

I realize people think of this as a difference between c and c++, but if you go to compile C code with a thread_local variable and throw -nostdlib you’re going to have a bad day. Same for atomics, sometimes complex numbers, floating point exceptions, even receiving arguments and other core language features require some crt code. Removing the runtime library guts the implementation regardless of your language. The question is, does your language provide ways to deal with this? Rust has core, c has alternate ad-hoc embedded libc implementations and a history of bootstrapping implementations long enough to have them be well understood, c++ has/will have free-standing.

rurban• 2 years ago

I'm unsure about the asserts instead of errors

Sane C++ Libraries

Related Stories