EASTL: An Alternative C++ Standard Library

by optimalsolver • 5 years ago

144 79 5 years ago

Is this really necessary? How many implementations of the standard library do we need? I get the feeling that engineers love reinventing the wheel so much that they forget to ask themselves what's wrong with the current wheel. My last company had its own implementation of the stl and it was a piece of crap, productivity would've gone way up if we just used the normal implementation and boost threading. Maybe I'm just projecting, but I get the feeling this company wasn't the only one that has made that mistake.

w_t_payne• 5 years ago

Every C++ shop that I've worked in has done the same thing.

cle• 5 years ago

Their docs clearly explain the difference and why this implementation exists, and reasons to use (and not use) EASTL: https://eastl.docsforge.com/master/faq/#info2-what-uses-are-...

JohnFen• 5 years ago

The STL is problematic and not a great solution for a lot of things, so replacing it with something else is very common practice.

agent327• 5 years ago

Could you show how it is problematic? And why replacing it with this one is better?

otabdeveloper4• 5 years ago

It was probably made in the stone age sometime in the 1990's.

maxlybbert2• 5 years ago

EA and Bloomberg happened to make similar extensions to the standard library. Then, when the C++0x revision started, they both submitted proposals explaining their extensions including why they felt the extensions were necessary ( http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n227... , http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n446... ). The approach eventually made it into the standard ( https://en.cppreference.com/w/cpp/memory/memory_resource ).

typon• 5 years ago

I suggest watching this talk:

https://www.youtube.com/watch?v=vElZc6zSIXM

Or pretty much any other by Chandler to understand why the STL sucks and why you should write better datastructures for your own use cases.

pjmlp5• 5 years ago

All nice and good if one is a compiler expert, except most people end up writing datastructures that perform worse than STL.

ryani• 5 years ago

Full disclosure: I used to work at EA and while I was there I contributed a tiny amount of code to EASTL.

EASTL provided a bunch of value that the standard STL would not. The biggest benefit was a unified implementation across all platforms. Standard library STLs all had their own idiosyncrasies and code that worked on one platform might not compile, or worse, have a bug on another.

At the time, EASTL was equal or higher quality than standard implementations. Performance was better, code quality was better, and it broke from the standard in some key ways that were important for performance, and it had some key upgrades that allowed usage patterns and data structures that the standard STL simply didn't allow:

Vectors supported "trivial relocation" before the existence of move constructors. While move constructors have ameliorated the problem, I argue that feature is still missing from c++. Please support P1144! http://open-std.org/JTC1/SC22/WG21/docs/papers/2020/p1144r5....

Intrusive containers (in particular, intrusive linked lists in eastl::intrusive_list) embed the container overhead into objects themselves. This allows non-movable objects to be stored in these lists without requiring an extra pointer dereference on access. It also lets you convert an object reference into an iterator over the list that contains it. There are tons of uses for this. It also allows polymorphic lists (e.g. intrusive_list<BaseClass> that actually holds instances of various subclasses, again without an extra pointer dereference)

If you want more information, EASTL's lead programmer Paul Pedriana (rip) goes into detail in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n227...

sho_hn• 5 years ago

Looks interesting, but the README is thin on espousing the benefits. Questions that come to mind:

- What stands out about it?

- Benchmarks?

- What platforms are supported?

- How is it differentiated today vs. 15 years ago? Qt wound up with its own 'standard library'-alike types mainly because the STL (and its implementations across platforms) were not really up to snuff when it started out. I suspect this may be similar.

- Does anyone have direct experience to share? :-)

Edit: Found some info at https://eastl.docsforge.com/master/faq/#info2-what-uses-are-... and benchmarks at https://eastl.docsforge.com/master/benchmarks/

Especially interesting for me as Linux dev: "[...] EASTL is significantly more efficient than Dinkumware STL, and Microsoft Windows STL [...] EASTL is roughly equal in efficiency to STLPort and GCC 3.x+ STL, though EASTL has some optimizations that these do not."

zrail• 5 years ago

It's been a long time but from what I remember the selling point for EASTL is the ability to specify your own allocator instance for every container. EA writes portable code across lots of different architectures, many of which share a lot more with embedded systems than with PCs. They have constrained memory or special dedicated memory areas. Also, iirc EA uses tagging allocators extensively letting them trace against memory budgets down to the team level.

Edit: there's an extensive FAQ in the docs folder: https://github.com/electronicarts/EASTL/blob/master/doc/FAQ....

jeffbee• 5 years ago

The FAQ seems to answer the question of why EASTL is better than some extremely old and bad standard libraries, not why it is better than modern ones.

fnord123• 5 years ago

Specifying your own allocator is like a main feature of bde from Bloomberg:

https://github.com/bloomberg/bde

maxlybbert2• 5 years ago

That is accurate. That general model eventually made it into the standard, because of EA and Bloomberg ( https://en.cppreference.com/w/cpp/memory/memory_resource ).

mseidl• 5 years ago

Don't forget about micro transactions and loot boxes.

npalli• 5 years ago

They benchmark against Windows STL from Visual C++ 7.1 that came out in 2003!. Not sure how it compares to more modern versions much less something like abseil or Folly.

jbluepolarbear• 5 years ago

I used EASTL on a few projects and it was 10x faster in debug mode. VS had a lot of checks in debug mode and it was almost impossible to play games in debug mode. I’m haven’t used it since 2012.

Const-me• 5 years ago

Does anyone have direct experience to share?

Used it a few times for some medium-complexity stuff, here’s an open-source example: https://github.com/Const-me/vis_avs_dx/tree/master/avs_dx/Dx...

Works OK, and indeed faster than VC++ (I was using VS2017 at that time), especially in debug builds.

One interesting feature missing from the standard library is segmented_vector container. https://github.com/electronicarts/EASTL/blob/master/include/...

obviouslynotme• 5 years ago

Main Benefits:

- customizable allocators and alignment

- designed with turning off exceptions in mind

- performance improvements(especially map and list)

- portable between different C++ vendors

jsnell• 5 years ago

Previous discussion (2016): https://news.ycombinator.com/item?id=11094436

darzu• 5 years ago

For Azure Sphere OS’s user-land components we opted to use EASTL b/c of its small footprint. I don’t recall any major issues arising in the two years I worked there.

pjmlp5• 5 years ago

What userland, when the only thing available is a C SDK?

Subsentient2• 5 years ago

An innovative new STL with unlockable features, such as:

std::vector: $5

std::string: $15

std::map: $30

std::unordered_map: $100

sigzero• 5 years ago

That's just funny.

teeray• 5 years ago

Those are per-month, I assume. Subscriptions are all the rage in software these days.

microtherion• 5 years ago

And occasional loot crates containing optimizations?

marcodiego• 5 years ago

Actually some historic IBM computers were like that. You leased them and could pay more to enable some instructions on the processor.

mike_hock• 5 years ago

std::cout << std::vector<char>{'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '\0'}.data() << std::endl;

I'm too cheap to pay triple the price for a little bit of convenience.

Subsentient2• 5 years ago

static const char StringLiteral[] = "hello world";

std::vector<char> Derp;

Derp.resize(sizeof StringLiteral);

memcpy(Derp.data(), StringLiteral, sizeof StringLiteral);

glouwbug• 5 years ago

Just get std::map. You can get std::vector<int> if you do std::map<int, int>

MauranKilom• 5 years ago

Yikes for allocating all your elements separately, and with 400% - 700% memory overhead.

TeMPOraL2• 5 years ago

That's prevented by kernel-mode anti-cheat that's statically linked into the runtime.

phone8675309• 5 years ago

There's also a new version with renamed methods every year, and some of the methods move to other libraries in the middle of the project.

paulddraper• 5 years ago

Correction: $5/month

jhgb• 5 years ago

Is that a pay-to-Win32 library?

TeMPOraL2• 5 years ago

std::unordered_map: $100

+ in-template purchases for every specialization of std::hash for user type.

danuker• 5 years ago

List size is O(n).

https://eastl.docsforge.com/master/design/#listsize-is-on

I am surprised compilers don't magically optimize the code and remove the size-remembering variable if it is not needed.

gpderetta• 5 years ago

These sort of layout optimizations are extremely hard in C-like languages as the compiler needs to prove that the program can't tell the difference. At the very least you need whole program optimizations.

T-zex• 5 years ago

Isn't space complexity for a list with cached count (size) property "c" still O(n + c) ~ O(n)?

jcelerier3• 5 years ago

I am surprised compilers don't magically optimize the code and remove the size-remembering variable if it is not needed.

the problem is that technically anyone can come and do

    auto lib = dlopen("my_lib.so", 0);
    auto sym = dlsym(lib, "_ZNKSt7__cxx114listIiSaIiEE4sizeEv"); // std::list<int>::size()
    auto func = static_cast<void(*)(std::list<int>*)>(sym);
    std::list<int> some_local_list = ...;
    (*func)(some_local_list);

and that is expected to not crash - template symbols are generally part of your shared library API (at least that's been the default, however bad it is, on Linux, for a very long time). If compilers were optimizing the layout of individual list instances, then the above wouldn't work anymore (unless the compiler would inline / create new symbols for each individual cases in your code which would make the object sizes go through the roof).

Kranar2• 5 years ago

There's no requirement or even expectation that your snippet of code in principle works (even after working out the issues in the snippet you posted, such as needing to use the .* operator, the fact that member functions are not the same size as void*, and other quirky details).

The only way something somewhat similar to what you posted could possibly work would be to have some_local_list allocated/constructed dynamically from the same shared object that contains the the member function and to expose the member function using extern "C".

For example something like:

    auto some_local_list = create_list(...);
    list_size(some_local_list);

Where create_list is a function loaded from the same dlopen and returns a dynamically allocated and opaque handle to a list, and list_size is also a function loaded from dlopen that is exported using extern "C".

With this approach it's certainly possible for a compiler to optimize out unused member variables. Any other approach is undefined behavior and may or may not work.

secondcoming• 5 years ago

Having compilers remove members would be a backwards compatibility shitshow.

josefx• 5 years ago

However list splice is O(1), since it doesn't have to count how many items it moves.

IvanAchlaqullah• 5 years ago

I wonder why EA need this when modern console (except Nintendo Switch) are all x86.

corysama• 5 years ago

1. It was started back in the days of the PS2, I believe.

2. https://eastl.docsforge.com/master/faq/

maccard• 5 years ago

Ea were shipping games on PS3 and Xbox 360 until 2017. A modern game likely has code that datds back to the early days of that console, so it's a choice between maintaining eastl or porting all of their games to the standard library, along with the tooling theuve built around it (as someone else mentioned, tagged allocators are a big thing that aren't supported by the standard library.

Also, just because something is x64, doesn't mean it's standard library is up to snuff, or that it has the same features. You might be using a 5 year old compiler with a standard library to match it.

immmmmm• 5 years ago

somewhat unrelated but came across the Embedded TL recently, mostly for use on very small CPU, not exactly the same functionality but might be useful elsewhere as well

https://www.etlcpp.com/

belazeebub• 5 years ago

It should be noted that Paul Pedriana, the original developer of EASTL and many other things, just recently passed away unexpectedly. He was an EA programming legend.

Note of passing: https://www.facebook.com/groups/20296764839/posts/1015929527...

EASTL paper: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n227...

Partial list of Paul's programming credits: https://www.mobygames.com/developer/sheet/view/developerId,2...

vlovich123• 5 years ago

Shit. I just worked with Paul not too long ago at Oculus. That’s a real loss.

adamrezich• 5 years ago

Partial list of Paul's programming credits

MS Hearts, wow!

pcwalton• 5 years ago

Oh, that's terrible news :( We had some good conversations about the EASTL design with him early on designing the Rust standard library. Very smart and articulate guy.

amatecha• 5 years ago

Sad news. It sounds like he must have been a really influential and inspiring person to work with. We could all hope to have the fortune of having people like this in our cohort of colleagues around us! :)

I love this bit of the 1993 bio on Mobygames: "I do this computer work on the side as a hobby"

wyldfire• 5 years ago

It would be interesting to see how well it performs on the libc++ test suite. I know Microsoft recently explored running their C++ lib against this test suite and I think libstdc++ has been run against it too.

StephanTLavavej• 5 years ago

Well beyond exploring - microsoft/STL runs libc++’s test suite for every PR, skipping a set of known failures (for product bugs, test bugs, etc.), see https://github.com/microsoft/STL/blob/main/tests/libcxx/expe... . (I work on MSVC’s STL.)

waynecochran• 5 years ago

Somewhere near the first paragraph there should be something that tells me what I get beyond STL and Boost. I still have no idea.

pjmlp5• 5 years ago

This is pre-historic by now, in C++ history.

Quite appropriate to this subject are the talks from Mathieu Ropert.

Here is one of the latest ones,

"This Videogame Developer Used the STL and You'll Never Guess What Happened", ACCU21

https://www.youtube.com/watch?v=xoEUO9DezV8

Arech• 5 years ago

Note to fellow devs: if you are familiar with modern CPU properties, C++ and STL, you most likely won't find anything new there, so save your time.

ncmncm• 5 years ago

There is little reason to pay this any attention anymore. The world has moved on.

jsjsjsjsjsjs• 5 years ago

EASTL is great, but we call it UBSTL for a reason :D

EASTL: An Alternative C++ Standard Library

Related Stories