How to write better game libraries

https://handmade.network/wiki/7138-how_to_write_better_game_libraries

by LucaSas • 7 years ago

146 123 7 years ago

edem• 7 years ago

Please add (C) to the title of the article. In this form, it is misleading because most of the advice only applies to C.

kd5bjo5• 7 years ago

The target audience isn’t C programmers, but library authors that want their library to be usable with multiple programming languages. Most of the advice applies to C libraries because the article starts by advising you to write your library in C, and gives several reasons.

rootlocus• 7 years ago

The (games) in the title pretty much limits the article to C / C++

nurettin2• 7 years ago

> C is the lingua franca of programming

I don't have the exact numbers, but this seems debatable.

Reelin• 7 years ago

It's the lingua franca in that it's the standard for interoperating. Every high level language I've used has had at least one C FFI readily available.

nurettin2• 7 years ago

We do have a mutual understanding on whether C foreign function interfaces are available for other programming languages, which does enable languages to send system requests (for systems which are written in C) or to create optimized C code to parse documents or calculate formulas. I am aware that the concept of making languages talk with C interfaces exist. And I made the above statement knowing this.

rustybolt• 7 years ago

I think the term lingua franca is most often used to indicate a commonly understood language, not one that everybody actively uses.

Six of the eight most popular programming languages (from https://www.tiobe.com/tiobe-index/) have C-like syntax. I think most people would agree.

Edit: I think that in this context, Reelin's comment is more appropriate: This is not about understanding, but about interoperability.

fortran90• 7 years ago

Since this article lays so much emphasis on C, I have an honest question to everyone. What is a good way for a beginner to learn C in the current time? The minefield of undefined behavior is really overwhelming to a beginner. Are there any good resources that teach C the right way with good advice and best practices to navigate the UB minefield?

kd5bjo5• 7 years ago

I can’t advise on modern resources as I’ve known C for a while now, but I can speak a bit about the mindset required. Programming in C isn’t much different than programming in any other language, just without the safety railings. If you’re at the stage where you are typically fixing bugs by understanding what is happening and then making a single targeted change to correct the issue, you should have no significant trouble working in a C codebase.

If, on the other hand, you’re in the habit of shotgun debugging, where you make repeated changes to the problem code until it appears to work, you’re quite likely to leave behind various problems that will be hard to figure out.

Often, experience is the best teacher. If you’re not exposing your program to malicious users (aka. the public at large), the most serious issue you’re likely to run into is either a program crash or data corruption— nothing that will really harm your computer, but may cause you grief as you try to figure it out. In that process, though, you’ll learn an awful lot about how everything works. So, go write some programs for your own use and see how they crash and burn so that the next thing you make is more stable. Eventually you’ll start to intuitively spot trouble before it actually happens.

tick_tock_tick• 7 years ago

Being careful with malloc/free and using a modern toolchain that will warn on 99% of stuff is pretty much all you need. People way overemphasize UB

pjmlp12• 7 years ago

Yes, you need to dive into books like

Writing Secure Code

https://www.amazon.com/Writing-Secure-Second-Developer-Pract...

Secure Programming Cookbook for C and C++

http://shop.oreilly.com/product/9780596003944.do

SEI CERT C Coding Standard

https://wiki.sei.cmu.edu/confluence/display/c

huhtenberg4• 7 years ago

the UB minefield

While UB quirks exist, they are WAY off the beaten path and it takes an effort to run into them. Doubly so if you are just starting with the language.

Just treat C as a thin convenient layer over the hardware that expects you to think and act responsibly in exchange for this nearly raw access.

Profan• 7 years ago

In C (and C++) something as simple as forgetting to initialize a variable lands you in UB-land already, nearly every person I knew when I was learning myself (and still now) ran into these things _very quickly_.

Not to mention compilers make fun-times out of this by sometimes zeroing memory in debug and then not doing so for release builds (Hi MSVC!)..

The "thin layer over hardware" idea is a thing of the past as soon as optimizations come into the picture, and even then.

js8• 7 years ago

If you already know it a bit, I think https://modernc.gforge.inria.fr/ is pretty good.

LucaSas8• 7 years ago

As a good resource for learning C I would recommend Casey Muratori's tutorials: https://www.youtube.com/watch?v=F3ntGDm6hOs&list=PLEMXAbCVnm...

If you are interested in learning more about writing complex software in C consider checking out HandmadeHero.

Sadly there need to be more good resources on learning how to write good C and low level software. I am hoping my article can be a starting point for people who wish to learn about library design for example.

jandeboevrie• 7 years ago

This is bad advice on the c++ part. If you don't use RAII, then you could've just not done the c++ lib (and have just a c lib). Exceptions, raii and smart pointers are what makes modern c++ so pleasant, next to templates.

a_t482• 7 years ago

FWIW I’ve never been at a job that allows exceptions...but smart pointers and RAII are great.

jstanley• 7 years ago

No exceptions. No exceptions.

EliRivers2• 7 years ago

On the other side of the coin, I don't think I've ever had a job in which anyone would have ever considered banning exceptions (although I have worked at some where people did know that some other companies did ban them). The world of programming is a very broad church indeed.

LessDmesg4• 7 years ago

modern C++
pleasant

Nice joke. Hour-long compilations, type-unsafe templates with unreadable errors, and circular references are considered pleasant now?

je422• 7 years ago

- reasonable organized code doesn't take hours. even template heavy code is quite fast these days

- templates are type safe. it is just the error messages are not helpful a lot of times. C++20's concepts should help

- circular references. that's a thing in C as well. as soon as you need interact with your libraries user to allocate/deallocate resources.

C++ has a large surface to interact with. Yes you can shot yourself in your foot. However, the more you know you can use the features to your advantage instead of shotting yourself in the foot. At the point you gain more than you lose.

rumanator4• 7 years ago

Hour-long compilations,

If your project takes hours to compile them you only have yourself to blame.

type-unsafe templates

There is no such thing.

with unreadable errors,

That's an implementation issue, not a language issue.

and circular references

If you write buggy code then you only have yourself to blame.

LucaSas8• 7 years ago

Hi, I am the author of the article. Regarding RAII, what I mention in the article is that it is ok to provide RAII wrappers as long as you also provide C++ wrappers for the non-RAII plain structs and functions from C. I think this is the best way to satisfy everyone.

When it comes to exceptions and rtti the sad reality is that about 50% of the C++ community doesn't use (for one reason or another), so if you do use them in a library it can have an impact.

wokwokwok• 7 years ago

Exceptions, raii and smart pointers are what makes modern c++ so pleasant

Point in case for lack of portability, UE4 doesn't support exceptions unless you do a custom build.

saagarjha• 7 years ago

I don't see it mentioned, but it's important to note that you don't have to write your library in the same language that you expose to your users. You can write everything in C++/Rust/Python and expose something compatible with the C ABI and people will be able to use it (provided they have your toolchain…)

pjmlp12• 7 years ago

Fully agree with you.

Anyone providing a library written in pure C, better be serious about security and prove that they have taken all the required steps to handle memory corruption and UB exploits.

We really need more liability on software development.

erikbye• 7 years ago

Do note where this article resides. Casey Muratori (of Handmade Hero, a game dev tutorial project) is a self-proclaimed anything-not-C hater. A community has formed around his project, and has, of course, adopted his hard-line stance.

sharpneli2• 7 years ago

Indeed. And a C style api is still valid C++ too. Nothing in C++ mandates one to design a boost-esque inheritance and template monstrosity.

Functions and opaque handles are often way simpler for the user.

seba_dos1• 7 years ago

provided they have your toolchain…

That's important here. Some of my projects have something around 10 cross-compilation toolchains for various platforms. If a library is written in standard C99, adding it to the project is a breeze. Otherwise...

Const-me11• 7 years ago

Some of these recommendations are awesome, others aren’t good, IMO.

Always prefix your names to avoid name collisions

Solved by C++ namespaces.

Use header guards instead of #pragma once.

#pragma once is supported by all mayor compilers, header guards can introduce bugs, also #pragma once builds measurably faster: https://github.com/electronicarts/EASTL/blob/3.15.00/include...

Expose constants to the user using constexpr variables.

Modern C++ has strongly-typed scoped enums for such constants.

kd5bjo5• 7 years ago

C++ features don’t help when you’re programming in straight C, as the article advises.

pjmlp12• 7 years ago

Other than UNIX like kernels and tiny PIC micro-controllers no one else should still be programming in straight C in 21st century.

sharpneli2• 7 years ago

And then there is things like windows.h defining max which breaks std::max. Sure that one has another define which prevents it but not all libraries have.

C++ defines don’t respect namespaces so you are pretty much screwed in anycase.

Const-me11• 7 years ago

The OP’s advice to avoid macros in the library’s API is a good one. Please read my previous comments, it begins with “Some of these recommendations are awesome”.

BTW, you can disable these windows.h macros by defining NOMINMAX. I usually do, because I prefer min/max from <algorithm>; in some edge cases std::min / std::max can be twice as fast because compiler guarantees to compute arguments exactly once.

rehemiau• 7 years ago

C++ is not being used by all indie game developers so I imagine the article was written with that in mind, that the library needs to work with C code as well.

jstimpfle9• 7 years ago

Solved by C++ namespaces.

I've never understood what's the practical difference between NS::foo() and NS_foo() with regards to preventing name collisions. Can someone enlighten me?

I know some disadvantages of the namespacing variant, though. There are now multiple names for the items defined in the namespace: the qualified one and the unqualified one. The latter is often not unique in practice, since the programmer relies on the qualified name for uniqueness. In effect, making simple text searches for identifiers is very unreliable. Note I do use IDEs, but I also code in vim, and I need to do simple text searches even when working in Visual Studio.

Additionally, there's no guarantee that there are no spaces around the scoper. I believe "NS :: foo" is just as valid, which makes me uneasy with regards to text search, as well.

Another issue I have:

    $ cat test.cpp 
    namespace NS { int foo() { return 0; } };
    $ g++ -c -o test.o test.cpp
    $ nm test.o
    0000000000000000 T _ZN2NS3fooEv  # I hate my life

> Modern C++ has strongly-typed scoped enums for such constants.

I haven't found those working for me. Apart from the namespacing issue described above, I have issues with explicit enum types. One issue is that I often need to put sentinel / "missing" values (typically the value is -1) where an enum value is expected. Even more often, I want to iterate over the values of an enum. C++'s enum "type safety" makes working like this really unergonomic.

The way I go about this is I don't even use names for my enum types, and I fully qualify the enumeration values.

    enum {
        FPGAPARAM_BLA,
        FPGAPARAM_BLUB,
        FPGAPARAM_FOO,
        NUM_FPGAPARAM_KINDS
    };

    struct ASDF {
        int fpgaparamKind;   // obvious what kind of values are expected here...
    };

    for (int i = 0; i < NUM_FPGAPARAM_KINDS; i++) {
        struct ASDF asdf;
        asdf.fpgaParamKind = i;
        do_asdf(&asdf);
    }

In programming, the slightest mistakes, like putting a "-" instead of a "+", result in program bugs. These mistakes are much more likely to be made (and much harder to spot) than mistakes involving enum values from the wrong set. I won't let programming ergonomics be ruined in the name of "type safety".

Const-me11• 7 years ago

what's the practical difference between NS::foo() and NS_foo() with regards to preventing name collisions.

They both do the job. There’re 2 practical differences.

1. You can write `using namespace` inside functions or the whole .cpp files. This often makes the consuming code more readable.

2. Sometimes you want to replace implementations. With prefixes it gonna be massive changes likely to introduce new bugs. With namespaces, replace `using std::vector` with `using eastl::vector` and you’re done.

Even more often, I want to iterate over the values of an enum.

I only need to do that rarely. When I do, I cast types like this:

    enum struct eParamKind : uint8_t
    {
        Bla, Blub, Foo, valuesCount
    };
    for( uint8_t i = 0; i < (uint8_t)eParamKind::valuesCount; i++ )
    {
        const eParamKind pk = (eParamKind)i;
        // Whatever
    }

> I won't let programming ergonomics be ruined in the name of "type safety".

I disagree on ergonomics. VS makes much easier to consume API with strongly typed enums: ePar<Ctrl+Space>::f<Enter>, to type eParamKind::Foo It’s similar with namespaces versus prefixes BTW, IDE will first auto-complete the namespace, then only list members of that namespace once you type the `::`

Update: another C++ feature relevant for game development is overloaded operators. Games often do non-trivial amount of math on small vectors, matrices and quaternions. Overloaded operators make sense for them.

LucaSas8• 7 years ago

Hi, thanks for your feedback. I am the original author.

Regarding prefixes, I advice that you start by writing the library in C and then wrap it in C++ for a variety of reasons that you might want to consider. In C++ you should indeed always use namespace.

Header guards have the advantage over pragma once that they are standard and you can also use them to check if a library is included. I might remove that since maybe it's not that important and people might different views.

Regarding constants, I was referring to things such as numeric constants for which you would constexpr in C++. Maybe I can be more explicit there. Thanks for the feedback.

Const-me11• 7 years ago

I advise you start by writing the library in C++ and then wrap it in C if you need C ABI of the library.

It’s very hard to write correct C code which does IO and supports multithreading. Take a look at Microsoft’s implementation of fprintf, copy-pasted from Windows 10 SDK: https://gist.github.com/Const-me/f1bb320969adde6c79694265ea6... They use RAII to set & revert the locale, and to lock RAM buffer to avoid corruption by another threads. They use C++ lambda for exception handling. They even use C++ template to avoid code duplication between printf and wprintf.

But these C++ shenanigans are not exposed to user, user calls their `printf` (possibly in a code built by C compiler) and it just works.

astrobe_2• 7 years ago

Make is a much the lingua franca of build systems as C is for programming, so there's no reason not to provide a (simple) makefile. The alternative they suggest is worse anyway.

huhtenberg4• 7 years ago

Windows is the reason make is not a good choice. Visual Studio is a de-facto standard for Windows development and it's not exactly makefile-oriented.

The lowest common denominator for build instructions that works everywhere is literally "set your include paths like so and compile these sources".

astrobe_2• 7 years ago

How strange. I have tinkered with a couple of FOSS games of Windows and none of them required VS (not to mention another couple of other non-game projects). Maybe that's because they often are multi-platform projects so Make is the best choice at the global level for them?

The lowest common denominator for build instructions that works everywhere is literally "set your include paths like so and compile these sources".

Yes, but that's quite inconvenient and error-prone ; a typo in a -D flag is very likely to not compile what you wanted.

Make is not a multi-megabyte program with a frack tonne of dependencies. Having your users to install it in order to build your project is far from being completely outrageous (unless for coughmicrosoftcough cults that consider the command-line is evil).

diamondo25• 7 years ago

Funny thing is that AtmelStudio (for embedded development of Atmel/Microchip chips) is basically a VisualStudio shell with a makefile generator (built in c#). This is then pushed through make and invokes gcc...

killmov3s2• 7 years ago

build tools are a mistake in the majority of cases

rienbdj• 7 years ago

Why not a C++ library with an optional C API?

LessDmesg4• 7 years ago

Because C++ is one of the most complex languages in the world, hence a pain to interop with. C FFIs, on the other hand, are ubiquitous.

izacus• 7 years ago

The idea (and one that worked well in several projects I've worked on) is that you use C++ as main language of the library and make outside bindings C.

It does require some discipline, but C++ has many portable constructs that make life significantly easier when writing business logic.

pnako• 7 years ago

It's possible to use the "hourglass" pattern where your implementation is written in C++, you export a C API (and ABI), and then you can provide an additional, light C++ wrapper based on the C API.

shmerl2• 7 years ago

Write the library in Standard C99

Why not write it in Rust, and provide C99 bindings? Using stock C for the actual library doesn't sound like a good idea to me, when today there are better alternatives.

And for sure avoid using Metal lock-in directly. Use Vulkan, which Apple should have supported from the beginning. For lock-in targets, there are translation options from Vulkan.

LucaSas8• 7 years ago

Hi, the reason for using C99 is that it is fully compatible with other languages. If you write a library in Rust you might be tempted to use Rust only feature which will then make it hard to wrap the library for other languages. And if you don't use those features from Rust in your library people will comment that your library isn't Rust enough. Same thing applies to C++. To that extent my advice is to write C in C rather than C in Rust or C++.

Also if people don't have a Rust toolchain setup, compiling the library and using it from source would be hard. In some cases integrating Rust in their toolchain could be hard.

Regarding metal and vulkan I will edit the article to mention Vulkan there too alongside metal. Thanks for your feedback.

shmerl2• 7 years ago

I suppose wrapping it for other languages can be harder, but should be doable I think. The benefit of Rust though is using a much better language with rich standard library.

As for toolchain, Rust can be set up basically anywhere llvm can, which is quite a lot. There are rare cases where llvm wasn't ported yet to, but I don't think they are enough to make C a compelling option in general, and I don't think any of them are gaming related.

If you are in such case - then sure, but otherwise, I'd still prefer Rust.

NohatCoder• 7 years ago

It always baffles me when a library comes with its own arcane build code. Instructions like "Requires Python" invariably turn out to actually mean "Requires hours of debugging".

ginko• 7 years ago

IMO the ideal way to get your library used is to generate a single .h and .c file.

FreeFull• 7 years ago

Some libraries don't even ship a .c file, all the code is in the header

derefr• 7 years ago

The problem is that C is a "lingua franca" and yet has no inherent facility for compile-time codegen (i.e. "real", grammar-level macros, as opposed to C's lever-level macros), so you need something else (whether that be the venerable m4 or yacc, or random Python scripts) to serve as your substitute for "real" macros.

In languages that do have "real" macros, you don't see the same problem. (On the other hand, in most of them, you see a need to call out to a C compiler to generate FFI code, which is sometimes just as bad when you're using a PL with a managed runtime and your build-env didn't otherwise have any need for a C toolchain.)

flohofwoe3• 7 years ago

As a warning: While C99 is a really nice language - especially over C89 - please note that the Visual Studio C compiler still does not support the full C99 specification, and it is not possible at all to compile C99 as C++ code (which sometimes is useful).

I usually recommend to write library-level code in the "subset of C99 which compiles both in C and C++ mode on GCC, clang and MSVC". This is basically a version of C that's somewhere C89 and C99 (basically a "C95").

Another quite valid option is to define a pure C-API first, but implement the "inner library code" in a simple C++ (just be careful with "modern C++", since this usually results in increased compilation time and binary size).

jstimpfle9• 7 years ago

The one feature from C99 that I rely on (from the top of my head) is designated initializers which is still not in C++...

    enum {
        FPGAPARAM_FOO,
        FPGAPARAM_BAR,
        FPGAPARAM_BAZ,
        NUM_FPGAPARAM_KINDS
    };

    struct FpgaparamInfo {
        const char *name;
        int address;
        int args;
    };

    static struct FpgaparamInfo fpgaParamInfo[NUM_FPGAPARAM_KINDS] = {
        [FPGAPARAM_FOO] = { "FOO", 0x1337, -1 },
        [FPGAPARAM_BAR] = { "BAR", 0x666, -1 },
        [FPGAPARAM_BAZ] = { "BAZ", 0x42, -1 },
    };

The FpgaParamInfo is basically a mapping from a FPGAPARAM_??? value to additional information. We can make as many of these mappings as we want, and can define them where we want, which means it's all nicely modular. That's not really possible when modelling in a OOP fashion.

I really like this style of programming since it's data first and it minimizes the amount of actual code. Designated initializers are important because the order in which the items in "fpgaParamInfo" are given doesn't matter. Without designated initializers, programming in this style would probably lead to many hard to find bugs when the enum is changed and not all associated data items are updated.

LucaSas8• 7 years ago

Thanks for the comment, I might steal your "subset of C99 which compiles both in C and C++ mode on GCC, clang and MSVC" quote and put it in the article if that's ok ;)

matheusmoreira2• 7 years ago

Is there a benefit to building C code with a C++ compiler?

Keyframe• 7 years ago

And then the author provides example where there’s Dear IMGUI which is c++.

flohofwoe3• 7 years ago

Dear ImGui has a very C-style API though, it's just flat a function API wrapped in a namespace, and about the only other C++ feature that's used in the API is default function arguments.

For usage from C code, there's cimgui, which is an automatically generated C-API wrapper:

https://github.com/cimgui/cimgui

yoz-y2• 7 years ago

Not everyone wants to use C++ (some prefer C).

This could be very well flipped over. If I have the source available, I'd much rather deal with C++ bindings.

LucaSas8• 7 years ago

Hi, I mention in the article that "It is easier in general for a C++ user to use a C library than it is for a C user to use a C++ library." which is where the advice comes from.

yoz-y2• 7 years ago

I agree. But "wants" is a term indicating personal preference. I understand that C is best suited for maximum portability and performance. But as a C++ developer I prefer libraries that come with C++ interfaces (and you do mention that it is good to provide a C++ wrapper as well).

matheusmoreira2• 7 years ago

Because the binary interfaces generated by C libraries are simple and therefore universally supported. It's just normal symbols and calling conventions. Conforming to this ABI benefits users of every other language, not just C++.

C++ ABIs are rather complex and much harder to interface with. The existence of concepts like virtual functions and exceptions significantly complicates the implementation of foreign language interfaces.

https://wiki.osdev.org/System_V_ABI https://itanium-cxx-abi.github.io/cxx-abi/abi.html

rehasu• 7 years ago

The problem for game libraries in general is that game dev lives in c++ world and c++ is awful for libraries. Most c++ devs I know would rather start writing a program by defining string than by learning to handle proper library management tooling. I'm not an expert so I'm not sure if it's just a culture thing or if there are inherent features in the language that make library usage hard, but yeah. That's that.

overgard• 7 years ago

It’s pretty inherent to the tooling around the language. Basically because theres a lack of “standard” tooling, so you either go lowest-common-denominator (header only), or you have to fuss with very gross build systems that have weird quirks on different platforms.

C is easier in this regard just because the language evolves so slowly that even though the build and library systems are bad, theyre a known bad that people can work around. No such luck with C++, where its bad and ever changing.

novok• 7 years ago

I wish the C++ world would adopt bazel eventually as their standard tooling base.

rumanator4• 7 years ago

I wish the C++ world would adopt bazel

Why do you believe picking a specific build system is relevant wrt libraries, particularly a build system whose main selling point is build speed.

grawprog• 7 years ago

Honestly, for me all I would like is the ability to include/import one or more files as needed, helpfully separated by basic functionality, without needing to include the entire library, have a bare minimum of dependencies, hopefully, module independent, the ability to build painlessly for different platforms with a minimum of code changes and a painless build process for the library itself, ideally one I can include in my own build process without a lot of effort.

I've used some libraries that do some of these things, but I can't think of any that does all of them.

killmov3s2• 7 years ago

We just need more single-file headers like nothings' stb libraries

drjeats• 7 years ago

Nice article :)

Might be worth touching on error callbacks/logging as an error handling strategy.

Sometimes an error is not recoverable in the sense that the calling code can't really do anything about it, but the library should attempt to make progress anyway instead of halting the entire program.

By allowing users to specify an error callback, this means they can log errors, capture stack traces, assert, or whatever.

This isn't that helpful for smaller libraries with smaller-scoped processes, but if it's something like a renderer or interactive audio lib, those often just need to be given a bunch of frame time to do work with the complex input you've prepped and fed to it, and trying to propagate error codes up out of that simulation step would both contort the inner code and not be as helpful as an error callback.

With an error callback you can assert, set breakpoints, or do whatever. But more importantly, by default you can have it just log so when you inevitably in a bug it doesn't prevent everyone else from getting work done while it keeps asserting until you fix your shit.

This will be a less common need than the other standard error reporting mechanisms, but is important to get right if your library has these complex internal preconditions that you want to make visible to the client when violated.

How to write better game libraries

Related Stories