Libvirt: Adoption of GLib library to replace GNULIB and home grown code

https://www.berrange.com/posts/2020/01/30/libvirt-adoption-of-glib-library-to-replace-gnulib-home-grown-code/

by adgnaf • 6 years ago

79 65 6 years ago

These problems are common to many applications / libraries that are written in C and thus there are a number of libraries that attempt to provide a high level “standard library”. The GLib library is one such effort from the GNOME project developers that has long been appealing.

I've been a fan of glib for some time, other than glib and apache libapr, what other "high level standard libraries" for C should I know about?

loeg3• 6 years ago

If your C umbrella includes C++, Qt has a number of (non-GUI) high-level libraries that can be useful for applications.

lrossi6• 6 years ago

Qt is more like gtk rather than glib. Although it does come with a fantastic core library.

But if you can move to C++, the standard library already offers great replacements for most glib features. Glib exists mostly because the C standard library is lacking in many aspects.

ognarb• 6 years ago

There is QtCore that is more similar in scope to glib and only containers, json and a few more useful stuff. It's a sort of alternate C++ standard library.

Personally I wonder why people would choose today using C and Glib over C++ for system programming. I could understand why not Rust but for having to deal with glib in the past its so much of a pain.

PaulDavisThe1st2• 6 years ago

Even the Qt folks accepted, though, that for run/event-loop integration between glib and Qt code, it was Qt that provided a way to use the glib event loop not vice versa (because Qt doesn't offer a sufficiently hook-able event loop abstraction).

JNRowe• 6 years ago

I can't offer extra recommendations, but just wanted to offer full agreement.

GLib is the awesome standard library I get to use everywhere thanks to gobject-introspection. From work projects in Vala, window manager¹, image viewer², video player³. It is especially useful with lua configurable projects, given the sparsity of the language itself.

¹ https://awesomewm.org/

² https://github.com/muennich/sxiv - my user configs are lua & lgi, extended with gexiv2 also via gobject-introspection.

³ https://mpv.io/

stretchcat2• 6 years ago

Incidentally, mpv is extensible enough to replicate sxiv's features with the right configuration and scripts. I like to think of mpv as the emacs of multimedia.

saurik• 6 years ago

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NS...

Netscape Portable Runtime (NSPR) provides a platform-neutral API for system level and libc-like functions. The API is used in the Mozilla clients, many of Red Hat's and Oracle's server applications, and other software offerings.

tyingq• 6 years ago

Qlibc: http://wolkykim.github.io/qlibc/

lrossi6• 6 years ago

Glib is OK except for the fact that it has a memory cache on top of that of malloc. This prevents tools like asan or valgrind from detecting memory related bugs. It caused my team a lot of grief, to the point that we regretted the choice of using glib in the first place.

I am not sure why there is a memory cache in the first place. Malloc may have been slow in the 90s, but these days there is no reason to cache and reuse allocations.

It’s also a major security risk, since it nullifies hardening measures from the standard library, as we have seen with openssl/heartbleed recently.

lrossi6• 6 years ago

Another problem is that there is no type checking in the containers. So you have to cast everything to/from void pointers, even for basic structures like lists and arrays. This makes it easy to introduce tricky bugs.

wahern• 6 years ago

The BSDs basically solved this problem decades ago: <sys/queue.h> and <sys/tree.h> provide type-safe core data structures sufficient for most C application needs. The amount of wheel reinvention and dependency complexity outside the BSD universe blows my mind. (Though, FreeBSD projects do have a greater tendency to complicate things, perhaps owing to the stronger corporate-induced feature chasing.)

The only real universal sore spot IME has been arrays and vectors. But nobody seems to pitch glib as a way to get a fast and ergonomic FIFO buffer. There are many other areas without simple, go-to solutions, but then that's the nature of C programming. Most of the C programmers I interact with are multi-language programmers, as opposed to many C++, Java, etc engineers who lean toward single-language, monolithic approaches.

I can understand using glib for GUI applications, considering it's already a requirement for Gtk, and because of the OOP emphasis in GUI programming. But IMNSHO, in most other areas the right reasons for selecting C as your implementation language are mutually exclusive with the need for cookie-cutter, void-pointer heavy data structure implementations a la glib.

EDIT: Removed outdated discussion of systemd + glib.

attractivechaos• 6 years ago

In addition, using void pointers harms performance as this strategy often incurs additional heap allocations, hurts memory locality and prevents compiler optimization. Personally, I avoid glib like a plague.

asveikau3• 6 years ago

Curious for some details. On what layer does that caching occur?

I immediately thought g_malloc but it seems to call directly to libc: https://github.com/GNOME/glib/blob/master/glib/gmem.c

the_why_of_y3• 6 years ago

Did you try to disable the slab allocator?

https://developer.gnome.org/glib/stable/glib-running.html#G_...

lrossi6• 6 years ago

I am aware of that environment variable. The problem is that I think almost nobody is using it, so it could come with a risk of introducing more bugs.

By the way, the example shown in the documentation page that you link to seems ridiculous:

  void *slist = g_slist_alloc (); /* void* gives up type-safety */
  g_list_free (slist);            /* corruption: sizeof (GSList) != sizeof (GList) */

This bug should actually be caught during compilation if we store the list in a GSList* typed variable instead of void*. You’d think: who does that?

Except that this is what you actually end up doing when using any kind of nested glib containers. Elements are always void*, so you have to cast them correctly. So for non-trivial applications, it’s very easy to make mistakes, since you lose the type checking support of the compiler.

C is already a tricky language. Removing the type checker makes it even worse.

pmiller2• 6 years ago

I don't know anything about this cache you're referring to, but, is it so pervasive within the library that you can't easily do a patch to remove it? Would such a patch be accepted by upstream? If you're opposed to using the environment variable, those would be my next thoughts if you wanted to get back the ability to use valgrind et al.

ibotty2• 5 years ago

You only need an environment variable. It is already implemented. They just did not try it.

inetknght2• 6 years ago

these days there is no reason to cache and reuse allocations.

No, you are wrong. Ideas like that are a big reason that apps get slower despite hardware getting faster.

stretchcat2• 6 years ago

Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc?

asveikau3• 6 years ago

I think it becomes an application specific or domain specific thing.

For example, I've done some work with audio or video. Nobody working on that goes straight to malloc on every packet or frame. It'd just be asking for pain.

But a general purpose allocator doing its own free-list on the assumption that libc is going to suck? I think that's outdated. If you do want to support it, I think it's better to allow the caller to replace the allocator through a function pointer, rather than just do it by default in a library.

nn3• 6 years ago

At least for valgrind it's fairly easy to annotate custom mallocs in the source. Just someone needs to do that.

qalmakka2• 5 years ago

I think that 3/4 of their issues would probably be solved by a well thought use of modern C++. I've seen too many C projects (including some of mine) starting by swearing out C++ for one reason or another and then ending out reimplementing half of it using macros or poorly written hashmap implementations taken from somewhere.

Glib2 is one excellent example of how people have been shunning C++ due to its complexity, while on the other hand implementing overly complex libraries to mitigate the fact it's too barebones, which is oxymoronic to me. Either you say C is better because its simplicity and you keep stuff simple, or you are just being a zealot for the sake of it.

Basically everything I can put my hands on supports C++, I've been running massive applications on embedded microcontrollers and it works as fine as C. If you don't like some features, just write C-style C++, use the C ABI and #include <> all the containers you need.

jononor2• 5 years ago

Agreed in general on using C++ over C, or C-style C++ in constrained environments. One thing that is nice about using C though is that it is quite easy (compared to C++) to expose to other languages. Glib, and most libraries built on it like GTK, GStreamer, etc can be used in languages such as Python, JavaScript et.c.. And a lot of development of applications and tools has been moving to such languages over say C++. So C is still useful for low-level libraries, due to being the lowest common denominator. Though arguably one could today write the library core in a better language like Rust maybe, and expose a C API/ABI from that. Rsvg is growing into an example of that.

pjmlp8• 5 years ago

I look forward to it eventually to grow into OS vendors SDKs, at very least those that provide llvm based toolchains.

Until then C++ will do.

samus• 5 years ago

C++ seems difficult to embed because there is no standard way to expose features that don't map well to the C calling convention. Just use `extern "C"` if you want a C API. For everything else you'd have to commit to an application binary interface first, such as GObject, COM or the CLR. You need these to define calling conventions and semantics (such as initialization, exceptions, memory and resource management) well enough that other languages can bind to them.

rleigh2• 5 years ago

Absolutely. It makes perfect sense, and in an ideal world this would be the direction which would benefit many, if not most, projects using C and/or GLib2. Unfortunately, there are far too many people who are wedded to the philosophy that C is perfect for every task, and C++ is the devil. Despite the fact that GLib/GObject are more complex than C++, more error-prone than C++, slower than C++ and make static code analysis impossible (due to all of the unsafe typecasting).

A few years ago now, I ported a C GLib/GObject-based application to C++. In removing all of the unnecessary typecasts I found a couple of minor (but real) bugs which were previously hidden from the compiler. Simple use of real classes, along with basic containers like vector and map, was the vast majority of the C++ usage in the whole application. It benefitted greatly in becoming smaller, simpler, easier to read, easier to maintain, and having the compiler able to typecheck everything.

Like yourself, I've also used C++ on MCUs. Some vendors even provide an "Embedded C++" C++ subset you can use, which is "safe" for safety-critical real-time code. Works fine. A lot of C embedded projects would benefit from the extra safety it provides. So long as you don't go overboard with the features; stick to a simple and easy to understand subset.

pjmlp8• 5 years ago

I gateway drug to C++, was Turbo C++ 1.0 for MS-DOS, released in 1990 as per Wikipedia. I got my copy in 1992.

Basically C++ARM as per language standard, on a 386SX running at 20 MHz, 2 MB but 640 KB was more than enough, right? :)

Our high school teacher giving us C classes with Turbo C 2.0 also had it around, so as it were back in those days, I eventually got a copy.

My gateway drug to programming until then were Turbo Pascal 6.0 and TASM, and C++ was in the same ballpark of features and culture for safer systems programming.

Never had any issues using it in such kind of PCs, including with my own bounds checked string and array classes, hardware that most modern MCUs can easily outperform.

rlkxx• 5 years ago

For long applications you get into he realm of tool chain bugs with C++. C compilers are more reliable.

If you write C-style C++, which parts of libstdc++ can you safely use without exceptions?

netheril96• 5 years ago

Google writes a lot of C++ code, disables exceptions entirely, and it does not shy away from STL at all.

pansa22• 5 years ago

which parts of libstdc++ can you safely use without exceptions?

Almost all of it, as long as you're happy to abort on memory allocation failure - which, according to the article, libvirt is now willing to do.

In fact for me, that's one of the main questions to ask when deciding between C and C++ for a project. Is it OK to abort on allocation failure? If so, use C++ (without using exceptions). If not, use C.

pjmlp8• 5 years ago

C++ was already everywhere on desktop computing back in the 90's, with Apple, IBM, Microsoft, BeOS adopting it on their frameworks, and even on the mobile with Symbian.

Then FOSS happened with its manifesto to use C for portability, war on KDE due to licensing gave raise to Gtk and related eco-system, and here we are.

Thankfully I learned C++ on MS-DOS and became enlighted, even with its 640 KB limit it was already so much better than the primitive C, in regards to type safety, generic code and yes RAII was already a thing.

the_why_of_y3• 5 years ago

The main practical benefit of C is the simplicity and stability of its ABI.

There are currently 8 different language bindings listed on LibVirt's website. It's not clear to me that this situation would improve by switching to a language as notoriously difficult to interop with as C++.

https://libvirt.org/bindings.html

qalmakka2• 5 years ago

The point is, C++ supports the C ABI. You can write your entire application in C++ and expose only a C-style API with the stable ABI you wish. I've done it a million times and it's absolutely fine.

sovande• 5 years ago

There is one significant thing that C has and C++ has not; a stable ABI. This is particularly important if you want to create a library that can be shared and used by everyone. You cannot do that with C++ without a lot of caveats. Google, for instance, forbid creating libraries in C++ for good reasons. Creating a C++ library with a C ABI sounds stupid. I.e. if the library’s functions and types can be presented as simple C, you have already folded down to C and C++ is just an implementation detail and you will probably also have to static link libstdc++. That’s no longer a small, simple library anymore. The stable ABI makes C also famously easy to include and inter opt with other languages and frameworks. Swift for example can easily utilize C libraries and code but cannot inter opt with C++ (yet, apparently it’s wip)

h_anna_h3• 5 years ago

I sure love gvoid and gint. I also love my program crashing instead of returning an error when I use glib lists and its internal malloc invocation fails.

ibotty2• 5 years ago

This article is nearly a year old. Is there any news on the progress?

Libvirt: Adoption of GLib library to replace GNULIB and home grown code

Related Stories