Rob Pike: Notes on Programming in C

http://www.lysator.liu.se/c/pikestyle.html

by ssp • 16 years ago

122 111 16 years ago

lysator.liu.se

rcfox3• 16 years ago

I disagree with the include files section.

As long as each file has a proper include guard, it won't matter how many times you've included a file. I'd much rather have the preprocessor figure out when and where to include a file than do it myself. Maybe this was a significant performance issue in 1989, but it shouldn't be anymore.

That said, you should only include files that are direct dependencies. There's no need to include stdio.h in every other file.

meese_• 16 years ago

I think that advice is just outdated. It's a different situation now that it's a standard practice and there are optimizations supporting it (http://gcc.gnu.org/onlinedocs/gcc-2.95.3/cpp%5F1.html#SEC8).

palish3• 16 years ago

That's a naive viewpoint. It's more of a concern in C++ ... if your codebase grows to massive proportions, and you #include headers within headers, then modifying a header can result in ~60% of your codebase being rebuilt. (At work, I deal with this pretty much every day.)

Including header files within headers is simply unnecessary, especially in C. You need to forward declare in header files, then include in source files where you actually use that header file.

rcfox3• 16 years ago

Suppose you've written some key data structure in its own file. Let's say it's a binary tree. The binary tree's functions are all implemented in bintree.c, and its functions, defines, and struct are all declared in bintree.h.

Now, you want to write some application-specific functionality that works on your binary tree. You need to pass a pointer to the binary tree to your new functions. Of course, you implement these new functions in app.c, and the functions are declared in app.h so that other parts of your application can call them.

You're going to need to include bintree.h in app.h to ensure that the compiler knows about the binary tree's structure. Otherwise, you'll have to make sure to always include bintree.h before you include app.h. Is that really something you want to be bothered with?

edsrzf• 16 years ago

I totally agree.

Not only will headers including headers cause files to rebuild unnecessarily, but it can cause build times to increase simply because the compiler has to compile all those extra headers each time they're included in a source file. (Precompiled headers can help, but they aren't always an option.)

gte910h7• 16 years ago

C is not C++.

In C you shouldn't be chaining header files because you don't make function calls in header files in C code (well most good C code at least, people abuse macros sometimes).

In C you should only include header files in header files if you have types in the other header files. It's okay to forward declare if you must, but its often quite a bit clearer to not.

Additionally, you should always include all the h files that have your undefined functions (only relying on include chaining for header files at most). So if you have:

DogModule.c calling Sounds.c functions and exporting a public function in DogModule.c called "PlayBark(struct soundNode);", DogModule.h and things that use DogModule should both import Sounds.h.

Good C habits and structure are very different than good C++ habits and structure. They're not the same language, not even close. Mangling your concepts of Good C and Good C++ is a great way to write both poorly.

jbn4• 16 years ago

yes, and that's using the Pimpl idiom to decouple things can improve build times. If your whole codebase rebuilds everytime a header is touched, then your headers are probably not correctly organized. Having minimal headers (using forwards decls as much as possible) is vital to keeping rebuild times down.

mwcremer• 16 years ago

I think Pike's point about "thousands of needless lines of code" is that people do this:

#include "foo.h"

instead of this:

#ifndef __FOO_H__

#include "foo.h"

#endif

where "foo.h" looks like this:

#ifndef __FOO_H__

#define __FOO_H__

/* contents /

#endif / __FOO_H__ */

which in the former case results in having to lex foo.h in order to skip the contents.

However, as pointed out below, modern compilers will effectively do the same thing for you automatically.

[Edited to make the code readable]

TheNewAndy5• 16 years ago

Just a quick pedantic point - __FOO_H__ is a reserved identifier (C90 reserved). You shouldn't be writing it in your code.

ori_b• 16 years ago

If you keep your header files independent, you have no reason to include a single header multiple times, and you can't do it accidentally through another header.

If your header files don't transitively include other headers, then

    #include "a.h"
    #include "b.h"

will never cause an error. Therefore, in well written code, the include guards will be no-ops, and for simplicity's sake, they may as well be left off.

rcfox3• 16 years ago

Sure, this isn't a problem in trivial projects. If you want to use types other than what the language gives you, you're going to have to introduce a dependency.

jbn4• 16 years ago

even with internal include guards, just the time to read through (and skip) the whole files for later inclusions can become dominant in cpp processing time (i.e. it's mostly waiting on I/O reading the same headers many times over). That's why Lakos advises external include guards for large systems.

cdavid4• 16 years ago

Unless you have gigantic projects, your project source code is likely to be mostly in memory through kernel IO buffers nowadays, so even this is not true anymore IMO.

EDIT: just for kicks, I compiled twice numpy (100 kLOC), first just after flushing IO buffers (echo 1 > /proc/sys/vm/drop_caches in recent linux kernels):

- cold case: 24 seconds - host case: 16 seconds

Of course, it is hard to say what contributes to the slowness (files themselves, loading in memory all the programs needed for compilation, etc...).

gte910h7• 16 years ago

C is not C++

C compilers are leagues, elephants and donuts faster than C compilers.

In C, internal guards are preferred and cause very little to no compilation time hit. In C++, external are often preferred, especially in large files.

Do not take C++ advice to apply to C. The best practices of both languages are very different.

drblast4• 16 years ago

Can anyone who's worked on a well-written, large C project comment on the best way to structure headers and include files?

I don't like including stdio.h and stdint.h in every .c and .h file, but it seems like the "standard" way to compile with Makefiles is to compile each .h and .c pair as an object then link them together at the end. So you need the header files to use includes and such in every file or the compiler barfs.

Every time I try to structure a C project I feel like I'm doing it wrong and that there must be an elegant solution that I'm missing.

gte910h7• 16 years ago

(Apoligies, HN is mangling my comment when I try to include C style comments, so I'm using C99/C++ style ones)

You should include whatever .h files you need in every file that uses them (things like stdlib and stdio for sure when you're using them). This makes it so you have fewer interfile dependencies which keeps your compile times down. So you only include other .h files in .h files if you need the types in them (or put a blank prototype in for it, aka struct foo;). Otherwise you include .h files only in .c files.

Feels hairy, yes, but that's the correct way to do it. By doing it creative ways, you cause great pain down the road (especially compilation times, and hunting for simple bugs). If you're only using one or two functions, sometimes people just prototype the function:

// some people just would stick this prototype in their .c

//file they're writing if they need nothing else in math.h

double sqrt(double x);

I think you're missing something about C there though the way you talk about compiling things.

h files aren't really compiled at all. They are substituted into files where they're #included. You can actually write your entire project without them if you really want to (not suggested, but you can) by just putting prototypes for the functions you call at the top of the c files that need to copy them. That's actually why its important to put guard include statements at the beginning and end of h files to make sure they aren't included more than once. A properly written h file shouldn't cause compiler errors when you do this to it:

#include "foo.h"

#include "foo.h"`

The way you make it proper is you do this the top:

#ifndef __FOO_H__

#define __FOO_H__

and this at the bottom

#endif // __FOO_H__

Many IDEs do this automatically and vim and emacs both have plugins you can get to do this. Robert Pike's piece is from 1989, when computers ran much slower. The lexical analysis time from this is not worth talking about anymore.

Structure:

Generally C is structured with a main .c file which has a few functions related to start up in it and includes of header files in other modules which are needed. Each of the .c files includes the header files it needs, from either internal project modules, or from system level header files (aka <thingsInAlligatorBrackets.h>). Generally each .c file is a module, although in Object Oriented C, you may find modules built up of several different c files with each "class family" in a single .c files. Modules are purely conceptual in C.

Some C people get macro happy. Do no do this. It is unclear to the rest of the C developing universe. Modern C compilers can do inline substitution quite well, often giving better performance than your macros. When people get macro happy, they end up making a file like "GlobalMacros.h" or even if they don't, they just have a lot of constants and end up with "LotsOfConstants.h" included in everything.

Resist the urge to put your .h includes in there but NOT in the files that actually depend on it. If you do that you break code if that .h file is ever taken out.

As far as final compilation goes: Every C file is compiled into an object file then linked using the linker. Alternatively, groups of .c files are compiled into .o files then linked into .a files (static libraries). These libraries are then linked in during the final compilation step.

loup-vaillant5• 16 years ago

To write code, skip 2 or more characters at the beginning of the line, Markdown style.

  #include "foobar.h" /* no mangling */

http://news.ycombinator.com/formatdoc

TheNewAndy5• 16 years ago

Your header guard is bad (but not horribly so). Identifiers with two underscores at the beginning are reserved in the C90 standard (as are identifiers with one underscore followed by an upper case letter).

Your comments are also only valid in C99. Unless you have a really good reason, then you should use the block style comments (and I can't think of any good reason). (oh, maybe I should have read your very first sentence... yes that seems like a good reason :-)

tedunangst3• 16 years ago

I'm not positive what you mean "you need the header files to use includes". header files should generally not include other files. C files include headers. If the headers have a dependency that can't be resolved by a forward declaration, it's often best to make the C file include the headers in the right order.

headers including headers leads to messes where people just include a random selection of headers and hope that pulls in everything they want. It's very annoying to untangle years later.

drblast4• 16 years ago

If I have data types defined as function arguments in header files, I need to include the headers for those data types. There's apparently no way around this, and it does result in messy dependencies.

I'm just curious if large projects have a systematic way of dealing with the dependency tree created by having fine-grained compilation units.

The simplest way to write a C program is as a single file. Then you have none of these problems, other than it's very difficult to navigate through the code as nothing is separated by functional area, and you have to recompile the whole program for every small change.

Next in the complexity chain, you could break apart the single C file into multiple .c files and include them in the right order, and not use header files at all. However, that still requires that you recompile the whole program for every change.

So now we're down to compiling every separate .h/.c entity as its own object file to speed up compilation, but structuring these files so that they compile and include everything in the correct order becomes a much bigger problem.

anonymousDan• 16 years ago

I'm not a C/C++ expert but I can highly recommend the book "Large-Scale C++ Software Design" by John Lakos. It gives an extensive discussion on how to structure programs and addresses the issue above in detail, in addition to many others. I'd be very interested to hear the opinions of any HN C++ experts on the principles espoused in this book (assuming a few have read it!).

jbn4• 16 years ago

It's a great book with valuable insights, i have successfully used them multiple times to decrease build times of multi-million LoC projects.

drblast4• 16 years ago

Thanks, just what I was looking for!

rbetts2• 16 years ago

Absolutely - a wonderful book!

sixtofour• 16 years ago

Logged in specifically to up vote Lakos.

Will log out and log back in again to attempt a second up vote.

signa11• 16 years ago

if you have not looked at 'c interfaces and implementation' i would heartily recommend it. it is very, very good.

tptacek• 16 years ago

Forget what 'gte910h says. Here's what every project I've worked on professionally has ended up doing:

  #ifndef MYPROJECT_H
  #define MYPROJECT_H

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <unistd.h>
  #include <fcntl.h>
  #include <sys/types.h>
  #include <sys/time.h>
  #include <sys/socket.h>

  #include "trie.h"
  #include "hash.h"

  #endif

Then, in each project file, at the top, just include:

  #include "myproject.h"

There are all sorts of niggling little reasons why you aren't supposed to do this, just like there are niggling little reasons why you're supposed to cast functions that return values to (void) when you don't use their values. Ignore them and get on with your life.

wtracy• 16 years ago

"Thus I say maxphysaddr (not MaximumPhysicalAddress) for a global variable, but np not NodePointer for a pointer locally defined and used."

Yes, because "maxphysaddr" is extremely readable for people who are not native English speakers.

Wait, was that "maxiphysadd" or "maxmphyaddrss"? Better stop coding (breaking up my train of thought) and scroll back up to the variable declaration to double-check how I spelled it.

(Yes, I know that modern IDEs can remember variable names for you, but there's always those emergency situations where you wind up trying to debug something with VIM over SSH via a slow-as-molasses hotel wifi service.)

tedunangst3• 16 years ago

If you can't remember maxphysaddr vs maxiphysadd, I doubt you'll do much better remembering MaxPhysAddr vs MaxiPhysAdd.

chc9• 16 years ago

MaxPhysicalAddress is more readable than all the above. Pike simply had it wrong, IMO, probably from having been immersed in a culture of poor readability for so long that he had become blind to some of it.

drv• 16 years ago

I don't have any first-hand experience with the non-native English aspect, but I doubt such a person can remember how to spell MaximumPhysicalAddress without looking either.

If you stick with a consistent mapping of terms to shortened version ("maximum" always becomes "max") throughout a code base, these names read (and can be typed) much more smoothly than the 30-character monstrosities common in big frameworks.

cma• 16 years ago

ctrl-x p

greyman• 16 years ago

I program in C++ and not C, but in my opinion, unless we are talking about a very small project, it is even wrong to have a global variable, not to mention naming it maxphysaddr! If it is some utility with 50 lines of code, then ok, but otherwise it's a maintenance nightmare to have such names.

As a local variable, it is ok to have "ii" for loops or "p" for pointers, though.

eru6• 16 years ago

Global variables are semi-OK as long as you don't mutate them.

Lots of project have global functions and global classes, and nobody cares.

rauljara• 16 years ago

I think it's helpful to have criteria like this out there, so that we can discuss them. That said, what's with the bias against capital letters? maxphysaddr is more readable than MaxPhysAddr? godisnowhere is more readable than GodIsNowHere? Or is that GodIsNowhere? I'm personally a fan of coding styles that cut down on ambiguity.

amh• 16 years ago

I think "godisnowhere" is more an example of a straight-up badly chosen name (identifiers should not be complete sentences) rather than a case where caps are better than lowercase.

I'm not against camelcase and its variants, but there are a wide variety of naming standards using caps, it's quite possible to get mixed up between them and introduce ambiguity or worse (referring to the wrong variable).

TheNewAndy5• 16 years ago

I believe the intention of this name was the parsing ambiguity of the word "nowhere" when you don't have word delimiters, not as an example of an actual name, or a statement about the location of God :)

jmulho• 16 years ago

I prefer god_is_nowhere. The easiest thing to read is what you are used to. To fairly compare godisnowhere, GodIsNowhere, and god_is_nowhere, you need to stick with each for several days. I bet you'll discover, as I have, that god_is_nowhere is the easiest to read.

TheNewAndy5• 16 years ago

It makes the distinction between god_is_no_where and god_is_nowhere much clearer than GodIsNowhere and GodIsNowHere. A slightly more subjective one is for things with acronyms in them. Compare: check_url_index, CheckURLIndex and CheckUrlIndex.

antirez• 16 years ago

iLoveFunctionLikeThis() but I must admit this is one of the rare case of useless tip in this article. All the rest is almost gold for C programmers.

What is funny is that reading it will hardly make me or you a better programmer because most of this things will take anyway years to be fully understood / mastered, and of course it's an endless process of refinement.

But still very useful to read what we should try to focus on when writing good C code.

georgecmu• 16 years ago

We use capital letters, and so half the timestamp variables are named xxxTimestamp and the other half yyyTimeStamp, which makes it a fun guessing game.

greatfog• 16 years ago

The yyyTimeStamp form could be useful if the program also dealt with postage stamps, rubber stamps, and food stamps.

pgbovine• 16 years ago

''' The following data structures are a complete list for almost all practical programs:

array linked list hash table binary tree '''

this is why it's such a pleasure to program in modern high-level languages (e.g, python, ruby) --- these basic data structures (essentially sequences and mappings) are baked into the language with simple, concise syntax

chc9• 16 years ago

Except for one. Quick: What's the syntax for a binary tree in Python?

OK, I'll give away the answer: It's not there. I don't think binary trees are even in the standard library for Python or Ruby. It's a sadly neglected data structure because, when exposed for what it is, it's a little bit too computer science-y for most people's taste.

archangel_one• 16 years ago

I think it's simpler than that: when you have arrays, linked lists and hash tables there aren't _that_ many cases when you need a binary tree, so I guess the designers felt they weren't worth baking into the language at a syntax level. I agree it would have been nice to have in the standard library though.

jemfinch10• 16 years ago

Except for two, actually: there aren't any linked lists in Python, either.

Why? Because there's only one thing that a linked list is better at than a growable array: O(1) splice. Everything else is better with a growable array.

scott_s• 16 years ago

Balanced binary tree versus hash table is an implementation choice for the associative array abstract data type. Similarly, growable array versus linked list are implementation choices for the list abstract data type.

I don't buy that balanced binary trees are "too computer sciencey for most people's taste" as an argument against not have them as implementations of native types in Python. Designing a good hash table is equally as subtle in terms of both the theory and the implementation itself. But those details are not exposed through the interface.

derefr3• 16 years ago

Python:

    conn = sqlite3.connect('foo')
    curs = conn.cursor()
    curs.execute('create table things (id INTEGER PRIMARY KEY, stuff TEXT)')
    curs.execute('CREATE UNIQUE INDEX tree ON things (id ASC)')

Now "tree" is your tree. ;)

Quite seriously, though, people (that aren't systems programmers) don't think "this is a load of data—I need a binary tree!" People instead think "this is a load of data—I need a database!" and then, when the database isn't actually help their queries go faster, they slap their foreheads and exclaim "oh, wait, I need an indexed database!" Of course, the index just happens to be stored as a b-tree—but they don't know that, and they don't care.

So, to reinterpret, this is basically the data-structure-usage flowchart for the average app programmer:

1. Need quick random access? Use an array.

2. Need quick insertion and removal, or lazy reification (to support, i.e., infinite sequences)? Linked lists, then.

3. Need quick search by key? Hash tables have you covered. (Well, these days, any "hash table" ADT also carries around its keys, so you also get O(N) enumeration of pairs.)

4. And, if your data set makes any of the above solutions choke? Just throw a database at it, and hope it helps.

acqq• 16 years ago

Still, Pike writes from the perspective of a C programmer. Having the pointers and 1-1 mapping between the types and the memory layout of the can combine the principles of the structures in something bigger and more complex. For example, the tree with where some elements are also linked in different lists. I'm not saying that you can't have some logical equivalent in higher languages, but you can't have that expressiveness and efficiency.

contol-m• 16 years ago

There is a lot more in his book, "The Practice of Programming" - http://cm.bell-labs.com/cm/cs/tpop/toc.html

kqueue• 16 years ago

I disagree with him on the include part. It is very difficult to remember the order of includes + dependencies for each file you want to include. It becomes very messy.

__david__• 16 years ago

I agree. I have better things to do than to memorize the random prerequisites of header files. Not to mention cluttering my C source with junk that isn't important to the source in that particular file. My rule is that I should be able to make an empty C file and put a single #include "header.h" line. If it doesn't compile then there is a bug in that .h file.

rbetts2• 16 years ago

One also sees the common advice to include the paired header (ie, foo.c should include foo.h) as the first #include to verify that foo.h is dependency-complete.

The problem with dependency-complete headers is that once a .c file relies on that header for a symbol, if the header is changed to no longer require the symbol, all those .c files have to have their #includes fixed. This can take a lot of time to iterate through in a projects that require hours to compile.

yason2• 16 years ago

I agree with about everything but the best part is the function pointer chapter.

Function pointers are an insanely novel concept. They're not even a C thing per se but since C has first-class pointers they fit right it.

Nevertheless, most design patterns, closures, higher-order functions, more complex dispatching, etc. are effectively just a function pointer and possibly some userdata. You can think in Lisp but do it all in C with function pointers.

viraptor2• 16 years ago

> They're not even a C thing per se

Really? from stdlib.h:

    void
    qsort(void *base, size_t nel, size_t width,
        int (*compar)(const void *, const void *));

yason2• 16 years ago

Not even a C thing per se, as in "C has function pointers but it's not really a C specific feature as some other languages have them too. Like assembly, very obviously."

Many other languages don't. If they're sufficiently high-level they might have function objects or closures, which often cover everything you want. If they're Java they have neither but you can write a Visitor-style class or some other scheme from the design patterns book that emulates the behaviour of a function pointer.

But fundamentally (and when the program is compiled down to machine code also hopefully) they're just function pointers.

eru6• 16 years ago

I agree that the paragraphs about tables driven programs and function pointers are the best part. You can read "Combining datadriven programs with function pointers leads to an astonishingly expressive way of working, a way that, in my experience, has often led to pleasant surprises." as much as an endorsement of functional languages than the endorsement of OO-like techniques it was meant to.

Though C is not really adequate to do Lisp-like programming. Because you can't capture free variables with C function pointers. Of course you can get around this restriction with lambda lifting [1], but why would you want to do the job of the compiler?

As an example, it is quite hard to encode function composition in C, i.e. write a function that takes two function and returns their composition. Or the same specification in (non-idiomatic [2]) Haskell:

  composition :: (b -> c) -> (a -> b) -> a -> c
  composition f g =
      let h a = f (g a)
      in h

I don't know of any way to make up new functions / function pointers in C at runtime.

[1] http://en.wikipedia.org/wiki/Lambda_lifting [2] Idiomatic would be

  composition :: (b -> c) -> (a -> b) -> a -> c
  composition f g a = f (g a)

but that's harder to understand for a C programmer who's not familiar with currying. Of course Haskell also has composition as a build-in.

Or in Scheme (sorry, I am getting carried away with this):

  (define ((composition f g) a)
    (f (g a)))

basman• 16 years ago

The rule against nested includes seems dated — he doesn't like include guards in header files because "[t]he result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase." Surely not any more?

cdavid4• 16 years ago

Indeed. I have read that parsing is the slowest part of compilation several times, but I guess this is very dated. For example, compiling numpy (a python package with ~ 100 kLOC of C code):

  - CFLAGS="-O0" -> ~ 20 sec
  - CFLAGS="-O1" -> ~ 35 sec
  - CFLAGS="-O3" -> ~ 60 sec

Since the amount of parsed code for those modes is approximately the same (for numpy it is exactly the same, but one could imagine differences in glibc headers, etc...), this shows that parsing is not the bottleneck anymore if you build non-debug builds.

mfukar• 16 years ago

No, that shows that the optimizer work takes a non-trivial amount of time. Optimizations (how often?) happen after parsing, type-checking and constructing the intermediate representation.

gte910h7• 16 years ago

Yeah, the parsing argument is bunk anyhow.

The pre-processor parsing is trivial and fast. Its the actual language which is the bulk of the time.

gte910h7• 16 years ago

Notice the date folks: 1989.

Lots of development has gone into C for the better since then.

Rob Pike: Notes on Programming in C

Related Stories