MinUnit – A minimal unit testing framework for C (2002)

http://www.jera.com/techinfo/jtns/jtn002.html

by hazz99 • 8 years ago

108 55 8 years ago

pavlov2• 8 years ago

So, the revelation is that a unit test is an if statement, and if you wrap that into a macro you can call it a framework.

ticviking• 8 years ago

For many novice engineers this is a huge revelation.

pavlov2• 8 years ago

Really? Is it perhaps because of those libraries that do their best to make test call sites look like weird pseudo-English. Stuff like having a library function named it():

it(“should be confusing”, function() { ... })

phkahler• 8 years ago

And any small set of functions is an API.

hinkley• 8 years ago

Except that’s not a whole test any more than the punchline is the joke.

It’s sample data, verification, analysis, and reporting. The if statement is just the verification step.

davydog1872• 8 years ago

Whats up with the do/while loops? Does that just make the return syntactically valid?

Apologies, its been a while since I've touched C/C++

pdexter• 8 years ago

https://stackoverflow.com/questions/154136/why-use-apparentl...

tom_3• 8 years ago

This is dealt with at the bottom of the article. It's a C FAQ: http://c-faq.com/cpp/multistmt.html

okl2• 8 years ago

GCC has an extension that can be used instead of the do-while loop, see example macro definition here: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html

jwilk2• 8 years ago

(2002) according to the HTTP headers:

    $ curl -s --head http://www.jera.com/techinfo/jtns/jtn002.html | grep ^Last
    Last-Modified: Fri, 06 Sep 2002 07:16:46 GMT

vortico2• 8 years ago

Why not just 1 line? Of course, the application will halt after one test fails, but some people like it this way.

    #include <assert.h>

Usage:

    void test_foo() {
        assert(foo() == 4 /* foo should be 4 */);
    }

Output would be something like

    Assertion failed at "foo() == 4 /* foo should be 4 */"

If you run the application in a debugger like GDB, you can see the frame when the abort trap is called.

jwilk2• 8 years ago

Comment won't be included the assertion message.

If you want to put arbitrary string there, you need something like this:

    assert("foo should be 4" && (foo() == 4));

vortico2• 8 years ago

Ah, thanks, I had forgotten.

stevefan1999• 8 years ago

I was about the point this out! At the end of the day, unit testing is just contract assertions, and contract assertions are usually conditions! But actually this isn't really the crux of unit testing, the value of unit testing is to iterate and refactor your program for a more predictable and bug-free application, not just writing a bunch of conditions.

simias3• 8 years ago

I actually implemented a basic unit test framework in C yesterday and started with basic asserts like you point out, however once I got it working I switched to return values not unlike TFA. The obvious problem with assertions in that they kill your test program immediately at the first failure instead of doing additional tests and potentially give a clearer picture of what's broken exactly. They're also not very flexible if you want to customize the error message since they only stringify the assert parameter. "Assertion `ret == 0' failed." is not super helpful. As sibling comments mention there are ways to work around that by using literal strings and && but it doesn't help if you want to wrap around tests using helper functions etc...

I'm also tempted to say that if you need to use GDB to figure out where your tests have failed exactly your framework is not particularly user friendly. Normally when a test fails it should be pretty clear where it failed and why. The root cause of the issue might be further away of course but then again assertions won't help with that either.

ainar-g2• 8 years ago

I mostly agree with you, but I would like to make something clear.

I'm also tempted to say that if you need to use GDB to figure out where your tests have failed exactly your framework is not particularly user friendly.

Am I missing something?

  #include <assert.h>
  
  int main(void)
  {
  	assert(0);
  	return 0;
  }

When using clang I get a clear message that includes the filename, the line, and the function:

  assert.bin: assert.c:5: int main(void): Assertion `0' failed.

kernelbandwidth• 8 years ago

Assertions don't actually have to kill the program. They send an abort signal, which means you can catch them with a signal handler.

I similarly wrote a simple C testing framework for a little project I was working on, based on assertions. The framework uses signal handlers for aborts and segfaults, and then setjmp/longjmp to handle resuming the test suite at the next test case. This has the particularly nice effect of turning segfaults into (marked as such) test failures, instead of just terminating everything. It probably wouldn't be too hard to fit custom messages in too, but I hadn't felt the need yet.

tom_3• 8 years ago

One thing I'm pretty doctrinaire about when it comes to this sort of thing is printing out more than just a simple message. Quite often, this makes the problem obvious with no need for deeper investigation.

To do this I have a bunch of macros like this:

    /* check A and B are equal. */
    #define EQ_II(A,B,M) (CheckEQII((A),(B),M,#A,#B,__FILE__,__LINE__))

You use it like this:

    EQ_II(i,3,"blah blah blah");

CheckEQII looks roughly like this:

    void CheckEQII(int64_t a,int64_t b,const char *message,const char *a_str,const char *b_str,const char *file,int64_t line) {
        if(a!=b) {
            printf("%s:%" PRId64 ": test failed: %s\n",file,line,message);
            printf("    Values not equal.\n");
            printf("    Got expr    : %s\n",a_str);
            printf("    Wanted expr : %s\n",b_str);
            printf("    Got value   : %" PRId64 " (0x%" PRIx64 ")\n");
            printf("    Wanted value: %" PRId64 " (0x%" PRIx64 ")\n");
            DEBUG_BREAK();
            exit(1);
        }
    }

(DEBUG_BREAK breaks into the debugger if you're running in the debugger.)

The FILE:LINE notation is probably clickable in your favourite text editor. (For VC++, use "FILE(LINE):". Just do #ifdef _MSC_VER or something.) Very convenient if you run tests as part of the build.

And you can flesh it out for strings, arrays, floats, doubles, and all the rest. You can fit everything you need into about 500 lines.

This isn't quite as impressive as the 3 lines here, but compared to something like Catch - which is a huge amount of C++ code, crazy C++ code to boot, that adds literally seconds to your build time - and, no, the fact that seconds is a drop in the ocean in C++land is not an excuse - it's in the same ballpark. At least, its extra utility should prove, over the course of a project, in my view, commensurate with the extra LOC.

masklinn• 8 years ago

That's why I love pytest.

You just `assert` an expression, and on failure it'll break down each sub-expression and show whatever it yielded, plus by default it captures all output (stdout and stderr) and prints it out only on failure (so logging and other console output is not painfully annoying while testing) and the fixtures systems is just fantastic and parameterised tests make for much clearer data-driven (/table-driven) tests and the tests selection at the CLI is awesome.

And all the complexity is in the test framework and runner, all you do is write

    def test_thing(a_fixture):
        assert foo(bar) == a_fixture.some_thing()

or whatever.

makmanalp• 8 years ago

On top of that, it'll compare complex structures to each other, so you could have a very large list of dicts that contain a custom data structure and it'll show you a diff and exactly where the mismatch is. Then it can auto-drop to the debugger and you can inspect previous variables to inspect the root cause, if not immediately apparent. Going back to C land, as much as I enjoy it, the tooling feels like the stone ages. I get that it's a lot harder for a language that compiles down to machine code, but dang.

gHosts• 8 years ago

pretty doctrinaire about

And that is the problem.

I have met teams that have gone into analysis paralysis about which unit test framework to use.

I have met teams where one of the impediments to unit testing was "we don't have time to learn something that complicated, we don't understand why all this stuff is necessary".

Answer:

It isn't necessary. Get off your ass and start unit testing with the simplist framework possible.

It's more important, way way more important to just do it than fuss about the framework.

Like TFA.

Once you have actually done that for awhile, someone will say, "Wouldn't it be nice if..."

And _that's_ the time to introduce a framework. Or grow what you have.

On a C project I elected to grow, the nice thing about it is test setup and teardown? It's exactly the same thing as process set up and teardown.

So valgrind will tell you _exactly_ if anything leaked or was uninitialized!

tom_3• 8 years ago

Oops, missed this at the time. Rant on... though only for posterity.

There's various reasons why one might be doctrinaire about stuff. One common reason: you're an arsehole. Another one: you're an engineer, and this was what you were trained to do. A bonus third: you had it beaten into you by bitter experience.

Me? Well, I will say that I'm not clever enough to be an engineer.

crescentfresh• 8 years ago

I'm not very familiar with C and macros so it took me a while to realize that this line:

  mu_assert("error, bar != 5", bar == 5);

will return out of the function before reaching the next line.

For anyone else like me, I put together the "inlined" version of the code to help me understand what is happening:

  #include <stdio.h>
  #include "minunit.h"

  int tests_run = 0;
   
  int foo = 7;

  static char * test_foo() {
    do {
      if (!(foo == 7))
        return "error, foo != 7";
    } while (0);
    return 0;
  }

  static char * all_tests() {
    do {
      char *message = test_foo();
      tests_run++;
      if (message)
        return message;
    } while (0);

    // ... more tests here

    return 0;
  }

giancarlostoro3• 8 years ago

And then in D it's built-in to the compiler so it's a 1 liner of sorts:

unittest { ... }

and they all run during compilation.

https://dlang.org/spec/unittest.html

tonyedgecombe• 8 years ago

I'm surprised that isn't used in other languages, having the test right next to the code seems such a good idea.

giancarlostoro3• 8 years ago

Rust went this path too, there's something I love about having unittesting readily available. I can choose to use it or not use it. My asserts are available to me as I so desire. Python too sorta has some unittesting in the standard library, but that's not a compiler level thing.

blt2• 8 years ago

what happens if you write a really slow test? is there some way to take advantage of the inbuilt console reporting, etc. without running them after every compile?

giancarlostoro3• 8 years ago

It's not compiled into your binary unless you want it to be. It's ran at compile time every time you compile, though I'm sure there's a flag to not run unit testing, and maybe with version() you might be able to block out the time ocnsuming tests. Not sure what you'd do in D that'd be so time consuming though.

Edit:

URL to version() docs:

https://dlang.org/spec/version.html

Yuioup• 8 years ago

I did something like that once in Classic ASP (VBScript). I think I only wrote two Assert functions and that was enough. The code ended up being really robust after I wrote a bunch of test cases that eventually passed (after writing the code of course).

VictorSCushman2• 8 years ago

I have a library [link redacted] that was originally designed to extend MinUnit. I love the idea of an all-in-one unit testing library without complicated set up.

mikestew2• 8 years ago

Well, there's a fine welcome from HN: new account that's fifteen minutes old, which posts what could be useful information (I just skimmed it) in a non-controversial manner and...insta-dead.

I obviously vouched for the comment, but yeesh.

sctb• 8 years ago

I hear you, but since we don't have software that can determine whether posts follow the guidelines, sometimes (especially for brand new accounts) the anti-abuse software gets it wrong so then we have vouching.

VictorSCushman2• 8 years ago

Thank you :)

okl2• 8 years ago

For everybody who demands from their test framework that it prints expected vs actual values and file/line references, and that it can be adapted to embedded and bare metal targets I recommend the excellent Unity framework[1], example[2].

[0] http://www.throwtheswitch.org/unity/

[1] https://gitlab.com/oliver117/blut/blob/master/test/test_math...

bfrog• 8 years ago

Libcheck does the nice thing and let's you optionally fork each test resulting in parallelism and test failures from segfaults and such not causing the whole suite failing. Does this do something similar?

blt2• 8 years ago

I did mine using assert.h and a signal handler. More code, but advantage is you can also use assert.h internally anywhere in your program without it needing to know about the test framework. On other hand, makes testing optimized builds difficult.

hector_ka• 8 years ago

Works great in Embedded Linux, and it is easy to crosscompile.

gon1332• 8 years ago

Well, big minds meet.

Here is mine Minimal Unit Testing FW for C:

https://github.com/gon1332/minut

codeplea• 8 years ago

Why stick to three lines? With only a couple more you could reduce the boiler-plate needed to use it.

I wrote a minimal C test framework too: https://github.com/codeplea/minctest It's got some real-world usage. Still only just one header file, but it'll time each test and if an equal assertion fails it'll print the variable values.

RhysU• 8 years ago

FCTX is in a similar space, and quite good: https://github.com/imb/fctx/blob/master/README.rst

makecheck• 8 years ago

The examples suggest that maybe the implementation should use “#str” in the macros (i.e. when you want your string to contain the same thing as the expression itself).

MinUnit – A minimal unit testing framework for C (2002)

Related Stories