What afl-fuzz is bad at

http://blog.regehr.org/archives/1238

by doty • 11 years ago

54 10 11 years ago

blog.regehr.org

I don't see why afl-fuzz needs to be turned into a general-purpose Swiss Army Knife framework. It excels quite well at mutation-based techniques that train themselves on resulting outputs. For problems that involve specifically triggering code branches, then a tool that employs symbolic execution methods via constraint solving should be used instead, as the author implies.

f-• 11 years ago

(afl-fuzz author here)

The basic idea for AFL is that it's supposed to be reliably better than dumb fuzzers without requiring you to think too much about the problem space, fuzzing settings, grammar definitions, harness design, etc.

Anything that makes it retain these properties while improving results is probably worth pursuing. Conversely, anything that sounds cool but ultimately doesn't meet that criteria is probably a better fit for other tools.

regehr• 11 years ago

The "without requiring you to think too much about the problem space" is pretty huge and it is where solver-based testing currently falls over (in my experience).

dfc3• 11 years ago

I wish there was an afl-fuzz like tool for fuzzing network traffic. By afl-fuzz for network traffic I mean a general purpose network fuzzer; not protocol specific fuzzer like the codenomicon ipsec fuzzer. Does anyone know of anything close? Remotely close?

There seems to be a significant overlap in what you would want from a network protocol fuzzer and a tool to reverse engineer a network protocol. Netzob is the only protocol RE tool that I know of and it seems that development has stalled.

yeukhon• 11 years ago

But what exactly are you fuzzing without being protocol specific as you start fuzzing? How do you eliminate false positive? You can be generative with grammar or with some learning capability (as "dumb" as aligning similar traffic as a cluster for example).

dfc3• 11 years ago

I am not sure how you are distinguishing my theoretical protocol agnostic fuzzer from afl. Is afl a jpg/png specific fuzzer? One of the things that is great about afl is that it is not targeted towards a specific file format it can work with a small corpus or thorough dictionary of the target format. eg I can give afl-fuzz a snippet of two of markdown and it will go nuts or if I want to be exhaustive I can feed afl the afl "dictionary" from the commonmark testsuite.

How do you ever eliminate false positives when fuzzing? And what constitutes a false positive?

detaro• 11 years ago

This was posted a few days back, I don't know if you've seen it? https://lolware.net/2015/04/28/nginx-fuzzing.html (discussion: https://news.ycombinator.com/item?id=9463785)

jagger11• 11 years ago

You can try https://github.com/google/honggfuzz - The basic idea is pretty much the same as with afl-fuzz - maximization of code coverage (more or less).

Some old write-up on the technique can be found here: https://github.com/google/honggfuzz/blob/wiki/AttachingToPid....

You can attach to a process (-p pid) and then feed it with an external, initial input (from pcap, or hand-crafted). Honggfuzz will modify it to maximize code coverage in the network server. I got pretty decent results with e.g. apache (in must be executed with -X, so it doesn't fork/daemonize).

What afl-fuzz is bad at

Related Stories