In my previous post I walked through CVE-2015-7392, the remote heap buffer overflow in FreeSWITCH's JSON parser that I disclosed in September 2015. What I didn't cover was how the bug was actually found. The short version: I isolated FreeSWITCH's JSON and XML parsers, wrote a small fuzzing harness around them, and let AFL (American Fuzzy Lop) run against them for two weeks. Here's the longer version, including the parts I'd do the same way again.

Why fuzz the parsers

FreeSWITCH takes structured text input in a lot of places, and two parsers stood out as worth a look: its JSON parser (a modified cJSON, with copies in src/switch_json.c, libs/libks/src/ks_json.c and libs/esl/src/esl_json.c) and its XML parser. Parsers are where memory-safety bugs concentrate: they allocate buffers based on attacker-controlled length assumptions, they handle escape sequences and nested structures, and the code paths multiply faster than anyone can review by hand.

Manual review of the JSON parser turned up nothing obvious — the bug that eventually fell out was a one-line interaction between sscanf and pointer arithmetic that was easy to read past. That's exactly the class of bug fuzzing exists for. So instead of staring longer, I decided to make the computer stare.

Isolating the JSON and XML parsers

Fuzzing all of FreeSWITCH wasn't practical: it's a large codebase with threads, networking and configuration to get through before you ever reach a parser. The standard answer is to cut the parsing code out of the project and build it into a small standalone program — a harness — that you can compile with instrumentation and run thousands of times per second.

The extraction was mostly mechanical. I pulled the cJSON-derived code (switch_json.c and its siblings) and the XML parser into their own directory, then stubbed out whatever they depended on from the rest of FreeSWITCH — mainly logging macros and the memory-pool wrappers, which I redirected to plain malloc/free. Where FreeSWITCH used its own allocator with pools, plain libc allocation was actually preferable for fuzzing: it lets AddressSanitizer see the real heap boundaries instead of a pool's.

The goal at this stage isn't elegance. It's a target that compiles cleanly, parses one input file, and exits.

Writing the harness

The harness for the JSON parser was deliberately minimal — read a file, hand it to the parser, walk and free the result:

int main(int argc, char **argv)
{
    char *buf = read_file(argv[1]);   /* slurp the fuzz case */
    if (!buf) return 0;

    cJSON *json = cJSON_Parse(buf);   /* the code under test */
    if (json)
        cJSON_Delete(json);           /* exercise the free path too */

    free(buf);
    return 0;
}

A few things mattered here:

  • Parse and destroy. Calling cJSON_Delete on a successfully parsed tree exercises the cleanup paths, where use-after-free and double-free bugs live. Parsing alone would have missed those.
  • No premature validation. The harness doesn't reject malformed input — that's the parser's job. Every weird byte sequence goes straight in.
  • Return 0 on everything non-fatal. AFL treats crashes (signals) as findings; a parser returning an error for bad input is correct behavior, not a crash.

I built the same style of harness for the XML parser, then compiled both with AFL's instrumentation (afl-gcc) and AddressSanitizer. ASan is what turns a silent heap overflow into a loud, immediate crash at the exact instruction that steps out of bounds — without it, a fuzzer can run for weeks over corrupting inputs that never visibly crash.

Two weeks of AFL

With harnesses built, the setup was routine AFL: a small seed corpus of valid JSON and XML documents — nested objects, arrays, escaped strings, Unicode escapes, numbers in odd formats, deeply nested tags — trimmed down with afl-cmin and afl-tmin so every seed earned its place, then afl-fuzz left to mutate from there.

afl-fuzz -i seeds_json -o findings -- ./json_harness @@

Then I waited. AFL ran on the JSON and XML harnesses for about two weeks. Fuzzing is mostly patience: the status screen cycles through mutations, the queue grows as new coverage is found, and you check in once in a while to see whether the crash counter has moved. For days it didn't. The XML harness produced nothing of interest. And then the JSON harness started collecting crashes — all variations of the same theme, inputs where a string ended in a truncated \u escape.

The crash that mattered

I took the crashing inputs, minimized them with afl-tmin, and reproduced under ASan to get a clean report: a heap-buffer-overflow write in parse_string(), a few bytes past the end of the allocated string buffer.

The root cause was the escape-handling code I had read straight past during manual review. When the parser hit a \u escape, it stepped over the u and called sscanf to convert the following four hex digits — but the pointer advanced by four whenever sscanf returned greater than zero, even if it had only consumed a single character. With a truncated escape like \u1 followed by more string data, the parser walked past the closing quote and kept writing attacker-controlled bytes beyond the buffer it had sized by counting characters between the quotes.

Two weeks of fuzzing had found, deterministically and reproducibly, a bug that human review had missed — and it wasn't just a crash. Because the overflow size and content were controlled by the input, this was a remote code execution candidate, reachable wherever untrusted JSON met the parser.

From crash to CVE

From there the process was standard coordinated disclosure: I reported the issue to the FreeSWITCH project, and disclosed it publicly on Bugtraq on September 14, 2015. It was assigned CVE-2015-7392. Tenable Research later showed the first patch was incomplete and found additional remote vectors — that part of the story, and the fix versions, are in the CVE-2015-7392 post.

Lessons for your own fuzzing

If you're thinking about fuzzing your own code — and if you maintain a parser, you should be — this campaign distilled into a few rules I still use:

  • Isolate before you fuzz. Cutting the parser out of the monolith is boring work, but it's what makes the difference between dozens and thousands of executions per second. Speed is coverage.
  • Always fuzz with sanitizers. AFL finds inputs; AddressSanitizer makes the bugs visible. A heap overflow that silently corrupts memory is a missed finding.
  • Seed with real, valid inputs. The crashing cases were mutations of legitimate JSON with escape sequences. Starting from garbage wastes cycles on inputs the parser rejects on line one.
  • Exercise the full lifecycle. Parse and free. Some of the most exploitable bugs live in cleanup code.
  • Give it time. Two weeks felt long. The crash counter sat at zero for days before the mutation engine walked the input into exactly the right shape. Coverage-guided fuzzing is a marathon by design.
  • Triage every crash to root cause. "It crashes" is not a finding. The finding is the one-line pointer bug, and you can't write a good advisory — or verify a patch — without it.

We apply the same thinking on the consulting side today: manual secure code review for the bugs that require context, and tooling — including fuzzing — for the bug classes that only exhaustiveness can find. The two reinforce each other.

Key takeaways

  • CVE-2015-7392 was found by fuzzing: I extracted FreeSWITCH's JSON and XML parsers into standalone harnesses and ran AFL against them for two weeks.
  • The harness was minimal — parse a file, then free the result — compiled with AFL instrumentation and AddressSanitizer.
  • The winning input family was valid JSON with a truncated \u escape, which exposed a pointer-arithmetic bug in parse_string().
  • Fuzzing found in two weeks what manual review had read straight past — but root-cause analysis of the crash is what turned it into a fixable, disclosable bug.