1# Test Flatbuffers library with help of libFuzzer
2Test suite of Flatbuffers library has fuzzer section with tests are based on libFuzzer library.
3
4> LibFuzzer is in-process, coverage-guided, evolutionary fuzzing engine.
5LibFuzzer is linked with the library under test, and feeds fuzzed inputs to the library via a specific fuzzing entrypoint (aka “target function”);
6the fuzzer then tracks which areas of the code are reached, and generates mutations on the corpus of input data in order to maximize the code coverage.
7The code coverage information for libFuzzer is provided by LLVM’s SanitizerCoverage instrumentation.
8
9For details about **libFuzzer** see: https://llvm.org/docs/LibFuzzer.html
10
11To build and run these tests LLVM compiler (with clang frontend) and CMake should be installed before.
12
13The fuzzer section include four tests:
14- `annotator_fuzzer` checks that inputs given to the flatc --annotate are always parsable;
15- `verifier_fuzzer` checks stability of deserialization engine for `Monster` schema;
16- `parser_fuzzer` checks stability of schema and json parser under various inputs;
17- `scalar_parser` focused on validation of the parser while parse numeric scalars in schema and/or json files;
18- `flexverifier_fuzzer` checks stability of deserialization engine for FlexBuffers only;
19
20## Build
21```sh
22cd tests/fuzzer
23CC=clang CXX=clang++ cmake . -DCMAKE_BUILD_TYPE=Debug -DUSE_ASAN=ON
24```
25
26## Run tests with a specific locale
27The grammar of the Flatbuffers library is based on printable-ASCII characters.
28By design, the Flatbuffers library should be independent of the global or thread locales used by an end-user application.
29Set environment variable `FLATBUFFERS_TEST_LOCALE` to run a fuzzer with a specific C-locale:
30```sh
31>FLATBUFFERS_TEST_LOCALE="" ./scalar_parser
32>FLATBUFFERS_TEST_LOCALE="ru_RU.CP1251" ./parser_fuzzer
33```
34
35## Run fuzzer
36These are examples of running a fuzzer.
37Flags may vary and depend on a version of the libFuzzer library.
38For details, run a fuzzer with `-help` flag: `./parser_fuzzer -help=1`
39
40`./verifier_fuzzer ../.corpus_verifier/ ../.seed_verifier/`
41
42`./parser_fuzzer -only_ascii=1 -max_len=500 -dict=../parser_fbs.dict ../.corpus_parser/ ../.seed_parser/`
43
44`./monster_fuzzer -only_ascii=1 -max_len=500 -dict=../monster_json.dict ../.corpus_monster/ ../.seed_monster/`
45
46`./scalar_fuzzer -use_value_profile=1 -max_len=500 -dict=../scalar_json.dict ../.corpus_scalar/ ../.seed_scalar/`
47
48Flag `-only_ascii=1` is useful for fast number-compatibility checking while run `scalar_fuzzer`.
49
50Run with a specific C-locale:
51`FLATBUFFERS_TEST_LOCALE="ru_RU.CP1251" ./scalar_fuzzer -reduce_depth=1 -use_value_profile=1 -shrink=1 -max_len=3000 -timeout=10 -rss_limit_mb=2048 ../.corpus_parser/ ../.seed_parser/`
52
53
54## Merge (minimize) corpus
55The **libFuzzer** allow to filter (minimize) corpus with help of `-merge` flag:
56> -merge
57 If set to 1, any corpus inputs from the 2nd, 3rd etc. corpus directories that trigger new code coverage will be merged into the first corpus directory.
58 Defaults to 0. This flag can be used to minimize a corpus.
59
60Merge several corpuses to a seed directory (a new collected corpus to the seed collection, for example):
61`./verifier_fuzzer -merge=1 ../.seed_verifier/ ../.corpus_verifier/`
62`./parser_fuzzer -merge=1 ../.seed_parser/ ../.corpus_parser/`
63`./monster_fuzzer -merge=1 ../.seed_monster/ ../.corpus_monster/`
64`./scalar_fuzzer -merge=1 ../.seed_scalar/ ../.corpus_scalar/`
65
66## Know limitations
67- LLVM 7.0 std::regex library has problem with stack overflow, maximum length of input for `scalar_fuzzer` run should be limited to 3000.
68 Example: `./scalar_fuzzer -max_len=3000`
69
70# Fuzzing control
71
72## Set timeout or memory limit
73
74`-timeout=10 -rss_limit_mb=2048 -jobs=4 -workers=4`.
75
76## Force stop on first UBSAN error
77
78- `export UBSAN_OPTIONS=halt_on_error=1`
79- `export ASAN_OPTIONS=halt_on_error=1`
View as plain text