1# Go-118-fuzz-build
2
3
4
5Go-118-fuzz-build is a tool to compile native Golang fuzzers to libFuzzer fuzzers. The tool was initially developed because continuous and CI fuzzing providers have developed platforms that depend on features in fuzzing engines that the native Go engine was not released with. To accomodate this, Go-118-fuzz-build changes the fuzz harnesses into libFuzzer harnesses that can then be intrumented with libFuzzer.
6
7While it is not necessary to run Go-118-fuzz-build in a container, it is recommended, since it will change the source code of the project being built. See THIS_SECTION for an explanation on how it changes the project.
8
9## Table of contents
10- [Workflow](https://github.com/AdamKorcz/go-118-fuzz-build#workflow)
11- [Goals of Go-118-fuzz-build](https://github.com/AdamKorcz/go-118-fuzz-build#goals-of-go-118-fuzz-build)
12- [To-Dos](https://github.com/AdamKorcz/go-118-fuzz-build#to-dos)
13- [Dependencies](https://github.com/AdamKorcz/go-118-fuzz-build#dependencies)
14- [How to use](https://github.com/AdamKorcz/go-118-fuzz-build#how-to-use)
15
16## Workflow
17
18Say we have a native Go fuzz harness as such that we want to build as a libFuzzer fuzzer:
19```go
20package mypackage
21
22import(
23 "testing"
24)
25
26func FuzzMyApi(f *testing.F) {
27 f.Fuzz(func(t *testing.T, name string, raw []byte) {
28 ApiOne(name)
29 ApiTwo(raw)
30 })
31)
32```
33
34To do that, Go-118-fuzz-build does several things.
35
36First, it declares a libFuzzer entrypoint:
37
38```go
39// Code generated by go-118-fuzz-build; DO NOT EDIT.
40// +build ignore
41package main
42import (
43 "runtime"
44 "strings"
45 "unsafe"
46 target "mypackage"
47 "github.com/AdamKorcz/go-118-fuzz-build/testing"
48)
49// #include <stdint.h>
50import "C"
51//export LLVMFuzzerTestOneInput
52func LLVMFuzzerTestOneInput(data *C.char, size C.size_t) C.int {
53 s := (*[1<<30]byte)(unsafe.Pointer(data))[:size:size]
54 defer catchPanics()
55 LibFuzzerFuzzMyApi(s)
56 return 0
57}
58func LibFuzzerFuzzMyApi(data []byte) int {
59 fuzzer := &testing.F{Data:data, T:&testing.T{}}
60 target.FuzzMyApi(fuzzer)
61 return 1
62}
63func catchPanics() {
64 if r := recover(); r != nil {
65 var err string
66 switch r.(type) {
67 case string:
68 err = r.(string)
69 case runtime.Error:
70 err = r.(runtime.Error).Error()
71 case error:
72 err = r.(error).Error()
73 }
74 if strings.Contains(err, "GO-FUZZ-BUILD-PANIC") {
75 return
76 } else {
77 panic(err)
78 }
79 }
80}
81func main() {
82}
83```
84
85You may notice that the libFuzzer harness creates a `github.com/AdamKorcz/go-118-fuzz-build/testing.F{}` which holds the data from the fuzzing engine. This is passed onto the native Go harness. But how does that work, since the native Go harness accepts a std lib `testing.F{}` type? Go-118-fuzz-build changes our target harness from this:
86
87```go
88package mypackage
89
90import(
91 "testing"
92)
93
94func FuzzMyApi(f *testing.F) {
95 f.Fuzz(func(t *testing.T, name string, raw []byte) {
96 ApiOne(name)
97 ApiTwo(raw)
98 })
99)
100```
101
102... to this:
103
104```go
105package mypackage
106
107import(
108 "github.com/AdamKorcz/go-118-fuzz-build/testing"
109)
110
111func FuzzMyApi(f *testing.F) {
112 f.Fuzz(func(t *testing.T, name string, raw []byte) {
113 ApiOne(name)
114 ApiTwo(raw)
115 })
116)
117```
118
119`github.com/AdamKorcz/go-118-fuzz-build/testing"` implements an `f.Fuzz()` that loops through all the parameters in the `f.Fuzz()` function (except the first which is alway a `testing.T` type. For each parameter, Go-118-fuzz-build will create a value of the specified type based on the libFuzzer input. For example, say we want a string and a []byte in our native Go harness, and the libFuzzer testcase is `0x03 0x41 0x42 0x43 0x03 0x44 0x45 0x46`: In that case our `string` will be `"ABC"` and our `[]byte` will be `[]byte(0x44, 0x45, 0x46)`. Go-118-fuzz-build uses [go-fuzz-headers](https://github.com/AdaLogics/go-fuzz-headers) to get the values in `f.Fuzz()`
120
121You will notice that when we change `testing` to `github.com/AdamKorcz/go-118-fuzz-build/testing`, all uses of `testing`, for example `testing.T` and subsequently `t.Skip()`, `t.Error()` will come from the `github.com/AdamKorcz/go-118-fuzz-build/testing` library. As such, Go-118-fuzz-build implements a custom `testing` package that mimics the standard library testing package in a way that makes sense for libFuzzer. You could say that we translate all `testing.T` and `testing.F` methods in ways that make sense in a libFuzzer context. For example, currently, when you fuzz with a native Go harness, `t.Error()` will report an error after the whole fuzz run is over. This is not practical in a libFuzzer fuzz run, so instead we terminate immediately.
122
123## Goals of Go-118-fuzz-build
124The fundamental high-level goal of Go-118-fuzz-build is to provide developers a similar experience when building and running their fuzzers using the native Go engine and libFuzzers engine. It should be as easy to build libFuzzer harnesses as it is to run `go test -c -fuzz=FuzzMyApi`. It is currently not as easy, for the reasons listed below.
125
126## To-Dos
127
128Go-118-fuzz-build does not handle the following cases which causes the build process to not be as seamless as is the goal:
129
130- `_test.go` files are currently not included in the scope of the build. These will have to be manually changes to not end in `_test.go`.
131- Helper functions in separate files will not be modified to accept `github.com/AdamKorcz/go-118-fuzz-build/testing`.T type instead of `testing.T`.
132- The `github.com/AdamKorcz/go-118-fuzz-build/testing` dependency needs to manually be added to the `go.mod` file. This is not always easy, especially with projects using vendoring.
133
134## Dependencies
135Currently, to use this tool, you *must* have `github.com/AdamKorcz/go-118-fuzz-build/testing` in your `go.mod`. The easiest way to do this independently of whether you use vendoring is to create a dummy file that imports `github.com/AdamKorcz/go-118-fuzz-build/testing` somewhere in your project. This can be done with this line:
136
137```bash
138printf "package main\nimport _ \"github.com/AdamKorcz/go-118-fuzz-build/testing\"\n" > register.go
139```
140
141When you then run `go mod tidy`, Go will handle the dependencies.
142
143## How to use
144
145Here we enumerate common ways to use go-118-fuzz-build.
146
147### Preparation
148Any usage requires the go-118-fuzz-build binary:
149
150```bash
151git clone https://github.com/AdamKorcz/go-118-fuzz-build
152cd go-118-fuzz-build
153go build
154mv go-118-fuzz-build $GOPATH/bin/ # or add to $PATH instead
155```
156
157With that, let's discover some ways to use the tool:
158
159### The easy
160A simple fuzzer can be built like this:
161
162```bash
163git clone https://github.com/my/project
164cd project/package1
165printf "package package1\nimport _ \"github.com/AdamKorcz/go-118-fuzz-build/testing\"\n" > registerfuzzdependency.go
166go-118-fuzz-build -o fuzz_archive_file.a -func FuzzMyApi github.com/my/project
167clang -o fuzz_binary fuzz_archive_file.a -fsanitize=fuzzer
168```
169
170### Using test utils from other `*_test.go` files
171go-118-fuzz-build cannot read any `*_test.go` files. These will need to be renamed so they don't end in `_test.go`.
172In this example our fuzzer uses utilities from `utils_test.go`, so we rename that file to `utils_test_fuzz.go`.
173```bash
174git clone https://github.com/my/project
175cd project/package1
176printf "package package1\nimport _ \"github.com/AdamKorcz/go-118-fuzz-build/testing\"\n" > registerfuzzdependency.go
177mv utils_test.go utils_test_fuzz.go # has some functions that we use in our fuzzer.
178go-118-fuzz-build -o fuzz_archive_file.a -func FuzzMyApi github.com/my/project
179clang -o fuzz_binary fuzz_archive_file.a -fsanitize=fuzzer
180```
View as plain text