Integrating fuzzing by way of OSS-Fuzz

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Integrating fuzzing by way of OSS-Fuzz

david korczynski
Hi Lua team,

I have worked on Lua security recently with the intentions of setting up
continuous fuzzing of Lua by way of OSS-Fuzz. The goal is to use
automated test-case generation by way of fuzzing to catch any
undesirable bugs in Lua.

OSS-Fuzz is a service provided by Google
(https://github.com/google/oss-fuzz) that performs continuous fuzzing of
important open source projects. In essence, the idea is that you can
integrate a project by writing fuzzers for it and then have Google run
the fuzzers over and over again. This is an excellent way to find bugs
and security vulnerabilities in projects, and many well-known open
source projects are integrated (see full list here:
https://github.com/google/oss-fuzz/tree/master/projects).

Once a bug is found, an email is sent with information about bug
details, e.g. stack trace, trigger input and sanitizers reports, which
is very helpful in the root-cause analysis process. The only caveat is
that there is a 90 day disclosure time, so bugs will be made visible to
the public within after 90 days of being found. It also possible to set
it up such that bugs are made visible to all instantly when they are found.

I have done the necessary work to integrate Lua into OSS-Fuzz, meaning I
have written a fuzzer for Lua as well as the infrastructure necessary to
integrate into OSS-Fuzz. You can find this logic in a PR on the OSS-Fuzz
repository here: https://github.com/google/oss-fuzz/pull/4653

Would you be happy to integrate Lua into this project? If so, the only
thing I would need is an email(s) that will receive the bug-reports, or,
alternatively a "go" from the maintainers that bugs should be made
visible to the public when found. I am happy to maintain the fuzzer and
infrastructure from the OSS-Fuzz side of things.

Let me know what you think.

Kind regards,

David

ADA Logics Ltd is registered in England. No: 11624074.
Registered office: 266 Banbury Road, Post Box 292,
OX2 7DL, Oxford, Oxfordshire , United Kingdom
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

Roberto Ierusalimschy
> I have worked on Lua security recently with the intentions of setting up
> continuous fuzzing of Lua by way of OSS-Fuzz. The goal is to use
> automated test-case generation by way of fuzzing to catch any
> undesirable bugs in Lua.
>
> [...]
>
> Would you be happy to integrate Lua into this project? If so, the only
> thing I would need is an email(s) that will receive the bug-reports, or,
> alternatively a "go" from the maintainers that bugs should be made
> visible to the public when found. I am happy to maintain the fuzzer and
> infrastructure from the OSS-Fuzz side of things.

Sounds interesting, bug how bugs are defined, or who decides that some
specific behavior is a bug?

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

david korczynski
In this case bugs are defined as the sanitizers integrated into LLVM,
which essentially revolves around memory corruption bugs. Example bugs
include overflows of various sorts, such as:
- stack-based buffer overflows
- heap-based buffer overflows
- global-buffer overflows

Other memory corruption bugs are also detected such as use-after frees,
double frees and bugs of this nature. The sanitizers can also detect
memory leaks as well and undefined behaviour such as signed integer
overflows.

In case you are unfamiliar with sanitizers, these are the ones I am
speaking of:

Address sanitizer: https://clang.llvm.org/docs/AddressSanitizer.html
Memory sanitizer: https://clang.llvm.org/docs/MemorySanitizer.html
Undefined behaviour sanitizer:
https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

Kind regards,

David

On 19/11/2020 13:45, Roberto Ierusalimschy wrote:

>> I have worked on Lua security recently with the intentions of setting up
>> continuous fuzzing of Lua by way of OSS-Fuzz. The goal is to use
>> automated test-case generation by way of fuzzing to catch any
>> undesirable bugs in Lua.
>>
>> [...]
>>
>> Would you be happy to integrate Lua into this project? If so, the only
>> thing I would need is an email(s) that will receive the bug-reports, or,
>> alternatively a "go" from the maintainers that bugs should be made
>> visible to the public when found. I am happy to maintain the fuzzer and
>> infrastructure from the OSS-Fuzz side of things.
> Sounds interesting, bug how bugs are defined, or who decides that some
> specific behavior is a bug?
>
> -- Roberto
> ADA Logics Ltd is registered in England. No: 11624074.
> Registered office: 266 Banbury Road, Post Box 292,
> OX2 7DL, Oxford, Oxfordshire , United Kingdom
ADA Logics Ltd is registered in England. No: 11624074.
Registered office: 266 Banbury Road, Post Box 292,
OX2 7DL, Oxford, Oxfordshire , United Kingdom
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

Roberto Ierusalimschy
> In this case bugs are defined as the sanitizers integrated into LLVM,
> which essentially revolves around memory corruption bugs.
> [...]

As far as I can remember, the only issue we've had with sanitizers
is float-divide-by-zero, because Lua assumes IEEE behavior (NaN
result). There are other options that complain about standard behavior
used by Lua (e.g., unsigned-integer-overflow, unsigned-shift-base).
Can we assume they will not be used?

Another recent issue we had with some sanitizer (or maybe it was
a static analyzer?)  was the read of an uninitialized union. When
we use a tagged union, it is common that some tags don't need any
field in the union. The C standard is somewhat vague about this case
(trap representation vs unions), but it does not make sense to
initialize some arbitrary field only to satisfy a tool.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

david korczynski
Yes, we can disable the use of undefined behavior sanitizer to avoid
errors like the arithmetic ones. About the use of uninitialised union
then I believe we can also discard those, but naturally we can ignore
any bugs if we don't find them relevant. This will not stop the fuzzers
of OSS-Fuzz in looking for more interesting bugs.

The read of uninitialized union I am pretty sure we can also avoid

On 19/11/2020 15:22, Roberto Ierusalimschy wrote:

>> In this case bugs are defined as the sanitizers integrated into LLVM,
>> which essentially revolves around memory corruption bugs.
>> [...]
> As far as I can remember, the only issue we've had with sanitizers
> is float-divide-by-zero, because Lua assumes IEEE behavior (NaN
> result). There are other options that complain about standard behavior
> used by Lua (e.g., unsigned-integer-overflow, unsigned-shift-base).
> Can we assume they will not be used?
>
> Another recent issue we had with some sanitizer (or maybe it was
> a static analyzer?)  was the read of an uninitialized union. When
> we use a tagged union, it is common that some tags don't need any
> field in the union. The C standard is somewhat vague about this case
> (trap representation vs unions), but it does not make sense to
> initialize some arbitrary field only to satisfy a tool.
>
> -- Roberto
> ADA Logics Ltd is registered in England. No: 11624074.
> Registered office: 266 Banbury Road, Post Box 292,
> OX2 7DL, Oxford, Oxfordshire , United Kingdom
ADA Logics Ltd is registered in England. No: 11624074.
Registered office: 266 Banbury Road, Post Box 292,
OX2 7DL, Oxford, Oxfordshire , United Kingdom
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

Roberto Ierusalimschy
> Yes, we can disable the use of undefined behavior sanitizer to avoid
> errors like the arithmetic ones. About the use of uninitialised union
> then I believe we can also discard those, but naturally we can ignore
> any bugs if we don't find them relevant. This will not stop the fuzzers
> of OSS-Fuzz in looking for more interesting bugs.
>
> The read of uninitialized union I am pretty sure we can also avoid

I am afraid we left this unfinished. I'm sorry.

We are always in favor of finding bugs :-)  What exactly do we have
to do?

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Integrating fuzzing by way of OSS-Fuzz

david korczynski
I just need a set of emails that will receive access to the bug reports :)

These emails will be in the `project.yaml` configuration file on the
OSS-Fuzz repository, you can see the PR here:
https://github.com/google/oss-fuzz/pull/4653

I will also put in my own email so I can see if errors happen in the set
up, and thus fix them if need be.


On 26/11/2020 22:44, Roberto Ierusalimschy wrote:

>> Yes, we can disable the use of undefined behavior sanitizer to avoid
>> errors like the arithmetic ones. About the use of uninitialised union
>> then I believe we can also discard those, but naturally we can ignore
>> any bugs if we don't find them relevant. This will not stop the fuzzers
>> of OSS-Fuzz in looking for more interesting bugs.
>>
>> The read of uninitialized union I am pretty sure we can also avoid
> I am afraid we left this unfinished. I'm sorry.
>
> We are always in favor of finding bugs :-)  What exactly do we have
> to do?
>
> -- Roberto
> ADA Logics Ltd is registered in England. No: 11624074.
> Registered office: 266 Banbury Road, Post Box 292,
> OX2 7DL, Oxford, Oxfordshire , United Kingdom
ADA Logics Ltd is registered in England. No: 11624074.
Registered office: 266 Banbury Road, Post Box 292,
OX2 7DL, Oxford, Oxfordshire , United Kingdom