Skip to content

ci: run the fuzz target on PRs #4378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 1, 2024
Merged

ci: run the fuzz target on PRs #4378

merged 2 commits into from
Jul 1, 2024

Conversation

evverx
Copy link
Contributor

@evverx evverx commented May 3, 2024

using https://google.github.io/oss-fuzz/getting-started/continuous-integration/

It downloads the corpus OSS-Fuzz has accumulated so far (including the test cases that triggered issues in the past) and runs the fuzz target with it. It should help to catch most regressions when PRs are opened.

Prompted by #4373.

It's a draft because to make it more useful in terms of testing the dissectors should probably be covered first (The dissectors were (partly) covered in google/oss-fuzz#11912).

As expected it triggered

 === Uncaught Python exception: ===
error: unpack requires a buffer of 2 bytes
Traceback (most recent call last):
  File "pcap_fuzzer.py", line 29, in TestOneInput
  File "scapy/utils.py", line 1259, in rdpcap
  File "scapy/utils.py", line 1319, in __call__
  File "scapy/utils.py", line 1883, in __init__
  File "scapy/utils.py", line 1584, in __init__
  File "scapy/utils.py", line 1648, in _read_block_shb
error: unpack requires a buffer of 2 bytes

==38== ERROR: libFuzzer: fuzz target exited
    #0 0x7f1941d7e7f1 in __sanitizer_print_stack_trace /src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
    #1 0x7f1941c816e8 in fuzzer::PrintStackTrace() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
    #2 0x7f1941c644cc in fuzzer::Fuzzer::ExitCallback() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:248:3
    #3 0x7f1941a318a6  (/lib/x86_64-linux-gnu/libc.so.6+0x468a6) (BuildId: 87b331c034a6458c64ce09c03939e947212e18ce)
    #4 0x7f1941a31a5f in exit (/lib/x86_64-linux-gnu/libc.so.6+0x46a5f) (BuildId: 87b331c034a6458c64ce09c03939e947212e18ce)
    #5 0x7f193fa4eaa8 in Py_Exit /tmp/Python-3.8.3/Python/pylifecycle.c:2299:5
    #6 0x7f193fa534b1 in handle_system_exit /tmp/Python-3.8.3/Python/pythonrun.c:658:9
    #7 0x7f193fa534b1 in _PyErr_PrintEx /tmp/Python-3.8.3/Python/pythonrun.c:668:5
    #8 0x55ec2c838b73  (build-out/pcap_fuzzer.pkg+0x3b73)
    #9 0x55ec2c838f10  (build-out/pcap_fuzzer.pkg+0x3f10)
    #10 0x7f1941a0f082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 87b331c034a6458c64ce09c03939e947212e18ce)
    #11 0x55ec2c8374ad  (build-out/pcap_fuzzer.pkg+0x24ad)

https://github.com/secdev/scapy/actions/runs/8941500042/job/24561958924?pr=4378

Copy link

codecov bot commented May 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.46%. Comparing base (8ed8647) to head (84867e7).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4378      +/-   ##
==========================================
- Coverage   81.46%   81.46%   -0.01%     
==========================================
  Files         353      353              
  Lines       84477    84477              
==========================================
- Hits        68821    68818       -3     
- Misses      15656    15659       +3     
Files Coverage Δ
scapy/layers/dcerpc.py 90.99% <100.00%> (ø)

... and 4 files with indirect coverage changes

@evverx evverx force-pushed the cifuzz branch 2 times, most recently from 5e247e7 to 6558716 Compare May 15, 2024 08:28
@evverx evverx force-pushed the cifuzz branch 2 times, most recently from 000abd7 to 954523e Compare May 23, 2024 08:56
@evverx evverx marked this pull request as ready for review May 23, 2024 09:06
@evverx evverx force-pushed the cifuzz branch 2 times, most recently from 54319af to c7b066e Compare June 15, 2024 20:05
@evverx evverx force-pushed the cifuzz branch 4 times, most recently from 8b2b098 to a1f6988 Compare June 28, 2024 04:23
evverx added 2 commits July 1, 2024 07:37
using https://google.github.io/oss-fuzz/getting-started/continuous-integration/

It downloads the corpus OSS-Fuzz has accumulated so far (including the
test cases that triggered issues in the past) and runs the fuzz target
with it. It should help to catch most regressions when PRs are opened.

Prompted by secdev#4373.
to make it possible to turn it off with logging.disable().

(it should help to make the fuzz target less chatty among other things
because it seems to be the only dissector (covered by the fuzz target)
printing messages like that directly)
@evverx
Copy link
Contributor Author

evverx commented Jul 1, 2024

Currently CIFuzz triggers

AttributeError: 'NoneType' object has no attribute 'val'
Traceback (most recent call last):
  File "pcap_fuzzer.py", line 31, in TestOneInput
  File "scapy/packet.py", line 1688, in summary
  File "scapy/packet.py", line 1662, in _do_summary
  File "scapy/packet.py", line 1662, in _do_summary
  File "scapy/packet.py", line 1662, in _do_summary
  File "scapy/packet.py", line 1665, in _do_summary
  File "scapy/layers/ldap.py", line 736, in mysummary
AttributeError: 'NoneType' object has no attribute 'val'

but since it's reproducible with the master branch it's ignored and the CI is green here as expected because it isn't introduced in this PR:

2024-07-01 07:43:55,579 - root - INFO - Done downloading latest build.
2024-07-01 07:43:55,580 - root - INFO - Trying to reproduce crash using: /tmp/tmpmj6t60wr/crash-32c922e403e265577a7a671701c4a984f455c808.
2024-07-01 07:44:44,733 - root - INFO - Reproduce command returned: 1. Reproducible on /github/workspace/cifuzz-prev-build/pcap_fuzzer.
2024-07-01 07:44:44,734 - root - INFO - The crash is reproducible on previous build. Code change (pr/commit) did not introduce crash.

All in all I think in its current form it can catch shallow issues when PRs are opened.

(Also google/oss-fuzz#12074 was merged so CIFuzz should be more or less in sync with the scapy repository)

Copy link
Member

@gpotter2 gpotter2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this work !

@gpotter2 gpotter2 merged commit 37d9412 into secdev:master Jul 1, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants