Fuzz testing

[mkg20001]

• Version: hopefully the next
• Platform: *
• Subsystem: *

Type: enhancment / question

Severity: medium

Description: Fuzz Testing

I have discovered many critical application crashes by accident after my node was crashed by the invalid data it recived.

Such kinds of crashes are likely unacceptable in production.
Discovering them by accident isn't really a solution to that. #1292 libp2p/js-libp2p#179

So I propose to add some tests that just bombard the node with random semi-valid or invalid data. Aka fuzz testing.

[daviddias]

//cc @fsdiogo

[fsdiogo]

Where do you guys suggest I start? I'm a bit lost to be honest.

[daviddias]

@fsdiogo are you familiar with the concept of fuzz testing -- https://en.wikipedia.org/wiki/Fuzzing --? It will be powerful for your goal of #1325.

@pgte and @satazor should/might have a lot of ideas to help you here :)

[fsdiogo]

Yes I'm familiar with the concept, although have never implemented it.

My question was a bit more specific, should I start in js-ipfs, libp2p?

What do you mean when a node receives invalid data?

[lidel]

I will go on a limb here and say that the low hanging fruit in js-ipfs is CLI and HTTP API. It may be easier to reason about creating fuzzing tests for them plus user/developer experience will directly benefit from that.

Fuzzing around messaging between nodes should happen in libp2p project, I think.

[mkg20001]

Fuzzing around messaging between nodes should happen in libp2p project, I think.

I have to disagree with that as for example #1292 clearly was an IPFS specific issue, although generic safeguards for not crashing the node should be implemented in libp2p. (for ex wrapping the libp2p protocol handler in a try/catch and printing to stderr instead of crashing)