Log and Screenshots are not following correct order as pushed to the scenario

[devqaio]

Describe the bug
As part of test, we push the logs and screenshots for a specific step in below order. The default out-of-box cucumber html report populates the details in that order

Log1
Screenshot1
Log2
Screenshot2

However, the generated report populates in below order
Log1
Log2
Screenshot1
Screenshot2

To Reproduce
Steps to reproduce the behavior:
Call below statements in step definition in multiple orders
ScenarioObject.log("MESSAGE : " + message);
ScenarioObject.attach(screenshot, "image/png", "");

Expected behavior
The scenario report will publish the entries in the order it was logged

Expected (From Cucumber Default HTML Report)

Actual:

Sample Json

1.json

[mpkorstanje]

What is the order in the .json file?

[devqaio]

@mpkorstanje : Didn't check the Json earlier. As I see, the Json has all messages under one json array and all embedding in a different json array. Could it be the reason for this?

The Cucumber HTML is capturing in proper sequence. Could be the logic for HTML plugin is different than that of Json with in Cucumber

I have logged cucumber/cucumber-jvm#3109 to understand the view of cucumber maintainers.

[mpkorstanje]

I'm the maintainer of Cucumber JVM. 😉

@bischoffdev logs and embeddings are shown in order in the cucumber-html report because it consumes cucumber messages. Which treats both of these as attachments.

Aside from starting to consume Cucumber messages, I don't think there is anything you can do to address this.

[bischoffdev]

Yes, if the order or any kind of timestamp or running id is not reflected in the JSON, this is not possible with the JSON dependent version of Cluecumber. For Cluecumber core, it would be possible to migrate it to messages eventually, but in Cluecumber Maven, which runs as a completely separate step after any test framework, this would not help.

[mpkorstanje]

For Cluecumber core, it would be possible to migrate it to messages eventually, but in Cluecumber Maven, which runs as a completely separate step after any test framework, this would not help.

You can swap json:target/report.json with message:target/report.ndjson.

[bischoffdev]

Ok, then this would be a major release... That would mean it would lose compatibility with Karate 🫣

[mpkorstanje]

Time to ask Peter Thomas if he wants to support Cucumber Messages. 😅

[mpkorstanje]

Done: karatelabs/karate#2724

[devqaio]

Yes, if the order or any kind of timestamp or running id is not reflected in the JSON, this is not possible with the JSON dependent version of Cluecumber. For Cluecumber core, it would be possible to migrate it to messages eventually, but in Cluecumber Maven, which runs as a completely separate step after any test framework, this would not help.

If we switch to the messages in core in upcoming major release, can we not point to messages in maven plugin to make it work?

[bischoffdev]

If we switch to the messages in core in upcoming major release, can we not point to messages in maven plugin to make it work?

I am not sure I understand the question, @devqaio