CORTX-32900 : RGW-SAL: Enable flag M0_ENF_GEN_DI for checksum generation in Motr client

[rajatpatil98]

• RGW-SAL: Enable flag M0_ENF_GEN_DI for checksum generation in Motr client

  [Support is added in Motr client to generate checksum; If M0_ENF_GEN_DI is set then Motr client will generate DI for data
  units along with Parity units. while If M0_ENF_GEN_DI is not set then Motr will not generate the checksum for data units.]

  Signed-off-by: Rajat Patil [email protected]

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox

[rajatpatil98]

Note :This PR is created for early review of RGW-SAL code. After PR approval the merge to main should in sync with Motr changes merge with Motr main.

[bkirunge7]

@andriytk @siningwuseagate @sjain09 @sumedhak27 please review

[siningwuseagate]

A question: what is the overhead of WRITE/READ after setting this flag compared to those without this flag?
From the code, look like this flag is set by default, is this required by RGW? or should it be a configuration option of RGW so users can choose if they want to set it?

[rajatpatil98]

A question: what is the overhead of WRITE/READ after setting this flag compared to those without this flag?

Read/Write is being measured by the component QA and preliminary measurement doesn't show any unexpected deviation.

From the code, look like this flag is set by default, is this required by RGW?

Yes, this was discussed with architect forum and DI will be enabled by default.

or should it be a configuration option of RGW so users can choose if they want to set it?

Yes, Configuration option of enabling/disabling DI is planned for later PI as it will involve other components and their bandwidth. In that configuration option DI will be enabled by default.

[cdeshmukh]

1N deployment had failed earlier, is it because Motr changes are not in main and this PR needs to go along with Motr PR. In that case, custom build with Motr & RGW branches should be confirmed here

[rajatpatil98]

1N deployment had failed earlier, is it because Motr changes are not in main and this PR needs to go along with Motr PR. In that case, custom build with Motr & RGW branches should be confirmed here

Yes, It is because of Motr changes are not there in main.
here is motr PR : Seagate/cortx-motr#1875

And here is build with RGW + Motr :
cortx-docker.colo.seagate.com/seagate/cortx-rgw:2.0.0-7519-custom-ci
cortx-docker.colo.seagate.com/seagate/cortx-all:2.0.0-7519-custom-ci
cortx-docker.colo.seagate.com/seagate/cortx-data:2.0.0-7519-custom-ci
cortx-docker.colo.seagate.com/seagate/cortx-control:2.0.0-7519-custom-ci

Test results on custom build 7519:
All jobs (divided as per feature) for Regression PASSED successfully :
https://eos-jenkins.colo.seagate.com/view/QA-R2/job/QA/job/QA_R2_Single_Runner/2295/ (Bucket Ops, Object Ops and Versioning)
https://eos-jenkins.colo.seagate.com/view/QA-R2/job/QA/job/QA_R2_Single_Runner/2296/ (Blackbox, Data path, Multipart, Copy Object)
https://eos-jenkins.colo.seagate.com/view/QA-R2/job/QA/job/QA_R2_Single_Runner/2298/ (CSM)
https://eos-jenkins.colo.seagate.com/view/QA-R2/job/QA/job/QA_R2_Single_Runner/2299/ (Longevity and HA)

Feature Sanity PASSED :
https://eos-jenkins.colo.seagate.com/job/QA-Sanity-Multinode-RGW/507/

I/O stability Type-1 is IN-PROGRESS, and running since 5 days 20 hr :
https://eos-jenkins.colo.seagate.com/job/QA/job/IOStabilityTestRuns/381/

[cdeshmukh]

And here is build with RGW + Motr :
cortx-docker.colo.seagate.com/seagate/cortx-rgw:2.0.0-7519-custom-ci
cortx-docker.colo.seagate.com/seagate/cortx-all:2.0.0-7519-custom-ci
cortx-docker.colo.seagate.com/seagate/cortx-data:2.0.0-7519-custom-ci

@rajatpatil98 thanks, can you also share link to custom build job which generated 7519 build

[cdeshmukh]

retest this please