rgw_sal_motr:[CORTX-33799] Handle progress_cb response in case of copy-object operation failure.

[shriya-deshmukh]

Problem:
If copy operation failed after copying some data, its throwing an invalid XML error due to partial progress response. i.e rgw was not closing 'CopyObjectResult' tag in case of failure.

progress_cb() is not sending copy progress percentage or the total size of the object being copied during the copy operation, but periodically keeps sending copied bytes till now, which is not enough to know copy progress anyways as total obj size not known, and in negative cases, no proper response is sent by Copy object, instead progress_cb() response is seen, which is originally meant as partial response only.

Solution:
Replace progress_cb() call from rgw_sal_motr layer, with dump_continue() signal, which responds with HTTP 100 Continue header to avoid client timeout during the long copy operation.

Behavior comparison with AWS S3 & RGW(RADOS)

1. AWS/RGW not sending the copy progress during copy operation, still no client connection timeout seen.
2. In MGW(Motr) case, progress_cb() is using chunked encoding, which is keeping the client connection alive, though error handling in in negative cases is missing.

Signed-off-by: Shriya Deshmukh [email protected]

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox

[sachitanands]

looks good.
please test with ceph test tool job

[shriya-deshmukh]

Sanity test -https://eos-jenkins.colo.seagate.com/job/Cortx-Automation/job/S3Server/job/S3-Compatibility-Test-Ceph-RGW/195/ (UNSTABLE)
Manual Test result - CORTX-33799-test-logs

[cdeshmukh]

In PR solution, also mention what is the AWS & RADOS behavior and if anything expected from copy progress perspective, like copy % finished or remaining, todays progress_cb doesnt serve that since it doesnt tell what is the total size of object being copied, so remaining copy cant be determined anyways

[shriya-deshmukh]

In PR solution, also mention what is the AWS & RADOS behavior and if anything expected from copy progress perspective, like copy % finished or remaining, todays progress_cb doesnt serve that since it doesnt tell what is the total size of object being copied, so remaining copy cant be determined anyways

done

[sjain09]

Added comment.

[sjain09]

Looks good.