Skip to content

Segmentation fault in Aws::S3Crt::S3CrtClient::UploadPartCallable #1654

Closed
@Johnparke888

Description

@Johnparke888

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Describe the bug
A clear and concise description of what the bug is.
Attempting to do a multipart upload using the new S3Crt client. I started with a working version of code using the S3 client. When the code executes Aws::S3Crt::S3CrtClient::UploadPartCallable, I get a segmentation fault.
Address sanitizer indicates an attempt to read from memory address which points to the zero page:
==11503==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0000005888f3 bp 0x60300023b7b0 sp 0x7fff12af8210 T0)
==11503==The signal is caused by a READ memory access.
==11503==Hint: address points to the zero page.
#0 0x5888f3 in Aws::S3Crt::S3CrtClient::UploadPartCallable(Aws::S3Crt::Model::UploadPartRequest const&) const (.../Linux/test2+0x5888f3)
#1 0x4ddf5b in main /.../aws-multipart-upload-new/main.cpp:176
#2 0x7feaa9037544 in __libc_start_main (/lib64/libc.so.6+0x22544)
#3 0x410241 (.../Linux/test2+0x410241)

I would be very grateful for any help.

John
SDK version number
1.9.23
Platform/OS/Hardware/Device
What are you running the sdk on?
Red Hat Enterprise Linux Server release 7.5 (Maipo)
gcc version 11.1.0 (GCC)

To Reproduce (observed behavior)
Steps to reproduce the behavior (please share code)

int main ()
{

int i = 0;
int j = 0;
int position1 = 0;
int position2 = 0;

int partNumber = 0;
int outputBufferLength = 1024 * 1024 * 20;
unsigned long int contentLength = 0;

const unsigned long int fixedPartSize = 1024 * 1024 * 8;
int bytesRead = 0;
const double throughput_target_gbps = 5;

Aws::Utils::Logging::InitializeAWSLogging(
Aws::MakeSharedAws::Utils::Logging::DefaultLogSystem(
"RunUnitTests", Aws::Utils::Logging::LogLevel::Trace, "aws_sdk_"));

std::string inputFile = "/brixhome/brixton/pdmbuild/jparkeTest/books/43254.txt";

std::ifstream infile(inputFile.c_str(), std::ios::binary);
char outputBuffer = (char) malloc(outputBufferLength);

const Aws::String bucket_name = "test-cse-0cc24cfe-";
const Aws::String object_name = "test-file2.txt";
const std::string object_content = "text content.";
const Aws::String region = "us-east-1";
std::string bucket = bucket_name;
std::string key = object_name;

Aws::SDKOptions options;
Aws::InitAPI(options);
{

  Aws::S3Crt::ClientConfiguration clientConfiguration;
  clientConfiguration.region = region;
  clientConfiguration.throughputTargetGbps = throughput_target_gbps;
  clientConfiguration.partSize = fixedPartSize;
  Aws::S3Crt::S3CrtClient s3_client;

  Aws::S3Crt::Model::CreateMultipartUploadRequest multipartUploadRequest;
  multipartUploadRequest.SetBucket(bucket.c_str());
  multipartUploadRequest.SetKey(key.c_str());

  auto multipartUploadOutcome = s3_client.CreateMultipartUpload(multipartUploadRequest);
  std::string upload_id = multipartUploadOutcome.GetResult().GetUploadId();

  std::cout << "multipart upload id is:" << upload_id << std::endl;

  Aws::S3Crt::Model::UploadPartRequest uploadPartRequest;
  uploadPartRequest.SetBucket(bucket.c_str());
  uploadPartRequest.SetKey(key.c_str());

  auto stream_ptr =
     Aws::MakeShared<Aws::StringStream>("PutObjectInputStream",
     std::stringstream::in | std::stringstream::out | std::stringstream::binary);

  Aws::S3Crt::Model::CompleteMultipartUploadRequest completeMultipartUploadRequest;
  Aws::S3Crt::Model::CompletedMultipartUpload completedMultipartUpload;

  //**************************
  // Set Content Length
  //**************************    

  infile.read(outputBuffer, fixedPartSize);
  bytesRead = infile.gcount();
  std::cout << "read(" << bytesRead << ")" << std::endl;

  stream_ptr->write(reinterpret_cast<char*> (outputBuffer), bytesRead);

  uploadPartRequest.SetUploadId(upload_id.c_str());

  Aws::Utils::ByteBuffer part_md5(Aws::Utils::HashingUtils::CalculateMD5(*stream_ptr));
  uploadPartRequest.SetContentMD5(Aws::Utils::HashingUtils::Base64Encode(part_md5));

  uploadPartRequest.SetBody(stream_ptr);

  auto start_pos = stream_ptr->tellg();

  stream_ptr->seekg(0LL, stream_ptr->end);

  partNumber ++;
  contentLength = static_cast<long> (stream_ptr->tellg());

  std::cout << "tellg (" << static_cast<long> (stream_ptr->tellg()) << ")" << std::endl;
  std::cout << "start position (" << start_pos << ")" << std::endl;
  std::cout << "part Number (" << partNumber << ")" << std::endl;
  std::cout << "content length set to(" << contentLength << ")" << std::endl;

  uploadPartRequest.SetContentLength(contentLength);
  uploadPartRequest.SetPartNumber(partNumber);
  stream_ptr->seekg(start_pos);

  std::cout << "upload id(" <<  uploadPartRequest.GetUploadId () << ")" << std::endl;

SEGV ===> auto uploadPartOutcomeCallable = s3_client.UploadPartCallable(uploadPartRequest);

  //**************************
  // finish upload
  //**************************

  completeMultipartUploadRequest.SetBucket(bucket.c_str());
  completeMultipartUploadRequest.SetKey(key.c_str());
  completeMultipartUploadRequest.SetUploadId(upload_id.c_str());

  Aws::S3Crt::Model::UploadPartOutcome uploadPartOutcome = uploadPartOutcomeCallable.get();
  Aws::S3Crt::Model::CompletedPart completedPart;

  completedPart.SetPartNumber(partNumber);
  Aws::S3Crt::Model::UploadPartResult uploadPartResult = uploadPartOutcome.GetResult();
  auto etag = uploadPartOutcome.GetResult().GetETag();
  Aws::S3Crt::Model::UploadPartOutcome response = s3_client.UploadPart(uploadPartRequest);

  completedPart.SetETag(etag);
  completedMultipartUpload.AddParts(completedPart);

  /*
   * Complete the Multipart Upload
   *             
   */
  completeMultipartUploadRequest.WithMultipartUpload(completedMultipartUpload);

  auto completeMultipartUploadOutcome = s3_client.CompleteMultipartUpload(completeMultipartUploadRequest);

  if (!completeMultipartUploadOutcome.IsSuccess())
  {
     auto error = completeMultipartUploadOutcome.GetError();
     std::cout << error << error.GetExceptionName() << ": " << error.GetMessage() << std::endl;

  }

}
Aws::Utils::Logging::ShutdownAWSLogging();
Aws::ShutdownAPI(options);

Expected behavior
A clear and concise description of what you expected to happen.
File should upload.
aws_sdk_2021-05-21-13.log
asan.log.11503.log

Logs/output
If applicable, add logs or error output.

Program output:
multipart upload id is:fk95A9gWIUaO7KPwA462BgBBZjp8Bcq..Q4qxIO8IWbOZBkcsxQtQbWnQ_0c8zXQ0EeXWrqCQ8ESY3lS197dR4PT9.F2mdsHsmnVACv9Kxk32Y53Nmjfy8tlpDWgSNEI
read(1178782)
tellg (1178782)
start position (0)
part Number (1)
content length set to(1178782)
upload id(fk95A9gWIUaO7KPwA462BgBBZjp8Bcq..Q4qxIO8IWbOZBkcsxQtQbWnQ_0c8zXQ0EeXWrqCQ8ESY3lS197dR4PT9.F2mdsHsmnVACv9Kxk32Y53Nmjfy8tlpDWgSNEI)
AddressSanitizer:DEADLYSIGNAL
Aborted

To enable logging, set the following system properties:

REMEMBER TO SANITIZE YOUR PERSONAL INFO

options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
Aws::InitAPI(options)

Additional context
Add any other context about the problem here.
gdb output:

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00000000004aa783 in Aws::S3Crt::S3CrtClient::UploadPartCallable(Aws::S3Crt::Model::UploadPartRequest const&) const ()
[Current thread is 1 (Thread 0x7f4cc995f800 (LWP 10892))]
(gdb) where
#0 0x00000000004aa783 in Aws::S3Crt::S3CrtClient::UploadPartCallable(Aws::S3Crt::Model::UploadPartRequest const&) const ()
#1 0x000000000040da87 in main () at main.cpp:176

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a bug.p2This is a standard priority issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions