[Bug 2131790] Re: RGW - etag not returned on multipart upload
Ghadi Rahme
2131790 at bugs.launchpad.net
Thu Jun 25 11:15:44 UTC 2026
yoga verification:
ceph-mon:
$ dpkg -l | grep ceph
ii ceph 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 distributed storage and file system
ii ceph-base 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 common ceph daemon libraries and management tools
ii ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 common utilities to mount and interact with a ceph storage cluster
ii ceph-mds 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 metadata server for the ceph distributed file system
ii ceph-mgr 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 manager for the ceph distributed file system
ii ceph-mgr-modules-core 17.2.9-0ubuntu0.22.04.3~cloud0 all ceph manager modules which are always enabled
ii ceph-mon 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 monitor server for the ceph storage system
ii ceph-osd 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 OSD server for the ceph storage system
ii ceph-volume 17.2.9-0ubuntu0.22.04.3~cloud0 all tool to facilidate OSD deployment
ii libcephfs2 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Ceph distributed file system client library
ii libsqlite3-mod-ceph 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 SQLite3 VFS for Ceph
ii python3-ceph-argparse 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 utility libraries for Ceph CLI
ii python3-ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 all Python 3 utility libraries for Ceph
ii python3-cephfs 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 libraries for the Ceph libcephfs library
ceph-rgw:
$ dpkg -l | grep ceph
ii ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 common utilities to mount and interact with a ceph storage cluster
ii libcephfs2 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Ceph distributed file system client library
ii python3-ceph-argparse 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 utility libraries for Ceph CLI
ii python3-ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 all Python 3 utility libraries for Ceph
ii python3-cephfs 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 libraries for the Ceph libcephfs library
ceph-osd:
$ dpkg -l | grep ceph
ii ceph 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 distributed storage and file system
ii ceph-base 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 common ceph daemon libraries and management tools
ii ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 common utilities to mount and interact with a ceph storage cluster
ii ceph-mds 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 metadata server for the ceph distributed file system
ii ceph-mgr 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 manager for the ceph distributed file system
ii ceph-mgr-modules-core 17.2.9-0ubuntu0.22.04.3~cloud0 all ceph manager modules which are always enabled
ii ceph-mon 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 monitor server for the ceph storage system
ii ceph-osd 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 OSD server for the ceph storage system
ii ceph-volume 17.2.9-0ubuntu0.22.04.3~cloud0 all tool to facilidate OSD deployment
ii libcephfs2 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Ceph distributed file system client library
ii libsqlite3-mod-ceph 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 SQLite3 VFS for Ceph
ii python3-ceph-argparse 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 utility libraries for Ceph CLI
ii python3-ceph-common 17.2.9-0ubuntu0.22.04.3~cloud0 all Python 3 utility libraries for Ceph
ii python3-cephfs 17.2.9-0ubuntu0.22.04.3~cloud0 amd64 Python 3 libraries for the Ceph libcephfs library
2. Upload a 10MB test file:
$ ls -lah test.txt
-rw-rw-r-- 1 ubuntu ubuntu 10M Jun 25 11:05 test.txt
$ aws s3 --profile ceph --endpoint-url http://10.159.7.97 cp ./test.txt s3://my-test-bucket/ --debug
2026-06-25 11:11:01,077 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19
2026-06-25 11:11:01,077 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', '--profile', 'ceph', '--endpoint-url', 'http://10.159.7.97', 'cp', './test.txt', 's3://my-test-bucket/', '--debug']
2026-06-25 11:11:01,078 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x7f7383731040>
2026-06-25 11:11:01,078 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x7f7383bb8b80>
2026-06-25 11:11:01,078 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x7f7383b968b0>
2026-06-25 11:11:01,079 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2026-06-25 11:11:01,079 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x7f7383874790>
2026-06-25 11:11:01,079 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <function add_waiters at 0x7f73836b9940>
2026-06-25 11:11:01,079 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.s3.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,079 - MainThread - botocore.hooks - DEBUG - Event building-command-table.cp: calling handler <function add_waiters at 0x7f73836b9940>
2026-06-25 11:11:01,080 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.paths: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,080 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.dryrun: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.quiet: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.acl: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.follow-symlinks: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,081 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-key: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-kms-key-id: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-copy-source: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-copy-source-key: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.storage-class: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.grants: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.website-redirect: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-type: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,082 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.cache-control: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-disposition: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-encoding: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-language: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.expires: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.source-region: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.only-show-errors: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.page-size: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,083 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.ignore-glacier-warnings: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.force-glacier-transfer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7f7383bdd220>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.request-payer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.metadata: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.metadata-directive: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.expected-size: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7f7383673970>
2026-06-25 11:11:01,084 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2026-06-25 11:11:01,084 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2026-06-25 11:11:01,084 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2026-06-25 11:11:01,085 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2026-06-25 11:11:01,085 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3/dist-packages/botocore/data/endpoints.json
2026-06-25 11:11:01,088 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7f73840850d0>
2026-06-25 11:11:01,088 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3/dist-packages/botocore/data/s3/2006-03-01/service-2.json
2026-06-25 11:11:01,093 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x7f73840c4c10>
2026-06-25 11:11:01,093 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x7f73840c49d0>
2026-06-25 11:11:01,094 - MainThread - botocore.regions - DEBUG - Creating a regex based endpoint for s3, US
2026-06-25 11:11:01,096 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2026-06-25 11:11:01,096 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3/dist-packages/botocore/data/_retry.json
2026-06-25 11:11:01,096 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2026-06-25 11:11:01,097 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7f73840850d0>
2026-06-25 11:11:01,098 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x7f73840c4c10>
2026-06-25 11:11:01,098 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x7f73840c49d0>
2026-06-25 11:11:01,098 - MainThread - botocore.regions - DEBUG - Creating a regex based endpoint for s3, US
2026-06-25 11:11:01,099 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2026-06-25 11:11:01,099 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2026-06-25 11:11:01,100 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7f73840850d0>
2026-06-25 11:11:01,100 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x7f73840c4c10>
2026-06-25 11:11:01,101 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x7f73840c49d0>
2026-06-25 11:11:01,101 - MainThread - botocore.regions - DEBUG - Creating a regex based endpoint for s3, US
2026-06-25 11:11:01,102 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2026-06-25 11:11:01,103 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2026-06-25 11:11:01,103 - MainThread - awscli.customizations.s3.s3handler - DEBUG - Using a multipart threshold of 8388608 and a part size of 8388608
2026-06-25 11:11:01,104 - MainThread - s3transfer.utils - DEBUG - Acquiring 0
2026-06-25 11:11:01,104 - ThreadPoolExecutor-1_0 - s3transfer.tasks - DEBUG - UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f7383334fa0>}) about to wait for the following futures []
2026-06-25 11:11:01,104 - ThreadPoolExecutor-1_0 - s3transfer.tasks - DEBUG - UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f7383334fa0>}) done waiting for dependent futures
2026-06-25 11:11:01,104 - ThreadPoolExecutor-1_0 - s3transfer.tasks - DEBUG - Executing task UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f7383334fa0>}) with kwargs {'client': <botocore.client.S3 object at 0x7f738339a190>, 'config': <s3transfer.manager.TransferConfig object at 0x7f7383331580>, 'osutil': <s3transfer.utils.OSUtils object at 0x7f7383331610>, 'request_executor': <s3transfer.futures.BoundedExecutor object at 0x7f7383331850>, 'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f7383334fa0>}
2026-06-25 11:11:01,106 - ThreadPoolExecutor-1_0 - s3transfer.futures - DEBUG - Submitting task CreateMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {'ContentType': 'text/plain'}}) to executor <s3transfer.futures.BoundedExecutor object at 0x7f7383331850> for transfer request: 0.
2026-06-25 11:11:01,106 - ThreadPoolExecutor-1_0 - s3transfer.utils - DEBUG - Acquiring 0
2026-06-25 11:11:01,106 - ThreadPoolExecutor-0_0 - s3transfer.tasks - DEBUG - CreateMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {'ContentType': 'text/plain'}}) about to wait for the following futures []
2026-06-25 11:11:01,106 - ThreadPoolExecutor-1_0 - s3transfer.futures - DEBUG - Submitting task UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}}) to executor <s3transfer.futures.BoundedExecutor object at 0x7f7383331850> for transfer request: 0.
2026-06-25 11:11:01,106 - ThreadPoolExecutor-1_0 - s3transfer.utils - DEBUG - Acquiring 0
2026-06-25 11:11:01,106 - ThreadPoolExecutor-0_0 - s3transfer.tasks - DEBUG - CreateMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {'ContentType': 'text/plain'}}) done waiting for dependent futures
2026-06-25 11:11:01,106 - ThreadPoolExecutor-0_0 - s3transfer.tasks - DEBUG - Executing task CreateMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {'ContentType': 'text/plain'}}) with kwargs {'client': <botocore.client.S3 object at 0x7f738339a190>, 'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {'ContentType': 'text/plain'}}
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <function validate_ascii_metadata at 0x7f73840a4160>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <function sse_md5 at 0x7f73840a3550>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <function validate_bucket_name at 0x7f73840a34c0>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f738339af10>>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateMultipartUpload: calling handler <function generate_idempotent_uuid at 0x7f73840a30d0>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-call.s3.CreateMultipartUpload: calling handler <function add_expect_header at 0x7f73840a3820>
2026-06-25 11:11:01,107 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-call.s3.CreateMultipartUpload: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-call.s3.CreateMultipartUpload: calling handler <function inject_api_version_header_if_needed at 0x7f73840a4b80>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateMultipartUpload) with params: {'url_path': '/my-test-bucket/test.txt?uploads', 'query_string': {}, 'method': 'POST', 'headers': {'Content-Type': 'text/plain', 'User-Agent': 'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19'}, 'body': b'', 'url': 'http://10.159.7.97/my-test-bucket/test.txt?uploads', 'context': {'client_region': 'US', 'client_config': <botocore.config.Config object at 0x7f738339a340>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'my-test-bucket'}}}
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event request-created.s3.CreateMultipartUpload: calling handler <function signal_not_transferring at 0x7f738398bb80>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event request-created.s3.CreateMultipartUpload: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f738339a2b0>>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event choose-signer.s3.CreateMultipartUpload: calling handler <function set_operation_specific_signer at 0x7f738409df70>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event before-sign.s3.CreateMultipartUpload: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f738339afa0>>
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.utils - DEBUG - Using S3 path style addressing.
2026-06-25 11:11:01,108 - ThreadPoolExecutor-0_0 - botocore.auth - DEBUG - Calculating signature using v4 auth.
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_0 - botocore.auth - DEBUG - CanonicalRequest:
POST
/my-test-bucket/test.txt
uploads=
content-type:text/plain
host:10.159.7.97
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20260625T111101Z
content-type;host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_0 - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20260625T111101Z
20260625/US/s3/aws4_request
79af72826a5601914700be8ea8744fca54ad718ac695abea9bbbd40d74bd53dc
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_0 - botocore.auth - DEBUG - Signature:
39fc4c550611a7dc8acca6e95442e3d1fb644ed01260d490f8dfb0623cea8031
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event request-created.s3.CreateMultipartUpload: calling handler <function signal_transferring at 0x7f738398bc10>
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_0 - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=http://10.159.7.97/my-test-bucket/test.txt?uploads, headers={'Content-Type': b'text/plain', 'User-Agent': b'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'X-Amz-Date': b'20260625T111101Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NVHH09CP5XR9G4H9UKXV/20260625/US/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=39fc4c550611a7dc8acca6e95442e3d1fb644ed01260d490f8dfb0623cea8031', 'Content-Length': '0'}>
2026-06-25 11:11:01,110 - ThreadPoolExecutor-0_0 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 10.159.7.97:80
2026-06-25 11:11:01,109 - ThreadPoolExecutor-0_1 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}}) about to wait for the following futures [<s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>]
2026-06-25 11:11:01,110 - ThreadPoolExecutor-1_0 - s3transfer.futures - DEBUG - Submitting task UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}}) to executor <s3transfer.futures.BoundedExecutor object at 0x7f7383331850> for transfer request: 0.
2026-06-25 11:11:01,110 - ThreadPoolExecutor-1_0 - s3transfer.utils - DEBUG - Acquiring 0
2026-06-25 11:11:01,110 - ThreadPoolExecutor-0_1 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}}) about to wait for <s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>
2026-06-25 11:11:01,111 - ThreadPoolExecutor-0_2 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}}) about to wait for the following futures [<s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>]
2026-06-25 11:11:01,111 - ThreadPoolExecutor-1_0 - s3transfer.futures - DEBUG - Submitting task CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) to executor <s3transfer.futures.BoundedExecutor object at 0x7f7383331850> for transfer request: 0.
2026-06-25 11:11:01,111 - ThreadPoolExecutor-1_0 - s3transfer.utils - DEBUG - Acquiring 0
2026-06-25 11:11:01,111 - ThreadPoolExecutor-0_2 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}}) about to wait for <s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>
2026-06-25 11:11:01,112 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) about to wait for the following futures [<s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>, <s3transfer.futures.ExecutorFuture object at 0x7f738315a370>, <s3transfer.futures.ExecutorFuture object at 0x7f73810f6df0>]
2026-06-25 11:11:01,112 - ThreadPoolExecutor-1_0 - s3transfer.utils - DEBUG - Releasing acquire 0/None
2026-06-25 11:11:01,112 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) about to wait for <s3transfer.futures.ExecutorFuture object at 0x7f7383127b20>
2026-06-25 11:11:02,600 - ThreadPoolExecutor-0_0 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "POST /my-test-bucket/test.txt?uploads HTTP/1.1" 200 253
2026-06-25 11:11:02,601 - ThreadPoolExecutor-0_0 - botocore.parsers - DEBUG - Response headers: {'x-amz-request-id': 'tx0000016022614f61c194b-006a3d0cc5-1368-default', 'Content-Type': 'application/xml', 'Content-Length': '253', 'Date': 'Thu, 25 Jun 2026 11:11:02 GMT', 'Connection': 'Keep-Alive'}
2026-06-25 11:11:02,601 - ThreadPoolExecutor-0_0 - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>my-test-bucket</Bucket><Key>test.txt</Key><UploadId>2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA</UploadId></InitiateMultipartUploadResult>'
2026-06-25 11:11:02,601 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event needs-retry.s3.CreateMultipartUpload: calling handler <botocore.retryhandler.RetryHandler object at 0x7f738339adf0>
2026-06-25 11:11:02,601 - ThreadPoolExecutor-0_0 - botocore.retryhandler - DEBUG - No retry needed.
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event needs-retry.s3.CreateMultipartUpload: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event after-call.s3.CreateMultipartUpload: calling handler <function enhance_error_msg at 0x7f738372daf0>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_0 - s3transfer.utils - DEBUG - Releasing acquire 0/None
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}}) done waiting for dependent futures
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - s3transfer.tasks - DEBUG - Executing task UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}}) with kwargs {'client': <botocore.client.S3 object at 0x7f738339a190>, 'fileobj': <s3transfer.utils.ReadFileChunk object at 0x7f7383127eb0>, 'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 1, 'extra_args': {}, 'upload_id': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA'}
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function sse_md5 at 0x7f73840a3550>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function convert_body_to_file_like_object at 0x7f73840a4a60>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function validate_bucket_name at 0x7f73840a34c0>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,603 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f738339af10>>
2026-06-25 11:11:02,603 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function generate_idempotent_uuid at 0x7f73840a30d0>
2026-06-25 11:11:02,603 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function conditionally_calculate_md5 at 0x7f73840a3430>
2026-06-25 11:11:02,602 - ThreadPoolExecutor-0_2 - s3transfer.tasks - DEBUG - UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}}) done waiting for dependent futures
2026-06-25 11:11:02,605 - ThreadPoolExecutor-0_2 - s3transfer.tasks - DEBUG - Executing task UploadPartTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}}) with kwargs {'client': <botocore.client.S3 object at 0x7f738339a190>, 'fileobj': <s3transfer.utils.ReadFileChunk object at 0x7f73810f6160>, 'bucket': 'my-test-bucket', 'key': 'test.txt', 'part_number': 2, 'extra_args': {}, 'upload_id': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA'}
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function sse_md5 at 0x7f73840a3550>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function convert_body_to_file_like_object at 0x7f73840a4a60>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function validate_bucket_name at 0x7f73840a34c0>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f738339af10>>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-parameter-build.s3.UploadPart: calling handler <function generate_idempotent_uuid at 0x7f73840a30d0>
2026-06-25 11:11:02,606 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function conditionally_calculate_md5 at 0x7f73840a3430>
2026-06-25 11:11:02,609 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) about to wait for <s3transfer.futures.ExecutorFuture object at 0x7f738315a370>
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function add_expect_header at 0x7f73840a3820>
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.handlers - DEBUG - Adding expect 100 continue header to request.
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function inject_api_version_header_if_needed at 0x7f73840a4b80>
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.endpoint - DEBUG - Making request for OperationModel(name=UploadPart) with params: {'url_path': '/my-test-bucket/test.txt', 'query_string': {'uploadId': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA', 'partNumber': 2}, 'method': 'PUT', 'headers': {'User-Agent': 'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'Content-MD5': '+wrRd/DllzSOARvzjyKFzA==', 'Expect': '100-continue'}, 'body': <s3transfer.utils.ReadFileChunk object at 0x7f73810f6160>, 'url': 'http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=2', 'context': {'client_region': 'US', 'client_config': <botocore.config.Config object at 0x7f738339a340>, 'has_streaming_input': True, 'auth_type': None, 'signing': {'bucket': 'my-test-bucket'}}}
2026-06-25 11:11:02,614 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <function signal_not_transferring at 0x7f738398bb80>
2026-06-25 11:11:02,615 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f738339a2b0>>
2026-06-25 11:11:02,615 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event choose-signer.s3.UploadPart: calling handler <function set_operation_specific_signer at 0x7f738409df70>
2026-06-25 11:11:02,615 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event before-sign.s3.UploadPart: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f738339afa0>>
2026-06-25 11:11:02,617 - ThreadPoolExecutor-0_2 - botocore.auth - DEBUG - Calculating signature using v4 auth.
2026-06-25 11:11:02,621 - ThreadPoolExecutor-0_2 - botocore.auth - DEBUG - CanonicalRequest:
PUT
/my-test-bucket/test.txt
partNumber=2&uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA
content-md5:+wrRd/DllzSOARvzjyKFzA==
host:10.159.7.97
x-amz-content-sha256:f96cb00cb30ef61fe0101098b343421b4be0fe0decb9bf1f4088666b11abaab2
x-amz-date:20260625T111102Z
content-md5;host;x-amz-content-sha256;x-amz-date
f96cb00cb30ef61fe0101098b343421b4be0fe0decb9bf1f4088666b11abaab2
2026-06-25 11:11:02,622 - ThreadPoolExecutor-0_2 - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20260625T111102Z
20260625/US/s3/aws4_request
ce9befc115b9f1f60765b347bcd2ec05a2692a5063addf6cb82bb1c36c2a47cb
2026-06-25 11:11:02,622 - ThreadPoolExecutor-0_2 - botocore.auth - DEBUG - Signature:
186760b35a1b42c5ca3c88e236e90014af89a46cafc91888deea5ee352faebfa
2026-06-25 11:11:02,622 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <function signal_transferring at 0x7f738398bc10>
2026-06-25 11:11:02,622 - ThreadPoolExecutor-0_2 - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=2, headers={'User-Agent': b'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'Content-MD5': b'+wrRd/DllzSOARvzjyKFzA==', 'Expect': b'100-continue', 'X-Amz-Date': b'20260625T111102Z', 'X-Amz-Content-SHA256': b'f96cb00cb30ef61fe0101098b343421b4be0fe0decb9bf1f4088666b11abaab2', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NVHH09CP5XR9G4H9UKXV/20260625/US/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=186760b35a1b42c5ca3c88e236e90014af89a46cafc91888deea5ee352faebfa', 'Content-Length': '2097152'}>
2026-06-25 11:11:02,623 - ThreadPoolExecutor-0_2 - botocore.awsrequest - DEBUG - Waiting for 100 Continue response.
2026-06-25 11:11:02,623 - ThreadPoolExecutor-0_2 - botocore.awsrequest - DEBUG - 100 Continue response seen, now sending request body.
2026-06-25 11:11:02,635 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function add_expect_header at 0x7f73840a3820>
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.handlers - DEBUG - Adding expect 100 continue header to request.
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-call.s3.UploadPart: calling handler <function inject_api_version_header_if_needed at 0x7f73840a4b80>
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.endpoint - DEBUG - Making request for OperationModel(name=UploadPart) with params: {'url_path': '/my-test-bucket/test.txt', 'query_string': {'uploadId': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA', 'partNumber': 1}, 'method': 'PUT', 'headers': {'User-Agent': 'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'Content-MD5': 'N9FX7Y3ENVDur3OnqQqrTQ==', 'Expect': '100-continue'}, 'body': <s3transfer.utils.ReadFileChunk object at 0x7f7383127eb0>, 'url': 'http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=1', 'context': {'client_region': 'US', 'client_config': <botocore.config.Config object at 0x7f738339a340>, 'has_streaming_input': True, 'auth_type': None, 'signing': {'bucket': 'my-test-bucket'}}}
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <function signal_not_transferring at 0x7f738398bb80>
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f738339a2b0>>
2026-06-25 11:11:02,638 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event choose-signer.s3.UploadPart: calling handler <function set_operation_specific_signer at 0x7f738409df70>
2026-06-25 11:11:02,639 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event before-sign.s3.UploadPart: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f738339afa0>>
2026-06-25 11:11:02,645 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - Calculating signature using v4 auth.
2026-06-25 11:11:02,645 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - CanonicalRequest:
PUT
/my-test-bucket/test.txt
partNumber=1&uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA
content-md5:N9FX7Y3ENVDur3OnqQqrTQ==
host:10.159.7.97
x-amz-content-sha256:c401681cc5b84c6973a843e8a8e4bb3c16d23a67cb148eced51f2b0349750ba7
x-amz-date:20260625T111102Z
content-md5;host;x-amz-content-sha256;x-amz-date
c401681cc5b84c6973a843e8a8e4bb3c16d23a67cb148eced51f2b0349750ba7
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20260625T111102Z
20260625/US/s3/aws4_request
502873523b75fe2a7d76c22616b95f7d78fb11cea5a8bc34459a4f739d432773
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - Signature:
28ce96e3364df64e4245fae8e72a4d879d98d3ee2c243cec6e066a0d471771f8
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <function signal_transferring at 0x7f738398bc10>
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=1, headers={'User-Agent': b'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'Content-MD5': b'N9FX7Y3ENVDur3OnqQqrTQ==', 'Expect': b'100-continue', 'X-Amz-Date': b'20260625T111102Z', 'X-Amz-Content-SHA256': b'c401681cc5b84c6973a843e8a8e4bb3c16d23a67cb148eced51f2b0349750ba7', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NVHH09CP5XR9G4H9UKXV/20260625/US/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=28ce96e3364df64e4245fae8e72a4d879d98d3ee2c243cec6e066a0d471771f8', 'Content-Length': '8388608'}>
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (2): 10.159.7.97:80
2026-06-25 11:11:02,647 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - Waiting for 100 Continue response.
2026-06-25 11:11:02,648 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - 100 Continue response seen, now sending request body.
2026-06-25 11:11:02,760 - ThreadPoolExecutor-0_2 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "PUT /my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=2 HTTP/1.1" 200 0
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response headers: {'Content-Length': '0', 'ETag': '"fb0ad177f0e597348e011bf38f2285cc"', 'Accept-Ranges': 'bytes', 'x-amz-request-id': 'tx00000963969c7cdbc5172-006a3d0cc6-1368-default', 'Date': 'Thu, 25 Jun 2026 11:11:02 GMT', 'Connection': 'Keep-Alive'}
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response body:
b''
2026-06-25 11:11:02,647 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - Waiting for 100 Continue response.
2026-06-25 11:11:02,648 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - 100 Continue response seen, now sending request body.
2026-06-25 11:11:02,760 - ThreadPoolExecutor-0_2 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "PUT /my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=2 HTTP/1.2026-06-25 11:11:02,648 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - 100 Continue response seen, now sending request body.
content-md5;host;x-amz-content-sha256;x-amz-date
c401681cc5b84c6973a843e8a8e4bb3c16d23a67cb148eced51f2b0349750ba7
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20260625T111102Z
20260625/US/s3/aws4_request
502873523b75fe2a7d76c22616b95f7d78fb11cea5a8bc34459a4f739d432773
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.auth - DEBUG - Signature:
28ce96e3364df64e4245fae8e72a4d879d98d3ee2c243cec6e066a0d471771f8
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event request-created.s3.UploadPart: calling handler <function signal_transferring at 0x7f738398bc10>
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=1, headers={'User-Agent': b'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'Content-MD5': b'N9FX7Y3ENVDur3OnqQqrTQ==', 'Expect': b'100-continue', 'X-Amz-Date': b'20260625T111102Z', 'X-Amz-Content-SHA256': b'c401681cc5b84c6973a843e8a8e4bb3c16d23a67cb148eced51f2b0349750ba7', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NVHH09CP5XR9G4H9UKXV/20260625/US/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=28ce96e3364df64e4245fae8e72a4d879d98d3ee2c243cec6e066a0d471771f8', 'Content-Length': '8388608'}>
2026-06-25 11:11:02,646 - ThreadPoolExecutor-0_1 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (2): 10.159.7.97:80
2026-06-25 11:11:02,647 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - Waiting for 100 Continue response.
2026-06-25 11:11:02,648 - ThreadPoolExecutor-0_1 - botocore.awsrequest - DEBUG - 100 Continue response seen, now sending request body.
2026-06-25 11:11:02,760 - ThreadPoolExecutor-0_2 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "PUT /my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=2 HTTP/1.1" 200 0
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response headers: {'Content-Length': '0', 'ETag': '"fb0ad177f0e597348e011bf38f2285cc"', 'Accept-Ranges': 'bytes', 'x-amz-request-id': 'tx00000963969c7cdbc5172-006a3d0cc6-1368-default', 'Date': 'Thu, 25 Jun 2026 11:11:02 GMT', 'Connection': 'Keep-Alive'}
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response body:
b''
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x7f738339adf0>
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.retryhandler - DEBUG - No retry needed.
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - botocore.hooks - DEBUG - Event after-call.s3.UploadPart: calling handler <function enhance_error_msg at 0x7f738372daf0>
2026-06-25 11:11:02,761 - ThreadPoolExecutor-0_2 - s3transfer.utils - DEBUG - Releasing acquire 0/None
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "PUT /my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA&partNumber=1 HTTP/1.1" 200 0
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.parsers - DEBUG - Response headers: {'Content-Length': '0', 'ETag': '"37d157ed8dc43550eeaf73a7a90aab4d"', 'Accept-Ranges': 'bytes', 'x-amz-request-id': 'tx00000ea16a22abf8a6208-006a3d0cc6-1368-default', 'Date': 'Thu, 25 Jun 2026 11:11:02 GMT', 'Connection': 'Keep-Alive'}
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.parsers - DEBUG - Response body:
b''
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <botocore.retryhandler.RetryHandler object at 0x7f738339adf0>
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.retryhandler - DEBUG - No retry needed.
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event needs-retry.s3.UploadPart: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - botocore.hooks - DEBUG - Event after-call.s3.UploadPart: calling handler <function enhance_error_msg at 0x7f738372daf0>
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_1 - s3transfer.utils - DEBUG - Releasing acquire 0/None
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) about to wait for <s3transfer.futures.ExecutorFuture object at 0x7f73810f6df0>
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) done waiting for dependent futures
2026-06-25 11:11:02,839 - ThreadPoolExecutor-0_3 - s3transfer.tasks - DEBUG - Executing task CompleteMultipartUploadTask(transfer_id=0, {'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}}) with kwargs {'client': <botocore.client.S3 object at 0x7f738339a190>, 'bucket': 'my-test-bucket', 'key': 'test.txt', 'extra_args': {}, 'upload_id': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA', 'parts': [{'ETag': '"37d157ed8dc43550eeaf73a7a90aab4d"', 'PartNumber': 1}, {'ETag': '"fb0ad177f0e597348e011bf38f2285cc"', 'PartNumber': 2}]}
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CompleteMultipartUpload: calling handler <function validate_bucket_name at 0x7f73840a34c0>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CompleteMultipartUpload: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CompleteMultipartUpload: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f738339af10>>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-parameter-build.s3.CompleteMultipartUpload: calling handler <function generate_idempotent_uuid at 0x7f73840a30d0>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-call.s3.CompleteMultipartUpload: calling handler <function add_expect_header at 0x7f73840a3820>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-call.s3.CompleteMultipartUpload: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-call.s3.CompleteMultipartUpload: calling handler <function inject_api_version_header_if_needed at 0x7f73840a4b80>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.endpoint - DEBUG - Making request for OperationModel(name=CompleteMultipartUpload) with params: {'url_path': '/my-test-bucket/test.txt', 'query_string': {'uploadId': '2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA'}, 'method': 'POST', 'headers': {'User-Agent': 'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19'}, 'body': b'<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Part><ETag>"37d157ed8dc43550eeaf73a7a90aab4d"</ETag><PartNumber>1</PartNumber></Part><Part><ETag>"fb0ad177f0e597348e011bf38f2285cc"</ETag><PartNumber>2</PartNumber></Part></CompleteMultipartUpload>', 'url': 'http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA', 'context': {'client_region': 'US', 'client_config': <botocore.config.Config object at 0x7f738339a340>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'my-test-bucket'}}}
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event request-created.s3.CompleteMultipartUpload: calling handler <function signal_not_transferring at 0x7f738398bb80>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event request-created.s3.CompleteMultipartUpload: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f738339a2b0>>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event choose-signer.s3.CompleteMultipartUpload: calling handler <function set_operation_specific_signer at 0x7f738409df70>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event before-sign.s3.CompleteMultipartUpload: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f738339afa0>>
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.auth - DEBUG - Calculating signature using v4 auth.
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.auth - DEBUG - CanonicalRequest:
POST
/my-test-bucket/test.txt
uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA
host:10.159.7.97
x-amz-content-sha256:0fdd3972b29ba794f9ba43f4a2ca6e6526fa0bdaa13f1cd6dcf8a7f01e902f86
x-amz-date:20260625T111102Z
host;x-amz-content-sha256;x-amz-date
0fdd3972b29ba794f9ba43f4a2ca6e6526fa0bdaa13f1cd6dcf8a7f01e902f86
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20260625T111102Z
20260625/US/s3/aws4_request
ff8296e88e042e994ce1b938416fd148bf84708e219abaf043a2b797d21bc8c4
2026-06-25 11:11:02,840 - ThreadPoolExecutor-0_3 - botocore.auth - DEBUG - Signature:
def0976306a1ba81b9efd9c5404668c2f3d7d044f9da8a5f4543c3a216665379
2026-06-25 11:11:02,841 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event request-created.s3.CompleteMultipartUpload: calling handler <function signal_transferring at 0x7f738398bc10>
2026-06-25 11:11:02,841 - ThreadPoolExecutor-0_3 - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=http://10.159.7.97/my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA, headers={'User-Agent': b'aws-cli/1.18.69 Python/3.8.10 Linux/5.4.0-216-generic botocore/1.16.19', 'X-Amz-Date': b'20260625T111102Z', 'X-Amz-Content-SHA256': b'0fdd3972b29ba794f9ba43f4a2ca6e6526fa0bdaa13f1cd6dcf8a7f01e902f86', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NVHH09CP5XR9G4H9UKXV/20260625/US/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=def0976306a1ba81b9efd9c5404668c2f3d7d044f9da8a5f4543c3a216665379', 'Content-Length': '271'}>
2026-06-25 11:11:02,877 - ThreadPoolExecutor-0_3 - urllib3.connectionpool - DEBUG - http://10.159.7.97:80 "POST /my-test-bucket/test.txt?uploadId=2~bGJ7i9R8uEK3Z4vzQAZpkAPCIE8tbeA HTTP/1.1" 200 309
2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response headers: {'x-amz-request-id': 'tx00000073eca4b39ab23c2-006a3d0cc6-1368-default', 'Content-Type': 'application/xml', 'Content-Length': '309', 'Date': 'Thu, 25 Jun 2026 11:11:02 GMT', 'Connection': 'Keep-Alive'}
2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.97/my-test-bucket/test.txt</Location><Bucket>my-test-bucket</Bucket><Key>test.txt</Key><ETag>652843ebc17dfc4b9d477c40f199cc1a-2</ETag></CompleteMultipartUploadResult>'
2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event needs-retry.s3.CompleteMultipartUpload: calling handler <function check_for_200_error at 0x7f738409de50>
2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event needs-retry.s3.CompleteMultipartUpload: calling handler <botocore.retryhandler.RetryHandler object at 0x7f738339adf0>
2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.retryhandler - DEBUG - No retry needed.
2026-06-25 11:11:02,879 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event needs-retry.s3.CompleteMultipartUpload: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f738339ae50>>
2026-06-25 11:11:02,879 - ThreadPoolExecutor-0_3 - botocore.hooks - DEBUG - Event after-call.s3.CompleteMultipartUpload: calling handler <function enhance_error_msg at 0x7f738372daf0>
upload: ./test.txt to s3://my-test-bucket/test.txt
2026-06-25 11:11:02,879 - ThreadPoolExecutor-0_3 - s3transfer.utils - DEBUG - Releasing acquire 0/None
2026-06-25 11:11:02,879 - Thread-1 - awscli.customizations.s3.results - DEBUG - Shutdown request received in result processing thread, shutting down result thread.
The output line: 2026-06-25 11:11:02,878 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.97/my-test-bucket/test.txt</Location><Bucket>my-test-bucket</Bucket><Key>test.txt</Key><ETag>652843ebc17dfc4b9d477c40f199cc1a-2</ETag></CompleteMultipartUploadResult>'
correctly shows the final ETag.
3. Downloading the file from ceph and confirming no corruption took
place:
$ aws s3 --profile ceph cp s3://my-test-bucket/test.txt ./test-from-ceph.txt --endpoint-url http://10.159.7.97
download: s3://my-test-bucket/test.txt to ./test-from-ceph.txt
$ sha256sum ./test.txt test-from-ceph.txt
fb144bfdb960a7faf9c9406faf32488d2f75955e5400476e8363c53500bbf478 ./test.txt
fb144bfdb960a7faf9c9406faf32488d2f75955e5400476e8363c53500bbf478 test-from-ceph.txt
Since they have the same sha256 we can confirm they are the same.
** Tags removed: verification-yoga-needed
** Tags added: verification-yoga-done
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2131790
Title:
RGW - etag not returned on multipart upload
Status in Ubuntu Cloud Archive:
Fix Released
Status in Ubuntu Cloud Archive yoga series:
Fix Committed
Status in ceph package in Ubuntu:
Fix Released
Status in ceph source package in Jammy:
Fix Released
Status in ceph source package in Noble:
Fix Released
Bug description:
Thank you @pponnuvel for the original description!
[ Impact ]
Currently in ceph 17.2.9, when doing multipart uploads, the etag of the multipart upload is not returned. This causes issues with some applications that rely on ceph and the etag which now have to calculate the etag manually causing performance issues.
The issue was fixed upstream through https://github.com/ceph/ceph/pull/56453.
However this commit caused regressions and three other commits that are part of this PR are also needed to resolve them: https://github.com/ceph/ceph/pull/57257
A total of 4 commits are needed to resolve the issue.
[ Test Plan ]
1- Deploy ceph 17.2.9
2- Use the awscli tool to upload a file large enough that it would be sent as part of a multipart upload. Make sure to use the "--debug" argument to verify if the etag is being returned or not:
aws s3 --profile ceph --endpoint-url <ceph URL> cp ./test.txt s3://test-bucket/ --debug
Currently the following output is seen towards the end of the debug
log:
2026-04-14 22:47:57,150 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.70/test-bucket/test.txt</Location><Bucket>test-bucket</Bucket><Key>test.txt</Key><ETag></ETag></CompleteMultipartUploadResult>'
Where the <ETag></ETag> clause is empty.
The output should look like the following:
2026-04-14 22:47:57,150 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.70/test-bucket/test.txt</Location><Bucket>test-bucket</Bucket><Key>test.txt</Key><ETag>9609801fffde7f91a34c9686065d9785-2</ETag></CompleteMultipartUploadResult>'
With the <ETag> clause being populated.
3- Finally download the file from ceph to confirm that it is not corrupted:
aws s3 --profile ceph cp s3://test-bucket/test.txt ./test-from-ceph.txt --endpoint-url <ceph URL>
4- Compare both sha256 of the files to make sure they are the same
sha256sum test.txt test-from-ceph.txt
Alternatively use the command "cmp -l test.txt test-from-ceph.txt" to
confirm that they are the same. No diff should be printed in the
terminal if they are the same.
[ Where problems could occur ]
* Since the fix touches on the upload logic of ceph, there is a chance that issues related to uploading data to ceph fails.
* ETag might still fail to be included similarly to previous patches that tried resolving this issue but failed
[Original Description]
With 17.2.9, when doing multipart uploads, etags don't get returned. And the etag body is empty.
This was supposed to be fixed in the bug:
https://github.com/ceph/ceph/pull/51446
However, we still see the issue with that fix too (which is part of
17.2.9).
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2131790/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list