Object Storage

python script to delete s3 objects and bucket

manistorage
2,428 Views

I had putt gather from public forum. but its erroring with certificate. would appreciate any assistance.

 

[test-profile1]
aws_access_key_id=myAccessKey
aws_secret_access_key=mySecretAccessKey

import sys
import boto3
import boto3.session

## Configure the StorageGRID S3 endpoint and profile name ##
session = boto3.session.Session(profile_name='test-profile1')
endpoint = 'https://s3.mycompany.com:8080/'

# Ignore SSL verification
s3 = session.resource(service_name='s3', endpoint_url=endpoint, verify=False)
client = s3.meta.client

# Get the target bucket name from the command line argument
if len(sys.argv) < 2:
print("Please provide the target bucket name as a command line argument")
sys.exit(1)

target_bucket_name = sys.argv[1]

# Create an S3 client
s3_client = boto3.client("s3")

# create an S3 resource
s3 = boto3.resource("s3")

# Get a list of all S3 buckets
response = s3_client.list_buckets()

# Iterate over the buckets
for bucket in response["Buckets"]:
bucket_name = bucket["Name"]

# Check if the bucket name contains the target string
if target_bucket_name in bucket_name:
print(f"Found bucket: {bucket_name}")

versioning = s3_client.get_bucket_versioning(Bucket=bucket_name)
bucket = s3.Bucket(bucket_name)

if versioning.get("Status") == "Enabled":
bucket.object_versions.delete()
else:
bucket.objects.delete()

# Finally, delete the bucket
s3_client.delete_bucket(Bucket=bucket_name)
print(f"Deleted bucket: {bucket_name}")

execute the script - python search_bucket_and_delete.py test-bucket

ERROR - File "C:\Program Files\Python38\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://s3.eu-west-1.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

3 REPLIES 3

elementx
2,420 Views

So your configured domain is mycompany.com and you're trying to use the script to delete data at amazonaws.com. That won't work. You need to fill in the correct, valid values.

 

manistorage
2,385 Views

hi,

 

Hi,

appreciate the response. i have defined the endpoint.

## Configure the StorageGRID S3 endpoint and profile name ##
session = boto3.session.Session(profile_name='test-profile1')
endpoint = 'https://s3.mycompany.com:8080/'

 

i believe the script hits SLL cert authentication error. the output might be generic.

 

Regards,

Mani

 

 

elementx
2,375 Views

The error isn't generic.

If the lines took effect, validation should not have happened:

s3 = session.resource(service_name='s3', endpoint_url=endpoint, verify=False)

But you instead got CERTIFICATE_VERIFY_FAILED.

 

A step-by-step Boto 3 example for StorageGRID 11 can be found here:

https://github.com/NetApp/storagegrid-examples/blob/master/s3-python/README.md

 

I tried to connect with certificate validation off and my certificate was ignored like it's supposed to.

elementx_0-1692717058547.png

All S3-compliant storage should work the same way; you don't need a NetApp-specific example.

(I'm not sure what product you're inquiring about, StorageGRID or ONTAP S3, but I haven't asked because it shouldn't matter. In fact I used that StorageGRID example with ONTAP S3 (9.12.1), and it worked fine.

Public