BlueXP Services

Deletion of large S3 bucket, best way

Sander
3,516 Views

Hi,

need to destroy a tenant in our SGW with a 80TB bucket, now question is how?

How i've started is with S3 browser-pro and just select "delete bucket". Now this starts but takes a very long time, even with multiple threads.

Other suggestion i've seen was to put a lifecycle policy on the bucket of 1 day (was an org. AWS bucket in the example), does that work with S3 browser also on a SGW bucket?

 

Very curious if there are alternatives that work a bit faster.

1 REPLY 1

elementx
3,218 Views

You could certainly create a policy that would delete files on the storage, but if you did this often you'd end up with a ton of archived rules.

 

It's probably better to use a S3 client like PowerShell with S3 module, or try this https://stackoverflow.com/questions/10054985/how-to-delete-files-recursively-from-an-s3-bucket/18698235#18698235 

aws s3 rm --recursive s3://your_bucket_name/foo/

Or take a look at https://github.com/eschwim/s3wipe (I don't know if it works).

I haven't tried these on SG.  

Public