How to Write Files to S3 Using Boto3

If you’re transitioning from Boto 2 to Boto 3 for interacting with AWS S3, you might find that the methods for uploading data to S3 objects have changed. In Boto 2, you might have used methods like set_contents_from_string or set_contents_from_file. In Boto 3, the workflow is a bit different, but don’t worry—it’s straightforward.

Writing to S3 with Boto3

With Boto3, you primarily have two methods to write data to S3: Object.put and Client.put_object. Both are versatile and can handle different types of data uploads, whether it’s binary data, files, or streams.

Method 1: Using Object.put()

If you’re using the higher-level resource interface of Boto3, Object.put is the way to go. Here’s how you can use it:

import boto3

# Initialize Boto3 S3 resource
s3 = boto3.resource('s3')
# Specify bucket name and object key (path)
object = s3.Object('my_bucket_name', 'my/key/including/filename.txt')

# Example data to be uploaded
some_binary_data = b'Here we have some data'

# Upload the binary data
object.put(Body=some_binary_data)

The Body parameter takes in the data you want to upload, which could be a byte string, a file-like object, or file data.

Method 2: Using Client.put_object()

For a lower-level approach, you might prefer the Client.put_object method. This method provides more granular control, which is beneficial if you need to configure additional settings for your request.

import boto3

# Initialize Boto3 S3 client
client = boto3.client('s3')

# Example data to be uploaded
more_binary_data = b'Here we have some more data'

# Upload the binary data
client.put_object(Body=more_binary_data, Bucket='my_bucket_name', Key='my/key/including/anotherfilename.txt')

This method is similar in use to Object.put(), but it allows you to specify additional parameters, such as metadata and ACL settings.

Uploading Files

If you’re looking to upload the contents of a file, you’re in luck; it’s just as simple. Here’s how you’d do that:

import boto3

# Initialize Boto3 S3 resource
s3 = boto3.resource('s3')

# Upload file data using Object.put
s3.Object('my_bucket_name', 'my/key/including/filename.txt').put(Body=open('/path/to/file.txt', 'rb'))

Note: When handling files, ensure they are opened in binary mode ('rb') for reading.

Transitioning from Boto 2 to Boto 3

If you’re coming from Boto 2, it’s important to familiarize yourself with Boto3’s methods. Boto3 integrates more cleanly with Python’s features and is thread-safe. For more on transitioning, visit the official AWS Boto3 documentation.

Now you’re ready to efficiently upload data to S3 with Boto3, streamlining your data storage needs.