Introducing Amazon S3 Switch Supervisor within the AWS SDK for Java 2.x

We’re happy to announce the Developer Preview launch of the Amazon S3 Switch Supervisor – a excessive degree file switch utility for the Amazon Easy Storage Service (Amazon S3) within the AWS SDK for Java 2.x.

Utilizing Switch Supervisor’s easy API, now you can carry out accelerated uploads and downloads of objects to and from Amazon S3 and profit from enhanced throughput and reliability, which is achieved via concurrent transfers of a set of small elements from a single object. The Switch Supervisor is constructed on prime of the Java bindings of the AWS Frequent Runtime S3 consumer and leverages Amazon S3 multipart add and byte-range fetches for parallel transfers.

Parallel add through multipart add

For add operation, the Switch Supervisor makes use of the Amazon S3 multipart add API; it converts one single PutObjectRequest to a number of MultiPartUpload requests and sends these requests concurrently to attain excessive efficiency.

Parallel obtain through byte-range fetches

For obtain operation, the Switch Supervisor makes use of byte-range fetches. It splits a GetObjectRequest to a number of smaller requests, every of which retrieves a selected portion of the item. These requests are additionally executed via concurrent connections to Amazon S3.

In case you are utilizing Switch Supervisor 1.x, an object should be uploaded utilizing multipart add for it to be eligible to be downloaded with optimized efficiency. If an object was initially uploaded as a single object, the Switch Supervisor will be unable to speed up the downloading course of.

With the brand new Switch Supervisor 2.x, that is now not a limitation – downloading an object doesn’t rely on how the item was initially uploaded, and all downloads can profit from excessive throughput and concurrency.

Getting Began

Add a dependency for the Switch Supervisor

First, you’ll want to embody the separate dependency in your challenge.

<dependency>
  <groupId>software program.amazon.awssdk</groupId>
  <artifactId>s3-transfer-manager</artifactId>
  <model>2.17.123-PREVIEW</model>
</dependency>

Instantiate the Switch Supervisor

You’ll be able to instantiate the Switch Supervisor simply utilizing the default settings

S3TransferManager transferManager = S3TransferManager.create();

When you want to configure settings, we advocate utilizing the builder as an alternative:

S3TransferManager transferManager =
    S3TransferManager.builder()
                     .s3ClientConfiguration(cfg -> cfg.credentialsProvider(credentialProvider)
                                                      .area(Area.US_WEST_2)
                                                      .targetThroughputInGbps(20.0)
                                                      .minimumPartSizeInBytes(10 * MB))
                     .construct();

Add a file to Amazon S3

To add a file to S3, you’ll want to present the supply file path and the PutObjectRequest that must be used for the add.

FileUpload add = transferManager.uploadFile(b -> b.supply(Paths.get("myFile.txt"))
                                                     .putObjectRequest(req -> req.bucket("bucket")
                                                                         .key("key")));

add.completionFuture().be part of();

Obtain an Amazon S3 object to a file

To obtain an object, you’ll want to present the vacation spot file path and the GetObjectRequest that must be used for the obtain.

FileDownload obtain = 
    transferManager.downloadFile(b -> b.vacation spot(Paths.get("myFile.txt"))
                                       .getObjectRequest(req -> req.bucket("bucket")
                                                                   .key("key")));
obtain.completionFuture().be part of();

Conclusion

To learn to arrange and start utilizing the Switch Supervisor for the AWS SDK for Java 2.x, go to our Developer Information. If you wish to study extra, you may take a look at the supply code on GitHub. Check out the brand new Switch Supervisor at this time and tell us what you assume through the GitHub points web page!