TNS
VOXPOP
Favorite Social Media Timesink
When you take a break from work, where are you going?
Instagram/Facebook
0%
Discord/Slack
0%
LinkedIn
0%
Video clips on TikTok/YouTube
0%
X, Bluesky, Mastodon et al...
0%
Web surfing
0%
I do not get distracted by petty amusements
0%
Cloud Native Ecosystem / Storage

Amazon S3 Express One Zone Introduces Near-Real Time Object Storage

Able to process millions of requests-per-second, Amazon S3 Express One Zone promises to hasten data access speeds by 10x over the AWS s3 Standard edition, offering single-digit millisecond latency while reducing request costs by 50%.
Nov 28th, 2023 1:20pm by
Featued image for: Amazon S3 Express One Zone Introduces Near-Real Time Object Storage
AWS CEO Adam Selipsky introduces the new S3 bucket Tuesday at AWS Re:Invent 2023.

Amazon Web Services has created a high-performance version of its veritable S3 (Simple Storage Service), optimized for high-throughput and low latency, making it a candidate for large-scale parallel operations such as machine learning training and real-time data analysis.

Able to process millions of requests per second, Amazon S3 Express One Zone promises to hasten data access speeds by 10x over the AWS s3 Standard edition, offering single-digit millisecond latency while reducing request costs by 50%.

With Express, “You can bring your frequently accessed data next to your high-performance computing resources,” explained AWS CEO Adam Selipsky, during the company’s AWS Re:Invent user conference, being held this week in Las Vegas.

The new offering was designed to meet an emerging set of data-intensive applications that require fast runtimes. It can manage storage objects of any size but works best with large numbers of small objects. In these cases, the data may only be needed for a short period of time, though it is consulted frequently.

Think AI/ML training, which runs across hundreds or thousands of parallel compute nodes. Financial modeling, media processing, and real-time ad placement are other possible cases where it could be used.

“The extremely low latency provided by Amazon S3 Express One Zone, combined with request costs that are 50% lower than for the S3 Standard storage class, means that your Spot and On-Demand compute resources are used more efficiently and can be shut down earlier, leading to an overall reduction in processing costs, “explained AWS Chief Analyst Jeff Barr, in a blog post.

Each Amazon S3 Express One Zone directory bucket is confined to a single Availability Zone, and can be accessed using the usual assortment of S3 API functions: CreateBucket, PutObject, GetObject, ListObjectsV2, etc. What is new is a new bucket type and a new authentication model optimized for these workloads.

Pinterest, Ansatz and Clickhouse have already started using the storage technology. Pinterest, for instance, tested S3 Express One Zone in conjunction with its own open source MemQ pub/sub software, and saw 10x faster writes, while reducing total processing costs for an ML workload by 40%, Selipsky said.

Pricing is on a pay-as-you-go basis. It i $0.16/GB/month in the U.S. East (N. Virginia) Region. Amazon S3 Express One Zone is currently available in the U.S. East (N. Virginia), U.S. West (Oregon), Asia Pacific (Tokyo), and Europe with (Stockholm), with more regions to be added.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.