Amazon S3 Files: Access S3 Buckets as File Systems

Amazon has announced Amazon S3 Files, a new file system designed to connect any AWS compute resource with Amazon Simple Storage Service (Amazon S3). This innovation aims to provide fully-featured, high-performance file system access to data stored in Amazon S3, allowing buckets to be accessed as file systems.

With S3 Files, changes to data within the file system are automatically reflected in the S3 bucket, providing fine-grained control over synchronization. The system supports attachment to multiple compute resources, facilitating data sharing across clusters without the need for duplication.

The service eliminates the traditional trade-off between the cost-effectiveness and durability of Amazon S3 and the interactive capabilities of a file system. Amazon S3 can now serve as a central hub for an organization’s data, accessible from any AWS compute instance, container, or function, supporting production applications, machine learning model training, and agentic AI systems.

S3 Files allows any general-purpose bucket to be accessed as a native file system on Amazon Elastic Compute Cloud (Amazon EC2) instances, containers running on Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Lambda functions. The file system presents S3 objects as files and directories, supporting Network File System (NFS) v4.1+ operations, including creating, reading, updating, and deleting files.

The system uses high-performance storage for frequently accessed file metadata and content, ensuring low-latency access. For files requiring large sequential reads, S3 Files serves them directly from Amazon S3 to maximize throughput. Byte-range reads minimize data movement and costs by transferring only the requested bytes.

S3 Files employs intelligent pre-fetching to anticipate data access needs and provides control over what is stored on the file system’s high-performance storage, allowing users to optimize for specific access patterns by loading either full file data or metadata only.

Underneath, S3 Files utilizes Amazon Elastic File System (Amazon EFS) to deliver latencies of approximately 1ms for active data. The file system supports concurrent access from multiple compute resources with NFS close-to-open consistency, making it suitable for interactive, shared workloads that involve data mutation.

S3 Files integrates with AWS Identity and Access Management (IAM) for access control and encryption, allowing the use of identity and resource policies to manage permissions at both the file system and object level. Data is encrypted in transit using TLS 1.3 and at rest using Amazon S3 managed keys (SSE-S3) or customer-managed keys with AWS Key Management Service (AWS KMS).

The service uses POSIX permissions for files and directories, checking user ID (UID) and group ID (GID) against file permissions stored as object metadata in the S3 bucket. It can be monitored using Amazon CloudWatch metrics for drive performance and updates, and AWS CloudTrail for logging management events.

Amazon S3 Files is available in all commercial AWS Regions. Pricing is based on the portion of data stored in the S3 file system, small file read operations, all write operations to the file system, and S3 requests during data synchronization between the file system and the S3 bucket.