S3 buckets are the oldest and simplest of cloud concepts: hosted storage, any time. Simple is even in the name (Super Simple Storage). However, as the market penetration of S3 has grown, so has the demands of its users for more customization.

While S3 may seem straightforward to deploy, it doesn’t change the fact that complex, impactful, and potentially harmful settings such as versioning, ACLs, logging, and lifecycle configuration still need to be navigated. You need not look far to find many examples of improperly configured S3 buckets causing production incidents or costly data breaches.

Find below our guide to these configuration parameters: who cares about them, why they would be used, and some advice on best practices. We’ll be using the Terraform provider for S3 and its related sub-resources for this guide, as our opinion is infrastructure as code is the best way to deploy cloud infrastructure!

S3 Bucket Primary Parameters

bucket and bucket_prefix

• Teams That Care: Developers, Platform (or other central) Teams
• Why:
  • Developers need to name and organize buckets for application functionality.
  • Platform/Other Teams have an opinionated naming convention (usually a prefix) that they want teams to adhere to.
• Best Practice:
  • Consistent Naming Conventions: Implement and enforce standardized naming conventions to avoid conflicts and enhance security.

force_destroy

• Teams That Care: Developers, SREs
• Why:
  • Allows an S3 bucket to be destroyed without error, by enabling all objects in a bucket to be destroyed when the bucket is destroyed.
  • Developers might need to delete buckets during development or testing, and they don’t want to go through the hassle of deleting all objects first.
  • SREs ensure that deletions don't occur accidentally in production environments.
• Best Practice:
  • Default to false: Set force_destroy to false by default to prevent accidental deletions.
  • Explicit Overrides: Require explicit actions or approvals to set force_destroy to true when necessary.

tags

• Teams That Care: Everybody!
• Why:
  • Developers use tags for resource management within applications.
  • FinOps Teams track costs and optimize spending based on tags.
  • Security and Compliance Teams enforce policies and manage access based on tags.
• Best Practice:
  • Standardized Tagging Strategy: Add tags to any resource you create (Developers), following the tagging strategy set by central teams (FinOps/Security/Compliance/Platform).

Configuration Blocks

aws_s3_bucket_accelerate_configuration

• Teams That Care: Developers, FinOps
• Why:
  • Developers enable faster data transfers for performance.
  • FinOps Teams monitor and control the additional costs incurred.
• Best Practice:
  • Enable Only When Needed: Activate transfer_acceleration for buckets that require enhanced transfer speeds.

aws_s3_bucket_acl

• Note: aws_s3_bucket_acl resources replace acl and grant inline configuration blocks on the s3 resource.
• Teams That Care: Developers and Security
• Why:
  • Developers need their ACLs to be set up correctly to use the bucket.
  • Security Teams prefer using IAM and bucket policies over ACLs for better security and manageability. ACLs are a riskier way to manage access than roles and policies.
• Best Practice:
  • Use IAM and Bucket Policies instead: Favor IAM roles and bucket policies over ACLs to manage access controls.

aws_s3_bucket_cors_configuration

• Teams That Care: Security and Developers
• Why:
  • Developers need CORS configuration to be set up correctly to make use of the bucket.
  • Security and Platform Teams only care that developers aren’t using overly permissive or risky CORS configuration
• Best Practice:
  • Specific to use case: CORS policies are very dependent on use case, and while developers may understand what they want to achieve it is best to work with security teams or experts to verify what they are doing is safe.

aws_s3_bucket_lifecycle_configuration

• Teams That Care: Developers, FinOps, Compliance
• Why:
  • Developers want to make sure their application doesn’t break based on business requirements (historical data access, for example)
  • FinOps Teams want to optimize storage costs by transitioning or expiring costly objects, specifically by setting Glacier policies
  • Compliance Teams will ensure data retention policies meet regulatory standards.
• Best Practice:
  • Set abort_incomplete_multipart_upload and transition lifecycle rules.

aws_s3_bucket_logging

• Teams That Care: Everybody!
• Why:
  • Developers use logs for debugging and monitoring application behavior. Often they don’t set it up until after they need it, and then they set it up every time.
  • SREs leverage logs for system reliability and performance monitoring.
  • Security and Compliance Teams use logs for auditing and ensuring compliance.
  • FinOps Teams want to limit the potential cost of logging on high traffic buckets
• Best Practice:
  • Enable Server Access Logging: Activate logging for audit trails.

aws_s3_bucket_object_lock_configuration

• Teams That Care: Developers, Security, Compliance
• Why:
  • Developers use Object Locks to enforce data immutability.
  • Security and Compliance Teams require immutability for regulatory compliance and data protection.
• Best Practice:
  • Enable Object Lock for Sensitive Data: Use object_lock_enabled for buckets storing data that must remain immutable.
  • Configure Retention Policies: Define appropriate retention periods based on compliance requirements.

aws_s3_bucket_policy

• Teams That Care: Developers and Security
• Why:
  • Developers will use this to define access to their S3 bucket
  • Security teams will want to enforce broad controls over what resources can access S3 buckets in general
• Best Practice:
  • Principle of least privilege: Define specific actions (s3:GetObject, s3:PutObject, etc.) rather than broad permissions (s3:*), and specific resources rather than all resources.

aws_s3_bucket_replication_configuration

• Teams That Care: Developers care, and sometimes SRE…in unique cases
• Why:
  • Developers use this to enable replication, set a source, and destination, and any other rules (filtering, etc.).
  • SRE teams will care if your organization is running a gigantic resiliency program, or if your SRE is local to your team and knows about your buckets (unlikely)
• Best Practice:
  • Enable Replication: Turn it on if you want backups! You’re probably going to want backups.

aws_s3_bucket_request_payment_configuration

Almost nobody sets this anymore, it is related to an old billing model.

aws_s3_bucket_server_side_encryption_configuration

• Teams That Care: Compliance
• Why:
  • Compliance Teams require encryption to meet regulatory standards. It really only matters in a region where hard drives are likely to walk out of data centers.
• Best Practice:
  • Enable SSE-KMS: Use SSE-KMS for enhanced control over encryption keys.
  • Key Rotation: Implement regular key rotation policies to maintain security.

aws_s3_bucket_versioning

• Teams That Care: Developers, FinOps
• Why:
  • Developers may require versioning so that individual objects can be quickly restored after change or deletion.
  • Versioning can be expensive, so FinOps will look to make sure developers aren’t overzealous in their use.
• Best Practice:
  • Only use versioning where necessary, because it is expensive. If it has a good use case, it is probably worth paying for.

aws_s3_bucket_website_configuration

• Teams That Care: Developers, Security
• Why:
  • Developers may want to enable website mode
  • Security may not want to allow static website hosting based on S3 buckets
• Best Practice:
  • This one is pretty agnostic - totally your preference!

Conclusion

It may be tempting to think of S3 as a straightforward concept: turn it on, and put data in it. However, the sheer number and complexity of configuration options for S3 mean that it can easily be incorrectly deployed, with a significant negative impact.

At any organization, a variety of teams will be stakeholders in how S3 buckets are configured. While the primary challenge may appear to be configuring a single S3 bucket one time, the real challenge that companies face is repeatably configuring S3 that meets the expectations of everybody involved: from developers, to platform teams, to security, to FinOps and beyond.

If you have any feedback on these configuration perspectives, don’t hesitate to reach out and let us know!

‍