How to create AWS S3 Buckets with vRA Cloud /8.x

Business Requirement:

The customer was using vRA Cloud with extensive hybrid cloud offerings. The customer had a legacy 7.x vRA environment connected to the public clouds (AWS and Azure). They were using custom vRO objects based on dynamic types for AWS S3 storage catalog. After moving to vRA Cloud they continued using the dynamic types for S3 objects in vRA Cloud. I decided to do a demo for the customer just to show them how easy it is to now consume some of these public cloud objects like S3 storage with vRealize Automation Cloud.

Solution

Onboarding

The solution starts with creating and onboarding the AWS cloud account to vRA C.

  1. Create an AWS Account.
  2. Create an “IAM” user with appropriate permissions. For my demo I provided AdministratorAccess
  3. Copy the access key id and secret access key for the user. It will be required while onboarding the account to vRA.
  4. Go to vRA Cloud –> Infrastructure –> Amazon Web Services
  5. Add the Cloud Account Name, Access key ID , Secret access key copied in the above steps.
  6. Configure the regions in which provisoning will be allowed.
  7. Create the cloud zone for the selected region.
  8. I usually add my lab tags for the capabilities.

Cloud Template

vRA Cloud now provides all the AWS S3 objects as simple drag and drop items to the canvas. It’s as easy as creating a vsphere machine with a cloud template now. For the purpose of this demo, I created the following.

  1. AWS S3 bucket with the following features
    • Bucket version enabled.
    • Object Lock enabled.
    • Force Destory enabled.
    • Lifecycle rule.
  2. AWS S3 bucket policy with public access ( Please note that this policy was just to demo the policy feature. This should not be used unless there is a specific requirement)
  3. AWS S3 bucket object with a static key (filename) and fixed content.

Cloud template

Custom Form

AWS Console

S3 Object has been created with the required properties and policy

S3 Bucket created

Bucket properties: Versioning has been enabled as per the cloud template.

Lifecycle rule created

Bucket Policy Created

File object in the S3 bucket created.

Additional Resources

If you are looking for the available properties and their accepted values, check the VMware documentation. Since the OOB S3 objects use terraform in the backend, terraform provides better documentation and example for these properties.

AWS S3 Buckets:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket

https://registry.terraform.io/modules/terraform-aws-modules/s3-bucket/aws/latest

AWS S3 Bucket Policy :

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy

You can also generate the policy using the native AWS console and pass it in your cloud template as JSON.

AWS Policy Generator :

https://awspolicygen.s3.amazonaws.com/policygen.html

Bucket Policy Examples :

https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html

AWS S3 bucket object:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket_object

vRA Cloud Template (blueprint)

Conclusion

Out Of Box S3 integration provides most of the required features. However, I found that “Intelligent-Tiering Archive configurations” are currently not available. Customers can configure the day 2 operations on these objects and unlock a range of operations that can be done by API. It is important that we avoid dynamic types for anything that is readily available as an OOB integration from stability and future scalability perspective.

Author

Barjinder Singh

This Post Has 2 Comments

  1. Jaspreet

    Very informative and well documented as always

  2. Jason

    Good article, How can you turn off public access on S3 bucket creation? the acl property doesn’t really do it. Is there some other property that would? can you show an example of how to handle that?

    When you login to the aws console it would be under bucketname > permissions > Block public access (bucket settings)

    I tried adding these properties, and they show up as properties inside VRA after build, but they didn’t seem to do anything on the AWS side.

    PublicAccessBlockConfiguration:
    BlockPublicAcls: true
    BlockPublicPolicy: true
    IgnorePublicACLs: true
    RestrictPublicBuckets: true

Leave a Reply