|
|
As someone who used to work at Sun Microsystems, where ZFS was invented, I’ve always loved working with storage systems that offer instant volume copies for my development and testing needs.
Today, I’m excited to share that AWS is bringing similar functionality to the Amazon Elastic Block Store (Amazon EBS) with the launch of Amazon EBS Volume Clones, a new capability that lets you create instant copies of your EBS volumes within the same Availability Zone.
Many customers need to create copies of their production data to support development and testing activities in a separate non-production environment. Until now, this process required taking an EBS snapshot (stored in Amazon Simple Storage Service (Amazon S3)) and then creating a new volume from that snapshot. Although this approach works, the process creates operational overhead due to the multiple steps.
With Amazon EBS Volume Clones, you can now create copies of EBS volumes with a single API call or console click. Copied volumes are available in seconds, providing instant access to your data with single-digit millisecond latency. This makes Volume Clones particularly useful for quickly setting up test environments with production data or creating temporary copies of databases for development purposes.
Let me show you how Volume Clones works
For this post, I created a small Amazon Elastic Compute Cloud (Amazon EC2) instance with a volume attached. I created a file in the root of the file system using the command echo "Hello CopyVolumes" > hello.txt.
To start the copy, I open a browser in the AWS Management Console and go to EC2, Elastic Block Shop, Volumes. I select the volume I want to copy.
Please note that at the time of this post, only encrypted volumes can be copied.
On Action menu, I choose Copy volume choice.
Next, I choose the details of the target volume. I can change Volume type and edit Size, IOPSand Throughput parameters. i choose Copy volume to start the Volume Clone operation.
The copied volume will enter the Creating status and will be available within seconds. I can then connect it to an EC2 instance and start using it immediately.
Data blocks are copied from the source volume and written to the background volume copy. The volume remains in Initialization status until the process is complete. I can follow its progress using describe-volume-status API. The initialization operation does not affect the performance of the source volume. I can use it normally during the copying process.
I like that the copied volume is immediately available. I don’t have to wait for it to finish initializing. During the initialization phase, my copied volume provides performance based on the lowest of: a baseline of 3000 IOPS and 125 MiB/s, the guaranteed performance of the source volume, or the guaranteed performance of the copied volume.
After the initialization is complete, the copied volume becomes fully independent of the source volume and provides its full performance.
Alternatively, I can use the AWS Command Line Interface (AWS CLI) to start the copy:
aws ec2 copy-volumes \
--source-volume-id vol-1234567890abcdef0 \
--size 500 \
--volume-type gp3After creating a copy of the volume, I attach it to the EC2 instance and mount it. I can check if the file I created at the beginning is present.
First I mount the volume from my laptop using attach-volume command:
aws ec2 attach-volume \
--volume-id 'vol-09b700e3a23a9b4ad' \
--instance-id 'i-079e6504ad25b029e' \
--device '/dev/sdb'Then I connect to the instance and enter these commands:
$ sudo lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1
├─nvme0n1p1 xfs / 49e26d9d-0a9d-4667-b93e-a23d1de8eacd 6.2G 22% /
└─nvme0n1p128 vfat FAT16 3105-2F44 8.6M 14% /boot/efi
nvme1n1
├─nvme1n1p1 xfs / 49e26d9d-0a9d-4667-b93e-a23d1de8eacd
└─nvme1n1p128 vfat FAT16 3105-2F44
$ sudo mount -t xfs /dev/nvme1n1p1 /data
$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.0M 0 4.0M 0% /dev
tmpfs 924M 0 924M 0% /dev/shm
tmpfs 370M 476K 369M 1% /run
/dev/nvme0n1p1 8.0G 1.8G 6.2G 22% /
tmpfs 924M 0 924M 0% /tmp
/dev/nvme0n1p128 10M 1.4M 8.7M 14% /boot/efi
tmpfs 185M 0 185M 0% /run/user/1000
/dev/nvme1n1p1 8.0G 1.8G 6.2G 22% /data
$ cat /data/home/ec2-user/hello.txt
Hello CopyVolumesThings you should know
Cloning a volume creates copies in the same Availability Zone as your source volume. You can only create copies from encrypted volumes, and the size of your copy must be equal to or larger than the source volume.
Volume cloning creates crash-consistent copies of volumes, just like snapshots. For application consistency, you must suspend application I/O operations before creating a copy. For example, with PostgreSQL databases you can use pg_start_backup() and pg_stop_backup() function to suspend writing and create a consistent copy. At the operating system level on Linux with XFS you can use xfs_freeze command to temporarily suspend and resume access to the file system and ensure that all cached updates are written to disk.
Although Volume Clones creates point-in-time copies, it supplements rather than replaces EBS snapshots for backup purposes. EBS snapshots remain the recommended solution for data backup and failover protection at the AZ and volume level. Snapshots provide incremental backups to Amazon S3 with 11 nines durability compared to Volume Clones, which maintains EBS volume durability (99.999% for io2, 99.9% for other volume types). Consider using Volume Clones specifically for test and development environment scenarios where you need immediate access to volume copies.
Copied volumes exist independently of their source volumes and continue to incur standard EBS volume charges until you delete them. To effectively manage costs, implement governance rules to identify and remove duplicate volumes that are no longer needed for your development or testing activities.
Price and availability
Volume Clones supports all EBS volume types and works with volumes in the same AWS account and Availability Zone. This new feature is available in all AWS commercial regions, select on-premises zones, and AWS GovCloud (US).
For pricing, you will be charged a one-time fee per GiB of data on the source volume at launch and the standard EBS price for the new volume.
I find Volume Clones particularly valuable for database workloads and continuous integration (CI) scenarios. For example, you can quickly create a copy of your production database to test new features or troubleshoot problems without affecting your production environment or waiting for data to be hydrated from Amazon S3.
To get started with Amazon EBS Volume Clones, visit the Amazon EBS section of the console or view the EBS documentation. I look forward to hearing how you use this capability to improve your development workflows.
— self