AWS ‘s Storage Gateway solutions are designed to be used as a backup destinations for your infrastructure. There are 3 types of Storage Gateway solutions offered by AWS: File, Volume, and Tape Gateway.
Overall process is, you either deploy a local on-premises VM ( Hyper-V/Vmware VM), or a cloud based one which is in turn of course runs on AWS EC2 instance. You need to add an additional virtual disk to the Storage Gateway, to cache the data before it uploads it. The disk size has to be a minimum of 150GB, and you can add several drives for a total of 16 TB in size across all drives. You can’t allocate the drive with 150GB to begin with, and then increase its size down the road, you will have to add a new disk, if you want to increase the cache size.
There is an additional requirement for Volume and Tape Storage gateways; you will need to have an “Upload Buffer” drive(s) along with caching drives. Upload buffer drive has to be a minimum of 150GB and a maximum of 2TB in size.
As name suggests, Upload buffer’s purpose is straight forward; backup data from the cache drives are transferred to the upload buffer drive, and afterwards it gets copied to AWS’s storage, then buffer gets re-filled from the cache drive with more data, and so on.
Cache drives purposes is on the other hand is twofold; besides pumping more data into Upload Buffer, it keeps the cache of your most recent backup data, depending on the cache drive size. It will check the cache drive to see if the data is still available on the cache drive, if that is the case, then you don’t have to pull data down from AWS storage, and of course not incurring data transfer (charged per GB of data retrieved) charges from AWS.
Data Storage:Compression, de-duplication or deltas ?
File based Storage gateway (NFS) doesn’t make use of any compression or de-duplication mechanisms. But as per FAQ ” uses multipart uploads and copy put, so only changed data is uploaded to S3 which can reduce data transfer”. Basically, it will compare your current file with the one that was already uploaded, and upload only changed bits, which is still good, and should reduce Continue reading →