Backing-up to AWS: Basics of Storage Gateways Types

AWS ‘s Storage Gateway solutions are designed to be used as a backup destinations for your infrastructure. There are 3 types of Storage Gateway solutions offered by AWS:  File, Volume, and Tape Gateway.

Overall process is, you either deploy a local on-premises VM ( Hyper-V/Vmware VM), or a cloud based one which is in turn of course runs on AWS EC2 instance. You need to add an additional virtual disk to the  Storage Gateway,  to cache the data before it uploads it. The disk size has to be a minimum of 150GB, and you can add several drives for a total of 16 TB in size across all drives. You can’t allocate the drive with 150GB to begin with, and then increase its size down the road, you will have to add a new disk, if you want to increase the cache size.

There is an additional requirement for Volume and Tape Storage gateways; you will need to have an “Upload Buffer” drive(s) along with caching drives.  Upload buffer drive has to be a minimum of  150GB and a maximum of 2TB in size.
As name suggests, Upload buffer’s purpose is straight forward; backup data from the cache drives are transferred to the  upload buffer drive, and afterwards it gets copied to AWS’s storage, then buffer gets re-filled from the cache drive with more data, and so on.
Cache drives purposes is on the other hand is twofold;  besides pumping more data into Upload Buffer, it keeps the  cache of your most recent backup data, depending on the cache drive size. It will check the cache drive to see if the data is still available on the cache drive, if that is the case,  then  you don’t have to  pull data down from AWS storage, and of course not incurring  data transfer (charged per GB of data retrieved) charges from  AWS.

Data Storage:Compression, de-duplication or deltas ?

File based Storage gateway (NFS) doesn’t make use of any compression or de-duplication mechanisms. But as per FAQ ” uses multipart uploads and copy put, so only changed data is uploaded to S3 which can reduce data transfer”. Basically, it will compare your current file with the one that was already uploaded, and upload only changed bits, which is still good, and should reduce the amount of traffic that traverses your network, and amount of data stored.
Volume based Storage (SAN snapshots) gateway compresses all the data, prior to uploading it to AWS. This potentially should reduce your  data transfer and storage charges.
VTL (Virtual Tape Library) based gateway doesn’t de-duplicate  or use compression for data storage.

Cache Drive size

Amazon suggests cache drive size equal of 20% of your backup data.  It might be wise to move the cache to a higher capacity and low cost storage solution , and  increase the cache size to 30-35%, If you recover data quite often.  Local cache size doesn’t mean you will be paying less for the AWS storage.  As previously mentioned, the larger the cache means the less time you will need to wait for the  data (recent backup) to be pulled down from AWS, and of course this should incur less data transfer charges overall.    Let’s not forget that it cost more to download the data from AWS, than it cost to store or upload.
Make sure to setup the CloudWatch to monitor the Storage Gateway  to identify necessary metrics. Over time this should help with narrowing down the size for  cache drive size, as well as how much cache you use locally vs download from AWS storage while restoring  new/old data.

Fees and Pricing

Storage gateway has a per Gb of data transfer price associated with it. But, the price per/Gb turns into a monthly flat $125 fee if you upload more than $125 worth of data for that month.

Then there are various kinds of fees based on the Storage Gteway type; Tape, File, it Volume. Fees are for  data storage, type of S3 storage, number of requests made, location (region) where the data stored,  data transfer (download), and archived data retrieval  (if data is archived, which cost much less than regular storage) fees.

Accessing the Storage Gateway

There are couple of things you need to keep in mind, if you are looking to manage the SGs remotely.  Storage Gateway will not be visible /accessible in your AWS Storage Gateway panel if you try to view them outside of your network.  To be more precise, your Storage Gateway it will not show up in your AWS Storage Gateway dashboard, unless you accessing the AWS dashboard from within your company network, and network you are in can access the SG/Backup Network. This might happen if the network with servers is heavily firewalled, and isolated  from the user side of the network.

This might happen if the network with servers is heavily firewalled, and isolated  from the user side of the network.

 The Storage Gateway must be able to access the several AWS endpoints for it to function properly. These are:
  • anon-cp.storagegateway.region.amazonaws.com:443
  • client-cp.storagegateway.region.amazonaws.com:443
  • proxy-app.storagegateway.region.amazonaws.com:443
  • dp-1.storagegateway.region.amazonaws.com:443
  • storagegateway.region.amazonaws.com:443

As well as the following endpoint on the CloudFront, it contains the list of regions and required endpoints for Storage Gateways.

 

 Region indicates your gateway region endpoint. If your Storage Gateway  deployed in US West Oregon region, then the endpoint will look like the following: storagegateway.us-west-2.amazonaws.com:443.

You could either allow all AWS regions or only the ones your SG needs an access to,  depending in your security requirements.

Monitoring the Gateway

You can monitor each gateway for number of various metrics using CloudWatch Metrics. You will need to identify GatewayID and GatewayName before being able to do so.  As a minimum you should monitor  how much data is used from local cache to restore new/old data, Cache and Buffer drives usage, data transferred, queue writes, working storage and Upload buffer free/used .
You should keep an eye on  the monitoring screen, at least first half a dozen backups or so. This will help you to identify any bottlenecks during your backups.  Bottlenecks could be on the cache/buffer drives; disk size is not large enough, or  a network related; not enough or a throttled bandwidth.

Would I use AWS to backup my data to it? It depends.  If you remember there are 3 types of Storage Gateways; File, Volume, and Tape.  You could even use the Volume  SG to send your hourly/daily/weekly SAN snapshots to AWS.  AWS is capable of delivering  any amount of storage you might need. As long you are solvent to pay them for it. But I doubt I would use it for my daily backups or SAN based snapshots. It might end up costing me more in time that it  takes to retrieve the data and in Uncle Sam’s currency  ( $$$) than it’s worth.

Given all that, I do see myself using File or VTL Storage Gateways  for monthly, off-site backups. Just make sure you have enough $$$ to pay for all those TBs of data.  Just 100 TB of archived data on VTL ( the most cheapest solution) will cost you about $525 per month, that is $400 for archives, and $125 monthly Storage Gateway fee.

 

References:

http://docs.aws.amazon.com/storagegateway/latest/userguide/AWSStorageGatewayMetricsList-common.html

http://docs.aws.amazon.com/storagegateway/latest/userguide/Main_monitoring-gateways-common.html#UsingCloudWatchConsole-common

https://aws.amazon.com/getting-started/projects/replace-tape-with-cloud/services-costs/

https://aws.amazon.com/storagegateway/faqs/

http://docs.aws.amazon.com/storagegateway/latest/userguide/Requirements.html#requirements-host

http://docs.aws.amazon.com/storagegateway/latest/userguide/Main_TapesIssues-vtl.html#creating-recovery-tape-vtl

http://docs.aws.amazon.com/storagegateway/latest/userguide/StorageGatewayConcepts.html#storage-gateway-vtl-concepts

 

Upgrading firmware and bios on Cisco’s C220/C240 series servers

 

In order to upgrade the Bios and hardware firmware on Cisco C series series we need to have the following:

  • To know the c-series product model # you looking to upgrade the firmware on
  • Download the latest iso image with server firmware.

How do we achieve it:

  • download HUU ( Host Upgrade Utility) ISO image to your workstation
  • login to C-IMC ( Cisco Integrated Management Controller) via a browser
  • open up Java based virtual KVM
  • attach the iso image from your workstation
  • reboot server
  • Press F6 during the boot
  • boot from virtual DVD
  • run the pre-upgrade check
  • upgrade bios and firmware on raid controller, mlom, NIC cards, and etc reboot the system

Server Product model and version can easily be identified under “Server Summary” when you login to the server via CIMC.

You will need to have an account with Cisco to download the server Continue reading →

WSUS 2016 error: Report Viewer 2012 Redistributable is required for this feature.

Trying to generate reports on SUS 2016 gives out  an error stating that its missing  “The Microsoft Report Viewer  2012 Redistributable is required for this feature”

Head over to Microsoft’s website to download and install the “The MICROSOFT® REPORT VIEWER 2012 RUNTIME”  https://www.microsoft.com/en-us/download/details.aspx?id=35747

During its installation  you might get another error, this time warning you about missing ” Microsoft System CLR Types for SQL Server 2012

And then Report Viewer installations fails with below error.

 

If that is the case, then you will need to do the following:

TL;DR

  1. Download and install Microsoft® System CLR Types for Microsoft® SQL Server® 2012
  2. Re-install the Report Viewer 2012

Long version

Continue reading →

Migrating WSUS to a new Windows 2016 server, and 2014 standalone SQL server

Plan is to retain current WSUS data and configuration while moving the SUS service from old Windows 2016 TP 5 server to an new fully licensed Windows 2016 Standard server, and move database from WID to a standalone SQL 2014 server.

1)   Setup a new Windows 2016 Server, update. Patch, reboot. Install the WSUS role on it, choose the WID database during the install. Make sure to point to a drive\folder for wsusContent.
a. Copy the wsusContent folder from old server to the new one. Make sure you placing it to the proper drive\path, you identified during post-install configuration for new WSUS service.
Continue reading →

Teradici APEX 2800: how can I quickly confirm if the offloading works or not.

I recently got my hands on teradici’s 2800-lp (low profile) offloading card for the a Horizon view 7 VDI PoC implementation. The one I am using is a PCI Express version of the card that can be installed on any server with PCIe gen2 x4/x8 or x16 slot.

There are also MXM Type A with Mezzanine Adapter and an Amulet Hotkey DXM-A versions. They are designed for HP’s Gen8 and 9 blade servers, and for Dell M Series blades respectively. In either case, standalone, mezzanine adapter, or an amulet hot key, you can install up to 2 of such cards per server.

There are plenty of choices on the market for a GPU offloading, some of which are Nvidia’s Grid K1/K2, Nvidia Tesla K40 /K80, and AMD’s FirePro S7150 (x2) GPU cards.

Continue reading →

Categories: vdi

VCDX workshop notes

Below are the notes I have taken during the NJ/NY VCDX workshop at @iamAntonZ  ‘s place. We had a great opportunity to have 2 VCDX panelists giving the VCDX workshop, Niran Even-Chen ( @NiranEC) and Agustin Malanco (@agmalanco).  You could find more info on this at http://nycvmug.blogspot.com/2016/06/re-cap-of-njnyv-vcdx-workshop.html 

 These notes are by no means to be used or relied on as primary source for your VCDX preparation.

Your design must include all the VCDX blueprint points.

Don’t use any kind of blogs as an official source for your references . Official VMWARE docs are the only sources that you can quote or reference to in your VCDX design.

One of the main changes to VCDX is removal of the “Troubleshooting” scenarios from the defense.

One design can be submitted 2 more times, if it did not get accepted the 1st time around. This means that you will need to make some modifications to your design, before resubmitting it again. You won’t get any detailed information on why it failed the submission, but you might get a generic response such as ” Storage or Networking needs more info”.

Same design could potentially be submitted by 3 different people, and they must submit it at the same time for the same VCDX track. Also, each and every applicant must know the design inside out, not just the portion he or she design.

It takes an average of 4-7 month to prepare the design and validate it. Once you got accepted for defense, get yourself an official VCDX mentor. The mentor will not be fixing your design for you, but rather guide you and advise you on proper documentation. Lookup the directory, and work with him to help you out on this. Just keep in mind that they are ( VCDX mentors) are not being paid for it and do it rather on their own time.

Continue reading →

Recovering VMs after a vmware’s Purple Screen of Death (PSOD)

I had an interesting case a while ago. One of our test ESXI hosts running ESXi version 5.5 has crashed taking down number of test environments with it.

All the attempts to bring the host back to life was in vain, as each reboot was giving us a Purple screen of Death.  We needed these test environments up and running ASAP, and due to time limit on hands, it was decided to :

  1. keep the current VMFS datastore and install partition intact,
  2. Install ESXI 5.5 from scratch onto a USB flash drive, and
  3. Re-create the vSwitches
  4. Re-import the VMs into inventory
  5. Re-import and start up the vCenter
  6. Login to vCenter and bring up the test environment back online

Luckily this test server- a Cisco UCS C220- had its CIMC enabled, and IP configured for remote access.  So, I was able to connect to the hosts’s remote management panel (CIMC) and install the new Esxi via Continue reading →

Some useful UNIX shell commands for VMware admins

These are the esxi host log files one needs to be quite familiar with. These logs should be checked depending on the issue you facing, and trying to troubleshoot.

  1. /var/log/auth.log: ESXi Shell authentication success and failure.
  2. /var/log/lacp.log: Link Aggregation Control Protocol logs.
  3. /var/log/hostd.log: Host management service logs, including virtual machine and host Task and Events, communication with the vSphere Client and vCenter Server vpxa agent, and SDK connections. Continue reading →

Error while upgrading from VMware vCenter 5.1 to vCenter 6.0

 

I have not seen any special feature improvements in 5.5 over 5.1 that would have benefited our environment, and as with any other major new releases  was patiently waiting for VMware to come out with Update 1 for vSphere 6. Originally I had the SSO, VMware Update Manager, and vCenter each running on its own server. Databases for SSO and vCenter are separated to a standalone SQL 2008 R2 server.   I ended up combining the VUM and vCenter on one single server, and upgrading the SSO to PSC (Platform Services Controller) and kept it separate, in case if we go with 2nd vCenter in the future.

Upgrading vCenter from 5.1.x to 6.0 is quite straight forward process, mount the vCenter 6 VMware-VIMSetup iso image to the vCenter server, and run the installer. It will recognize that there is a previous version installed and offer you to upgrade it.But first, make a backup of working production servers, before the upgrade. Shutdown the VMs, and copy the SSO and vCenter vmdks to a separate folder – in case if snapshots decide to take a break from work. Take the SSO, vCenter snapshots, and of course backup the SQL databases, if you have them running separately.

Don’t despair, if you receive the below error during vCenter upgrade: Continue reading →

Working with OpenDJ Server configs

This post provides examples of dsconfig usage for OpenDJ config management. I might expand and update the post in the future, as i find more info.

All the logs audit trails will be saved as long as you use OpenDJ supplied tools for modifying the DJ configs:
Tools are:

  • OpenDJ Control Panel and
  • openDJ command line tools, located under /bin directory, of the OpenDJ install

You could also modify the config files directly, via vi or nano, if you don’t care about proper audit trail or automatic config file backups, each time you modify something. Continue reading →