VMware vCloud Availability Installation-Part-6-Create vSphere Replication Cloud Service Host & Replication Server

The vSphere Replication Cloud Service is a tenancy aware replication manager that provides the required API for managing the service and all the components. vSphere Replication Cloud Service registers it self as a vCloud Director extension and will be  accessible through the vCloud Director interface.

Lets Deploy vSphere Replication Cloud Service Host using below command..




Create a new Variable with below information.

#export HCS01_ADDRESS=

we will use this variable in subsequent commands.

Next is to configure trust for vSphere replication certificate by vSphere using below command.


if command response is “OK” , that means we have successfully deployed vSphere Replication Cloud Service Host.

Create vSphere Replication Server

As we know vSphere Replication Server handles the replication process for each protected virtual machine. ideally it should be deployed one per manager instance.Run command as below to deploy HBR01.


if the deployment has been successfully completed , then you would get VM IP as success  message.


Next is to  create a variable with the IP address of above deployed VM, you can create additional variable if you have deployed multiple hms. this variable we will use in further commands. (Variables are casesensitive)

#export HBR01_ADDRESS=

Next step is to trust the vSphere replication certificate by vSphere using below command and it should return “OK”


This completes deployment of vSphere Replication Server appliance for vCAV.



VMware vCloud Availability Installation-Part-5-Deploy vSphere Replication Manager

vSphere Replication Manager manages and monitors the replication process from tenant VMs to the cloud provider environment. A vSphere Replication management service runs for each vCenter Server and tracks changes to VMs and infrastructure related to replication. these appliances can be horizontally scaled based on the requirement.

In production environment we must deploy one vSphere Replication Manager for each Resource vCenter Server but in this lab i will be deploying in my management vCenter only as i don’t have two separate vCenter one for management and another one for tenant called resource vCenter.

Let’s Start the deployment , again make a SSH connection to VCAV appliance and run below command to deploy replication manager.

you do not need to specify the location of Replication manager appliance location as described in the documentation , command picks up automatically from within the appliances.

Location of appliances on the VCAV appliance is – /opt/vmware/share/vCAvForVCD/latest


Run command as below to deploy HMS01 on vCAV appliance.


I am using –debug just to understand what is happening behind the scene but you can ignore it if you want and monitor the progress in vCenter, it must be deploying a VM with Name called “hms01” with IP “” as specified in –vm-address option.


once succeed and result on appliance will be displayed as deployed Virtual Machine IP address , that means it is successfully deployed virtual Machine.


Repeat the same process to deploy additional hms ,if you have many resource vCenter ideally you should have one per vCenter..

Next is to  create a variable with the IP address of above deployed VM, you can create additional variable if you have deployed multiple hms. this variable we will use in further commands.

#export HMS01_ADDRESS=

Next step is to trust the vSphere replication certificate by vSphere using below command and it should return “OK”


This completes deployment of vSphere Replication Manager appliance for vCAV, which will help us  in managing and monitoring the replication process from tenant VMs to the service provider environment.

VMware vCloud Availability Installation-Part-4-vCD Configuration and IP Plan

In continuation to deploy and configure vCloud availability , till now we deployed vCAV appliance and prepared its dependencies. in this post we will configure vCD to be used as DR site and will Plan IP schema for vCAV appliances which will be deployed next.

First setup a trusted connection between the RabbitMQ host and the vCloud Availability Installer Appliance.


Register RabbitMQ host with vCloud Director by running the following command on the vCloud Availability Installer Appliance.


if command responds “OK” then configuration has been successfully applied. you can also verify in vCD UI.

3.pngRestart vCloud Director Service after configuring AMQP settings, by using

#Service vmware-vcd restart

Check vCD Endpoints:

This step to verify that our environment is properly configured for vCloud Availability installation, by checking the vCloud Director endpoints for known problems.


if everything  has been done properly then we should get response as “OK”. This completes pre-configuration before proceeding with the installation of VCAV Replication/UI Virtual Machines but before we get into the installation of appliances , we need to plan IP address and DNS names for those appliances.

Here is my IP planning sheet for your reference.

                                                  Planning Sheet  
Machine Type DNS Name IP Address
vCloud Availability Portal vcav.corp.local
Docker Host for Cassandra and RabbitMQ docker01.corp.local
HMS hms01.corp.local
HCS hcs01.corp.local
HBR hbr01.corp.local
UI01 ui01.corp.local
UI02 ui02.corp.local

This Completes this post , in next post we will install appliances using above Table.


VMware vCloud Availability Installation-Part-3-Install Cassandra and RebbitMQ


RabbitMQ is an Open Source AMQP server that can be used to exchanges messages within a vCloud Director environment.  in production environments for high availability and scalability purposes, you can configure the RabbitMQ servers in a cluster.


Cassandra is a free and open-source distributed NoSQL database management system that stores metadata and supports storage of the metadata for replication services. for High availability you must deploy 3 Clustered nodes.

Since i don’t have resources in my Lab, so i am going ahead and deploy Cassandra and RabbitMQ in a single VM using containers and this is enough for our Lab deployment.

In our Part-1 we deployed VCAV, connect to vCloud Availability using SSH and run below commands to start docker service on vcav host.

#systemctl start docker – and once commands succeed check status using..


Create Password Files on Your vCloud Availability Installer Appliance

  • # mkdir ~/.ssh = > Create a directory  called “ssh”.
  • # chmod 0700 ~/.ssh -> Changes the directory permission.
  • # echo ‘VMware1!’ > ~/.ssh/.root – creates a  file names “root”  with having password. “VMware1!”
  • # echo ‘VMware1!’ > ~/.ssh/.vcd  – create a file named “vcd” with having vCD admin password stored.
  • # echo ‘VMware1!’ > ~/.ssh/.sso – This file will store “SSO” password.
  • # echo ‘VMware1!’ > ~/.ssh/.vsphere.mgmt – This file will store “vSphere” password
  • # echo ‘VMware1!’ > ~/.ssh/.cassandra.root.password
  • # find ~/.ssh -type f -name ‘.*’ -print0 | xargs -0 chmod 0600


This completes creation of password files. now lets create a IP pool.

Add a Network Protocol Profile

Basically a vSphere network protocol profile contains a pool of IPv4 and IPv6 addresses, IP subnet, DNS, and HTTP proxy server.VC assigns those resources to vApps or to virtual machines with vApp functionality that are connected to port groups associated with the profile. let’s create a network profile which our VM’s will use during their deployment.

  1. Go to data center click the Configure tab , click Network Protocol Profiles and edit Default profile.
  2. docker03
  3. associate a port group with profiles , on which you want your deployed vms get connected.
  4. docker04
  5. Enter your Subnet, Gateway, DNS server address , don’t forget to enable the pool and specify the IP range , so in my case i have assigned 20 IPs start with .160.
  6. docker05
  7. Specify DNS domain name and DNS search path.
  8. docker06

this completes creation of Network IP pool and its setting that VMs will use while deploying vCAV component vms.

Deploy a Docker Host

To deploy a docker host on vSphere Management Cluster run below command on vCAV appliance.


but before running this command , you can see certain variables has been used in the command, so first lets create those variables..

  • $MGMT_VSPHERE_ADDRES -> export MGMT_VSPHERE_ADDRESS=vcsa-01a.corp.local
  • $MGMT_VSPHERE_USER -> export MGMT_VSPHERE_USER=administrator@vsphere.local


so after creation of variables we run the above command on vcav vm using vcav docker create… which successfully created a docker VM in our management cluster.


Download Rabbitmq container on vCAV appliance using below command. for this step your vCAV appliance must able to reach to internet or if you have your own registry like VMware Harbor then you can pull from there.


Download Cassandra container on vCAV appliance using below command.for this step your vCAV appliance must able to reach to internet or if you have your own registry like VMware Harbor then you can pull from there.


Create three new Variables and password file as below:

  • export AMQP_ADDRESS=
  • export DOCKER01_ADDRESS=
  • echo ‘VMware1!’ > ~/.ssh/.amqp

Create RabbitMQ Container

Now lets create RabbitMQ Container using below command on vCAV appliance. and command returned “OK” that means my container creation was successful.


trust the vCAV connection with RabbitMQ as below.


Create Cassandra Container

Now lets create Cassandra Container using below command on vCAV appliance. and command returned “OK” that means my container creation was successful.


You can check the connectivity using telnet with particular port numbers for RabbitMQ and Cassandra servers. this post completes RabbitMQ and Cassandra containers deployment , we will configure these in subsequent posts.



VMware vCloud Availability Installation-Part-2-Configure SAML Federation

Using the vSphere SSO service as the SAML identity provider for the vCloud Director System organisation can be a more secure alternative to LDAP or a local account. When vCloud Director is federated with vCenter SSO, enables you to import system administrators from vSphere and this is required for VCAV to work properly. so let’s configure it.

Login to vCD as system admin user and navigate to Administration > System Settings > Federation and click on Metadata (3) and download Metadata.  it will be like this



then go to vSphere and upload this downloaded vCD Metadata.


Choose the File downloaded by Clicking in “Import from File” which we have downloaded and click on “Import”. This will complete the the metadata import from vCD to vSphere.


Now we need to Download SSO metadata file and need to import to vCD. login to vSphere , Go to “Configuration” -> SAML Service Providers -> Click on “Download”


Go to vCD login with Administrator , then go “Administration” -> “Federation” -> Tick on “Use SAML Identity Provider” – > then Browse the File which we have downloaded in previous step – Click “Upload” and Click “Finish”



Once mutual metadata sharing is completed , on vCD go to Administrator -> Users -> Import Users – you will see new Source called “SAML”


Choose SAML and manually enter “administrator@vsphere.local”  and click ok.


and new user has been added to vCD with System administrator  role.

10logout and login with vSphere SSO credential like”administrator@vsphere.local” and its password , it should be a successful login.

There is one more important setting that we need to do on vCD appliances , go to /opt/vmware/vcloud-director/etc/global.properties and add – extensibility.timeout=60.


This completes our vCD pre-requisite configuration , in the next post i will deploy cassandra and rabbitmq.







VMware vCloud Availability Installation-Part1-Deploy Appliance

So in Previous post i tried to explain what problem VMware solving with vCAv , now let’s get in to the installing of the components , there are two ways to install vCAv either using automated way or manually running few commands on vCAv appliance which will automatically install and configure stuff , this we can call semi automated and in next few posts i will be installing is using semi automated way as this gives me more comfort around understanding of what component is getting installed and integrate with whom…..

So lets get into the installation mode and first thing we need few Linux VM for

Cassandra 01 Nodes
RabbitMQ 01 Nodes
Cloud Proxy 02 Nodes

Since this is Demo environment , so i am not considering HA for any VMs.So first lets create a Cent OS VM with all the required pre-requisite installed on this and then template it , which will help us in saving considerable amount of time. same approach can be taken to your production deployment with customisation specification.

For this Demo , i am creating a new Linux VM based on CentOS-7-x86_64-Minimal-1804.iso  and install the OS, Once OS installation is completed,  Connect to the VM with SSH and first update yum:

  • #yum update yum and Reboot the guest OS.

Install the packages required by vCD.


#yum install alsa-lib bash chkconfig coreutils findutils glibc grep initscripts krb5-libs libgcc libICE libSM libstdc++ libX11 libXau libXdmcp libXext libXi libXt libXtst module-init-tools net-tools pciutils procps redhat-lsb sed tar wget which

I would suggest to install NTP to keep the VM clock in sync:

  • #yum install ntp

Configure ntp servers ,using VI, change the lines beginning with server to NTP servers. All components connecting to vCD should share the same NTP servers for accurate timekeeping:

  • #vi /etc/ntp.conf

Start the ntpd service

  • #systemctl start ntpd
  • #systemctl enable ntpd

Check ntpd is syncing to correct ntp servers using – #ntpq -p

There are lots of features depend on DNS , so i would suggest to install DNS bind utilities and verify that VM is able to resolve DNS queries.

  • #yum install bind-utils (to install nslookup)
  • #nslookup VMNAME
  • #nslookup VMNAME.DOMAIN.COM
  • #nslookup

For this lab environment, I turn off selinux as well as firewall, while in production deployment please choose correct configuration.

Go to selinux file and edit – #vi /etc/sysconfig/selinux and change SELINUX=enforcing to “SELINUX=disabled”.


To disable firewalld, run the following command as root: #systemctl disable firewalld and  stop firewalld, run the following command as root: #systemctl stop firewalld

Install VMware tools , reboot the VM to takes effect all the above changes. now we are done with OS configuration , shutdown the VM and Change the VM as template and deploy 7 VMs from this template. while i am deploying These external VMs, also we need to deploy vCAv Appliance. download the appliance from here.  and deploy the appliance by following below steps:

Choose OVF


Select appropriate Cluster/ host location to deploy35

Accept EULA67

Enter Domain name , IP address and others settings as per requirement.8

and Click Finish to Deploy.9

This Completes Template preparation and deployment of vCAv Appliance.


What is VMware vCloud Availability ?

While in Previous posts i was talking about VMware Based Cloud to Cloud DR  which means on One of the VMware Cloud Provider data center is having your VM running and you need to disaster recovery to another data center of the same provider thats where vCAv-C-to-C helps. but what about Customers who are having lots of VMs inside their own data center and wanted to move to VMware Cloud Provider data center or want to have DR as a Service on VMware Provider Datacenter. This is where VMware vCloud Availability helps.

vCloud Availability is a Disaster Recovery-as-a-Service (DRaaS) and  Migration Service solution that provides simple and secure asynchronous replication and failover  and fail back for vSphere managed workloads.  it supports vSphere version 5.5, 6.0, 6.5 and 6.7 environments. but only vSphere version 6.x provides replication in both directions.



  • Multi-tenant support for Provider Environment
  • Self-service portal for Migration,Protection Failover and Fail back per VM basis.
  • Initial Data seeding can be done using disk shipping to the provider and tenant.
  • RPO from 15 Minutes to 24 Hours .( if you have latest version of vSphere it supports 5 Minute RPO also).
  • Build in support for encryption of replication traffic.
  • Simple to deploy , manager and support compare to using other solutions and getting in to integration and support issues.
  • Supports up to 24 previous restore points.
  • efficient , robust and proven as it uses existing vSphere Replication engine.

Architecture & Components:-

The architecture of the vCAv relies on the service provider environment that will be considered as the replication target and the customer/tenant environment that deploys vSphere Replication to move the data to the service provider considers as source. In the service provider environment, multiple components operate together to support replication, secure communication, and storage of the replicated data. Each service provider can support recovery for multiple customer environments that can scale to handle increasing loads for each tenant, and for multiple tenants.

Cloud/Service provider environment runs multiple components listed as below together to support replication, secure communication with compute and storage. Provider must running production grade vCD environment to successfully run vCloud availability. i am not here listing down vCD components as i assume vCD is deployed and configured successfully. i will be covering only vCAv components only..

vCAv Components Components Use
Cloud Proxy

Cloud Proxy creates a public listening  TCP port to which vCloud Tunneling Agents connect  and communicate using secure web sockets. Multiple  instances can be deployed behind a load balancer to  support scaling.

vCloud Tunnelling Agent

It communicates with the Cloud Proxy and  is responsible for orchestrating tunnel creation for both  to-the-cloud and from-the-cloud tunnels.


An open source distributed database that stores  metadata related to the configured replications.


An open source Message Queue Service.  All requests to the Cloud Service are routed  through the messaging service.

vSphere Replication Cloud Services

Rest API Entry Point for Replication Operations , interacts with vCloud director for Multi-tenancy and placement.

vSphere Replication Manager

Existing component of Sphere Replication , Manager and Monitors Replication.


e Replication Server

Existing Component of vSphere Replication, handles replication of each protected VM.

vCloud Availability Portal

Tenet UI portal to manage replication on Cloud.


Above is the vCAv Diagram in conjunction with vCD and its components for your reference, thats cover the basic information about vCAv. below is the list of Service Provider Components needs to be deployed for successful DRaaS from your Cloud.

vCAv Components No of Nodes to be Installed
Cassandra 03 Nodes
RabbitMQ 02 Nodes
Cloud Proxy 02 Nodes
vSphere Replication Cloud Services 02 Nodes
vCloud Availability Administration Portal 01 Nodes
vSphere Replication Manager 01 Node
vSphere Replication Server 02 Nodes
vCloud Availability Portal 01 Node

in next few days i will be installing in my Lab based on above table and posting here in series of posts. In the mean time please refer documentation on installation and configuration on VMware website and get ready for installation and configuration.



Installing and Configuring VMware vCloud Availability for Cloud-to-Cloud DR – Part2

In Previous Post we had successfully installed VCAv-C2C on Site1 and following same steps i have installed on Site2 as well. now Next step is to establish a trust between vCloud Availability vApp Replication Service/Manager instances in these two different sites.we can initiate  site pairing from either one of the sites.In this post i will cover Pairing between sites and Administrator GUI walk through , so let’s Start….

In your Browser, Login to  https://Appliance-IP-Address:8046.


Go to Sites , Click on New site and In the Sites administration window, enter the site “B” vCloud Availability vApp Replication Service/Manager url , enter siteB appliance password and click OK.


Accept Certificate and if everything entered is correct and goes fine then we will see “Pairing Successful” message :).


To verify that the trust , Lets Go to Sites > Show all sitesThe new site displays in the Sites administration window.


We can reverify by going into “Diagnostics” -> Health -> Site Status


This completes Pairing , here is the Site2 View…


Login as Service Provider (SP admin GUI Walkthrough)

Cloud Provider System administrators can log in to the vCAv-C2C Portal to view information about DR workloads from the vCloud Director instances, monitor services health status and work as tenant organisation administrators.

Lets login https://Appliance_IP_Address:8443 with login as administrator@system or adminuser@system.

7 .            9

On vCloud Availability Portal  , after login as a service provider, you can review the Home dashboard. It contains read-only information that presents a site health summary.

A rolling bar on top of the dashboard view given over all health of the solution and we can pause/resume this operation.


Cloud Topology pane shows the incoming and outgoing vApps replications for both, the source and destination sites.


vApp Workload Status window lists the current health of vApp replications.

12.pngVM Workload Information window shows the health of the site you are currently logged in for VMs incoming and outgoing replications.


Provider vDC System Resource Utilization window tracks how much storage, memory, and CPU is used from the allocated system resources.


Top 3 Orgs window helps you to view the overall vCloud Director used storage by those organizations, and their vApps and VMs replication states.


Orgs page we can track the number of vCD orgs, usage of total storage, total pre-allocated CPU, and total pre-allocated memory for those orgs and other information.


DR Workloads page lists down workloads protected on source site1 and workloads protected to destination site2 respectively.


Administration page, to add a service component for monitoring and Configuration tab to Edit the vDC resource usage threshold shared by CPU, memory and storage. Optionally, you enable tenant vDC monitoring.


This post covers pairing and Admin walkthrough of Interface , in next post i will cover configuration of vApp/VM DR and tenant portal walkthrough…