Whilst updating my ApplicationHA lab images for various Symantec Partner events that i will be running later this year I noticed that I was seeing a “This Host has no Management network redundancy” message on my ESX servers, whilst it doesn’t effect my labs from running properly it is a bit off putting seeing warning triangles next to the ESX servers. I typically use a single NIC for creating ESX servers for the lab and this is the reason why it gets displayed, in real production and test environments you would normally have redundant NICs configured and this message wouldn’t appear.
Here are the steps that will remove the warning using the “das.ignoreRedundantNetWarning” attribute.
There are a couple of ways to remove the message from appearing, the correct way is to add redundancy to the management networks by adding in another NIC to the configuration or alternativeliy you can use the “das.ignoreRedundantNetWarning” attribute in the advanced vMwareHA/vSphere HA settings.
1. From within the vSphere Client right click on the cluster and edit settings.
2. Select (Depending on your version) the VMwareHA or vSphere HA feature and click “Advanced Options”
3. In the options column enter the tag “das.ignoreRedundantNetWarning”
4. In the value column enter “true” for the value.
5. Click on “OK” and then “OK” again to make the changes in the advanced options
6. right click on the ESX server which is diplaying the warning symbol and select “Reconfigure for vSphere HA” or “Reconfigure for VMware HA”
Thats it, once the reconfigure task completes the warning should disappear and you are left with a warning icon clean ESX server.
of course on the screen shot I do have a warning sign on the Win2003-SQL virtual machine but thats a story for another day….
Well it’s certainly been a busy couple of months, October had kept me very busy what with Symantec Vision conference in Barcelona, I presented in various breakout sessions and also ran the ApplicationHA lab which showed off how to configure Backup Exec with auto recovery of VMware VMs. Then within a couple of weeks we had the VMworld conference in Copenhagen which saw great attendee crowds on the Symantec stand as well as an unforgettable Symantec party, and not forgetting the number of discussions I had for ApplicationHA with attendees which was overwhelming and very enjoyable to say the least.
So whats kept me busy the last month or so has been our newest product release, on November 7th ApplicationHA 6.0 was released and the towards the end of that week the Backup Exec vSphere management plugin was available. So for me, most of November I have been busy getting product documents, whitepapers and demos ready for the launch and with that I thought I would screen record the lab I ran for auto recovery in Barclelona and just show how easy it really is to install and configure the whole thing in under 30 mins. I have compressed the whole 1 hour lab which includes how to test the whole setup in under 15 mins, which was to save time waiting for reboots and backup and restore times but hopefully you should see how quick and really easy the whole thing is to install, configure and manage.
YouTube Link :- http://youtu.be/x3RmCzlwrAI
for more information and to try this yourself download the plugin and trailware from symantec.com
Learn What’s New in the V-Ray update for Backup Exec 2010 R3! http://www.symantec.com/docs/TECH168425
Symantec Backup Exec™ Management Plug-in for VMware® http://www.symantec.com/docs/TECH169148
Symantec ApplicationHA 6.0 trailware https://www4.symantec.com/Vrt/offer?a_id=115758
It certaily feels like a Christmas tech fest this week what with Workstation8, SRM5, View 5 and now Msft Windows 8 Developer Preview all available to download. They can all join the new versions vSphere ESX 5.0 and vCloud Director in my test lab. but not wanting to be left out of the party, I have got a new build of Symantec ApplicationHA 6.0 pre Beta to play with, just in time for my lab at Vision EMEA in Barcelona in a couple of weeks time.
Upon installing the Developer Preview of Windows 8 onto my VMware Workstation 8 test system I did find one oddity, it was a bit frustrating especially during the final stages of the installation. The installation itself was a breeze as you would expect with any OS that VMware supports today, just remember to use “Windows 7” for 32bit or “Windows 7 x64” for 64bit as OS and your good to go. You can accept the majority of the remaining defaults when configuring the VM with just one exception, watch you memory allocation, I did see an issue when I allocated over 4Gb to my VM. All seemed fine during the actual Windows installation it was only when installing VMtools the installation hung and in one case when it did install successfully, upon the reboot I was presented with a perpetual blank screen.
I then brought down my memory allocation to 3Gb and all worked like a charm. I believe the tipping point was at 3840mb of RAM allocation so maybe if you want to set at 3.5Gb then all will be OK.
After the memory change everything worked as you would expect. The Metro UI interface moves really well once VMtools are installed.
Working with multiple hypervisors has been great in experimenting with certain features or just gaining an understanding how to configure something similar to a customers environment. Until now my lab environment is littered with multiple servers or harddisks with different images upon them but having the ability to work with these technologies with minimal hardware in lab environment or even mobile out and about would be great. Until recently this was not possible due to the fact that nested 64bit VMs were not possible with Workstation 7.x or even ESX 4.x for that matter. But now with ESX5i and Workstation 8 this is now possible.
One caveat to add though is that your hardware must be VT-x capable which of course most are these days but with one additional and really important CPU instruction set of EPT (Enhanced Page Tables) or RVI from AMD must be present, searching the various chip manufactures sites it seems that if your running Nehalem or i7 from Intel at least then your good to go. Luckily I have both an i7 on my laptop and Nehalem in my HP ML G6 home test lab. So lets take a look at ESX5i first and see what needs to be set for Hyper-v to load. VMware ESX5i configuration.
1. First you need to enable hardware virtualization by modifying the etc/vmware/config file. Enable SSH via tech support mode and putty to the ESX5i server
2. Once connected with putty : # echo ‘vhv.allow = “TRUE” ‘ >> /etc/vmware/config
3. Next create your Virtual Machine hardware, I personally used hardware version 8 to make things easier with configuration.
4. Before you get to booting up the VM and installing Hyper-V you need to add two lines the virtual machines config file .vmx
You can either add these via the vSphere Client in the settings of the virtual machine > Configuration Parameters, or doing it from command-line
To add them using command-line move back in SSH > change into the directory where you Hyper-V VM is installed
For example config file where my VM is located is called Hyper-V.vmx. Type the following commands: # echo ‘monitor.virtual_exec = “hardware” ‘ >> Hyper-V.vmx # echo ‘hypervisor.cpuid.v0 = “FALSE” ‘ >> Hyper-V.vmx
5. Next there are a couple of changes to be made with the CPU configuration.
in the VM settings > Options > CPU/MMU
Virtualization make sure you select the option to pass the Intel EPT feature.
6. Next move to the Options area > CPUID Mask click on Advanced
Add the following CPU mask Level ECX: —- —- —- —- —- —- –H- —-
8. Finally you are now ready to install Windows 2008/R2 and enable the Hyper-V role. VMware Workstation 8 configuration.
For configuration with Workstation 8.0 things are a little easier.
1. Start off by creating a new VM again I used version 8 hardware
2. Configure the VM with RAM and Disk etc….
3. Important step here select Windows 2008 R2 x64 and not ESX5i as you probably did with nested 32bit VMs.
4 Under the settings of the VM > CPU, make sure you have the option to pass-through the Intel VT-x/EPT feature.
6. Make sure you have set the VM to boot from Windows 2008 R2 x64 media ISO.
7. Before booting, you should edit the config file .vmx and add the parameter: hypervisor.cpuid.v0 = “FALSE”
8. Now Boot and Install Windows 2008 R2 x64.
Next stop for me is to see if I can get KVM working also and then I will be happy.
Finally finshed my latest video for VMworld US next week…
Watch a sneak preview of Symantec ApplicationHA 6.0 and Backup Exec 2010R3 with “Auto Recovery” via the VMware plug-in, The demo shows how ApplicationHA tries to keep a VM’s SQL databases online and calls upon VMwareHA as well as Backup Exec to Auto Restore the VM when it cannot due to application corruption. http://www.youtube.com/watch?v=DncQAYEhog4
It’s that time of year again; my car needs a service to ready it for the
coming winter. It’s only August and around 25°C here in the UK. It’s amazing
how quick the weather turns these days; on comes the wind and the rain and
before you know it there are signs for Halloween and Christmas in the shops. So
upon the topic of servicing the car, I mentioned in passing this morning to my
partner that I should book the car in for its service and her response was
“Can’t you do that yourself and save some money?”. Of course I probably could.
Thinking about it for that moment, but it’s much easier for someone who knows
what they are doing to do this for me. I may of course save some money if I did
it myself, but saving money doesn’t always mean saving me time. Also I’m sure
she was thinking of spending the savings on a pair of shoes or something else
and that’s one battle I would lose.
Let’s think about this task for a moment. To service the car I would need a
number of things: a service manual, service replacement parts and of course
some spare time, to think of a just a few. All of this would mean there would be
some initial cost outlay. I would also need time dedicated to this process and
some level of knowledge on how cars work.
So what’s all this got to do with my virtualized applications? I hear you
ask! Well, it’s actually fairly similar. It’s now very easy to provide
availability for your applications in a virtual environment without the need to
“do it yourself” with scripts or monitoring tools. ApplicationHA from Symantec
allows you to very easily configure and manage your enterprise applications with
ease and for a very low cost per virtual machine I may add. A bit like a car
servicing outlet, ApplicationHA does the work for you. The application
framework and wizards can have you on the next step of availability in minutes
without any impact to your running applications and with very little application
Before ApplicationHA, I had seen system administrators using Windows Service
Control Manager (SCM) and manually changing each application service to restart
in case of a fault. While this seems like a simple thing to do at the time, it
did raise complexity and add some operational time to a problem. The system
administrator would need to know where all the services resided that belong to a
particular application, and that’s not as easy as you may think. Then you have
to think about each application on each system, well that is a lot of leg work
if you have lots and lots of applications out there. What if the application is
being service packed or what if the application gets upgraded? All this would
have an effect on how this process works and how scalable it is not. So it
surely would be easier to use the same tool for all your applications and manage
them directly from the vSphere Client interface which you use on a daily
ApplicationHA came about with the Application
Awareness API which VMware added to vSphere 4.1. With vSphere 5.0 the API
allows anyone that wishes to use it to provide a “do it yourself” approach to
application monitoring. Think of the API like a service manual. The API gives
you the information you need to do the job yourself by way of scripting and
interacting with your applications. You will of course need some knowledge of
how your application works and how you can integrate it with the API. This can
be said for any type of monitoring tool, you still need to know what you’re
doing and have some scripting skill to do it also.
ApplicationHA from Symantec uses this API for heartbeating to VMwareHA on the
application health status. Some of the exciting features of ApplicationHA are
Visibility and control of applications running inside virtual machines from
Recovery coordinated with VMware HA
Fully supported with key VMware features such as vMotion, SRM, DRS, DPM,
Easy installation. Just a few clicks to install across all of your virtual
Application Support includes: SAP, Sharepoint 2010; SQL Server 2005, 2008,
2008 R2; Exchange 2007, 2010; Weblogic 9, 10, Oracle 10R2, 11R2; Custom
Applications & more
Guest platform support includes: Win32, Winx64, Red Hat Enterprise Linux
5.x, SUSE Linux Enterprise Server 11 SP1, and Oracle Enterprise Linux 5.x
Robust solution built from industry leading Veritas Cluster Server
ApplicationHA has the ability to centrally deploy its application agents
directly to all VMs including Windows or Linux in your Data Center along with
integrated management and dashboard views inside the vSphere GUI. Added to
this, ApplicationHA also interacts with Site Recovery Manager from VMware to
provide application health status and reporting with your DR solution. It
really provides a gold standard service for your applications.
So let’s just look beyond a general service of my car. What about the other
components such as suspension, electrics and heating? If any of these break down
then they can have a bearing on other users on the road especially if the driver
cannot indicate the direction he/she wishes to turn. Similarly with your Data
Center – applications normally have some level of interaction with other systems
be they virtual or physical. This is where ApplicationHA comes into its own with
its ability to work with other virtualization solutions as well as integrating
into Veritas Cluster Server environments being used for physical systems. It is
possible to tie these applications together into lines of business entities with
Veritas Operation Manager. Additionally, outages on certain systems can have
an effect on other systems. For example, a database going down on a physical
system could require middle and upper tier systems restarting their applications
once the database is back online; try doing that with scripts.
One of the new features coming later this year for ApplicationHA is the
ability to call upon BackupExec to restore the last known good copy of the
virtual machine in case the virtual machine cannot start due to OS or
application corruption. Wouldn’t it be great to be able to do that in real
life? Actually you can! Think of BackupExec as the automobile club, like AA or
AAA depending on your country. You give them a call when you’re stuck and
cannot move. ApplicationHA with BackupExec will be exactly that as BackupExec
will get you back on the road when your VM has broken down and is not moving.
It’s nice to look under the hood now and again to see how things work and
maybe have a tinker but in the end do you really want to be spending the weekend
servicing your car and getting dirty when you could be out taking a drive in the
country side maybe having lunch at a riverside restaurant
Here is the second video in my series of ApplicationHA tutorials, this one is for Microsoft SQL2008 and shows how to configure ApplicationHA for this and also what happens if any issues occur to the appplication. http://www.youtube.com/watch?v=oS7IZzlmF9Q
The installation video is now live on Youtube. This is the first in a series of videos which will guide you through the installation and configuration of applications inside VMware environments. Im putting the finishing touches to the Site Recovery Manager video which will be up shortly also.
VMwareHA is rewritten from the ground up for vSphere 5.0, some important features to bring to light are as follows.
• Provides a foundation for increased scale and functionality
• Eliminates common issues (DNS resolution)
• Multiple Communication Paths
• Can leverage storage as well as the mgmt network for communications
• Enhances the ability to detect certain types of failures and provides redundancy
• IPv6 Support
• Enhanced Error Reporting
• One log file per host eases troubleshooting efforts
• Enhanced User Interface
• Enhanced Deployment Mechanism
One of the major changes with VMwareHA 5.0 is the rewrite of the underlying code. AAM was the agent in 4.x which stands for “Automated Availability Manager” was responsible for communicating resource information, HA properties to other nodes in the cluster as well as virtual machine states. AAM also is responsible for failure/isolation heartbeats. With vSphere 5.0 there is no longer the AAM agent this has now been replaced by FDM agent or Fault Domain Manager. This agent is important because the concept of Primary/Slave have also gone and replaced with a Master/Slave concept of which FDM plays a major part. There is now only one Master in the cluster on which the FDM agent is set as a Master role, on all other nodes FDM agents on those nodes are changed to Slave roles. One of the Slave nodes can be promoted to a Master if the original Master node fails.
The Master continues to monitor the availability of ESXi 5 hosts and also gathers information on the VM availability. As the Master agent monitors all Slave nodes and in case this slave host fails, all of the VMs on that nodes are restarted on another node.
If the Master Node fails then there is a re-election process and the host which has access to the largest number of Datastores is elected as a master. There is a really good reason for this as there is a new feature which allows you to communicate via Datastores for heartbeating
This communication via the secondary channel through Datastores is known as a Heartbeat Datastores. This secondary network is not used though in normal situations, it will only be used if the primary network goes down. This secondary channel also allows the Master to be aware of all Slave nodes and also the VMs running on those hosts. The Heartbeat Datastores can also determine if host has become isolated or if a network partition has occurred for that host.
The Master node also sends reports states to vCenter. Information from the slaves which monitor the state of their running VMs is sent to the Master also the slaves are notified if the Master is alive via heartbeats. The Slave sends heartbeats to master and if master should fail then that’s when the re-election process occurs. vCenter will know if a new Master is elected as the Master will inform vCenter when its process has finished.