Posted by mattr on April 15, 2010 at 10:59 pm
With the advent of vSphere my company has been looking at how best to make the move from ESX 3. One obvious path under consideration was to look at migrating to ESXi, both from the perspective of reduced patching (the default view being a lower attack surface with the removal of the Service Console) and the fact that roadmaps have often alluded to it being the final destination for ESX anyway.
With that in mind, I took to having a look at ESX4i in more depth to fathom where it would fit into our environment. We are very much an HP house, housing both standard and blade servers, so one of the core management systems we use is HP Systems Insight Manager. Its primary purpose is hardware monitoring – knowing when the hardware is either failing or failed being a must for our service assurance.
It was therefore a surprise when I first managed to add my test ESXi server to our existing SIM server (version 5.3), that certain components were not showing up. Most concerning was the lack of a disk subsystem. No array information, no disk breakdown – nothing.
Now I don’t know about other companies, but by far the most common failure in a server for us tends to be a disk. Therefore for SIM not to see it appeared somewhat dangerous. Puzzled by this, I investigated further – my first thought was that perhaps it was a GUI issue and that the underlying monitor was present but not displayed.
So I wandered up to the test server in the CER and disengaged one of the drives to see what happened. The HP SIM display did not change – no email alert, no status change on the GUI – even after ten minutes. By contrast the default alerts inside my test vCenter server, which was seeing the entire disk array inside the Hardware Status tab, immediately started sending alerts and changing status on screen.
This was supremely confusing – obviously for the vCenter to be able to detect the disk and any alerts the correct CIM information was being produced on the ESXi host (I was using the correct HP flavour of ESXi) yet SIM was oblivious to a problem.
Sorry to say after some more fiddling with authentication on SIM, trawling through numerous HP forums and even starting a Google Wave on the subject, I had to call it a day and put the ESXi question on ice.
Then back in late March, I saw something on Twitter announcing the release of SIM version 6 – there were various references made to improved support for ESX4 so I was anxious to see if the monitoring components had been upgraded.
So a swift guest build later (SIM 6 supports 64 bit so a Windows 2008 R2 was my choice) I was busy installing the new version. Now SIM is still not that intuitive when it comes to ESXi – it uses WBEM as a method to query the system so the authentication needs to be set there.
Before setting up a discovery job, a change needed to be made to the event filtering. Under Options > Events > Event Filter Options the ‘Accept unregistered events’ needs to be enabled. Don’t ask me why but the HP document suggests it needs to be there for ESXi.
I then set up a discovery job to look for the server (note the port used for access below – you will need to add this under the Advanced protocols credentials > WBEM/WMI tab:
Once this had been saved the job was started– the feedback is vastly improved in version 6, and you see each type of discovery it is trying and whether it is good or bad. The one you are looking for is the WBEM entry.
Once complete I switched to the All Systems view to see if the server had appeared. It had and I was greeted with the following extra display window.
NB. The auto-discovery is also much improved in SIM 6 – if you discover a host it automatically works out what is hosted on it and adds it as well if required. Very nice.
On clicking any of these categories I was pushed to a status window for the host where I was delighted to see the following – noting the comparison between v6 and 5.3 …
 |
 |
| SIM 6 |
SIM 5.3 |
Success! Full access to the complete hardware layer in ESXi through SIM … looks like we can put ESXi back on the upgrade table 
Continue Reading
Posted by mattr on January 26, 2010 at 10:43 pm
I have been battling to try and find out where my HP servers store and provide serial numbers for vCentre in the Hardware Status plugin – to no avail I am sad to say so far. However while investigating I have had to work out how to use Wireshark to decode the SSL traffic between the vCentre and an ESX host, which is very useful for troubleshooting connectivity or other issues between a VC and ESX host, so I thought I would record the process for reference …
For Wireshark (Download the latest version here http://www.wireshark.org/download.html ) to be able to decode the SSL traffic between a VC and host, it needs the private key of the SSL certificate. To get this you will need to copy it from the host via an SCP client ..
The file you are looking for is rui.key and is located in /etc/vmware/ssl folder.
Once you have copied the file to your local machine you can fire up Wireshark and head to the Edit > Preferences.
In there under the Protocols menu on the left hand side, select SSL and you will see the following fields:
The RSA keys list field is where you tell Wireshark to look for the server source i.p. (the ESX host),port,protocol and location of the private key you want to use to decrypt.
In this example I am using the same key to decrypt both standard 443 and 5989 (Secure CIM) traffic – to do more that one you simply use a semi-colon to separate the string:
10.0.0.1,5989,wbem-https,c:\temp\rui.key;10.0.0.1,443,https,c:\temp\rui.key
The SSL debug file field allows Wireshark to write out how it is using the key to aid troubleshooting.
Once you have this you can load up a capture from your vCentre and look for some SSL traffic – you should see in the lower frame something like this:
As you expect the output is unreadable in the standard tab, but if you look at the Decrypted SSL data tab you should see the data magically become a lot more useful:
Wireshark helpfully will now also display a context menu called Follow SSL Stream, once it can decrypt the traffic, and will piece all the traffic it can find back together and pop up a window with the whole transaction in one place.
Once you get into the underlying transactions it is great to see what is going back and forth between the vCentre and hosts – I hope this helps you figure out whatever problem you may be having …
Continue Reading
Posted by mattr on November 24, 2009 at 9:38 pm
I was lucky enough to get out of the office and go to the London VMUG today – I had missed the previous one so was keen to catch up and meet some of the people I had conversed with previously over Twitter etc.
Needless to say it had a lot of great content – Jonathan Medd’s demo of PowerGUI, VESI and in particular for me Onyx Alpha really enforced the power of using PowerGUI to administer VMware. I am going to be trying to use Onyx to help me fathom some areas I wanted to look at with PowerGUI as a result ..
Symantec’s offerings around VMware were a surprise in terms of the breadth they seemed to cover so worthwhile seeing their current offerings as well as the coverage of the vStorage API which will release early next year.
Simon Gallagher’s T.A.R.D.I.S™ demo was very informative .. the sight of two HP ML115’s running ESXi with a further 10 ESXi servers virtualized on top of them was impressive! I am already working out how to hit my boss up for a couple for Lab Testing …
The final event with Mike Laverick and Guy Chapman discussing vSphere features and Cloud concepts was very interesting and gave good insight into what it means to different people, and quite how different that could be.
Thanks to Alaric for organising as usual – and for his interesting detour into the Large Hadron Collider
Now I need to see if I can think up some content for the next meeting to avoid Alaric waving the nagging finger again!
If you haven’t been before – keep an eye on this forum http://communities.vmware.com/community/vmug/emea/london?view=discussions or email londonvmug@yahoo.com to get yourself on the next one.
Continue Reading
