vCenter Inventory Snapshot - Alternative way of resolving vCenter's SQL issues

Somewhere on upgrade path from VI 3.5 to vSphere 4.0 and finally to vSphere 4.1 we screwed up Performance stats in vCenter. I could still see real time data, but I couldn't select any historical data. Considering my intention to disable Large Pages on all cluster hosts and prove huge RAM savings with TPS I needed that historical data. Surely, the first website I visited was VMware and I immediately found KB which describes all symptoms I experienced. With my about zero SQL experience I followed entire procedure and it even seemed that I fixed the issue, at least I could select historical data options.

However, after checking it out for couple of days I found out that historical data is broken and discontinuous. Moreover, when I tried to use Veeam Monitor for VMware I couldn't get any CPU or Memory usage data on vCenter level.  That is when I remembered that recently in someone's blog I have spotted Inventory Snapshot product in VMware Labs.
Generally speaking, Inventory Snapshot is a bunch of PowerCLI commands that retrieve all essential data from vCenter, such as Datacenters, Host, VMs, HA&DRS Cluster settings, Resource Pools, Permissions, Roles, etc. You can check the whole list of inventory yourself. Once it collected all data from vCenter it generates PowerCLI code that can reproduce your entire environment either on new vCenter or on existing one if you managed to corrupt your SQL database. As you already understood it doesn't save performance data, but in my case it was ideal tool. With all vCenter SQL databases backed up I could play around as much as I wanted.

Here is a brief overview of steps you need to pass through to get my problem solved. However, you definitely want  to read Inventory Snapshot documentation because I am not inclined to reproduce official documentation :)

1. Stop vCenter service and backup its SQL database.
2. Bring vCenter service back online.
3. Run Inventory Snapshot and provide administrator's credentials
4. Once all inventory is retrieved and saved to .ps file you need to provide root's password for ESXi hosts. This is the weakest part of this product since all password are stored in plain text, although we need to remember that this is a lab product.
5. Stop vCenter again, run command line, navigate to C:\Program Files\VMware\Infrastructure\VirtualCenter\ and run vpxd.exe - b. It will fully reinitialize vCenter database, as if it was newly installed. At this step I had some warning about licenses, but seems like it was not important
6. Start vCenter
7. Run PowerCLI, login to vCenter and apply .ps file you saved before to vCenter. With 10 hosts, 4 Resource Pools and 190 VMs it took Inventory Snapshot about 20 mins to fully import back all objects.

So, now I have all types of performance graphs and it was a way simpler than going through VMware KB to fix something in SQL database.

In comments on VMware Lab site I saw people's complaints about missing permissions after they imported Inventory, but in my case I have got all permissions back correctly implemented.

Very Important to know!!! - The problem I had was that Inventory Snapshot couldn't recreate some part of  dvSwitch. Luckily, it was used only for my couple of test hosts. The error says "The host proxy switch associated to dvSwitch no longer exists in vCenter". Today is too late to start investigation. Tomorrow I will try to inform developers if it is not known bug yet. Anyway, it is still kind of beta version and you need to take all precaution measures before you start playing with it.

PS. Veeam Monitor still shows zero CPU and Memory usage. According to Veeam Community forum I need to ask their support for a fix. Why can't they just post it on the website if it is well known issue?


If you find this post useful please share it with any of the buttons below.