Fixing VMware ESXi 6.5 Upgrade Issues

A few days ago VMWare released vSphere 6.5, and in it came a raft of improvements to vCenter and other fluffy features that everyone loves. I’ve been running a small dual host setup at home for a while now, and while not in anyway a “real production environment” its been the host of a lot of household services, most notibly (for my other half) a Plex server. Unfortunatly everything didn’t go to plan, my lab host (Obelisk) took the update without issue being managed by the now embedded update manager in vCenter, my other host (Anshar) didn’t take to it at all.

The error I encountered was “Cannot run upgrade script on host”, a lovely generic error which had me scrabbling around inside the ESXi logs to find the solution: It turns out that one time or another I put the USB stick with the ESXi install into my Mac, which in turn sprayed a collection of “.Spotlight-V100”, “.fseventd’, and various other Mac specific files in the local datastore and various critical folders in the filesystem. Thankfully the host still booted so I was able to resolve it.

  • Enable SSH on your ESXi host
  • Run find -name ".Spotlight-V100" -type d -exec rm -rf {} \;
  • Run find -name ".Trashes" -type d -exec rm -rf {} \;
  • Run find -name ".fseventd" -type d -exec rm -rf {} \;
  • Re-run the Upgrade

Now this should of all worked, and the logs indiciated it wasn’t failing on any silly parts, but again I was hit with the “Unable to run update script” error. Further digging was required.

It turns out that VUM writes out a very detailed log of exactly what it is doing into /var/log/vua.log and this should be your first port of call for debugging any issues. In my log it indiciated it was expecting that 6.5 was already installed and when it was comparing the list of VIBs to update it was extremely confused why everything was out of date. It seems that ESXi depends on a system called “locker” to store all its package information and one of the first VIBs that is updated includes the updated locker files. Somehow I had to revert these files back to the 6.0 files. VMWare itself seems to recommend copying over the files from a working host, which wasn’t possible in my case as the other host was already on 6.5. So I held my breath and did the following:

  • Remove tools-light using esxcli software via remove -n tools-light
  • Install the 6.0 version using update profile esxcli software profile update -p ESXi-6.0.0-20161004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

…and thankfully it seemed to complete without issue. Running VUM from then on updated to 6.5 without issue. Each situation is different, but looking at the logs can really give you some insight into what is going on, or you can run the installer ISO directly and watch it spit out the specific issue its hitting. But, here are some reference links for working out what is wrong:

Upgrading a Google GB-7007 / U1 firmware

For many years Google has produced a range of “Search Appliances”, the idea is that you have a miniture Google search engine within your business that is able to index all your internal files and make them available in a nice interface that everyone is use to. Over the years they’ve produced many iterations of the product with the most recent ones being essentially rebadged Dell hardware.

Resetting a HP ProCurve 2824

Another day, another switch. This time i’ve bought a second HP ProCurve 2824, they’re solid and reliable and with a quick replacement to the fans they’re damn near silent. Throw in the full Layer 2 feature set and a basic Layer 3 (named by HP as L3-lite) its a workhorse of a switch suited for small environment or edge switches on larger networks.

Main problem is that most of these ex-corporate switches come pre-configured with some setup you neither know or care about, thankfully resetting this switch is amazingly easy.

  1. With the power on, poke the Reset and Clear buttons at the same time with whatever pokey devices you can find.
  2. Release the Reset button
  3. Wait until the Test LED starts blinking
  4. Release the Clear button

Within a few seconds you’ll have a factory default switch, grab your straight through serial cable and have a play with the CLI.

Its worth taking the time to get the firmware up to date, remember to check the change logs and the documentation as some interim steps may be needed to bring it up to the current version. The software for this switch has progressed quite a bit, its still the same horrible Java based Web UI but little features introduced here and there really help out.

Fixing CIFS/Samba Browse Speed on OSX

One thing that has always frustrated me on Mac OS X is the impossible slow directory listing and browsing speeds on CIFS/SMB shares, Apple’s devices, such as the Time Capsule and OS X shares work perfectly, but anything running Samba has this amazingly slow response on any folder with more than 200 files.

Resetting the TP-Link TL-SG3210

In the hunt to introduce VLANs across all segments of my home network I managed to pick-up a L2 managed switch for a lot cheaper than I expected, the TL-SG3210 only offers the bare basics but its enough to get some control to the last remnant of the unmanaged network hidden behind a powerline ethernet adapter. At £36 I couldn’t say no to it.