Over the past week I have been trying to upgrade an ESXi 5.0 host to ESXi 5.5 Update 3d. Trying to upgrade it via VUM I would get “Cannot execute upgrade script on host”. Trying to patch the host to the latest 5.0 patches via VUM would generate the “The host returns esxupdate error code:15. The package manager transaction is not successful. Check the Update Manager log files and esxupdate log files for more details.” message.
Checking esxupdate.log, the only error I could find was:
ERROR: An esxupdate error exception was caught:
ERROR: Traceback (most recent call last):
ERROR: File “/usr/sbin/esxupdate”, line 216, in main
ERROR: cmd.Run()
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esx5update/Cmdline.py”, line 144, in Run
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py”, line 243, in InstallVibsFromSources
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py”, line 345, in _installVibs
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py”, line 388, in _validateAndInstallProfile
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esximage/HostImage.py”, line 630, in Stage
ERROR: File “/build/mts/release/bora-768111/bora/build/esx/release/python-2.6-lib-zip-stage/768111/visor/pylib/python2.6/site-packages/vmware/esximage/HostImage.py”, line 463, in _download_and_stage
ERROR: InstallationError: (‘VMware_locker_tools-light_5.0.0-3.90.3982828’, ‘[Errno 32] Broken pipe’)
In VUA.log, the only errors were related to PACKAGE_COMPLIANCE indicating that the 5.5.0 packages weren’t available (as it’s a 5.0.0 host).
So searching around a bit, I found this article:
https://kb.vmware.com/kb/2030665
I tried all the solutions offered in the article, but none worked. Checking the disk space with the vdf -h command showed that there was ample free space on all partitions, as did running df -h. What I had noticed was that the /locker/packages/5.0.0/ folder was missing so I recreated it and attempted to re-run the patching, which would fail. I ended up creating a VMware Workstation image of an ESXi 5.0 host so that I could grab the 5.0.0 folder from it to copy onto the existing host. However it was at this stage, when trying to copy 5.0.0 tools back onto the host using WinSCP, I’d get an error 4. This lead me to again look into the free space on the drive, which I did by logging onto the host via SSH and running:
df -h
This showed that the volume Hypervisor3 was 100% full!
Filesystem Size Used Available Use% Mounted on
vfat 285.9M 285.9M 16.0k 100% /vmfs/volumes/Hypervisor3
Investigating the Hypervisor3 store, there were a number of very old zdump.xxx files, amounting to around 100MB in the /vmfs/volumes/Hypervisor3/var/core/ folder. I copied these to my local machine, deleted from the host and re-copied the tools which then completed successfully.
After clearing the space I was still unable to upgrade the host to 5.5, but I was able to remidate with the latest 5.0 patches. Once the host was up to date on the patch level, re-running the upgrade to 5.5 was successful.
Lesson learned from this experience:
Check that there are no old dump files before upgrading a host!