How to fix: vSphere VUM: GUI error that prevents upgrades!

If you, as a vSphere Administrator, regularly update a vSphere environment you should be using vSphere Update Manager. vSphere Update Manager is the tooling that VMware made so that you don’t have to install patches through the command line on each host, or do updates on each host separately! And if you’re like me and do this a lot, you will probably get a lot of errors that you need to fix. I’ve posted several blogs since we started doing this about vSphere Update Manager errors, but this below error was a pain to fix!

The other day we were trying to update hosts again but then we saw the following error in the VMware vSphere Client (Flex):

Error in text:

interface com.vmware.vim.binding.integrity.VcIntegrity is not visible from class loader

So we started troubleshooting and did atleast the following, not in any particular order:

• Reboot VCSA;
• Restart vSphere-Client services with (service-control --stop vsphere-client and service-control --start vsphere-client);
• Check the logging in vsphere_client_virgo.log and others;
• Check VUM logging;
• Restart VUM services with (service-control --stop vmware-updatemgr and service-control --start vmware-updatemgr);
• Checked the certificates;
• Re-registered certificates and services for the vSphere Update Manager services;
  • This is actually really easy and all you have to do is (Make sure you have a backup just in case):
    • Go to: /usr/lib/vmware-updatemgr/bin/
    • Stop the vSphere Update Manager Service: service-control --stop vmware-updatemgr
    • Run: ./updatemgr-util refresh-certs
    • Run: ./updatemgr-util register-vc
    • Start the vSphere Update Manager Service: service-control --start vmware-updatemgr

None of this actually helped. Until I found a KB on the web which sort of explained the same error we were encountering, except the wrong scenario since we’ve upgraded the vCenter a long time ago.

Now if you are encountering the same error, you should atleast have the following similar entries in the following log files:

vsphere_client_virgo.log:
root@vcenter [ /var/log/vmware/vsphere-client/logs ]# vi vsphere_client_virgo.log
[2019-06-18T10:29:29.238+02:00] [ERROR] data-service-pool-798        71440426 354796 445147 com.vmware.vum.aspects.LoggingAspect                              Exception caught in class 'com.vmware.vum.dataservice.impl.CompliancePropertyProviderAdapterImpl', line 554 java.lang.IllegalArgumentException: interface com.vmware.vim.binding.integrity.VcIntegrity is not visible from class loader
        at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:581)
        at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557)
        at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230)
        at java.lang.reflect.WeakCache.get(WeakCache.java:127)
        at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
        at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719)
        at com.vmware.vim.vmomi.client.common.impl.ManagedObjectFactoryImpl.createManagedObject(ManagedObjectFactoryImpl.java:87)
        at com.vmware.vim.vmomi.client.http.impl.HttpClient.createStub(HttpClient.java:88)
        at com.vmware.vum.client.adapters.updatemanager.impl.UpdateManagerServiceImpl.login(UpdateManagerServiceImpl.java:174)
        at com.vmware.vum.client.adapters.updatemanager.impl.UpdateManagerServiceInstantiator.instantiate(UpdateManagerServiceInstantiator.java:58)
        at com.vmware.vum.client.adapters.updatemanager.impl.UpdateManagerServiceProviderImpl.createVumService(UpdateManagerServiceProviderImpl.java:273)
        at com.vmware.vum.client.adapters.updatemanager.impl.UpdateManagerServiceProviderImpl.getService(UpdateManagerServiceProviderImpl.java:137)
        at sun.reflect.GeneratedMethodAccessor772.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
        at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
        at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
        at com.sun.proxy.$Proxy637.getService(Unknown Source)
        at com.vmware.vum.dataservice.impl.CompliancePropertyProviderAdapterImpl.getIsVirtualAppliance(CompliancePropertyProviderAdapterImpl.java:551)
        at sun.reflect.GeneratedMethodAccessor1555.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.vmware.vise.data.query.impl.ServicePropertyProviderAdapter.invokeMethod(ServicePropertyProviderAdapter.java:285)
        at com.vmware.vise.data.query.impl.ServicePropertyProviderAdapter.getProperties(ServicePropertyProviderAdapter.java:127)
        at com.vmware.vise.data.query.impl.DataManager.getDataFromPropertyProvider(DataManager.java:1301)
        at com.vmware.vise.data.query.impl.DataManager.getResultFromPropertyProvider(DataManager.java:1269)
        at com.vmware.vise.data.query.impl.DataManager.access$000(DataManager.java:78)
        at com.vmware.vise.data.query.impl.DataManager$1.call(DataManager.java:784)
        at com.vmware.vise.data.query.impl.DataManager$1.call(DataManager.java:780)
        at com.vmware.vise.util.concurrent.ExecutorUtil$2.call(ExecutorUtil.java:693)
        at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingTask.call(ExecutorUtil.java:1060)
        at com.vmware.vise.data.query.impl.DataServiceThreadPoolDecorator$1.call(DataServiceThreadPoolDecorator.java:167)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)      

The issue it seems lies within your vSphere Client Profile which is stored in the SerenityDB folders/files in the following folder on the VCSA:

/storage/vsphere-client/SerenityDB/serenity/"USERNAME"

As you can see below there are several folders in which information is recorded.

root@vcenter [ /storage/vsphere-client/SerenityDB/serenity/username ]# ls
configuration  docking  recentitems  savedcolumnprefs  usersettings

If you browse through these folders you can find your vSphere Client Profile settings for the way you “docked” certain windows, your default UI settings, your recent items and if you choose to hide them, the Getting Started pages settings.

root@vcenter [ /storage/vsphere-client/SerenityDB/serenity/username/usersettings ]# ls
gettingStartedPages.neverShowAgain
root@vcenter [ /storage/vsphere-client/SerenityDB/serenity/username/recentitems ]# ls
vmware.vsphere.client.settings.recentitems

Within these files something about the vSphere Update Manager settings is corrupted, which gives you the error you can see in this post. Why this is happening hasn’t been disclosed, but a fix is known! Just follow the below steps and you are good to go again:

service-control --stop vsphere-client
rm -rf /storage/vsphere-client/SerenityDB/serenity/username
service-control --start vsphere-client

And after this you can happily update your ESXi hosts again:

Even though this fix has been described in the VMware KB I mentioned, I couldn’t actually match the circumstances so I thought I’d make a post for you guys to explain. I hope you can fix your issue with this guide!

Update on this issue 15-11-2019

This issue seems to have been fixed for us since we’ve updated our vCenter servers to VMware vCenter Server 6.7 U2c (Build 14070654). We’ve not had this issue since we’ve upgraded. So this could possibly be a fix for you guys!