With the release of 8.1 vSphere replication comes a ton of new upgrade and deployment issues. The one common issue is the Bad Exit Code: 1 error during the upgrade phase. This is valid for 6.1.2 or 6.5.x to 8.1 upgrade.
The first thing you will notice in the GUI is the following error message.
![]()
If you retry the upgrade will still fail and if you Ignore, the upgrade will proceed but then you will notice during during the configuration section.
![]()
Only after a "successful" failed upgrade we can access the logs to see what's the issue.
There is a log called hms-boot.log which records all these information and can be found under /opt/vmware/hms/logs
Here, the first error was this:
----------------------------------------------------
# Upgrade Services
Stopping hms service ... OK
Stopping vcta service ... OK
Stopping hbr service ... OK
Downloading file [/opt/vmware/hms/conf/hms-configuration.xml] to [/opt/vmware/upgrade/oldvr] ...Failure during upgrade procedure at Upgrade Services phase: java.io.IOException: inputstream is closed
com.jcraft.jsch.JSchException: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:315)
at com.jcraft.jsch.Channel.connect(Channel.java:152)
at com.jcraft.jsch.Channel.connect(Channel.java:145)
at com.vmware.hms.apps.util.upgrade.SshUtil.getSftpChannel(SshUtil.java:66)
at com.vmware.hms.apps.util.upgrade.SshUtil.downloadFile(SshUtil.java:88)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.downloadConfigFiles(Vr81MigrationUpgradeWorkflow.java:578)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.lambda$compileUpgrade$3(Vr81MigrationUpgradeWorkflow.java:1222)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.run(Vr81MigrationUpgradeWorkflow.java:519)
at com.vmware.jvsl.run.VlsiRunnable$1$1.run(VlsiRunnable.java:111)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.VlsiRunnable$1.run(VlsiRunnable.java:104)
at com.vmware.jvsl.run.ExecutorRunnable.withExecutor(ExecutorRunnable.java:17)
at com.vmware.jvsl.run.VlsiRunnable.withClient(VlsiRunnable.java:98)
at com.vmware.jvsl.run.VcRunnable.withVc(VcRunnable.java:139)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.launchMigrationUpgrade(Vr81MigrationUpgrade.java:62)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.access$100(Vr81MigrationUpgrade.java:21)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade$1.run(Vr81MigrationUpgrade.java:51)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.run(Vr81MigrationUpgrade.java:46)
at com.vmware.hms.apps.util.App.run(App.java:89)
at com.vmware.hms.apps.util.App$1.run(App.java:122)
at com.vmware.jvsl.run.ExceptionHandlerRunnable$1.run(ExceptionHandlerRunnable.java:47)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.ExceptionHandlerRunnable.withExceptionHandler(ExceptionHandlerRunnable.java:43)
at com.vmware.hms.apps.util.App.main(App.java:118)
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2911)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2935)
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:262)
... 24 more
So it looks like there is any issue with the copy of files from the old vR server to the new one. In the sshd_config file under /etc/ssh/ on the old vR server, the following was an entry:
Subsystem sftp /usr/lib64/ssh/sftp-server
Edit this line, so it will be:
Subsystem sftp /usr/lib/ssh/sftp-server
Then retry the Upgrade by deploying a fresh 8.1 and going through the "upgrade" process again and this time it should complete successfully.
Hope this helps!
The first thing you will notice in the GUI is the following error message.
If you retry the upgrade will still fail and if you Ignore, the upgrade will proceed but then you will notice during during the configuration section.
Only after a "successful" failed upgrade we can access the logs to see what's the issue.
There is a log called hms-boot.log which records all these information and can be found under /opt/vmware/hms/logs
Here, the first error was this:
----------------------------------------------------
# Upgrade Services
Stopping hms service ... OK
Stopping vcta service ... OK
Stopping hbr service ... OK
Downloading file [/opt/vmware/hms/conf/hms-configuration.xml] to [/opt/vmware/upgrade/oldvr] ...Failure during upgrade procedure at Upgrade Services phase: java.io.IOException: inputstream is closed
com.jcraft.jsch.JSchException: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:315)
at com.jcraft.jsch.Channel.connect(Channel.java:152)
at com.jcraft.jsch.Channel.connect(Channel.java:145)
at com.vmware.hms.apps.util.upgrade.SshUtil.getSftpChannel(SshUtil.java:66)
at com.vmware.hms.apps.util.upgrade.SshUtil.downloadFile(SshUtil.java:88)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.downloadConfigFiles(Vr81MigrationUpgradeWorkflow.java:578)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.lambda$compileUpgrade$3(Vr81MigrationUpgradeWorkflow.java:1222)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.run(Vr81MigrationUpgradeWorkflow.java:519)
at com.vmware.jvsl.run.VlsiRunnable$1$1.run(VlsiRunnable.java:111)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.VlsiRunnable$1.run(VlsiRunnable.java:104)
at com.vmware.jvsl.run.ExecutorRunnable.withExecutor(ExecutorRunnable.java:17)
at com.vmware.jvsl.run.VlsiRunnable.withClient(VlsiRunnable.java:98)
at com.vmware.jvsl.run.VcRunnable.withVc(VcRunnable.java:139)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.launchMigrationUpgrade(Vr81MigrationUpgrade.java:62)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.access$100(Vr81MigrationUpgrade.java:21)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade$1.run(Vr81MigrationUpgrade.java:51)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.run(Vr81MigrationUpgrade.java:46)
at com.vmware.hms.apps.util.App.run(App.java:89)
at com.vmware.hms.apps.util.App$1.run(App.java:122)
at com.vmware.jvsl.run.ExceptionHandlerRunnable$1.run(ExceptionHandlerRunnable.java:47)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.ExceptionHandlerRunnable.withExceptionHandler(ExceptionHandlerRunnable.java:43)
at com.vmware.hms.apps.util.App.main(App.java:118)
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2911)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2935)
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:262)
... 24 more
Then when I proceeded with an ignore, the error was this:
# Reconfigure VR
Failure during upgrade procedure at Reconfigure VR phase: null
java.lang.NullPointerException
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.vrReconfig(Vr81MigrationUpgradeWorkflow.java:1031)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.lambda$compileUpgrade$5(Vr81MigrationUpgradeWorkflow.java:1253)
at com.vmware.hms.apps.util.upgrade.Vr81MigrationUpgradeWorkflow.run(Vr81MigrationUpgradeWorkflow.java:519)
at com.vmware.jvsl.run.VlsiRunnable$1$1.run(VlsiRunnable.java:111)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.VlsiRunnable$1.run(VlsiRunnable.java:104)
at com.vmware.jvsl.run.ExecutorRunnable.withExecutor(ExecutorRunnable.java:17)
at com.vmware.jvsl.run.VlsiRunnable.withClient(VlsiRunnable.java:98)
at com.vmware.jvsl.run.VcRunnable.withVc(VcRunnable.java:139)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.launchMigrationUpgrade(Vr81MigrationUpgrade.java:62)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.access$100(Vr81MigrationUpgrade.java:21)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade$1.run(Vr81MigrationUpgrade.java:51)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.hms.apps.util.Vr81MigrationUpgrade.run(Vr81MigrationUpgrade.java:46)
at com.vmware.hms.apps.util.App.run(App.java:89)
at com.vmware.hms.apps.util.App$1.run(App.java:122)
at com.vmware.jvsl.run.ExceptionHandlerRunnable$1.run(ExceptionHandlerRunnable.java:47)
at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
at com.vmware.jvsl.run.ExceptionHandlerRunnable.withExceptionHandler(ExceptionHandlerRunnable.java:43)
at com.vmware.hms.apps.util.App.main(App.java:118)
When we still proceeded with ignore, the last stack was this:
Initialization error: Bad exit code: 1
Traceback (most recent call last):
File "/opt/vmware/share/htdocs/service/hms/cgi/boot.py", line 178, in main
__ROUTINES__[name]()
File "/opt/vmware/share/htdocs/service/hms/cgi/boot.py", line 86, in func
return fn(*args)
File "/opt/vmware/share/htdocs/service/hms/cgi/boot.py", line 86, in func
return fn(*args)
File "/opt/vmware/share/htdocs/service/hms/cgi/boot.py", line 714, in get_default_sitename
ovf.hms_cache_sitename()
File "/opt/vmware/share/htdocs/service/hms/cgi/ovf.py", line 686, in hms_cache_sitename
cache_f.write(hms_get_sitename(ext_key, jks, passwd, alias))
File "/opt/vmware/share/htdocs/service/hms/cgi/ovf.py", line 679, in hms_get_sitename
ext_key, jks, passwd, alias
File "/opt/vmware/share/htdocs/service/hms/cgi/ovf.py", line 412, in get_sitename
output = commands.execute(cmd, None, __HMS_HOME__)[0]
File "/opt/vmware/share/htdocs/service/hms/cgi/commands.py", line 324, in execute
raise Exception('Bad exit code: %d' % proc.returncode)
Exception: Bad exit code: 1
So it looks like there is any issue with the copy of files from the old vR server to the new one. In the sshd_config file under /etc/ssh/ on the old vR server, the following was an entry:
Subsystem sftp /usr/lib64/ssh/sftp-server
Edit this line, so it will be:
Subsystem sftp /usr/lib/ssh/sftp-server
Then retry the Upgrade by deploying a fresh 8.1 and going through the "upgrade" process again and this time it should complete successfully.
Hope this helps!