Citrix ADC SDX 15000 Bundle Upgrade Fail

by Carl Behrent, CTA

Environment:
Citrix ADC SDX 11.1 Build 61.112 (brand new appliance that came with 11.1 out of the box).

Issue:
After uploading Bundle 13.0 Build 36.27 to the SDX, I attempted to upgrade the SDX appliance. According to article https://docs.citrix.com/en-us/citrix-hardware-platforms/sdx/supported-versions.html you can go straight from 11.1 to 13.0.


The upgrade passes the ‘Upgrade initialization’ phase and after a short period of time, it fails on the ‘Management Service upgrade’ with the following error:

“Upgrade of Management Service to build-svm-13.0-36.27.tgz failed. Upgrade of Management Service should have rebooted the system but did not reboot even after 5 minutes. Manually reboot the Management Service. Restart Update Software operation. If problem persists contact Support.”

Run the following:

  • shell
  • /var/log
  • more messages

“Oct 23 13:30:09 <local0.err> <SDX Hostname> svm_event: 11.247.32.36 10/23/2019:00:30:09 GMT : SNMP TRAP_SENT : 127.0.0.1:BackupFailure:<SDX SVM IP> – Single Bundle Image was not found for SVM version 11.1-61.112
Oct 23 13:30:09 <local0.err><SDX Hostname>svm_event: <SDX SVM IP> 10/23/2019:00:30:09 GMT : EVENT BACKUPFAILED : 127.0.0.1:BackupFailure:11.247.32.36 – Single Bundle Image was not found for SVM version 11.1-61.112″


In the System > Events section it had the following event:

Troubleshooting

  • Logged back into the SDX SVM, and rebooted the Management Service via UI and SSH, attempted another SDX bundle upgrade and got the same error message.
  • Rebooted the whole SDX appliance then attempted the SDX bundle upgrade and the same error again.
  • Attempted to upgrade to Bundle 12.1 Build 54.13, same issue
  • Attempted to upgrade to Bundle 11.1 Build 63.9, failed as the ‘platform’ version in this build is lower than the version I’m on, even though its currently at Bundle 11.1 Build 61.112
  • Factory reset on SDX2, no customizations done, attempted an upgrade to Bundle 13.0, same issue
  • After the above factory reset, checked out /var/log/messages


Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: BEGIN_TIME 1571884055 Thu Oct 24 02:27:35 2019

Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: VERSION svm-13.0-36.27.gz
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: installsvm version (13.0-36.27) kernel (svm-13.0-36.27.gz)
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: inside check_sysid_white_list, sysid_to_check = –450097–
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: now reading sysid_white_list
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: currentLine: 450093:2:0/1,0/2
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: currentLine: 450087:2:0/1,0/2
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: currentLine: 450092:1:0/1
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: currentLine: 450096:2:0/1,0/2
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: currentLine: 450089:2:0/1,0/2
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: sysid=450097 NOT found in whitelist
Oct 24 02:27:35 <user.notice> nssdx-mgmt installsvm: [2699]: Error: This version of Management Service software is incompatible with the hardware platform 450097 , please contact Citrix support

  • Attempted to upgrade to Bundle v12.1 again, same GUI and ‘messages’ errors regarding the Management Service.


Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: BEGIN_TIME 1571886540 Thu Oct 24 03:09:00 2019
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: VERSION svm-12.1-54.13.gz
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: installsvm version (12.1-54.13) kernel (svm-12.1-54.13.gz)
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: inside check_sysid_white_list, sysid_to_check = –450097–
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: now reading sysid_white_list
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: currentLine: 450093:2:0/1,0/2
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: currentLine: 450087:2:0/1,0/2
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: currentLine: 450092:1:0/1
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: currentLine: 450096:2:0/1,0/2
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: currentLine: 450089:2:0/1,0/2
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: sysid=450097 NOT found in whitelist
Oct 24 03:09:00 <user.notice> nssdx-mgmt installsvm: [4908]: Error: This version of Management Service software is incompatible with the hardware platform 450097 , please contact Citrix support

Resolution:

Just found out that the new SDX 15xxx ONLY supports v11.1 build 61.112, the version it gets shipped with.

https://docs.citrix.com/en-us/citrix-hardware-platforms/sdx/hardware-platforms/sdx-15000.html

“Citrix ADC SDX 15000 appliance is supported only on SDX image version 11.1.61.112 (private build).”

Leave a Reply