Citrix ADC, Gateway and SD-WAN WANOP CVE-2020 Vulnerabilities

Citrix ADC, Gateway And SD-WAN WANOP CVE-2020 Vulnerabilities

Last Updated on July 13, 2020

With the help of this article, I intend to set some context and provide you with a systematic plan for the remediation of 11 vulnerabilities recently discovered in Citrix ADC, Gateway and SD-WAN WANOP.

Prologue

If not done already, configure your ADC MPX, VPX, and SDX in accordance with Citrix security recommendations – https://docs.citrix.com/en-us/citrix-adc/citrix-adc-secure-deployment/secure-deployment-guide.html. Had the security guidelines been implemented before those 11 vulnerabilities surfaced (most importantly, the separation of management interface from the network and protection through firewall), your overall risk would have reduced to only VIPs and Citrix Gateway plugin for Linux. The time for the action is now. It is never too late to make things right.

However, the need of the hour is to remediate vulnerabilities by upgrading Citrix ADC, Citrix Gateway and Citrix SD-WAN WANOP to fixed builds that can be downloaded from https://www.citrix.com/downloads/citrix-adc/ and https://www.citrix.com/downloads/citrix-gateway/ and https://www.citrix.com/downloads/citrix-sd-wan/. Users with Citrix Gateway Plug-in for Linux should login to an updated version of Citrix Gateway and select ‘Network VPN mode’. Citrix Gateway will then prompt the user to update.

The good news is that Citrix Gateway service is not affected. According to Fermin J. Serna, Chief Information Security Officer at Citrix, “Of the 11 vulnerabilities, there are six possible attack routes; five of those have barriers to exploitation”. It means that if untrustworthy traffic to management network is restricted then the risk is limited to Gateway and Authentication virtual servers only. Other virtual servers, for example, load balancing and content switching virtual servers, are not affected.

Citrix has decided to curb the public disclosure of the technical details of the vulnerabilities. “This measure is to protect our customers. Across the industry, today’s sophisticated malicious actors are using the details and patches to reverse engineer exploits”, says Fermin. Do you remember CVE-2019-19781? I wrote a detailed article about this – CVE-2019-19781 – What You Should Do And What’s Happening Around. The vulnerability was disclosed in mid-December; however, internet-wide attacks began after January 11, when proof-of-concept exploit code was published online. Citrix wants to avoid it this time and it actually makes sense.

The following versions of Citrix ADC, Citrix Gateway and Citrix SD-WAN WANOP remediate the vulnerabilities:

  • Citrix ADC and Citrix Gateway 13.0-58.30 and later releases
  • Citrix ADC and NetScaler Gateway 12.1-57.18 and later 12.1 releases
  • Citrix ADC and NetScaler Gateway 12.0-63.21 and later 12.0 releases
  • Citrix ADC and NetScaler Gateway 11.1-64.14 and later 11.1 releases
  • NetScaler ADC and NetScaler Gateway 10.5-70.18 and later 10.5 releases
  • Citrix SD-WAN WANOP 11.1.1a and later releases
  • Citrix SD-WAN WANOP 11.0.3d and later 11.0 releases
  • Citrix SD-WAN WANOP 10.2.7 and later 10.2 releases
  • Citrix Gateway Plug-in for Linux 1.0.0.137 and later versions

Remediation

Remediation of Citrix ADC, Gateway and SD-WAN WANOP vulnerabilities involves upgrading to the release that fixes vulnerabilities and staying vigilant about new updates.

Install Updates

  1. Note down appliance serial number and support agreement in case you need assistance from Citrix support at any point during the upgrade
  2. If the gateway logon page is customized then change the UI theme to default
  3. Download the firmware according to the version of your ADC, Gateway or WANOP. Here, I am going to download Citrix ADC firmware version 13.0 Build 58.32. See the download links and list of fixed builds in Prologue.
    • It is not advised to upgrade directly to the latest version. Upgrade to one major release at a time. For instance, if the Citrix ADC appliance is on release 12.0, and you want to upgrade to release 13.0, you must upgrade the appliance to release 12.1 first, and then to release 13.0
Citrix ADC, Gateway and SD-WAN WANOP CVE-2020 Vulnerabilities
Citrix ADC, Gateway and SD-WAN WANOP CVE-2020 Vulnerabilities1
  1. Take a full backup
    • Take a snapshot if it is VPX
    • SSH to the device
    • > save ns config
    • > create system backup -level full
    • > show system backup
    • A backup file in “backup_<level>_<nsip_address>_<date-timestamp>.tgz” format will show up.
    • Defining level of backup as full in command line is important because default value is basic
    • Back up file will be saved at /var/ns_sys_backup/
    • Connect to the device using a tool like WinSCP, navigate to /var/ns_sys_backup/ and download the recently created backup file
    • Do not rename or modify it, otherwise, restore will not be possible
      • Restore can be done using “restore system backup <filename> [-skipBackup]”. SkipBackup is not recommended if the backup was taken a long time back.
      • During the restore process, it is assumed that backup file is available at /var/ns_sys_backup/. If it is not, then restore from CLI is not possible
      • In that case, import the backup file using GUI and complete the restore process. Here the backup file is the one that you downloaded earlier.
    • Although full backup backs up nsconfig directory where ns.conf file resides, I prefer taking one extra measure of downloading that file to the local computer
  1. Take a backup of customizations
    • If you have introduced some modifications like custom scripts or a logo and background image on the login page, then it is recommended to delete them after taking a backup. While you are still connected using WinSCP, download the customizations folder at /var/. If the folder doesn’t exist or it is empty then don’t bother
    • Additionally, take a backup of following files and folders. If they don’t exist then no need to backup:
      • /nsconfig/monitors/*.pl
      • /nsconfig/htmlinjection/*
      • /nsconfig/rc.netscaler
    • If you have modified ttys, resolv.conf, sshd_config, host.conf, newsyslog.conf, host.conf, httpd.conf, rc.conf, syslog.conf, crontab, monitrc files at /etc and moved them to /nsconfig, then any update to the /etc directory will be lost. To avoid this, create a folder at /var/ by the name nsconfig_backup and move each file to that folder one by one
      • > cd var
      • > mkdir nsconfig_backup
      • > cp /nsconfig/resolv.conf /var/nsconfig_backup
      • > cp /nsconfig/host.conf /var/nsconfig_backup
      • and so on…
  1. Check available space and if it is below 90% then delete few files. Directories of interest are /var/nstrace, /var/log, /var/nslog, /var/tmp/support, /var/core, /var/crash, /var/nsinstall. Follow the instructions at How to free space on /var directory for logging issues with a Citrix ADC appliance. You may get a message like below image while logging in to a node where /var is full
  1. The most important thing while upgrading ADC aka NetScaler appliances in HA (High Availability) pair is that both nodes should be running on the same software release and builds. Otherwise, configuration, commands, state of the services, connection failover sessions and persistent sessions will be automatically synchronized.
    • Despite of different software release or builds on both nodes, everything can be synchronized using sync HA files full command
    • Files located on the secondary that are specific to the secondary (not present on the primary) are not deleted during the synchronization.
  1. SSH to the secondary node
    • > save config
    • > shell
    • # cd /var/nsinstall/
    • # mkdir 13_0nsinstall (for you, the release version maybe 12_0, 12_1, 11_1 or 10_5)
    • # cd 13_0nsinstall
    • # mkdir build58_32 (for you, the build can be 58_30, 57_18, 63_21, 64_14 or 70_18)
    • # cd build58_32
    • Upload the downloaded firmware in step 3 using WinSCP to the buildx_x folder
    • # tar –xvzf build-13.0-58.32_nc_64.tgz
    • # ./installns
    • # Reboot NOW? [Y/N] Y (once the installation is complete, press Y to reboot)
    • > show ha node (after reboot, login to the command line and execute show ha node. The output should show that this is the secondary node and synchronization is disabled)
    • > set ha node –hasync disabled (execute if sync is not auto-disabled)
    • Login to GUI and verify the status of the following entities:
      • Virtual servers are in UP state
      • Monitors are in UP state
      • GSLB sites synchronise without any issues
      • All certificates are present on the appliance
      • All the licenses are present on the appliance
    • > force failover (come back to command line and execute force failover to perform a force failover and takeover as primary appliance)
  1. SSH to the primary node ( after force failover, this node has become secondary)
    • Repeat step 8
    • After force failover, this node should become Primary again
  1. Go to secondary node
    • > show node (to verify that the node is secondary)
    • > set ha node -hasync enabled (to enable synchronization that we disabled in step 8)
    • > show ns runningconfig (to verify sync state is success)
  1. Congrats! Remediation of Citrix ADC, Gateway and SD-WAN WANOP vulnerabilities is completed

Keep an eye

Keep an eye on CTX276688 for any new update from Citrix. As of this writing, following CVE identifications have been assigned to the vulnerabilities:

CVE IDVulnerability TypeAffected Products
CVE-2019-18177Information disclosureCitrix ADC, Citrix Gateway
CVE-2020-8187Denial of serviceCitrix ADC, Citrix Gateway 12.0 and 11.1 only
CVE-2020-8190Local elevation of privilegesCitrix ADC, Citrix Gateway
CVE-2020-8191Reflected Cross Site Scripting (XSS)Citrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8193Authorization bypassCitrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8194Code InjectionCitrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8195Information disclosureCitrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8196Information disclosureCitrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8197Elevation of privilegesCitrix ADC, Citrix Gateway
CVE-2020-8198Stored Cross Site Scripting (XSS)Citrix ADC, Citrix Gateway, Citrix SDWAN WAN-OP
CVE-2020-8199Local elevation of privilegesCitrix Gateway Plug-in for Linux

Look at these tweets from Carl Stalhood and Leee Jeffries

Dear Past, thank you for the lesson. Dear Future, I am ready

Being ready before the storm strikes is sometimes the difference between life and death. Do not take it by its literal meaning but you know what I am trying to say. Here is what you should be doing:

  1. Deploy security guidelines as soon as you can – https://docs.citrix.com/en-us/citrix-adc/citrix-adc-secure-deployment/secure-deployment-guide.html
  2. In future, You can use Citrix ADM Service for simplified and bulk upgrade of all your Citrix ADC instances – https://www.citrix.com/blogs/2020/06/11/10-ways-citrix-adm-service-supports-easier-citrix-adc-upgrades/
  3. Subscribe to RSS feed of Security bulletin – https://support.citrix.com/feeds
Citrix ADC, Gateway and SD-WAN WANOP CVE-2020 Vulnerabilities - subscribe to RSS feed
  1. Update your support notifications to receive future security bulletins by email – https://support.citrix.com/user/alerts
Alerts settings
  1. Delivery Managers and Key Business decision makers should familiarize themselves with Citrix Trust Center. Read more about it at https://www.citrix.com/blogs/2020/06/16/keep-up-to-date-with-the-citrix-trust-center/
Citrix Trust Center

Good reads

Be the first to reply

Leave a Reply