Search
Switch Language

Exploiting a remote heap overflow with a custom TCP stack

Written by Etienne Helluy-Lafont , Luca Moro - 13/02/2023 - in Exploit - Download
In November 2021 our team took part in the ZDI Pwn2Own Austin 2021 competition [1] with multiple entries. One of them successfully compromised the Western Digital MyCloudHome connected hard drive via a 0-day in the Netatalk daemon. Our exploit was unusual because triggering the vulnerability required to mess with the remote TCP stack, so we wrote our own.
This blog post will provide some technical details about it.

Vulnerability details and analysis

Environment

The Western Digital MyCloudHome is a consumer grade NAS with local network and cloud based functionalities. At the time of the contest (firmware 7.15.1-101) the device ran a custom Android distribution on a armv8l CPU. It exposed a few custom services and integrated some open source ones such as the Netatalk daemon. This service was a prime target to compromise the device because it was running with root privileges and it was reachable from adjacent network. We will not discuss the initial surface discovery here to focus more on the vulnerability. Instead we provide a detailed analysis of the vulnerabilty and how we exploited it.

Netatalk [2] is a free and Open Source [3] implementation of the Apple Filing Protocol (AFP) file server. This protocol is used in networked macOS environments to share files between devices. Netatalk is distributed via the service afpd, also available on many Linux distributions and devices. So the work presented in this article should also apply to other systems.
Western Digital modified the sources a bit to accommodate the Android environment [4], but their changes are not relevant for this article so we will refer to the official sources.

AFP data is carried over the Data Stream Interface (DSI) protocol [5]. The exploited vulnerability lies in the DSI layer, which is reachable without any form of authentication.

Overview of server implementation

The DSI layer

The server is implemented as an usual fork server with a parent process listening on the TCP port 548 and forking into new children to handle client sessions. The protocol exchanges different packets encapsulated by Data Stream Interface (DSI) headers of 16 bytes.

A request is usually followed by a payload which length is specified by the dsi_len field.

The meaning of the payload depends on what dsi_command is used. A session should start with the dsi_command byte set as DSIOpenSession (4). This is usually followed up by various DSICommand (2) to access more functionalities of the file share. In that case the first byte of the payload is an AFP command number specifying the requested operation.

dsi_requestID is an id that should be unique for each request, giving the chance for the server to detect duplicated commands.
As we will see later, Netatalk implements a replay cache based on this id to avoid executing a command twice.

It is also worth mentioning that the AFP protocol supports different schemes of authentication as well as anonymous connections.
But this is out of the scope of this write-up as the vulnerability is located in the DSI layer, before AFP authentication.

Few notes about the server implementation

The DSI struct

To manage a client in a child process, the daemon uses a DSI *dsi struct. This represents the current connection, with its buffers and it is passed into most of the Netatalk functions. Here is the struct definition with some members edited out for the sake of clarity:

We mainly see that the struct has:

  • The command heap buffer used for receiving the user input, initialized in dsi_init_buffer() with a default size of 1MB ;
  • cmdlen to specify the size of the input in command ;
  • An inlined data buffer of 64KB used for the reply ;
  • datalen to specify the size of the output in data ;
  • A read ahead heap buffer managed by the pointers buffer, start, eof, end, with a default size of 12MB also initialized in dsi_init_buffer().

 

The main loop flow

After receiving DSIOpenSession command, the child process enters the main loop in afp_over_dsi(). This function dispatches incoming commands until the end of the communication. Its simplified code is the following:

 

The receiving process

In the previous snippet, we saw that an idling server will receive the client data in dsi_stream_receive(). Because of the buffering attempts this function is a bit cumbersome. Here is an overview of the whole receiving process within dsi_stream_receive().

 dsi_stream_receive(DSI* dsi)
 
  1. define char block[DSI_BLOCKSIZ] in its stack to receive a DSI header
 
  2. dsi_buffered_stream_read(dsi, block, sizeof(block)) wait for a DSI header
    
    1. from_buf(dsi, block, length)
       Tries to fetch available data from already buffered input
       in-between dsi->start and dsi->end
    
    2. recv(dsi->socket, dsi->eof, buflen, 0)
       Tries to receive at most 8192 bytes in a buffering attempt into the look ahead buffer
       The socket is non blocking so the call usually fails
    
    3. dsi_stream_read(dsi, block, len))
      
      1. buf_read(dsi, block, len)
        
        1. from_buf(dsi, block, len)
           Tries again to get data from the buffered input
        
        2. readt(dsi->socket, block, len, 0, 0);
           Receive data on the socket
           This call will wait on a recv()/select() loop and is usually the blocking one

  3. Populate &dsi->header from what has been received

  4. dsi_stream_read(dsi, dsi->commands, dsi->cmdlen)
        
    1. calls buf_read() to fetch the DSI payload
       If not enough data is available, the call wait on select()


The main point to notice here is that the server is only buffering the client data in the recv() of dsi_buffered_stream_read() when multiple or large commands are sent as one. Also, never more than 8KB are buffered.

The vulnerability

As seen in the previous snippets, in the main loop, afp_over_dsi() can receive an unknown command id. In that case the server will call dsi_writeinit(dsi, dsi->data, DSI_DATASIZ) then dsi_writeflush(dsi).

We assume that the purpose of those two functions is to flush both the input and the output buffer, eventually purging the look ahead buffer. However these functions are really peculiar and calling them here doesn't seem correct. Worst, dsi_writeinit() has a buffer overflow vulnerability! Indeed the function will flush out bytes from the look ahead buffer into its second argument dsi->data without checking the size provided into the third argument DSI_DATASIZ.

In the above code snippet, both variables dsi->header.dsi_len and dsi->header.dsi_data.dsi_doff were set up in dsi_stream_receive() and are controlled by the client. So dsi->datasize is client controlled and depending on MIN(dsi->eof - dsi->start, dsi->datasize), the following memmove could in theory overflow buf (here dsi->data). This may lead to a corruption of the tail of the dsi struct as dsi->data is an inlined buffer.

However there is an important limitation: dsi->data has a size of 64KB and we have seen that the implementation of the look ahead buffer will at most read 8KB of data in dsi_buffered_stream_read(). So in most cases dsi->eof - dsi->start is less than 8KB and that is not enough to overflow dsi->data.

Fortunately, there is still a complex way to buffer more than 8KB of data and to trigger this overflow. The next parts explain how to reach that point and exploit this vulnerability to achieve code execution.

Exploitation

Triggering the vulnerability

Finding a way to push data in the look ahead buffer

 

The curious case of dsi_peek()

While the receiving process is not straightforward, the sending one is even more confusing. There are a lot of different functions involved to send back data to the client and an interesting one is dsi_peek(DSI *dsi).

Here is the function documentation:



In other words, dsi_peek() will take a pause during a blocked send and might try to read something if possible. This is done in an attempt to avoid potential deadlocks between the client and the server. The good thing is that the reception is buffered:


Here we see that if the select() returns with dsi->socket set as readable and not writable, recv() is called with dsi->eof. This looks like a way to push more than 64KB of data into the look ahead buffer to later trigger the vulnerability.

One question remains: how to reach dsi_peek()?

 

Reaching dsi_peek()

While there are multiple ways to get into that function, we focused on the dsi_cmdreply() call path. This function is used to reply to a client request, which is done with most AFP commands. For instance sending a request with DSIFUNC_CMD and the AFP command 0x14 will trigger a logout attempt, even for an un-authenticated client and reach the following call stack:

afp_over_dsi()
dsi_cmdreply(dsi, err)
dsi_stream_send(dsi, dsi->data, dsi->datalen);
dsi_stream_write(dsi, block, sizeof(block), 0)

From there the following code is executed:

In the above code, we see that in order to reach dsi_peek() the call to send() has to fail.

 

Summarizing the objectives and the strategy

So to summarize, in order to push data into the look ahead buffer one can:

  1. Send a logout command to reach dsi_cmdreply.
  2. In dsi_stream_write, find a way to make the send() syscall fail.
  3. In dsi_peek() find a way to make select() only returns a readable socket.

Getting a remote system to fail at sending data, while maintaining the stream open is tricky. One funny way to do that is to mess up with the TCP networking layer. The overall strategy is to have a custom TCP stack that will simulate a network congestion once a logout request is sent, but only in one direction. The idea is that the remote application will think that it can not send any more data, while it can still receive some.

Because there are a lot of layers involved (the networking card layer, the kernel buffering, the remote TCP congestion avoidance algorithm, the userland stack (?)) it is non trivial to find the optimal way to achieve the goals. But the chosen approach is a mix between two techniques:

  • Zero'ing the TCP windows of the client side, letting the remote one think our buffer is full ;
  • Stopping sending ACK packets for the server replies.

This strategy seems effective enough and the exploit manages to enter the wanted codepath within a few seconds.

Writing a custom TCP stack

To achieve the described strategy we needed to re-implement a TCP networking stack. Because we did not want to get into low-levels details, we decided to use scapy [6] and implemented it in Python over raw sockets.

The class RawTCP of the exploit is the result of this development. It is basic and slow and it does not handle most of the specific aspects of TCP (such as packets re-ordering and re-transmission). However, because we expect the targeted device to be in the same network without networking reliability issues, the current implementation is stable enough.

The most noteworthy details of RawTCP is the attribute reply_with_ack that could be set to 0 to stop sending ACK and window that is used to advertise the current buffer size.

One prerequisite of our exploit is that the attacker kernel must be "muzzled down" so that it doesn't try to interpret incoming and unexpected TCP segments.
Indeed the Linux TCP stack is not aware of our shenanigans on the TCP connection and he will try to kill it by sending RST packets.

One can prevent Linux from sending RST packets to the target, with an iptables rule like this:

# iptables -I OUTPUT -p tcp -d TARGET_IP --dport 548 --tcp-flags RST RST -j DROP


Triggering the bug

To sum up, here is how we managed to trigger the bug. The code implementing this is located in the function do_overflow of the exploit:

  1. Open a session by sending DSIOpenSession.
  2. In a bulk, send a lot of DSICommand requests with the logout function 0x14 to force the server to get into dsi_cmdreply().
    From our tests 3000 commands seems enough for the targeted hardware.
  3. Simulate a congestion by advertising a TCP windows size of 0 while stopping to ACK reply the server replies.
    After a short while the server should be stuck in dsi_peek() being only capable of receiving data.
  4. Send a DSI dummy and invalid command with a dsi_len and payload larger than 64KB.
    This command is received in dsi_peek() and later consumed in dsi_stream_receive() / dsi_stream_read() / buf_read().
    In the exploit we use the command id DSIFUNC_MAX+1 to enter the default case of the afp_over_dsi() switch.
  5. Send a block of raw data larger than 64KB.
    This block is also received in dsi_peek() while the server is blocked but is consumed in dsi_writeinit() by overflowing dsi->data and the tail of the dsi struct.
  6. Start to acknowledge again the server replies (3000) by sending ACK back and a proper TCP window size.
    This triggers the handling of the logout commands that were not handled before the obstruction, then the invalid command to reach the overflow.

The whole process is done pretty quickly in a few seconds, depending on the setup (usually less than 15s).

Getting a leak

To exploit the server, we need to know where the main binary (apfd) is loaded in memory. The server runs with Address Space Layout Randomization (ASLR) enabled, therefore the base address of apfd changes each time the server gets started. Fortunately for us, apfd forks before handling a client connection, so the base address will remain the same across all connections even if we crash a forked process.

In order to defeat ASLR, we need to leak a pointer to some known memory location in the apfd binary. To obtain this leak, we can use the overflow to corrupt the tail of the dsi struct (after the data buffer) to force the server to send us more data than expected. The command replay cache feature of the server provides a convenient way to do so.

Here are the relevant part of the main loop of afp_over_dsi():

Here is the code for dsi_cmdreply():

When the server receives the same command twice (same clientID and function), it takes the replay cache code path which calls dsi_cmdreply() without initializing dsi->datalen. So in that case, dsi_cmdreply() will send  dsi->datalen bytes of dsi->data back to the client in dsi_stream_send().

This is fortunate because the datalen field is located just after the data buffer in the struct DSI. That means that to control datalen we just need to trigger the overflow with 65536 + 4 bytes (4 being the size of a size_t).

Then, by sending a DSICommand command with an already used clientID we reach a dsi_cmdreply() that can send back all the dsi->data buffer, the tail of the dsi struct and part of the following heap data. In the dsi struct tail, we get some heap pointers such as dsi->buffer, dsi->start, dsi->eof, dsi->end. This is useful because we now know where client controlled data is stored.
In the following heap data, we hopefully expect to find pointers into afpd main image.

From our experiments we found out that most of the time, by requesting a leak of 2MB+64KB we get parts of the heap where hash_t objects were allocated by hash_create():

The hash_t structure is very distinct from other data and contains pointers on the hnode_alloc() and hnode_free() functions that are located in the afpd main image.
Therefore by parsing the received leak, we can look for hash_t patterns and recover the ASLR slide of the main binary. This method is implemented in the exploit in the function parse_leak().

Regrettably this strategy is not 100% reliable depending on the heap initialization of afpd.
There might be non-mapped memory ranges after the dsi struct, crashing the daemon while trying to send the leak.
In that case, the exploit won't work until the device (or daemon) get restarted.
Fortunately, this situation seems rare (less than 20% of the cases) giving the exploit a fair chance of success.

Building a write primitive

Now that we know where the main image and heap are located into the server memory, it is possible to use the full potential of the vulnerability and overflow the rest of the struct *DSI to reach code execution.

Rewriting dsi->proto_close looks like a promising way to get the control of the flow. However because of the lack of control on the arguments, we've chosen another exploitation method that works equally well on all architectures but requires the ability to write arbitrary data at a chosen location.


The look ahead pointers of the DSI structure seem like a nice opportunity to achieve a controlled write.

By setting dsi->buffer to the location we want to write and dsi->end as the upper bound of the writing location, the next command buffered by the server can end-up at a controlled address.

One should takes care while setting dsi->start and dsi->eof, because they are reset to dsi->buffer after the overflow in dsi_writeinit():

As seen in the snippet, this is only a matter of setting dsi->start greater than dsi->eof during the overflow.

So to get a write primitive one should:

  1. Overflow dsi->buffer, dsi->end, dsi->start and dsi->eof according to the write location.
  2. Send two commands in the same TCP segment.

The first command is just a dummy one, and the second command contains the data to write.

Sending two commands here seems odd but it it necessary to trigger the arbitrary write, because of the convoluted reception mechanism of dsi_stream_read().

When receiving the first command, dsi_buffered_stream_read() will skip the non-blocking call to recv() and take the blocking receive path in dsi_stream_read() -> buf_read() -> readt().

The controlled write happens during the reception of the second command. Because the two commands were sent in the same TCP segment, the data of the second one is most likely to be available on the socket. Therefore the non-blocking recv() should succeed and write at dsi->eof.

Command execution

With the ability to write arbitrary data at a chosen location it is now possible to take control of the remote program.

The most obvious location to write to is the array preauth_switch:

As seen previously, this array is used in afp_over_dsi() to dispatch the client DSICommand requests. By writing an arbitrary entry in the table, it is then possible to perform the following call with a controlled function pointer:

One excellent candidate to replace preauth_switch[function] with is afprun(). This function is used by the server to launch a shell command, and can even do so with root privileges :)

So to get a command executed as root, we transform the call:

into

The situation is the following:

  •  function is chosen by the client so that afp_switch[function] is the function pointer overwritten with afprun ;
  •  obj is a non-NULL AFPObj* pointer, which fits with the root argument that should be non zero ;
  •  dsi->commands is a valid pointer with controllable content, where we can put a chosen command such as a binded netcat shell ;
  •  dsi->cmdlen must either be NULL or a valid pointer because *outfd is dereferenced in afprun.

Here is one final difficulty. It is not possible to send a dsi->command long enough so that dsi->cmdlen becomes a valid pointer.
But with a NULL dsi->cmdlen, dsi->command is not controlled anymore.

The trick is to observe that dsi_stream_receive() does not clean the dsi->command in between client requests, and afp_over_dsi() does not check cmdlen before using dsi->commands[0].

So if a client send a DSI a packet without a dsi->command payload and a dsi->cmdlen of zero, the dsi->command remains the same as the previous command.

As a result it is possible to send:

  • A first DSI request with dsi->command being something similar to <function_id> ; /sbin/busybox nc -lp <PORT> -e /bin/sh;.
  • A second DSI request with a zero dsi->cmdlen.

This ends up calling:

which is what was required to get RCE once afp_switch[function_id] was overwritten with afprun.


As a final optimization, it is even possible to send the last two DSI packets triggering code execution as the last two commands required for the write primitive.
This results in doing the preauth_switch overwrite and the dsi->command, dsi->cmdlen setup at the same time.
As a matter of fact, this is even easier to mix both because of a detail that is not worth explaining into that write-up.
The interested reader can refer to the exploit commentaries.

Putting things together

To sum up here is an overview of the exploitation process:

  1. Setting up the connection.
  2. Triggering the vulnerability with a 4 bytes overflow to rewrite dsi->datalen.
  3. Sending a command with a previously used clientID to trigger the leak.
  4. Parsing the leak while looking for hash_t struct, giving pointers to the afpd main image.
  5. Closing the old connection and setting up a new connection.
  6. Triggering the vulnerability with a larger overflow to rewrite the look ahead buffer pointers of the dsi struct.
  7. Sending both requests as one:
    1. A first DSICommand with the content "<function_id> ; /sbin/busybox nc -lp <PORT> -e /bin/sh;" ;
    2. A second DSICommand with the content &afprun but with a zero length dsi_len and dsi->cmdlen.
  8. Sending a DSICommand without content to trigger the command execution.

Conclusion

During this research we developed a working exploit for the latest version of Netatalk. It uses a single heap overflow vulnerability to bypass all mitigations and obtain command execution as root. On the MyCloud Home the afpd services was configured to allow guest authentication, but since the bug was accessible prior to authentication the exploit works even if guest authentication is disabled.

The funkiest part was undoubtedly implementing a custom TCP stack to trigger the bug. This is quite uncommon for an user land and real life (as not in a CTF) exploit, and we hope that was entertaining for the reader.

Our exploit will be published on GitHub after a short delay. It should work as it on the targeted device. Adapting it to other distributions should require some minor tweaks and is left as an exercise.

Unfortunately, our Pwn2Own entry ended up being a duplicate with the Mofoffensive team who targeted another device that shipped an older version of Netatalk. In this previous release the vulnerability was in essence already there, but maybe a little less fun to exploit as it did not required to mess with the network stack.

We would like to thank:

  • ZDI and Western Digital for their organization of the P2O competition, especially this session considering the number of teams and their help to setup an environment for our exploit ;
  • The Netatalk team for the considerable amount of work and effort they put into this Open Source project.

Timeline

  • 2022-06-03 - Vulnerability reported to vendor
  • 2023-02-06 - Coordinated public release of advisory


[1] https://www.zerodayinitiative.com/blog/2021/11/1/pwn2ownaustin


[2] http://netatalk.sourceforge.net/


[3] https://github.com/Netatalk/Netatalk


[4] https://support-en.wd.com/app/products/product-detail/p/1369#WD_downloa…


[5] https://en.wikipedia.org/wiki/Data_Stream_Interface


[6] https://scapy.net/

Other publications

Relaying Kerberos over SMB using krbrelayx

Kerberos authentication relay was once thought to be impossible, but multiple researchers have since proven otherwise. In a 2021 article, James Forshaw discussed a technique for relaying Kerberos over ...
Hugo Vincent - 20/11/2024 - Pentest

Forensic analysis of bitwarden self-hosted server

Bitwarden is a popular password managing software. Being open-source, it offers self-hosting capabilities with ease of use in a controlled office or home environment. Attackers might prioritize target ...
Noam Leipold - 14/10/2024 - CSIRT

Contact us

+33 1 45 79 74 75
contact@synacktiv.com
GPG key

PARIS

5 boulevard Montmartre
75002 Paris

TOULOUSE

4 Rue du Pont Guilhemery
31000 Toulouse

LYON

56 rue Smith
69002 Lyon

RENNES

7D Rue de Châtillon
35000 Rennes

LILLE

7 Boulevard Louis XIV
59000 Lille
Copyright © Synacktiv 2024