pkg/lwip: race condition (data loss) in async connection setup

[maribu]

Description

Taking the code

/* This callback is used when data is received over a TCP connection */
static void _server_tcp_data_cb(sock_tcp_t *sock, sock_async_flags_t flags,
                                void *arg)
{
    /* handle received data here */
}

/* This callback is used when a connection request on the TCP listen socket came in */
static void _server_tcp_listen_cb(sock_tcp_queue_t *queue,
                                  sock_async_flags_t flags, void *arg)
{
    foo_ctx_t *ctx = arg; /* app context in ctx, and event queue to use specified in ctx->evq */

    if (!(flags & SOCK_ASYNC_CONN_RECV)) {
        DEBUG("Unhandled event(s) on listen socket. Event flags = %x\n",
              (unsigned)flags);
        return;
    }

    sock_tcp_t *client;
    int res = sock_tcp_accept(queue, &client, 0);
    if (res != 0) {
        DEBUG("sock_tcp_accept failed with %d\n", res);
        return;
    }

    /* !!! RACE HERE !!! */

    sock_tcp_event_init(client, ctx->evq, _server_tcp_data_cb, ctx);
}

if data is received between sock_tcp_accept() and sock_tcp_event_init() (at !!! RACE HERE !!!), this data is lost and _server_tcp_data_cb() is never called

Steps to reproduce the issue

Open a TCP socket in async mode using lwIP (e.g. a TCP echo server). Connect with a client that directly starts to send data when the connection is established. Add a bit of delay before the sock_tcp_event_init(), e.g. by sending something out via TCP.

Expected results

Any data that already is in the queue upon call sock_tcp_event_init() would still be passed to _server_tcp_data_cb(), e.g. right when calling sock_tcp_event_init().

Actual results

The data received from the client is lost. The server will hang. Resources for the data in the TCP connection that never was passed to _server_tcp_data_cb() will not get freed until the client closes the connection.

A client triggering this issue and keeping connections open could quickly exhaust the memory of the TCP server.

Versions

Current RIOT master.