Photo

Algorithm Of Massively Parallel Networking In C++

Vladislav Shpilevoy

from VirtualMinds (Germany)

About speaker

Senior Developer at VirtualMinds

A senior developer in VirtualMinds and a developer in Tarantool DBMS.

About speakers company

Advertisement.

Abstracts

specific

This talk focuses on backend performance in C++ with an alternative to boost::asio for massively parallel networking. Boost::asio is the de facto standard for C++ network code, but in rare cases it isn't available or enough due to various reasons. Drawing from experience on high-performance projects, I developed a new task scheduling algorithm, built a networking library around it, and present them here. Key features include fair CPU scheduling, C++ coroutines support, formal verification with TLA+, and reproducible benchmarks showing N times speed up over boost::asio. The project is open-source and available at https://github.com/Gerold103/serverbox.

Abstract
There are 3 reasons to have a backend written in C/C++:
- Legacy;
- Truly exceptional team expertise in C/C++;
- Performance is the highest priority.

This talk mainly focuses on the last reason - performance. I've had the opportunity to work on projects where it was critical to maximize network throughput, minimize latency, and efficiently scale on many CPU cores with multithreading.

What I've learned is that in 99.999% of cases boost::asio will do the job. It is a stable battle-tested project that has become the de facto standard for async networking in C++. I would honestly stick to it when possible. However sometimes it can't be used. I've seen these common reasons:
- Boost is C++-only and might be not available at all for some platforms;
- Company has in-house "libraries of everything" and avoids large external dependencies like boost;
- Boost::asio performance in stress load is questionable, and the source code at times is hard to comprehend.

At one of my projects I had to find an alternative to boost::asio due to the facts above. As a result, I designed a new generic task scheduling algorithm and built a networking library around it. In this talk I am presenting my solution, focusing on the following features:
- Fair scheduling - even distribution of CPU usage across worker threads, enabling smooth scaling on many cores;
- C++ coroutine support - can write linear asynchronous code;
- Benchmarks - reproducible and automatic, the library outperforms boost::asio by N times.
- Formal correctness - the core algorithm is formally verified in TLA+.
- Simplicity - the algorithm is easy to understand, can be ported to other non-C++ languages.

The entire project is battle-tested, is open-source and available here: https://github.com/Gerold103/serverbox.

The Program Committee has not yet taken a decision on this talk