Photo

Algorithm Of Massively Parallel Networking In C++

Vladislav Shpilevoy

from VirtualMinds (Germany)

About speaker

Senior Developer at VirtualMinds

A senior developer in VirtualMinds and a developer in Tarantool DBMS.

About speakers company

Advertisement.

Abstracts

specific

This talk focuses on backend performance in C++ with an alternative to boost::asio for massively parallel networking. Boost::asio is the de facto standard for C++ network code, but in rare cases it isn't available or enough due to various reasons. Drawing from experience on high-performance projects, I developed a new task scheduling algorithm, built a networking library around it, and present them here. Key features include fair CPU scheduling, C++ coroutines support, formal verification with TLA+, and reproducible benchmarks showing N times speed up over boost::asio. The project is open-source and available at https://github.com/Gerold103/serverbox.

Abstract
There are 3 reasons to have a backend written in C/C++:
- Legacy;
- Truly exceptional team expertise in C/C++;
- Performance is the highest priority.

This talk mainly focuses on the last reason - performance. I've had the opportunity to work on projects where it was critical to maximize network throughput, minimize latency, and efficiently scale on many CPU cores with multithreading.

What I've learned is that in 99.999% of cases boost::asio will do the job. It is a stable battle-tested project that has become the de facto standard for async networking in C++. I would honestly stick to it when possible. However sometimes it can't be used. I've seen these common reasons:
- Boost is C++-only and might be not available at all for some platforms;
- Company has in-house "libraries of everything" and avoids large external dependencies like boost;
- Boost::asio performance in stress load is questionable, and the source code at times is hard to comprehend.

At one of my projects I had to find an alternative to boost::asio due to the facts above. As a result, I designed a new generic task scheduling algorithm and built a networking library around it. In this talk I am presenting my solution, focusing on the following features:
- Fair scheduling - even distribution of CPU usage across worker threads, enabling smooth scaling on many cores;
- C++ coroutine support - can write linear asynchronous code;
- Benchmarks - reproducible and automatic, the library outperforms boost::asio by N times.
- Formal correctness - the core algorithm is formally verified in TLA+.
- Simplicity - the algorithm is easy to understand, can be ported to other non-C++ languages.

The entire project is battle-tested, is open-source and available here: https://github.com/Gerold103/serverbox.

The Program Committee has not yet taken a decision on this talk

other talks of this topic

Photo
Continuous Profiling on K8s - why, when and how

Ant(on) Weiss

PerfectScale

specific
Photo
Why You Ignore Best Practices and How You Can Fix It

Annelore Egger

OpenValue Switzerland

broad
Photo
You don't need to implement GraphQL

Sefi Ninio

Tikal Knowledge

specific
Photo
C# 13 Unleashed: Live Demos of my Top 10 Cutting-Edge Features!

Ambesh Singh

Visionet Systems Deutschland

broad
Photo
Path to Golden Path

Daniel Drack

FullStackS GmbH

broad
Photo
Three Flavors of Pokémon - Framework Agnostic UI Testing

Shelly Goldblit

Dell Technologies

broad
Photo
UX at the centre of system development and design

Anesu Makwasha

Tose Technologies

specific
Photo
Collaborative applications and how to make them fast

Bartosz Sypytkowski

appflowy.io

specific
Photo
Deep dive into the postgres index types

Jesús Espino

Mattermost Inc.

specific
Photo
Become a Gen AI Bot Master in Just 50 Minutes – No Kidding!!

Ambesh Singh

Visionet Systems Deutschland

broad
Photo
What the @#!? is Auth

Warren Parad

Authress

specific
Photo
How we elevated tracking data accuracy from ~60% to ~80%

Alina Krasavina

Delivery hero

broad
Photo
An Efficient Git Workflow For High-Stakes Projects

Vladislav Shpilevoy

VirtualMinds

specific
Photo
Taking Shortcuts Beyond Your IDE

Annelore Egger

OpenValue Switzerland

broad
Photo
Pros and Cons of Jetpack Compose Toolkit

Stevan Milovanovic

InterVenture

specific
Photo
JavaScript is weird. MythBusters special.

Małgorzata Janeczek

Sector Alarm Tech

broad
Photo
From Server to Serverless - A story of saving Cost

Yoav Nordmann

Tikal Knowledge

specific
Photo
Sculpting Data for Machine Learning: Generative AI edition

Rishabh Misra

Attentive Mobile Inc

broad
Photo
From null to applications on Kubernetes

Roberth Strand

Sopra Steria

specific
Photo
Crafting the Ultimate Docker Image for Spring Applications

Pasha Finkelshteyn

BellSoft

specific
Photo
Putting the asm in Wasm: from bytecode to native

Edoardo Vacchi

Tetrate

specific
Photo
Throw exceptions... out of your codebase

Guillaume Faas

Vonage

specific
Photo
How Unit Testing Saved My Career

Annelore Egger

OpenValue Switzerland

broad
Photo
Workshop: Master Anti-Ban & Web Scraping Techniques (2h)

Fabien Vauchelles

Scrapoxy

specific
Photo
Simple and stable UI tests with Ultron

Aleksei Tiurin

Exness

specific
Photo
Go performance profiling in theory and practice

Alexey Palazhchenko

FerretDB Inc.

broad