Coroutines eclipsed: How NVIDIA is making C++ 900x faster
C++23 is just around the corner, with one of its most eagerly anticipated additions being full library support for coroutines. The antique technique allows for clean and useful asynchronous code, and a feature being worked on by C++ committee members at $1.1 trillion AI giant NVIDIA has the potential to do what coroutines do... only better.
At the Association of C and C++ Users conference 2023, keynote speaker and NVIDIA principal engineer Bryce Adelstein Lelbach spoke about his favorite features that could be added as early as C++26, one of which being Senders. "Today, C++ has no standard model for asynchrony and no standard way to express where things should execute," Lelbach says. Senders will change this.
They consist of three components: a scheduler, which creates senders, a sender, which produces and sends a signal (either success or a failure with an error message), and a receiver which processes the messages sent by the senders. This system allows individual nodes to operate efficiently across multiple threads and on multiple pieces of hardware.
Lelbach and NVIDIA already have an open source framework for this on GitHub called stdexec. When implementing it, the speed of execution for a single node can be over 900x faster and larger multi node projects can be implemented across thousands of GPUs.
Does this make senders better than coroutines? Not necessarily. "Coroutines are a much nicer way to write asynchronous code," Lelbach says, "because you can write code that appears synchronous." However, he says senders are more powerful as they can "express patterns you can't as easily express with coroutines." He says the intention is for the two systems to be interoperable.
What does this mean for finance? The purported speed increases will be an electronic trader's dream, particularly if they are hardware focused like the FPGA engineers at Optiver, Goldman Sachs and beyond. Implementing them won't necessarily be pretty or easy to do, but in the world of low latency trading, effectiveness trumps all. The implementation of senders in one of the next C++ releases may further cement C++ as the language of choice for superfast code.
Have a confidential story, tip, or comment you’d like to share? Contact: +44 7537 182250 (SMS, Whatsapp or voicemail). Telegram: @SarahButcher. Click here to fill in our anonymous form, or email editortips@efinancialcareers.com. Signal also available.
Bear with us if you leave a comment at the bottom of this article: all our comments are moderated by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libelous (in which case it won’t.)
