Rust + io_uring + ktls: How Fast Can We Make HTTP?
0 likes2,024 views
The document discusses the potential of using Rust and io_uring to improve HTTP performance, highlighting the current limitations and benchmarking challenges. It emphasizes the need for better hardware and testing practices while exploring various technical aspects of async programming and system calls. The author, Amos Wenger, invites collaboration and donations to enhance their testing setup and future developments.
![/// An HTTP status code (`status-code` in RFC 9110 et al.).
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct StatusCode(NonZeroU16);](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-12-320.jpg)
![// A string of packed 3-ASCII-digit status code
// values for the supported range of [100, 999]
// (900 codes, 2700 bytes).
const CODE_DIGITS: &str = "
100101102103104105106107108109110
✂ ✂ ✂
989990991992993994995996997998999";](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-14-320.jpg)
![fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-18-320.jpg)
![fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-19-320.jpg)
![fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-21-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-22-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-23-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-24-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<usize>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-25-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-26-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-27-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)
fn read(
&mut self,
buf: &mut [u8]
) -> Read<'_, Self>
where Self: Unpin
async](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-29-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn read(
&mut self,
buf: &mut [u8]
) -> Read<'_, Self>
where Self: Unpin
async](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-30-320.jpg)
![blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
async fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
where Self: Unpin { ... }
async](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-31-320.jpg)
![async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
s.read_exact(&mut buf).await?;
Ok(buf)
}](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-32-320.jpg)
![async stack trace
read(&mut [u8])
read_exact(&mut [u8])
mhhh()
// (not shown: tokio runtime internals)](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-33-320.jpg)
![async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
tokio::select! {
result = s.read_exact(&mut buf) => {
result?;
Ok(buf)
}
_ = sleep(Duration::from_secs(1)) => {
Err(timeout_err())
}
}
}](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-35-320.jpg)
![async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
tokio::select! {
result = s.read_exact(&mut buf) => {
result?;
Ok(buf)
}
_ = sleep(Duration::from_secs(1)) => {
Err(timeout_err())
}
}
}](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-36-320.jpg)
![let mut buf = vec![0u8; 4];
let mut read_fut = Box::pin(s.read_exact(&mut buf));
tokio::select! {
_ = &mut read_fut => { todo!() }
_ = sleep(Duration::from_secs(1)) => {
std::mem::forget(read_fut);
Err(timeout_err())
}
}](https://image.slidesharecdn.com/amoswengerp99conf2024-241011004039-d869a4e0/85/Rust-io_uring-ktls-How-Fast-Can-We-Make-HTTP-39-320.jpg)
Rust + io_uring + ktls: How Fast Can We Make HTTP?
- 1. A ScyllaDB Community Rust, io_uring, ktls: How fast can we make HTTP? Amos Wenger writer, video producer, cat owner bearcove
- 2. A ScyllaDB Community Nobody in the Rust space is going far enough with io_uring (as far as I'm aware) Amos Wenger writer, video producer, cat owner bearcove
- 3. Amos Wenger (they/them) aka @fasterthanlime writer, video producer, cat owner ■ Wrote "Making our own executable packer" ■ Teaching Rust since 2019 with Cool Bear ■ Fan of TLS (thread-local storage & the other one) bearcove
- 6. Define "HTTP"
- 7. Define "fast"
- 8. Rust HTTP is already fast
- 9. hyper on master is 📦 v1.4.1 via 🦀 v1.80.1 ❯ gl --color=always | tail -5 Commit: 886551681629de812a87555bb4ecd41515e4dee6 Author: Sean McArthur <sean.monstar@gmail.com> Date: 2014-08-30 14:18:28 -0700 (10 years ago) init
- 10. HTTP/1.1 200 OK Date: Fri, 31 Dec 1999 23:59:59 GMT Content-Type: text/html Content-Length: 1354
- 12. /// An HTTP status code (`status-code` in RFC 9110 et al.). #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] pub struct StatusCode(NonZeroU16);
- 13. mystery winner > itoa stack > itoa heap > std::fmt criterion bench: format_status_code, avg µs
- 14. // A string of packed 3-ASCII-digit status code // values for the supported range of [100, 999] // (900 codes, 2700 bytes). const CODE_DIGITS: &str = " 100101102103104105106107108109110 ✂ ✂ ✂ 989990991992993994995996997998999";
- 15. We're not bickering over assembly anymore
- 16. My hypothesis ● spectre, meltdown, etc => mitigations ● mitigations => more expensive syscalls ● more expensive syscalls => io_uring
- 17. Type systems are hard
- 18. fn read( &mut self, buf: &mut [u8], ) -> Result<usize>
- 19. fn read( &mut self, buf: &mut [u8], ) -> Result<usize>
- 20. Lifetimes exist in every language Rust merely explicits them
- 21. fn read( &mut self, buf: &mut [u8], ) -> Result<usize>
- 22. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( &mut self, buf: &mut [u8], ) -> Poll<Result<usize>> evented (O_NONBLOCK)
- 23. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( &mut self, buf: &mut [u8], ) -> Poll<Result<usize>> evented (O_NONBLOCK)
- 24. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( &mut self, cx: &mut Context<'_>, buf: &mut [u8], ) -> Poll<Result<usize>> evented (O_NONBLOCK)
- 25. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( &mut self, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>, ) -> Poll<Result<usize>> evented (O_NONBLOCK)
- 26. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( &mut self, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>, ) -> Poll<Result<()>> evented (O_NONBLOCK)
- 27. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>, ) -> Poll<Result<()>> evented (O_NONBLOCK)
- 29. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn poll_read( self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>, ) -> Poll<Result<()>> evented (O_NONBLOCK) fn read( &mut self, buf: &mut [u8] ) -> Read<'_, Self> where Self: Unpin async
- 30. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> fn read( &mut self, buf: &mut [u8] ) -> Read<'_, Self> where Self: Unpin async
- 31. blocking fn read( &mut self, buf: &mut [u8], ) -> Result<usize> async fn read( &mut self, buf: &mut [u8], ) -> Result<usize> where Self: Unpin { ... } async
- 32. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> { let mut buf = vec![0u8; 4]; s.read_exact(&mut buf).await?; Ok(buf) }
- 33. async stack trace read(&mut [u8]) read_exact(&mut [u8]) mhhh() // (not shown: tokio runtime internals)
- 34. real stack trace Read::poll(Pin<&mut Read>, &mut Context<'_>) ReadExact::poll(Pin<&mut ReadExact>, &mut Context<'_>) Mhh::poll(Pin<&mut Mhh>, &mut Context<'_>) // (not shown: tokio runtime internals)
- 35. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> { let mut buf = vec![0u8; 4]; tokio::select! { result = s.read_exact(&mut buf) => { result?; Ok(buf) } _ = sleep(Duration::from_secs(1)) => { Err(timeout_err()) } } }
- 36. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> { let mut buf = vec![0u8; 4]; tokio::select! { result = s.read_exact(&mut buf) => { result?; Ok(buf) } _ = sleep(Duration::from_secs(1)) => { Err(timeout_err()) } } }
- 37. rio::Uring pub fn recv<'a, Fd, Buf>( &'a self, stream: &'a Fd, iov: &'a Buf ) -> Completion<'a, usize>
- 38. rio::Uring impl<'a, C: FromCqe> Drop for Completion<'a, C> { fn drop(&mut self) { self.wait_inner(); } }
- 39. let mut buf = vec![0u8; 4]; let mut read_fut = Box::pin(s.read_exact(&mut buf)); tokio::select! { _ = &mut read_fut => { todo!() } _ = sleep(Duration::from_secs(1)) => { std::mem::forget(read_fut); Err(timeout_err()) } }
- 42. tokio_uring::net::TcpStream async fn read(&self, buf: T) -> (T, Result<usize>) where T: BoundedBufMut;
- 43. Fine, I'll rewrite everything on top of io-uring then. docs.rs/loona
- 45. load testing is hard ■ macOS = nice for dev, useless for perf ■ P-states ■ love your noisy neighbors ■ stats are hard (coordinated omission etc.)
- 46. the plan? ■ Intel(R) Xeon(R) CPU E3-1275 v6 @ 3.80GHz ■ h2load from another dedicated server ■ 16 clients virtual clients, max 100 streams per client ■ python for automation (running commands over SSH, CSV => XLS etc.) ■ perf for counting cycles, instructions, branches
- 49. what's next? ● more/better benchmarks ● …on hardware from this decade ● proxying to HTTP/1.1, serving from disk ● messing with: allocators, buffer size ● io_uring: provided buffers, multishot accept/read ● move off of tokio entirely (no atomics needed, no "write to unpark thread" needed)
- 50. how do we make that happen? ● money donations ● hardware donations ● expertise donations ● did I mention money
- 51. Thank you! Let’s connect. Amos Wenger amos@bearcove.net @fasterthanlime https://fasterthanli.me bearcove