jstrong.dev

series

Paging S3 Files With Glommio [ DRAFT ]

Lets use a bleeding edge IO framework to solve a simple task!

Fast Data Pipelines in Rust [ DRAFT ]

For when the data is big and dynamic scripting languages are slow...
posts
about

Preface
1. Introduction

1.1. Why "data pipelines"?
1.2. Why Rust?
1.3. The Data
1.4. The Task
1.5. Ground Rules

2. Baselines

2.1. Python/Pandas
2.2. Postgres
2.3. Influxdb
2.4. Clickhouse

3. Naive Rust

3.1. CSV Parsing
3.2. Query Execution: Easy
3.3. Query Execution: Hard

4. Single-Threaded

4.1. Binary serialization
4.2. Struct of Array
4.3. Explicit SIMD

5. Multi-Threaded

5.1. Rayon
5.2. Custom Work-Stealing Queue

7. Conclusion

series
fast data pipelines in rust
ch 3.1

Single-Threaded Optimizations

by Jonathan Strong 2020-04-16T02:38:54.007563707Z

It's all about the cache