Introducing Polars: The fast(est) DataFrame library you may have heard of
This talk will introduce Polars, a blazingly fast DataFrame library written in Rust on top of Apache Arrow. It’s a DataFrame library designed from the ground up for OLAP processing. Hoping to learn from accumulative research in data processing.
CPU’s today’s come with many cores and with their superscalar designs and SIMD registers allow for even more parallelism. Polars is written from the ground up to fully utilize the CPU’s of this generation.
Besides blazingly fast algorithms, cache efficient memory layout and multi-threading, it consists of a lazy query engine, allowing Polars to do several optimizations that may improve query time and memory usage.