About The RoleAt Massive, our live data products keep users current on financial markets, processing terabytes of data a day while serving results that are fresh, predictable, and fast.fireflow is the streaming dataflow engine behind those products.
Model the data. Design the entities, relationships, and grains that turn raw feeds into datasets people can reason about - and document why the model is shaped the way it is.
Build and own the transformation layer in Python and SQL: cleaning, normalization, validation, and reconciliation across overlapping sources.
Help architect, design, and implement critical components of our feed handling layer.
Build hardware and software capable of capturing 60M packets/sec. Analyze captures for dropped packets and merge/dedupe captures from multiple datacenters. ~5TB/day and growing.