Databento and QuestDB announce collaboration

We're excited to announce our latest collaboration with QuestDB, a high-performance, open source, and columnar database that specializes in handling time-series datasets. QuestDB is designed to solve ingestion speed bottlenecks and simplify SQL workflows, demonstrating incredible performance with high volumes of data.

Databento and QuestDB are building the modern infrastructure stack for finance, each offering complementary services used in environments managing billions of rows of data. The company's latest tutorial offers a step-by-step guide to efficiently aggregate and analyze Databento's live market data:

1. Request real-time market data using Databento's Python client library.
2. Ingest S&P 500 futures from our CME dataset into QuestDB.
3. Visualize the data in Grafana using QuestDB's plugin.
4. Calculate derived metrics with SQL, including each contract's mid-price, liquidity via the bid-ask spread, the minimum size quoted on the contracts, the roll yield, and the implicit annualized cost of holding a position.

Check out the example here.

The QuestDB integration introduces several exciting capabilities for users:

• Integration with other Apache tools and frameworks. QuestDB integrates with Kafka, Flink, and more.
• Visualize in Grafana as a drop-in replacement for InfluxDB. Create a real-time Grafana dashboard and enable time-series data analysis using the official QuestDB plugin. For InfluxDB users, QuestDB's support for schema-agnostic ingestion via the InfluxDB Line Protocol allows for flexible bulk imports and exports across various data sources.
• Use open standards like ANSI SQL for data preprocessing, while achieving lightning-fast speeds that effortlessly manage even the most granular data, such as Market By Order (MBO) and Level 2 (L2) data.
• Easy setup. Install with modern tools like Docker and Kubernetes (via Helm). Integrate using official language clients in Python, Go, Java, Rust, Node.js, C++, and more.

Databento is the only commercial market data solution that enables the delivery of L2 and MBO ("L3") feeds over the internet and public cloud, serving petabytes of data to thousands of customers.

QuestDB processes queries over billions of rows within milliseconds, taking advantage of a column-oriented storage model, parallelized vector execution, SIMD, and low-latency optimizations. It's used as a drop-in replacement for other high-performance financial databases like kdb and Vertica.