The amount billed will be based on the actual amount of bytes sent; see our pricing documentation for more details.
API reference - Historical
Databento's historical data service can be accessed programmatically over its HTTP API. To make it easier to integrate the API, we also provide official client libraries that simplify the code you need to write.
Our HTTP API is designed as a collection of RPC-style methods, which can be
called using URLs in the
form https://hist..
Our client libraries wrap these HTTP RPC-style methods with more idiomatic interfaces in their respective languages.
You can use our API to stream or load data directly into your application. You can also use our API to make batch download requests, which instruct our service to prepare the data as flat files that can downloaded from the Download center.
HISTORICAL DATA
Client libraries
Python
C++
Rust
APIs
HTTP
HTTP
$
git clone https://github.com/databento/databento-cpp.git
59
Basics
Overview
Our historical API has the following structure:
- Metadata provides information about the datasets themselves.
- Time series provides all types of time series data. This includes subsampled data (second, minute, hour, daily aggregates), trades, top-of-book, order book deltas, order book snapshots, summary statistics, static data and macro indicators. We also provide properties of products such as expirations, tick sizes and symbols as time series data.
- Symbology provides methods that help find and resolve symbols across different symbology systems.
- Batch provides a means of submitting and querying for details of batch download requests.
Authentication
Databento uses API keys to authenticate requests. You can view and manage your keys on the API keys page of your portal.
Each API key is a 32-character string.
The library will use the environment variable DATABENTO_API_KEY as your API key if the SetKeyFromEnv method is called.
However, you can pass an API key directly to the HistoricalBuilder through the SetKey method.
Calling the Build method constructs and returns an instance of Historical, the historical client.
Related: Securing your API keys.
Example usage
#include <databento/historical.hpp>
#include <iostream>
// Establish connection and authenticate
auto client =
databento::Historical::Builder().SetKeyFromEnv().Build();
// Authenticated request
auto datasets = client.MetadataListDatasets();
for (const auto& dataset : datasets) {
std::cout << dataset << ", ";
}
std::cout << '\n';
Schemas and conventions
A schema is a data record format represented as a collection of different data fields. Our datasets support multiple schemas, such as order book, tick data, bar aggregates, and so on. You can see a full list from our List of market data schemas.
You can get a list of all supported schemas for any given dataset using the MetadataListSchemas method. The same information can also be found on each dataset's detail page found through the Explore feature.
The following table provides details about the data types and conventions used for various fields that you will commonly encounter in the data.
| Name | Field | Description |
|---|---|---|
| Dataset | dataset |
A unique string name assigned to each dataset by Databento. Full list of datasets can be found from the metadata. |
| Publisher ID | publisher_id |
A unique 16-bit unsigned integer assigned to each publisher by Databento. Full list of publisher IDs can be found from the metadata. |
| Instrument ID | instrument_id |
A unique 32-bit unsigned integer assigned to each instrument by the venue. Information about instrument IDs for any given dataset can be found in the symbology. |
| Order ID | order_id |
A unique 64-bit unsigned integer assigned to each order by the venue. |
| Timestamp (event) | ts_event |
The matching-engine-received timestamp expressed as the number of nanoseconds since the UNIX epoch. |
| Timestamp (receive) | ts_recv |
The capture-server-received timestamp expressed as the number of nanoseconds since the UNIX epoch. |
| Timestamp delta (in) | ts_in_delta |
The matching-engine-sending timestamp expressed as the number of nanoseconds before ts_recv. See timestamping guide. |
| Timestamp out | ts_out |
The Databento gateway-sending timestamp expressed as the number of nanoseconds since the UNIX epoch. See timestamping guide. |
| Price | price |
The price expressed as signed integer where every 1 unit corresponds to 1e-9, i.e. 1/1,000,000,000 or 0.000000001. |
| Book side | side |
The side that initiates the event. Can be Ask for a sell order (or sell aggressor in a trade), Bid for a buy order (or buy aggressor in a trade), or None where no side is specified by the original source. |
| Size | size |
The order quantity. |
| Flag | flag |
A bit field indicating event end, message characteristics, and data quality. |
| Action | action |
The event type or order book operation. Can be Add, Cancel, Modify, cleaR book, Trade, Fill, or None. |
| Sequence number | sequence |
The original message sequence number from the venue. |
Datasets
Databento provides time series datasets for a variety of markets, sourced from different publishers. Our available datasets can be browsed through the search feature on our site.
Each dataset is assigned a unique string identifier (dataset ID) in the form PUBLISHER.DATASET, such as GLBX.MDP3.
For publishers that are also markets, we use standard four-character ISO 10383 Market Identifier Codes (MIC).
Otherwise, Databento arbitrarily assigns a four-character identifier for the publisher.
These dataset IDs are also found on the Data catalog and Download request features of the Databento user portal.
When a publisher provides multiple data products with different levels of granularity, Databento subscribes to the most-granular product. We then provide this dataset with alternate schemas to make it easy to work with the level of detail most appropriate for your application.
More information about different types of venues and publishers is available in our Knowledge base.
Symbology
Databento's historical API supports several ways to select an instrument in a dataset. An instrument is specified using a symbol and a symbology type, also referred to as an stype. The supported symbology types are:
- Raw symbology (
RawSymbol) original string symbols used by the publisher in the source data. - Instrument ID symbology (
InstrumentId) unique numeric ID assigned to each instrument by the publisher. - Parent symbology (
Parent) groups instruments related to the market for the same underlying. - Continuous contract symbology (
Continuous) proprietary symbology that specifies instruments based on certain systematic rules.
When requesting data from our timeseries.get_range or batch.submit_job endpoints, an input and output symbology type can be specified. By default, our client libraries will use raw symbology for the input type and instrument ID symbology for the output type. Not all symbology types are supported for every dataset.
The process of converting between one symbology type to another is called symbology resolution. This conversion can be done, for no cost, with the symbology.resolve endpoint.
For more about symbology at Databento, see our Knowledge base.
Encodings
DBN
Databento Binary Encoding (DBN) is an extremely fast message encoding and highly-compressible storage format for normalized market data. It includes a self-describing metadata header and adopts a binary format with zero-copy serialization.
We recommend using our Python, C++, or Rust client libraries to read DBN files locally. A CLI tool is also available for converting DBN files to CSV or JSON.
CSV
Comma-separated values (CSV) is a simple text file format for tabular data, CSVs can be easily opened with Excel, loaded into pandas data frames, or parsed in C++.
Our CSVs have one header line, followed by one record per line.
Lines use UNIX-style \n separators.
JSON
JavaScript Object Notation (JSON) is a flexible text file format with broad language support and wide adoption across web apps.
Our JSON files follow the JSON lines specification, where
each line of the file is a JSON record.
Lines use UNIX-style \n separators.
Compression
Databento provides options for compressing files from our API. Available compression formats depend on the encoding you select.
zstd
The zstd compression option uses the Zstandard format.
This option is available for all encodings, and is recommended for faster transfer speeds and smaller files.
Read more about working with Zstandard-compressed files.
none
The none compression option disables compression entirely, resulting
in significantly larger files.
However, this can be useful for loading small CSV files directly into Excel.
Dates and times
Our C++ client library has several methods with timestamp arguments. These arguments will implement separate overloads with types databento::UnixNanos and const std::string&.
The recommended interface to use is databento::UnixNanos. It is a type alias for std::chrono::time_point with nanosecond precision, represented as a uint64_t. The std::chrono section of the standard library contains functions for parsing and converting human-readable timestamps into UNIX nanoseconds.
These timestamps are always in UTC. To localize from other timezones, you will need to use features added in C++20, or a third-party library such as date.
The C++ client library supports const std::string& arguments, with supported formatting based on ISO 8601 as listed below.
yyyy-mm-dd, e.g."2022-02-28"(midnight UTC)yyyy-mm-ddTHH:MM, e.g."2022-02-28T23:50"yyyy-mm-ddTHH:MM:SS, e.g."2022-02-28T23:50:59"yyyy-mm-ddTHH:MM:SS.NNNNNNNNN, e.g."2022-02-28T23:50:59.123456789"
Timezone specification is also supported.
yyyy-mm-ddTHH:MMZyyyy-mm-ddTHH:MMĀ±hhyyyy-mm-ddTHH:MMĀ±hhmmyyyy-mm-ddTHH:MMĀ±hh:mm
Bare dates
Some parameters require a bare date, without a time. These arguments have type const std::string& and must be a string in yyyy-mm-dd format, e.g. "2022-02-28".
Errors
Our historical API uses HTTP response codes to indicate the success or failure of an API request. The client library provides exceptions that wrap these response codes.
2xxindicates success.4xxindicates an error on the client side. Represented as aHttpResponseError.5xxindicates an error with Databento's servers. Represented as aHttpResponseError.
You can check the HTTP response code of a HttpResponseError with its status_code method.
The full list of the response codes and associated causes is as follows:
| Code | Message | Cause |
|---|---|---|
| 200 | OK | Successful request. |
| 206 | Partial Content | Successful request, with partially resolved symbols. |
| 400 | Bad Request | Invalid request. Usually due to a missing, malformed or unsupported parameter. |
| 401 | Unauthorized | Invalid username or API key. |
| 402 | Payment Required | Issue with your account payment information. |
| 403 | Forbidden | The API key has insufficient permissions to perform the request. |
| 404 | Not Found | A resource is not found, or a requested symbol does not exist. |
| 409 | Conflict | A resource already exists. |
| 422 | Unprocessable Entity | The request is well formed, but we cannot or will not process the contained instructions. |
| 429 | Too Many Requests | API rate limit exceeded. |
| 500 | Internal Server Error | Unexpected condition encountered in our system. |
| 503 | Service Unavailable | Data gateway is offline or overloaded. |
| 504 | Gateway Timeout | Data gateway is available but other parts of our system are offline or overloaded. |
API method
class Exception : public std::exception;
class HttpRequestError : public Exception;
class HttpResponseError : public Exception;
class InvalidArgumentError : public Exception;
class JsonResponseError : public Exception;
class DbnResponseError : public Exception;
Example usage
#include <databento/exceptions.hpp> // HttpResponseError
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
// set invalid key
auto client = Historical::Builder().SetKey("invalid").Build();
try {
auto datasets = client.MetadataListDatasets();
} catch (const HttpResponseError& err) {
std::cout << err.what() << '\n';
}
Example response
Received an error response from request to /v0/metadata.list_datasets with status 401 and body '{"detail":{"case":"auth_authentication_failed","message":"Authentication failed.","status_code":401,"docs":"https://databento.com/docs/portal/api-keys","payload":null}}'
Rate limits
Our historical API allows each IP address up to:
- 100 concurrent connections.
- 100 time series requests per second.
- 100 symbology requests per second.
- 20 metadata requests per second.
- 20 batch list jobs requests per second.
- 20 batch submit job requests per minute.
When a request exceeds a rate limit, a HttpResponseCode exception is raised
with a 429 error code.
Size limits
There is no size limit for either stream or batch download requests. Batch download is more manageable for large datasets, so we recommend using batch download for requests over 5 GB.
You can also manage the size of your request by splitting it into
multiple, smaller requests. The historical API allows you to make stream and
batch download requests with time ranges specified up to nanosecond resolution.
You can also use the limit parameter in any request to limit the number of
data records returned from the service.
Batch download supports different
delivery methods which can be specified using the delivery parameter.
Example usage
#include <databento/historical.hpp>
using namespace databento;
auto client = Historical::Builder().SetKeyFromEnv().Build();
auto job = client.BatchSubmitJob(
"GLBX.MDP3", kAllSymbols, Schema::Trades,
{"2022-08-26", "2022-09-27"}, {}, {}, {}, {}, {}, {}, {}, {},
{}, SType::RawSymbol, SType::InstrumentId, 1000 /* limit */);
Metered pricing
Databento only charges for the data that you use. You can find rates (per MB) for the various datasets and estimate pricing on our Data catalog. We meter the data by its uncompressed size in binary encoding.
When you stream the data, you are billed incrementally for each outbound byte of data sent from our historical gateway. If your connection is interrupted while streaming our data and our historical gateway detects connection timeout over 5 seconds, it will immediately stop sending data and you will not be billed for the remainder of your request.
Duplicate streaming requests will incur repeated charges. If you intend to access the same data multiple times, we recommend using our batch download feature. When you make a batch download request, you are only billed once for the request and, subsequently, you can download the data from the Download center multiple times over 30 days for no additional charge.
You will only be billed for usage of time series data. Access to metadata, symbology, and account management is free.
Related: Billing management.
Versioning
Our historical API and its client libraries adopt MAJOR.MINOR.PATCH format
for version numbers. These version numbers conform to
semantic versioning. We are using major version 0 for
initial development, where our API is not considered stable.
Once we release major version 1, our public API will be stable. This means that
you will be able to upgrade minor or patch versions to pick up new functionality,
without breaking your integration.
Starting with major versions after 1, we will provide support for previous
versions for one year after the date of the subsequent major release.
For example, if version 2.0.0 is released on January 1, 2024, then all versions
1.x.y of the API and client libraries will be deprecated. However, they will
remain supported until January 1, 2025.
We may introduce backwards-compatible changes between minor versions in the form of:
- New data encodings
- Additional fields to existing data schemas
- Additional batch download customizations
Our Release notes will contain information about both breaking and backwards-compatible changes in each release.
Our API and official client libraries are kept in sync with same-day releases
for major versions. For instance, 1.x.y of the C++ client
library will use the same functionality found in any 1.x.y version of the Python client.
Related: Release notes.
Client
Historical
To access Databento's historical API, first create an instance of the
Historical client. The entire API is exposed through instance methods of
the client.
Note that the API key can be passed as an argument, which is
not recommended for production applications.
Instead, you can use the HistoricalBuilder class which includes the SetKeyFromEnv method for setting key from the DATABENTO_API_KEY environment variable.
Parameters
log_receiver
ILogReceiver*
Non-null, non-owning pointer to a log receiver. If you wish to disable logging, pass a
NullLogReceiver. By default it will use ILogReceiver::Default() which will log Info level and above messages to standard error.key
std::string
32-character API key. Found on your API keys page.
gateway
databento::HistoricalGateway
Site of historical gateway to connect to. Currently only
Bo1 is supported. If using HistoricalBuilder, it defaults to Bo1.API method
Historical(std::string key, HistoricalGateway gateway);
Example usage
#include <databento/historical.hpp>
#include <databento/log.hpp>
using namespace databento;
// Pass as an argument
auto client1 =
Historical{ILogReceiver::Default(), "$YOUR_API_KEY",
HistoricalGateway::Bo1};
// Or, pass as `DATABENTO_API_KEY` environment variable
auto client2 = Historical::Builder().SetKeyFromEnv().Build();
Metadata
Historical::MetadataListPublishers
List all publisher ID mappings.
Use this method to list the mappings of publisher names to publisher IDs.
Returns
std::vector<PublisherDetail>
A list of publisher details, where PublisherDetail is:
publisher_id
uint16_t
The publisher ID assigned by Databento.
dataset
std::string
The dataset ID for the publisher.
venue
std::string
The venue for the publisher.
description
std::string
The publisher description.
API method
std::vector<PublisherDetail> MetadataListPublishers();
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto publishers = client.MetadataListPublishers();
for (const auto& publisher : publishers) {
std::cout << publisher << '\n';
}
Example response
PublisherDetail { publisher_id = 1, dataset = "GLBX.MDP3", venue = "GLBX", description = "CME Globex MDP 3.0" }
PublisherDetail { publisher_id = 2, dataset = "XNAS.ITCH", venue = "XNAS", description = "Nasdaq TotalView-ITCH" }
PublisherDetail { publisher_id = 3, dataset = "XBOS.ITCH", venue = "XBOS", description = "Nasdaq BX TotalView-ITCH" }
PublisherDetail { publisher_id = 4, dataset = "XPSX.ITCH", venue = "XPSX", description = "Nasdaq PSX TotalView-ITCH" }
PublisherDetail { publisher_id = 5, dataset = "BATS.PITCH", venue = "BATS", description = "Cboe BZX Depth" }
PublisherDetail { publisher_id = 6, dataset = "BATY.PITCH", venue = "BATY", description = "Cboe BYX Depth" }
PublisherDetail { publisher_id = 7, dataset = "EDGA.PITCH", venue = "EDGA", description = "Cboe EDGA Depth" }
PublisherDetail { publisher_id = 8, dataset = "EDGX.PITCH", venue = "EDGX", description = "Cboe EDGX Depth" }
PublisherDetail { publisher_id = 9, dataset = "XNYS.PILLAR", venue = "XNYS", description = "NYSE Integrated" }
PublisherDetail { publisher_id = 10, dataset = "XCIS.PILLAR", venue = "XCIS", description = "NYSE National Integrated" }
PublisherDetail { publisher_id = 11, dataset = "XASE.PILLAR", venue = "XASE", description = "NYSE American Integrated" }
PublisherDetail { publisher_id = 12, dataset = "XCHI.PILLAR", venue = "XCHI", description = "NYSE Texas Integrated" }
PublisherDetail { publisher_id = 13, dataset = "XCIS.BBO", venue = "XCIS", description = "NYSE National BBO" }
PublisherDetail { publisher_id = 14, dataset = "XCIS.TRADES", venue = "XCIS", description = "NYSE National Trades" }
PublisherDetail { publisher_id = 15, dataset = "MEMX.MEMOIR", venue = "MEMX", description = "MEMX Memoir Depth" }
PublisherDetail { publisher_id = 16, dataset = "EPRL.DOM", venue = "EPRL", description = "MIAX Pearl Depth" }
PublisherDetail { publisher_id = 17, dataset = "XNAS.NLS", venue = "FINN", description = "FINRA/Nasdaq TRF Carteret" }
PublisherDetail { publisher_id = 18, dataset = "XNAS.NLS", venue = "FINC", description = "FINRA/Nasdaq TRF Chicago" }
PublisherDetail { publisher_id = 19, dataset = "XNYS.TRADES", venue = "FINY", description = "FINRA/NYSE TRF" }
PublisherDetail { publisher_id = 20, dataset = "OPRA.PILLAR", venue = "AMXO", description = "OPRA - NYSE American Options" }
PublisherDetail { publisher_id = 21, dataset = "OPRA.PILLAR", venue = "XBOX", description = "OPRA - BOX Options" }
PublisherDetail { publisher_id = 22, dataset = "OPRA.PILLAR", venue = "XCBO", description = "OPRA - Cboe Options" }
PublisherDetail { publisher_id = 23, dataset = "OPRA.PILLAR", venue = "EMLD", description = "OPRA - MIAX Emerald" }
PublisherDetail { publisher_id = 24, dataset = "OPRA.PILLAR", venue = "EDGO", description = "OPRA - Cboe EDGX Options" }
PublisherDetail { publisher_id = 25, dataset = "OPRA.PILLAR", venue = "GMNI", description = "OPRA - Nasdaq GEMX" }
PublisherDetail { publisher_id = 26, dataset = "OPRA.PILLAR", venue = "XISX", description = "OPRA - Nasdaq ISE" }
PublisherDetail { publisher_id = 27, dataset = "OPRA.PILLAR", venue = "MCRY", description = "OPRA - Nasdaq MRX" }
PublisherDetail { publisher_id = 28, dataset = "OPRA.PILLAR", venue = "XMIO", description = "OPRA - MIAX Options" }
PublisherDetail { publisher_id = 29, dataset = "OPRA.PILLAR", venue = "ARCO", description = "OPRA - NYSE Arca Options" }
PublisherDetail { publisher_id = 30, dataset = "OPRA.PILLAR", venue = "OPRA", description = "OPRA - Options Price Reporting Authority" }
PublisherDetail { publisher_id = 31, dataset = "OPRA.PILLAR", venue = "MPRL", description = "OPRA - MIAX Pearl" }
PublisherDetail { publisher_id = 32, dataset = "OPRA.PILLAR", venue = "XNDQ", description = "OPRA - Nasdaq Options" }
PublisherDetail { publisher_id = 33, dataset = "OPRA.PILLAR", venue = "XBXO", description = "OPRA - Nasdaq BX Options" }
PublisherDetail { publisher_id = 34, dataset = "OPRA.PILLAR", venue = "C2OX", description = "OPRA - Cboe C2 Options" }
PublisherDetail { publisher_id = 35, dataset = "OPRA.PILLAR", venue = "XPHL", description = "OPRA - Nasdaq PHLX" }
PublisherDetail { publisher_id = 36, dataset = "OPRA.PILLAR", venue = "BATO", description = "OPRA - Cboe BZX Options" }
PublisherDetail { publisher_id = 37, dataset = "OPRA.PILLAR", venue = "MXOP", description = "OPRA - MEMX Options" }
PublisherDetail { publisher_id = 38, dataset = "IEXG.TOPS", venue = "IEXG", description = "IEX TOPS" }
PublisherDetail { publisher_id = 39, dataset = "DBEQ.BASIC", venue = "XCHI", description = "DBEQ Basic - NYSE Texas" }
PublisherDetail { publisher_id = 40, dataset = "DBEQ.BASIC", venue = "XCIS", description = "DBEQ Basic - NYSE National" }
PublisherDetail { publisher_id = 41, dataset = "DBEQ.BASIC", venue = "IEXG", description = "DBEQ Basic - IEX" }
PublisherDetail { publisher_id = 42, dataset = "DBEQ.BASIC", venue = "EPRL", description = "DBEQ Basic - MIAX Pearl" }
PublisherDetail { publisher_id = 43, dataset = "ARCX.PILLAR", venue = "ARCX", description = "NYSE Arca Integrated" }
PublisherDetail { publisher_id = 44, dataset = "XNYS.BBO", venue = "XNYS", description = "NYSE BBO" }
PublisherDetail { publisher_id = 45, dataset = "XNYS.TRADES", venue = "XNYS", description = "NYSE Trades" }
PublisherDetail { publisher_id = 46, dataset = "XNAS.QBBO", venue = "XNAS", description = "Nasdaq QBBO" }
PublisherDetail { publisher_id = 47, dataset = "XNAS.NLS", venue = "XNAS", description = "Nasdaq Trades" }
PublisherDetail { publisher_id = 48, dataset = "EQUS.PLUS", venue = "XCHI", description = "Databento US Equities Plus - NYSE Texas" }
PublisherDetail { publisher_id = 49, dataset = "EQUS.PLUS", venue = "XCIS", description = "Databento US Equities Plus - NYSE National" }
PublisherDetail { publisher_id = 50, dataset = "EQUS.PLUS", venue = "IEXG", description = "Databento US Equities Plus - IEX" }
PublisherDetail { publisher_id = 51, dataset = "EQUS.PLUS", venue = "EPRL", description = "Databento US Equities Plus - MIAX Pearl" }
PublisherDetail { publisher_id = 52, dataset = "EQUS.PLUS", venue = "XNAS", description = "Databento US Equities Plus - Nasdaq" }
PublisherDetail { publisher_id = 53, dataset = "EQUS.PLUS", venue = "XNYS", description = "Databento US Equities Plus - NYSE" }
PublisherDetail { publisher_id = 54, dataset = "EQUS.PLUS", venue = "FINN", description = "Databento US Equities Plus - FINRA/Nasdaq TRF Carteret" }
PublisherDetail { publisher_id = 55, dataset = "EQUS.PLUS", venue = "FINY", description = "Databento US Equities Plus - FINRA/NYSE TRF" }
PublisherDetail { publisher_id = 56, dataset = "EQUS.PLUS", venue = "FINC", description = "Databento US Equities Plus - FINRA/Nasdaq TRF Chicago" }
PublisherDetail { publisher_id = 57, dataset = "IFEU.IMPACT", venue = "IFEU", description = "ICE Europe Commodities" }
PublisherDetail { publisher_id = 58, dataset = "NDEX.IMPACT", venue = "NDEX", description = "ICE Endex" }
PublisherDetail { publisher_id = 59, dataset = "DBEQ.BASIC", venue = "DBEQ", description = "Databento US Equities Basic - Consolidated" }
PublisherDetail { publisher_id = 60, dataset = "EQUS.PLUS", venue = "EQUS", description = "EQUS Plus - Consolidated" }
PublisherDetail { publisher_id = 61, dataset = "OPRA.PILLAR", venue = "SPHR", description = "OPRA - MIAX Sapphire" }
PublisherDetail { publisher_id = 62, dataset = "EQUS.ALL", venue = "XCHI", description = "Databento US Equities (All Feeds) - NYSE Texas" }
PublisherDetail { publisher_id = 63, dataset = "EQUS.ALL", venue = "XCIS", description = "Databento US Equities (All Feeds) - NYSE National" }
PublisherDetail { publisher_id = 64, dataset = "EQUS.ALL", venue = "IEXG", description = "Databento US Equities (All Feeds) - IEX" }
PublisherDetail { publisher_id = 65, dataset = "EQUS.ALL", venue = "EPRL", description = "Databento US Equities (All Feeds) - MIAX Pearl" }
PublisherDetail { publisher_id = 66, dataset = "EQUS.ALL", venue = "XNAS", description = "Databento US Equities (All Feeds) - Nasdaq" }
PublisherDetail { publisher_id = 67, dataset = "EQUS.ALL", venue = "XNYS", description = "Databento US Equities (All Feeds) - NYSE" }
PublisherDetail { publisher_id = 68, dataset = "EQUS.ALL", venue = "FINN", description = "Databento US Equities (All Feeds) - FINRA/Nasdaq TRF Carteret" }
PublisherDetail { publisher_id = 69, dataset = "EQUS.ALL", venue = "FINY", description = "Databento US Equities (All Feeds) - FINRA/NYSE TRF" }
PublisherDetail { publisher_id = 70, dataset = "EQUS.ALL", venue = "FINC", description = "Databento US Equities (All Feeds) - FINRA/Nasdaq TRF Chicago" }
PublisherDetail { publisher_id = 71, dataset = "EQUS.ALL", venue = "BATS", description = "Databento US Equities (All Feeds) - Cboe BZX" }
PublisherDetail { publisher_id = 72, dataset = "EQUS.ALL", venue = "BATY", description = "Databento US Equities (All Feeds) - Cboe BYX" }
PublisherDetail { publisher_id = 73, dataset = "EQUS.ALL", venue = "EDGA", description = "Databento US Equities (All Feeds) - Cboe EDGA" }
PublisherDetail { publisher_id = 74, dataset = "EQUS.ALL", venue = "EDGX", description = "Databento US Equities (All Feeds) - Cboe EDGX" }
PublisherDetail { publisher_id = 75, dataset = "EQUS.ALL", venue = "XBOS", description = "Databento US Equities (All Feeds) - Nasdaq BX" }
PublisherDetail { publisher_id = 76, dataset = "EQUS.ALL", venue = "XPSX", description = "Databento US Equities (All Feeds) - Nasdaq PSX" }
PublisherDetail { publisher_id = 77, dataset = "EQUS.ALL", venue = "MEMX", description = "Databento US Equities (All Feeds) - MEMX" }
PublisherDetail { publisher_id = 78, dataset = "EQUS.ALL", venue = "XASE", description = "Databento US Equities (All Feeds) - NYSE American" }
PublisherDetail { publisher_id = 79, dataset = "EQUS.ALL", venue = "ARCX", description = "Databento US Equities (All Feeds) - NYSE Arca" }
PublisherDetail { publisher_id = 80, dataset = "EQUS.ALL", venue = "LTSE", description = "Databento US Equities (All Feeds) - Long-Term Stock Exchange" }
PublisherDetail { publisher_id = 81, dataset = "XNAS.BASIC", venue = "XNAS", description = "Nasdaq Basic - Nasdaq" }
PublisherDetail { publisher_id = 82, dataset = "XNAS.BASIC", venue = "FINN", description = "Nasdaq Basic - FINRA/Nasdaq TRF Carteret" }
PublisherDetail { publisher_id = 83, dataset = "XNAS.BASIC", venue = "FINC", description = "Nasdaq Basic - FINRA/Nasdaq TRF Chicago" }
PublisherDetail { publisher_id = 84, dataset = "IFEU.IMPACT", venue = "XOFF", description = "ICE Europe - Off-Market Trades" }
PublisherDetail { publisher_id = 85, dataset = "NDEX.IMPACT", venue = "XOFF", description = "ICE Endex - Off-Market Trades" }
PublisherDetail { publisher_id = 86, dataset = "XNAS.NLS", venue = "XBOS", description = "Nasdaq NLS - Nasdaq BX" }
PublisherDetail { publisher_id = 87, dataset = "XNAS.NLS", venue = "XPSX", description = "Nasdaq NLS - Nasdaq PSX" }
PublisherDetail { publisher_id = 88, dataset = "XNAS.BASIC", venue = "XBOS", description = "Nasdaq Basic - Nasdaq BX" }
PublisherDetail { publisher_id = 89, dataset = "XNAS.BASIC", venue = "XPSX", description = "Nasdaq Basic - Nasdaq PSX" }
PublisherDetail { publisher_id = 90, dataset = "EQUS.SUMMARY", venue = "EQUS", description = "Databento Equities Summary" }
PublisherDetail { publisher_id = 91, dataset = "XCIS.TRADESBBO", venue = "XCIS", description = "NYSE National Trades and BBO" }
PublisherDetail { publisher_id = 92, dataset = "XNYS.TRADESBBO", venue = "XNYS", description = "NYSE Trades and BBO" }
PublisherDetail { publisher_id = 93, dataset = "XNAS.BASIC", venue = "EQUS", description = "Nasdaq Basic - Consolidated" }
PublisherDetail { publisher_id = 94, dataset = "EQUS.ALL", venue = "EQUS", description = "Databento US Equities (All Feeds) - Consolidated" }
PublisherDetail { publisher_id = 95, dataset = "EQUS.MINI", venue = "EQUS", description = "Databento US Equities Mini" }
PublisherDetail { publisher_id = 96, dataset = "XNYS.TRADES", venue = "EQUS", description = "NYSE Trades - Consolidated" }
PublisherDetail { publisher_id = 97, dataset = "IFUS.IMPACT", venue = "IFUS", description = "ICE Futures US" }
PublisherDetail { publisher_id = 98, dataset = "IFUS.IMPACT", venue = "XOFF", description = "ICE Futures US - Off-Market Trades" }
PublisherDetail { publisher_id = 99, dataset = "IFLL.IMPACT", venue = "IFLL", description = "ICE Europe Financials" }
PublisherDetail { publisher_id = 100, dataset = "IFLL.IMPACT", venue = "XOFF", description = "ICE Europe Financials - Off-Market Trades" }
PublisherDetail { publisher_id = 101, dataset = "XEUR.EOBI", venue = "XEUR", description = "Eurex EOBI" }
PublisherDetail { publisher_id = 102, dataset = "XEEE.EOBI", venue = "XEEE", description = "European Energy Exchange EOBI" }
PublisherDetail { publisher_id = 103, dataset = "XEUR.EOBI", venue = "XOFF", description = "Eurex EOBI - Off-Market Trades" }
PublisherDetail { publisher_id = 104, dataset = "XEEE.EOBI", venue = "XOFF", description = "European Energy Exchange EOBI - Off-Market Trades" }
Historical::MetadataListDatasets
List all valid dataset IDs on Databento.
Use this method to list the available dataset IDs (string identifiers), so you can use
other methods which take the dataset parameter.
Constants for dataset IDs are also available in <databento/constants.hpp>.
Parameters
date_range
databento::DateRange
The request range with an inclusive UTC start date and an optional exclusive UTC end date. Takes YYYY-MM-DD ISO 8601 date strings. If end is not specified, will return through the last date available.
Returns
std::vector<std::string>
A list of dataset IDs.
API method
std::vector<std::string> MetadataListDatasets();
std::vector<std::string> MetadataListDatasets(
const DateRange& date_range);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto datasets = client.MetadataListDatasets();
for (const auto& dataset : datasets) {
std::cout << dataset << '\n';
}
Example response
ARCX.PILLAR
BATS.PITCH
BATY.PITCH
DBEQ.BASIC
EDGA.PITCH
EDGX.PITCH
EPRL.DOM
EQUS.MINI
EQUS.SUMMARY
GLBX.MDP3
IEXG.TOPS
IFEU.IMPACT
MEMX.MEMOIR
NDEX.IMPACT
OPRA.PILLAR
XASE.PILLAR
XBOS.ITCH
XCHI.PILLAR
XCIS.TRADESBBO
XNAS.BASIC
XNAS.ITCH
XNYS.PILLAR
XPSX.ITCH
Historical::MetadataListSchemas
List all available schemas for a dataset.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.Returns
std::vector<databento::Schema>
A list of available data schemas.
API method
std::vector<Schema> MetadataListSchemas(
const std::string& dataset);
Example usage
#include <databento/constants.hpp> // kGlbxMdp3
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto schemas = client.MetadataListSchemas(dataset::kGlbxMdp3);
for (const auto& schema : schemas) {
std::cout << schema << '\n';
}
Example response
mbo
mbp-1
mbp-10
tbbo
trades
bbo-1s
bbo-1m
ohlcv-1s
ohlcv-1m
ohlcv-1h
ohlcv-1d
definition
statistics
status
Historical::MetadataListFields
List all fields for a particular schema and encoding.
Parameters
schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
encoding
databento::Encoding
The data encoding.
Encoding::Dbn is recommended.Returns
std::vector<FieldDetail>
A list of field details objects, where FieldDetail is:
name
std::string
The name of the field.
type
std::string
The type of the field.
API method
std::vector<FieldDetail> MetadataListFields(Encoding encoding,
Schema schema);
Example usage
#include <databento/constants.hpp> // kGlbxMdp3
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto fields =
client.MetadataListFields(Encoding::Dbn, Schema::Trades);
for (const auto& field_detail : fields) {
std::cout << field_detail << '\n';
}
Example response
FieldDetail { name = "length", type = "uint8_t" }
FieldDetail { name = "rtype", type = "uint8_t" }
FieldDetail { name = "publisher_id", type = "uint16_t" }
FieldDetail { name = "instrument_id", type = "uint32_t" }
FieldDetail { name = "ts_event", type = "uint64_t" }
FieldDetail { name = "price", type = "int64_t" }
FieldDetail { name = "size", type = "uint32_t" }
FieldDetail { name = "action", type = "char" }
FieldDetail { name = "side", type = "char" }
FieldDetail { name = "flags", type = "uint8_t" }
FieldDetail { name = "depth", type = "uint8_t" }
FieldDetail { name = "ts_recv", type = "uint64_t" }
FieldDetail { name = "ts_in_delta", type = "int32_t" }
FieldDetail { name = "sequence", type = "uint32_t" }
Historical::MetadataListUnitPrices
List unit prices for each data schema in US dollars per gigabyte.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.Returns
std::vector<UnitPricesForMode>
A list of objects with the unit prices for a feed mode, where UnitPricesForMode is:
mode
databento::FeedMode
The feed mode.
unit_prices
std::map<Schema, double>
The map of unit prices in US dollars by schema.
API method
std::vector<UnitPricesForMode> MetadataListUnitPrices(
const std::string& dataset);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
std::vector<UnitPricesForMode> unit_prices =
client.MetadataListUnitPrices("OPRA.PILLAR");
for (const auto& unit_prices_for_mode : unit_prices) {
std::cout << "- " << unit_prices_for_mode.mode << '\n';
for (const auto& schema_and_price :
unit_prices_for_mode.unit_prices) {
std::cout << " " << schema_and_price.first << ": "
<< schema_and_price.second << '\n';
}
}
Example response
- historical
trades: 280
ohlcv-1s: 280
ohlcv-1m: 280
ohlcv-1h: 600
ohlcv-1d: 600
definition: 5
statistics: 11
status: 5
cmbp-1: 0.16
cbbo-1s: 2
cbbo-1m: 2
tcbbo: 210
- historical-streaming
trades: 280
ohlcv-1s: 280
ohlcv-1m: 280
ohlcv-1h: 600
ohlcv-1d: 600
definition: 5
statistics: 11
status: 5
cmbp-1: 0.16
cbbo-1s: 2
cbbo-1m: 2
tcbbo: 210
- live
trades: 336
ohlcv-1s: 336
ohlcv-1m: 336
ohlcv-1h: 720
ohlcv-1d: 720
definition: 6
statistics: 13.2
status: 6
cmbp-1: 0.2
cbbo-1s: 2.4
cbbo-1m: 2.4
tcbbo: 252
Historical::MetadataGetDatasetCondition
Get the dataset condition from Databento.
Use this method to discover data availability and quality.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.date_range
databento::DateRange
The request range with an inclusive UTC start date and an optional inclusive UTC end date. Takes YYYY-MM-DD ISO 8601 date strings. If end is not specified, will return through the last date available.
Returns
std::vector<DatasetConditionDetail>
A list of conditions per date, where DatasetConditionDetail is:
date
std::string
The day of the described data, as an ISO 8601 date string.
condition
DatasetCondition
The condition code describing the quality and availability of the data on the given day. Possible values are listed below.
last_modified_date
std::optional<std::string>
The date when any schema in the dataset on the given day was last generated or modified as an ISO 8601 date string. Will be null when
condition is Missing.Possible values for condition:
available: the data is available with no known issuesdegraded: the data is available, but there may be missing data or other correctness issuespending: the data is not yet available, but may be available soonmissing: the data is not available
API method
std::vector<DatasetConditionDetail> MetadataGetDatasetCondition(
const std::string& dataset);
std::vector<DatasetConditionDetail> MetadataGetDatasetCondition(
const std::string& dataset, const DateRange& date_range);
Example usage
#include <databento/constants.hpp> // kGlbxMdp3
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto conditions = client.MetadataGetDatasetCondition(
dataset::kGlbxMdp3, {"2019-06-06", "2019-06-10"});
for (const DatasetConditionDetail& condition : conditions) {
std::cout << condition << '\n';
}
Example response
DatasetConditionDetail { date = "2019-06-06", condition = available, last_modified_date = "2024-05-13" }
DatasetConditionDetail { date = "2019-06-07", condition = available, last_modified_date = "2024-05-13" }
DatasetConditionDetail { date = "2019-06-09", condition = available, last_modified_date = "2024-05-13" }
DatasetConditionDetail { date = "2019-06-10", condition = available, last_modified_date = "2024-05-13" }
Historical.MetadataGetDatasetRange
Get the available range for the dataset given the user's entitlements.
Use this method to discover data availability.
The start and end values in the response can be used with TimeseriesGetRange and BatchSubmitJob endpoints.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets.
Returns
DatasetRange
The available range for the dataset.
start
std::string
The ISO 8601 start timestamp of the available range.
end
std::string
The ISO 8601 end timestamp of the available range (exclusive).
range_by_schema
std::unordered_map<Schema, DateTimeRange<std::string>>
A mapping of schema names to per-schema start and end timestamps.
API method
DatasetRange MetadataGetDatasetRange(const std::string& dataset);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto available_range =
client.MetadataGetDatasetRange("XNAS.ITCH");
std::cout << available_range << '\n';
Example response
DatasetRange {
start = "2018-05-01T00:00:00.000000000Z",
end = "2025-06-25T14:40:00.000000000Z",
range_by_schema = {
mbo: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
mbp-1: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
mbp-10: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
tbbo: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
trades: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
ohlcv-1s: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
ohlcv-1m: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
ohlcv-1h: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:00:00.000000000Z" },
ohlcv-1d: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T00:00:00.000000000Z" },
definition: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
statistics: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
status: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
imbalance: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
bbo-1s: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" },
bbo-1m: DateTimeRange { start = "2018-05-01T00:00:00.000000000Z", end = "2025-06-25T14:40:00.000000000Z" }
}
}
Historical::MetadataGetRecordCount
Get the record count of the time series data query.
This method may not be accurate for time ranges that are not discrete multiples of 10 minutes, potentially over-reporting the number of records in such cases. The definition schema is only accurate for discrete multiples of 24 hours.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.
symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
stype_in
databento::SType
limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit.Returns
std::uint64_t
The number of records.
API method
std::uint64_t MetadataGetRecordCount(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
std::uint64_t MetadataGetRecordCount(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
std::uint64_t MetadataGetRecordCount(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
std::uint64_t MetadataGetRecordCount(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto record_count = client.MetadataGetRecordCount(
"GLBX.MDP3", {"2022-01-06T12:00", "2022-03-10"}, {"ESM2"},
Schema::Mbo);
std::cout << record_count << '\n';
Example response
86913546
Historical::MetadataGetBillableSize
Get the billable uncompressed raw binary size for historical streaming or batched files.
This method may not be accurate for time ranges that are not discrete multiples of 10 minutes, potentially over-reporting the size in such cases. The definition schema is only accurate for discrete multiples of 24 hours.
Info
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.
symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
stype_in
databento::SType
limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit.Returns
std::uint64_t
The size in number of bytes used for billing.
API method
std::uint64_t MetadataGetBillableSize(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
std::uint64_t MetadataGetBillableSize(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
std::uint64_t MetadataGetBillableSize(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
std::uint64_t MetadataGetBillableSize(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto billable_size = client.MetadataGetBillableSize(
"GLBX.MDP3", {"2022-06-06T00:00", "2022-06-10T12:10"},
{"ESM2"}, Schema::Trades);
std::cout << billable_size << '\n';
Example response
99219648
Historical::MetadataGetCost
Get the cost in US dollars for a historical streaming or batch download request. This cost respects any discounts provided by flat rate plans.
This method may not be accurate for time ranges that are not discrete multiples of 10 minutes, potentially over-reporting the cost in such cases. The definition schema is only accurate for discrete multiples of 24 hours.
InfoThe amount billed will be based on the actual amount of bytes sent; see our pricing documentation for more details.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.
symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
stype_in
databento::SType
limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit.Returns
double
The cost in US dollars.
API method
double MetadataGetCost(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
double MetadataGetCost(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema);
double MetadataGetCost(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
double MetadataGetCost(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, std::uint64_t limit);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto cost = client.MetadataGetCost(
"GLBX.MDP3", {"2022-06-06T00:00", "2022-06-10T12:10"},
{"ESM2"}, Schema::Trades);
std::cout << cost << '\n';
Example response
2.58735
Time series
Historical::TimeseriesGetRange
Make a streaming request for time series data from Databento.
Primary method for getting historical intraday and daily market data, instrument definitions, and market status data directly into your application.
This method calls the callback as it streams data, it will only return once it has finished decoding all data. For large requests, consider using BatchSubmitJob instead. Consider using TimeseriesGetRangeToFile to save the data directly to a file.
Spawns another thread, however, the callbacks are called from the thread that made the method call.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Filters on
ts_recv if it exists in the schema, otherwise ts_event. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
stype_in
databento::SType
stype_out
databento::SType
The symbology type of output symbols. Defaults to
InstrumentId.limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit.metadata_callback
std::function<void(Metadata&&)>
A callback for receiving the
Metadata associated with the result of the query. This callback will be called only once, immediately before the first time record_callback is called.record_callback
std::function<KeepGoing(const Record&)>
A callback for receiving the
Records from the query. The particular record type will depend on the schema. The callback will be called as data is received. Return KeepGoing::Continue to continue receiving records or KeepGoing::Stop to stop streaming.Returns
Returns data via the callbacks metadata_callback and record_callback.
WarningThe reference passed to
record_callbackis only valid during the callback. Copy the record or particular fields to persist data between calls to the callback.
A full list of fields returned through the record_callback for each schema is available through
Historical.MetadataListFields
or the struct definitions in <databento/record.hpp>.
Won't return until all records for the query have been passed to record_callback or the callback
returned KeepGoing::Stop.
API method
void TimeseriesGetRange(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
const RecordCallback& record_callback);
void TimeseriesGetRange(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
const RecordCallback& record_callback);
void TimeseriesGetRange(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, SType stype_out, std::uint64_t limit,
const MetadataCallback& metadata_callback,
const RecordCallback& record_callback);
void TimeseriesGetRange(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, SType stype_out, std::uint64_t limit,
const MetadataCallback& metadata_callback,
const RecordCallback& record_callback);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto print_trades = [](const Record record) {
// Convert the generic `Record` to a `TradeMsg`, the record
// type associated with `Schema::Trades`
const auto& trade_msg = record.Get<TradeMsg>();
std::cout << trade_msg << '\n';
return KeepGoing::Continue;
};
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
client.TimeseriesGetRange(
"GLBX.MDP3", {"2022-06-06T00:00", "2022-06-10T00:10"},
{"ESM2"}, Schema::Trades, SType::RawSymbol,
SType::InstrumentId, 1, {}, print_trades);
Example response
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 1, instrument_id = 3403, ts_event = 2022-06-06T00:00:00.070033767Z },
price = 4108.500000000,
size = 1,
action = Trade,
side = Ask,
flags = 0,
depth = 0,
ts_recv = 2022-06-06T00:00:00.070314216Z,
ts_in_delta = 18681,
sequence = 157862
}
Historical::TimeseriesGetRangeToFile
Make a streaming request for time series data from Databento.
Primary method for getting historical intraday and daily market data, instrument definitions, and market status data directly into your application.
Data is streamed directly to the file at file_path.
If the file at file_path already exists, it will be overwritten.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Filters on
ts_recv if it exists in the schema, otherwise ts_event. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
stype_in
databento::SType
stype_out
databento::SType
The symbology type of output symbols. Defaults to
InstrumentId.limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit.file_path
std::filesystem::path
The path of the file where the data should be saved. If the file already exists, it will be overwritten. It is recommended to use the ".dbn.zst" suffix.
Returns
A DbnFileStore object.
A full list of fields returned through the record_callback for each schema is available through
Historical.MetadataListFields
or the struct definitions in <databento/record.hpp>.
API method
DbnFileStore TimeseriesGetRangeToFile(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
const std::filesystem::path& file_path);
DbnFileStore TimeseriesGetRangeToFile(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
const std::filesystem::path& file_path);
DbnFileStore TimeseriesGetRangeToFile(
const std::string& dataset,
const DateTimeRange<UnixNanos>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, SType stype_out, std::uint64_t limit,
const std::filesystem::path& file_path);
DbnFileStore TimeseriesGetRangeToFile(
const std::string& dataset,
const DateTimeRange<std::string>& datetime_range,
const std::vector<std::string>& symbols, Schema schema,
SType stype_in, SType stype_out, std::uint64_t limit,
const std::filesystem::path& file_path);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto file_bento = client.TimeseriesGetRangeToFile(
"GLBX.MDP3", {"2022-06-06T00:00", "2022-06-10T00:10"},
{"ESM2"}, Schema::Trades, SType::RawSymbol,
SType::InstrumentId, 1, "ESM2_20220606-20220610.dbn.zst");
file_bento.Replay([](const Record& record) {
// Convert the generic `Record` to a `TradeMsg`, the
// record type associated with `Schema::Trades`
const auto& trade_msg = record.Get<TradeMsg>();
std::cout << trade_msg << '\n';
return KeepGoing::Continue;
});
Example response
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 1, instrument_id = 3403, ts_event = 2022-06-06T00:00:00.070033767Z },
price = 4108.500000000,
size = 1,
action = Trade,
side = Ask,
flags = 0,
depth = 0,
ts_recv = 2022-06-06T00:00:00.070314216Z,
ts_in_delta = 18681,
sequence = 157862
}
Symbology
Historical::SymbologyResolve
Resolve a list of symbols from an input symbology type, to an output symbology type.
Take, for example, a raw symbol to an instrument ID: ESM2 ā 3403.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.date_range
databento::DateRange
The request range with an inclusive UTC start date and an optional exclusive UTC end date. Takes YYYY-MM-DD ISO 8601 date strings. If not specified, end defaults to the forward filled value of start based on the resolution provided.
symbols
std::vector<std::string>
The symbols to resolve. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols (not available for every dataset).stype_in
databento::SType
The symbology type of input
symbols.stype_out
databento::SType
The symbology type of output symbols.
Returns
SymbologyResolution, which is defined in <databento/symbology.hpp>
The results for the symbology resolution.
Can be converted to a TsSymbolMap via the CreateSymbolMap() method.
API method
SymbologyResolution SymbologyResolve(
const std::string& dataset,
const std::vector<std::string>& symbols, SType stype_in,
SType stype_out, const DateRange& date_range);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto resolution = client.SymbologyResolve(
"GLBX.MDP3", {"ESM2"}, SType::RawSymbol, SType::InstrumentId,
{"2022-06-01", "2022-06-30"});
std::cout << resolution << '\n';
Example response
SymbologyResolution {
mappings = {
{ "ESM2", { MappingInterval { start_date = 2022-06-01, end_date = 2022-06-26, symbol = "3403" } } }
},
partial = {},
not_found = {},
stype_in = raw_symbol,
stype_out = instrument_id
}
Batch downloads
Batch downloads allow you to download flat files directly from within your portal. For more information, see Streaming vs. batch download.
Historical::BatchSubmitJob
Make a batch download job request for flat files.
Once a request is submitted, our system processes the request and prepares the batch files in the background. The status of your request and the files can be accessed from the Download center from your user portal.
This method takes longer than a streaming request, but is advantageous for larger requests as it supports delivery mechanisms that allow multiple accesses of the data without additional cost for each subsequent download after the first.
Related: BatchListJobs.
Parameters
dataset
std::string
The dataset code (string identifier). Must be one of the values from MetadataListDatasets. Constants are provided in the
databento::dataset namespace in <databento/constants.hpp>.symbols
std::vector<std::string>
The product symbols to filter for. Takes up to 2,000 symbols per request. Use
databento::kAllSymbols from <databento/symbology.hpp> to request all symbols.schema
databento::Schema
The data record schema. Must be one of the values from MetadataListSchemas.
datetime_range
databento::DateTimeRange<UnixNanos> or databento::DateTimeRange<std::string>
The request range with an inclusive start and an optional exclusive end. Filters on
ts_recv if it exists in the schema, otherwise ts_event. Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. If not specified, end defaults to the forward filled value of start based on the resolution provided.compression
databento::Compression
The data compression mode. Defaults to Zstd for fastest transfer speed.
pretty_px
bool
If prices should be formatted to the correct scale (using the fixed-precision scalar 1e-9). Only applicable for
Csv or Json encodings. Defaults to false.pretty_ts
bool
If timestamps should be formatted as ISO 8601 strings. Only applicable for
Csv or Json encodings. Defaults to false.map_symbols
bool
If a symbol field should be included with each text-encoded record. Only applicable for
Csv or Json encodings. Defaults to false.split_symbols
bool
If files should be split by raw symbol. Defaults to
false. Cannot be used with limit.split_duration
databento::SplitDuration
The maximum time duration before batched data is split into multiple files. A week starts on Sunday UTC. Defaults to
Day.split_size
std::uint64_t
The maximum size (in bytes) of each batched data file before being split. Must be an integer between 1e9 and 10e9 inclusive (1GB - 10GB). Defaults to no split size.
delivery
databento::Delivery
The delivery mechanism for the batched data files once processed. Only
Download is supported at this time.stype_in
databento::SType
stype_out
databento::SType
The symbology type of output symbols. Defaults to
InstrumentId.limit
std::uint64_t
The maximum number of records to return. Use
0 or {} for no limit. Defaults to no limit. Cannot be used with split_symbols.Returns
BatchJob, which is defined in <databento/batch.hpp>.
The description of the submitted batch job.
id
std::string
The unique job ID for the request.
user_id
std::string
The user ID of the user who made the request.
cost_usd
double
The cost of the job in US dollars.
dataset
std::string
The dataset code (string identifier).
symbols
std::vector<std::string>
The list of symbols specified in the request.
stype_in
SType
The symbology type of input
symbols.stype_out
SType
The symbology type of output symbols.
schema
Schema
The data record schema.
start
std::string
The ISO 8601 timestamp start of request time range (inclusive).
end
std::string
The ISO 8601 timestamp end of request time range (exclusive).
limit
std::uint64_t
The maximum number of records to return or 0 if there's no limit.
encoding
Encoding
The data encoding.
compression
Compression
The data compression mode.
pretty_px
bool
If prices are formatted to the correct scale (using the fixed-precision scalar 1e-9).
pretty_ts
bool
If timestamps are formatted as ISO 8601 strings.
map_symbols
bool
If a symbol field is included with each text-encoded record.
split_symbols
bool
If files are split by raw symbol.
split_duration
SplitDuration
The maximum time interval for an individual file before splitting into multiple files.
split_size
std::uint64_t
The maximum size for an individual file before splitting into multiple files.
delivery
Delivery
The delivery mechanism of the batch data. Only
Download is supported at this time.record_count
std::uint64_t
The number of data records.
billed_size
std::uint64_t
The size of the raw binary data used to process the batch job (used for billing purposes).
actual_size
std::uint64_t
The total size of the result of the batch job after splitting and compression.
package_size
std::uint64_t
The total size of the result of the batch job after any packaging (including metadata).
state
JobState
The current status of the batch job.
ts_received
std::string
The ISO 8601 timestamp when Databento received the batch job.
ts_queued
std::string
The ISO 8601 timestamp when the batch job was queued.
ts_process_start
std::string
The ISO 8601 timestamp when the batch job began processing (if it's begun).
ts_process_done
std::string
The ISO 8601 timestamp when the batch job finished processing (if it's finished).
ts_expiration
std::string
The ISO 8601 timestamp when the batch job will expire from the Download center.
API method
BatchJob BatchSubmitJob(
const std::string& dataset,
const std::vector<std::string>& symbols, Schema schema,
const DateTimeRange<UnixNanos>& datetime_range);
BatchJob BatchSubmitJob(
const std::string& dataset,
const std::vector<std::string>& symbols, Schema schema,
const DateTimeRange<std::string>& datetime_range);
BatchJob BatchSubmitJob(
const std::string& dataset,
const std::vector<std::string>& symbols, Schema schema,
const DateTimeRange<UnixNanos>& datetime_range,
Encoding encoding, Compression compression, bool pretty_px,
bool pretty_ts, bool map_symbols, bool split_symbols,
SplitDuration split_duration, std::uint64_t split_size,
Delivery delivery, SType stype_in, SType stype_out,
std::uint64_t limit);
BatchJob BatchSubmitJob(
const std::string& dataset,
const std::vector<std::string>& symbols, Schema schema,
const DateTimeRange<std::string>& datetime_range,
Encoding encoding, Compression compression, bool pretty_px,
bool pretty_ts, bool map_symbols, bool split_symbols,
SplitDuration split_duration, std::uint64_t split_size,
Delivery delivery, SType stype_in, SType stype_out,
std::uint64_t limit);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto job = client.BatchSubmitJob(
"GLBX.MDP3", {"ESM2"}, Schema::Trades,
{"2022-06-06T12:00", "2022-06-10T00:00:00"});
std::cout << job << '\n';
Example response
BatchJob {
id = "GLBX-20240531-RUF8WUUAUT",
user_id = "NBPDLF33",
cost_usd = 0,
dataset = "GLBX.MDP3",
symbols = { "ESM2" },
stype_in = raw_symbol,
stype_out = instrument_id,
schema = trades,
start = "2022-06-06T12:00:00.000000000Z",
end = "2022-06-10T00:00:00.000000000Z",
limit = 0,
encoding = dbn,
compression = zstd,
pretty_px = false,
pretty_ts = false,
map_symbols = false,
split_duration = day,
split_size = 0,
split_symbols = false,
delivery = download,
record_count = 0,
billed_size = 0,
actual_size = 0,
package_size = 0,
state = queued,
ts_received = "2024-05-31T19:57:07.131684000Z",
ts_queued = "",
ts_process_start = "",
ts_process_done = "",
ts_expiration = ""
}
Historical::BatchListJobs
List batch job details for the user account.
The job details will be sorted in order of ts_received.
Related: Download center.
Parameters
states
std::vector<JobState>
The filter for job states as a list of comma separated values. Can include
Queued, Processing, Done, and Expired. Defaults to all except Expired.since
UnixNanos or std::string
The filter for timestamp submitted (will not include jobs prior to this). Takes an ISO 8601 string or a UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified.
Returns
std::vector<databento::BatchJob>
A list of batch job details.
API method
std::vector<BatchJob> BatchListJobs();
std::vector<BatchJob> BatchListJobs(
const std::vector<JobState>& states, UnixNanos since);
std::vector<BatchJob> BatchListJobs(
const std::vector<JobState>& states, const std::string& since);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto jobs = client.BatchListJobs(
{JobState::Queued, JobState::Processing, JobState::Done},
"2022-06-01");
for (const BatchJob& job : jobs) {
std::cout << job << '\n';
}
Example response
BatchJob {
id = "GLBX-20240509-T473UFYPSK",
user_id = "NBPDLF33",
cost_usd = 0.015481,
dataset = "GLBX.MDP3",
symbols = { "OZN.OPT" },
stype_in = parent,
stype_out = instrument_id,
schema = trades,
start = "2022-12-12T00:00:00.000000000Z",
end = "2022-12-17T00:00:00.000000000Z",
limit = 0,
encoding = dbn,
compression = zstd,
pretty_px = false,
pretty_ts = false,
map_symbols = false,
split_duration = day,
split_size = 0,
split_symbols = false,
delivery = download,
record_count = 12368,
billed_size = 593664,
actual_size = 458219,
package_size = 1019169,
state = done,
ts_received = "2024-05-09T00:20:45.473577000Z",
ts_queued = "2024-05-09T00:20:47.037110000Z",
ts_process_start = "2024-05-09T00:20:55.593991000Z",
ts_process_done = "2024-05-09T00:21:25.354614000Z",
ts_expiration = "2024-06-08T00:21:25.354614000Z"
}
...
Historical::BatchListFiles
List files for a batch job.
This will include all data files and support files.
Related: Download center.
Parameters
job_id
std::string
The batch job identifier.
Returns
std::vector<BatchFileDesc>, where BatchFileDesc is defined in <databento/batch.hpp>.
The file details for the batch job.
filename
std::string
The file name.
size
std::uint64_t
The size of the file in bytes.
hash
std::string
The SHA256 hash of the file.
https_url
std::string
The URL for the HTTPS download.
ftp_url
std::string
The URL for the FTP download.
API method
std::vector<BatchFileDesc> BatchListFiles(
const std::string& job_id);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
auto files = client.BatchListFiles("XNAS-20250108-VVS57U5PD8");
for (const BatchFileDesc& file : files) {
std::cout << file << '\n';
}
Example response
BatchFileDesc {
filename = "manifest.json",
size = 1889,
hash = "sha256:9f43e431be88c403e73ce244bb2e94b293c9c05ac74d93a00443311fc0c9ef09",
https_url = "https://api.databento.com/v0/batch/download/NBPDLF33/XNAS-20250108-VVS57U5PD8/manifest.json",
ftp_url = "ftp://ftp.databento.com/NBPDLF33/XNAS-20250108-VVS57U5PD8/manifest.json"
}
BatchFileDesc {
filename = "condition.json",
size = 122,
hash = "sha256:43dba9f90ba29334f233de4f76541f3d78f5378ed4baff0227673a680fde95d7",
https_url = "https://api.databento.com/v0/batch/download/NBPDLF33/XNAS-20250108-VVS57U5PD8/condition.json",
ftp_url = "ftp://ftp.databento.com/NBPDLF33/XNAS-20250108-VVS57U5PD8/condition.json"
}
BatchFileDesc {
filename = "metadata.json",
size = 699,
hash = "sha256:001a0e2b8e285875b8f1ac1aa5fa3dfadc39b4b92f664b12d99734c6a1bae148",
https_url = "https://api.databento.com/v0/batch/download/NBPDLF33/XNAS-20250108-VVS57U5PD8/metadata.json",
ftp_url = "ftp://ftp.databento.com/NBPDLF33/XNAS-20250108-VVS57U5PD8/metadata.json"
}
BatchFileDesc {
filename = "symbology.json",
size = 1753497,
hash = "sha256:3f3908205ec8b24def7cb3589d9f1be523ec64b4ac63be1c706d8dfca2051d78",
https_url = "https://api.databento.com/v0/batch/download/NBPDLF33/XNAS-20250108-VVS57U5PD8/symbology.json",
ftp_url = "ftp://ftp.databento.com/NBPDLF33/XNAS-20250108-VVS57U5PD8/symbology.json"
}
BatchFileDesc {
filename = "xnas-itch-20250106.imbalance.dbn.zst",
size = 88237480,
hash = "sha256:7d0945aa1f04dad3e263237dcbdb7529ce7f0f41d902e468bb99d686754ce599",
https_url = "https://api.databento.com/v0/batch/download/NBPDLF33/XNAS-20250108-VVS57U5PD8/xnas-itch-20250106.imbalance.dbn.zst",
ftp_url = "ftp://ftp.databento.com/NBPDLF33/XNAS-20250108-VVS57U5PD8/xnas-itch-20250106.imbalance.dbn.zst"
}
Historical::BatchDownload
Download a batch job or a specific file to {output_dir}/{job_id}/.
Will automatically generate output_dir and job_id directories if they do not already exist.
Related: Download center.
Parameters
output_dir
std::filesystem::path
The directory to download the file(s) to.
job_id
std::string
The batch job identifier.
filename_to_download
std::string
The specific file to download. If not specified, then will download all files for the batch job.
Returns
std::vector<std::filesystem::path> or std::filesystem::path
The paths of the downloaded files.
API method
std::vector<std::filesystem::path> BatchDownload(
const std::filesystem::path& output_dir,
const std::string& job_id);
std::filesystem::path BatchDownload(
const std::filesystem::path& output_dir,
const std::string& job_id,
const std::string& filename_to_download);
Example usage
#include <databento/historical.hpp>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
client.BatchDownload("my_data", "XNAS-20250108-VVS57U5PD8",
"metadata.json");
Helpers
DbnFileStore
The DbnFileStore object is an I/O helper class for reading DBN files.
This class provides both a callback API (Replay) similar to TimeseriesGetRange in historical data and LiveThreaded for live data, as well as a blocking API (GetMetadata and NextRecord) similar to that of LiveBlocking. Only one set of methods should be used on a given instance.
Parameters
log_receiver
ILogReceiver*
Non-null, non-owning pointer to a log receiver. To disable logging, pass a
NullLogReceiver.file_path
std::filesystem::path
File path to the DBN file (
.dbn) or Zstd-compressed DBN (.dbn.zst).upgrade_policy
VersionUpgradePolicy
How to handle records from previous DBN versions. By default, records from DBN versions 1 and 2 are upgraded to version 3.
API method
DbnFileStore(const std::filesystem::path& file_path);
DbnFileStore(ILogReceiver* log_receiver,
const std::filesystem::path& file_path,
VersionUpgradePolicy upgrade_policy);
DbnFileStore::Replay
Replay data by passing records sequentially to the given callback.
InfoShould only be called once per instance.
Parameters
metadata_callback
std::function<void(Metadata&&)>
A callback for receiving the
Metadata of the file. This callback will be called only once, immediately before the first time record_callback is called.record_callback
std::function<KeepGoing(const Record&)>
A callback for receiving the
Records from the file. The particular record type (or record types if saved live data) will depend on the schema in the metadata. Return KeepGoing::Continue to continue receiving records or KeepGoing::Stop to stop replaying.Returns
Returns data via the callbacks metadata_callback and record_callback.
WarningThe reference passed to
record_callbackis only valid during the callback. Copy the record or particular fields to persist data between calls to the callback.
A full list of fields returned through the record_callback for each schema is available in the
list of fields by schema or the struct definitions in
<databento/record.hpp>.
API method
void Replay(const MetadataCallback& metadata_callback,
const RecordCallback& record_callback);
void Replay(const RecordCallback& record_callback);
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto print_large_trades = [](const Record record) {
const auto& trade_msg = record.Get<TradeMsg>();
if (trade_msg.size > 100) {
std::cout << trade_msg << '\n';
}
return KeepGoing::Continue;
};
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
DbnFileStore dbn_file_store = client.TimeseriesGetRangeToFile(
"GLBX.MDP3", {"2022-06-06", "2022-06-10"}, {"ESM2"},
Schema::Trades, "ESM2_20220606-20220610.dbn.zst");
dbn_file_store.Replay(print_large_trades);
Example response
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 1, instrument_id = 3403, ts_event = 2022-06-06T06:52:51.478093125Z },
price = 4132.000000000,
size = 109,
action = Trade,
side = Ask,
flags = 0,
depth = 0,
ts_recv = 2022-06-06T06:52:51.478443918Z,
ts_in_delta = 18569,
sequence = 890656
}
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 1, instrument_id = 3403, ts_event = 2022-06-06T13:50:42.622492841Z },
price = 4140.000000000,
size = 165,
action = Trade,
side = Ask,
flags = 0,
depth = 0,
ts_recv = 2022-06-06T13:50:42.623069413Z,
ts_in_delta = 26961,
sequence = 3070868
}
...
DbnFileStore::GetMetadata
Decodes the DBN metadata if it has not already been decoded and returns a reference to it.
Returns
const Metadata&
DBN Metadata about the file.
API method
const Metadata& GetMetadata();
Example usage
#include <databento/historical.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
DbnFileStore dbn_file_store = client.TimeseriesGetRangeToFile(
"XNAS.ITCH", {"2024-06-24", "2024-06-29"}, {"META"},
Schema::Trades, "META_20240624-20240629.dbn.zst");
std::cout << dbn_file_store.GetMetadata() << '\n';
Example response
Metadata {
version = 3,
dataset = "XNAS.ITCH",
schema = trades,
start = 2024-06-24T00:00:00.000000000Z,
end = 2024-06-29T00:00:00.000000000Z,
limit = 0,
stype_in = raw_symbol,
stype_out = instrument_id,
ts_out = false,
symbol_cstr_len = 71,
symbols = { "META" },
partial = {},
not_found = {},
mappings = {
SymbolMapping { raw_symbol = "META", intervals = { MappingInterval { start_date = 2024-06-24, end_date = 2024-06-29, symbol = "10451" } } }
}
}
DbnFileStore::NextRecord
Replays the next record from the file.
Returns
const Record*
The next record.
The particular record type will depend on the schema in the metadata.
WarningThe returned reference or pointer is only valid until the next call to
NextRecord. Copy the record or relevant fields to persist data across calls toNextRecord.
Returns nullptr when all records have been decoded.
API method
const Record* NextRecord();
Example usage
#include <databento/historical.hpp>
#include <databento/record.hpp>
#include <iostream>
using namespace databento;
auto client =
Historical::Builder().SetKey("$YOUR_API_KEY").Build();
DbnFileStore dbn_file_store = client.TimeseriesGetRangeToFile(
"XNAS.ITCH", {"2024-06-24", "2024-06-29"}, {"META"},
Schema::Trades, "META_20240624-20240629.dbn.zst");
while (const Record* record = dbn_file_store.NextRecord()) {
const auto& trade_msg = record->Get<TradeMsg>();
if (trade_msg.size > 100) {
std::cout << trade_msg << '\n';
}
}
Example response
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 2, instrument_id = 10451, ts_event = 2024-06-24T08:01:59.771923047Z },
price = 498.380000000,
size = 228,
action = Trade,
side = Ask,
flags = LAST (130),
depth = 0,
ts_recv = 2024-06-24T08:01:59.772093551Z,
ts_in_delta = 170504,
sequence = 383997
}
TradeMsg {
hd = RecordHeader { length = 12, rtype = Mbp0, publisher_id = 2, instrument_id = 10451, ts_event = 2024-06-24T08:02:10.254849369Z },
price = 498.380000000,
size = 120,
action = Trade,
side = Ask,
flags = LAST (128),
depth = 0,
ts_recv = 2024-06-24T08:02:10.255020595Z,
ts_in_delta = 171226,
sequence = 397630
}
...
pretty::Px
A wrapper class for formatting fixed-precision prices returned from Databento APIs.
It supports configurable precision with std::setprecision; by default the precision is 9.
Width and fill can be configured with std::setw and std::setfill.
API method
namespace pretty {
struct Px {
std::int64_t val;
};
}
Example usage
#include <databento/pretty.hpp>
#include <iomanip>
#include <iostream>
std::cout << std::setprecision(2)
<< databento::pretty::Px{4'149'500'000'000} << '\n';
Example response
4149.50
Metadata
The contents of the header of a DBN stream.
Fields
version
std::uint8_t
The DBN schema version.
dataset
std::string
The dataset code.
schema
std::optional<Schema>
The data record schema. Will be
nullopt for live data which can mix schemas.start
UnixNanos
The timestamp of the start of request time range (inclusive) expressed as the number nanoseconds since the UNIX epoch.
end
UnixNanos
The timestamp of the end of request time range (exclusive) expressed as the number nanoseconds since the UNIX epoch.
limit
std::uint64_t
The maximum number of records to return.
stype_in
std::optional<SType>
stype_out
SType
The symbology type of output symbols.
ts_out
bool
Whether the stream contains live data with the send timestamps appended to every record.
symbol_cstr_len
std::size_t
The length of fixed-length symbol strings.
symbols
std::vector<std::string>
The product symbols from the original request.
partial
std::vector<std::string>
The symbols that did not resolve for at least one day in the query time range.
not_found
std::vector<std::string>
The symbols that did not resolve for any day in the query time range.
mappings
std::vector<SymbolMapping>
The symbol mappings for historical data.
API method
struct MappingInterval {
date::year_month_day start_date;
date::year_month_day end_date;
std::string symbol;
};
struct SymbolMapping {
std::string raw_symbol;
std::vector<MappingInterval> intervals;
};
struct Metadata {
std::uint8_t version;
std::string dataset;
std::optional<Schema> schema;
UnixNanos start;
UnixNanos end;
std::uint64_t limit;
std::optional<SType> stype_in;
SType stype_out;
bool ts_out;
std::size_t symbol_cstr_len;
std::vector<std::string> symbols;
std::vector<std::string> partial;
std::vector<std::string> not_found;
std::vector<SymbolMapping> mappings;
}
Metadata::CreateSymbolMap
Create a symbology mapping from instrument ID and date to text symbol from the mappings in the metadata.
Returns
A TsSymbolMap with the symbol mappings for the query range indexed by instrument ID and date.
API method
TsSymbolMap CreateSymbolMap() const;
Metadata::CreateSymbolMapForDate
Create a symbology mapping from the mappings in the metadata for the specified date.
Parameters
date
date::year_month_day
The date to create the symbol map for.
Returns
A PitSymbolMap with the symbol mappings for the query range indexed by instrument ID.
API method
PitSymbolMap CreateSymbolMapForDate(
date::year_month_day date) const;
TsSymbolMap
A timeseries symbol map, i.e. instrument IDs to text symbols by date. These objects can be obtained from Metadata::CreateSymbolMap.
API method
class TsSymbolMap{/* private fields */};
TsSymbolMap::At
Get the symbol mapping for a record.
Parameters
rec
R
The record to fetch a symbol mapping for.
Returns
const std::string&
The corresponding text symbol for the record's instrument ID and timestamp.
Raises an exception if no mapping is found.
API method
template <typename R>
const std::string& At(const R& rec) const;
PitSymbolMap
A point-in-time symbol map. Useful for working real-time symbology or a historical request over a single day and other situations where the symbol mappings are known to not change. These objects can be obtained from Metadata::CreateSymbolMapForDate for historical data.
API method
class PitSymbolMap{/* private fields */};
PitSymbolMap::At
Get the symbol mapping for a record.
Parameters
rec
R
The record to fetch a symbol mapping for.
Returns
const std::string&
The corresponding text symbol for the record's instrument ID.
Raises an exception if no mapping is found.
API method
template <typename R>
const std::string& At(const R& rec) const;
PitSymbolMap::OnRecord
Update the symbol map with the contents of the record.
Only SymbolMappingMsg records affect the map; all other record types will be ignored.
Parameters
record
Record
The record to update the map from.
API method
void OnRecord(const Record& rec);
PitSymbolMap::OnSymbolMapping
Update the symbol map with the contents of the record, either a SymbolMappingMsg or a SymbolMappingMsgV1.
Parameters
symbol_mapping
SymbolMappingMsgV1 or SymbolMappingMsg
The record to update the map from.
API method
void OnSymbolMapping(const SymbolMappingMsgV1& symbol_mapping);
void OnSymbolMapping(const SymbolMappingMsg& symbol_mapping);
ILogReceiver
The ILogReceiver is an interface for objects that accept log messages from the databento client library.
Two implementations are provided, but the interface can be extended for additional configuration.
Implementations
- ConsoleLogReceiver: a log receiver that forwards messages to a std::ostream,
like
std::clog,std::cerr, andstd::cout - NullLogReceiver: a log receiver that ignores all log messages
ILogReceiver::Receive
Receive a log message from the databento client library.
Parameters
level
LogLevel
The severity of the log event.
msg
std::string
The message describing the event.
Returns
void
API method
void Receive(LogLevel level, const std::string& msg);
ILogReceiver::Default
Get the default ILogReceiver implementation.
Returns
ILogReceiver*
Returns a pointer to the default log receiver, a [ConsoleLogReceiver] which logs info-level messages and above to std::clog.
API method
static ILogReceiver* Default();
ConsoleLogReceiver
The ConsoleLogReceiver implements the ILogReceiver interface forwarding messages to the console, one of the following std::ostream objects .
Parameters
min_level
LogLevel
The minimum severity required to be forwarded to
stream. Defaults to Info, meaning Debug messages will be ignored.stream
std::ostream&
The output stream to forward messages to. Defaults to std::clog which writes to standard error.
API method
ConsoleLogReceiver();
ConsoleLogReceiver(LogLevel min_level);
ConsoleLogReceiver(std::ostream& stream);
ConsoleLogReceiver(LogLevel min_level, std::ostream& stream);
ConsoleLogReceiver::Receive
Receive a log message from the databento client library.
Parameters
level
LogLevel
The severity of the log event.
msg
std::string
The message describing the event.
Returns
void
API method
void Receive(LogLevel level, const std::string& msg);