Schemas and data formats

What's a schema?

Databento supports over 15 different data formats of market data. When you make a request from Databento, you must usually specify which data format—also called a schema—to receive your data in.

Supported schemas and their fields

A schema represents a collection of data fields. The following is a summary of schemas supported by Databento. Click on any schema below for its details, the fields included, and a data dictionary that defines each field.

Market by order (MBO) provides every order book event across every price level, keyed by its order ID. This allows you to determine the queue position of each order and provides the highest level of granularity available.

Market by price (MBP-10) provides every order book event across the top ten price levels, keyed by price. This includes every trade and changes to aggregate market depth, alongside total size and order count at the top ten price levels.

Market by price (MBP-1) provides every order book event that updates the top price level, also known as the best bid and offer (BBO). This includes every trade and changes to book depth, alongside total size and order count at the BBO.

Consolidated market by price (CMBP-1) provides every order book event that updates the top price level across all venues in the dataset, also known as the consolidated best bid and offer (CBBO). This includes every trade and changes to book depth, alongside total size and publisher attribution at the CBBO.

BBO on trade (TBBO) provides every trade event alongside the BBO immediately before the effect of each trade. This is a subset of MBP-1.

Consolidated BBO on trade (TCBBO) provides every trade event alongside the consolidated BBO immediately before the effect of each trade. This is a subset of CMBP-1.

BBO on interval (BBO) provides the last best bid, best offer, and sale at 1-second or 1-minute intervals. This is a subset of MBP-1.

Consolidated BBO on interval (CBBO) provides the consolidated last best bid, best offer, and sale at 1-second or 1-minute intervals. This is a subset of CMBP-1.

Trades provides every trade event. This is a subset of MBO.

Aggregate bars (OHLCV) provide open, high, low, and close prices and total volume aggregated from trades at 1-second, 1-minute, 1-hour, or 1-day intervals.

Instrument definitions provide reference information about each instrument, which includes properties like symbol, instrument name, expiration date, listing date, tick size, and strike price.

Imbalance provides auction imbalance data such as paired quantity, total quantity, and auction status.

Statistics provides official summary statistics of each instrument that's published by the venue. This generally includes properties like daily volume, open interest, preliminary and final settlement prices, and official open, high, and low prices.

Status provides updates about the trading session, such as halts, pauses, short-selling restrictions, auction start, and other matching engine statuses. The granularity and frequency of these updates vary by publisher and dataset.

See also

The MBP-1, BBO and TBBO schemas, as well as the CMBP-1, CBBO, and TCBBO schemas, all provide top of book data with different update space and sampling intervals. Learn more about their differences in our MBP-1 vs. BBO vs. TBBO schemas guide.

Why are Databento's naming conventions different from third parties?

Databento avoids terms like Level 1 (L1) or Level 2 (L2) due to their inconsistent application. For example, some vendors refer to both MBO and MBP data as L2, while others strictly refer to MBP data as L2. More misleadingly, many vendors refer to MBO data as L3, even though this doesn't follow any major trading venue's naming convention.

Likewise, the term tick originates from the concept of a ticker tape and only refers to trades, not resting limit orders. This becomes a source of confusion when vendors use the term tick data to refer to either MBO or MBP data when it should be strictly reserved for trades data.

See also

For more Databento naming conventions and key terminology, visit our FAQs.

Special cases

Our MBO, MBP-1 and MBP-10 schemas adopt the following conventions in these special cases:

• Combining MBO with trades feed: Typically, MBO data provides the highest granularity, but certain venues enhance their trades feed with additional information like trades not reflected in the MBO feed, consolidated NBBO, and more. In these cases, we document the exception in our Venues and datasets section and recommend that you request both our MBO and trades schemas if you need the highest level of granularity.

Deriving one schema from another

Databento captures market data directly from the source and is only subscribed to the most granular feed(s) available from each publisher. Order book feeds are usually normalized into our MBO schema and top-of-book feeds are usually normalized into our MBP-1 schema.

To ensure consistency between schemas, Databento doesn't source the less granular schemas from separate feeds. Instead, Databento derives all of the less granular schemas starting from the most granular schema available. As you may have noticed from our schema's descriptions, the majority of them—MBP, BBO, TBBO, trades, and OHLCV—are simply derived from MBO data.

Likewise, you can also derive one schema from another losslessly on the client side, and you should expect your self-derived data to be consistent with ours. For example:

• MBP-1, BBO, and Trades can be derived from MBP-10.
• BBO, TBBO, and Trades can be derived from MBP-1.
• Trades and OHLCV can be derived from TBBO.
• OHLCV can be derived from Trades.

Deriving your own schema is useful for various reasons:

• The data needs to be defined differently for your application.
• Our derivation differs from those of another vendor and you want transparency.
• You can cut down the number of API requests made to Databento by getting the most granular schema that you need and deriving the rest yourself.
• Databento provides MBP-10 merely as a convenience feature. You can reduce bandwidth requirements, latency, and transfer time significantly by deriving MBP-10 yourself from MBO.

This is especially relevant for OHLCV, which can vary depending on how trade breaks or market halts are managed, how the start and end of each time interval are determined, and how illiquid instruments are handled if there are no trades over a given time interval. If these considerations are trivial for your use case, Databento offers OHLCV data in multiple time intervals (seconds, minutes, hours, and daily) for your convenience.

The table below summarizes which schemas can be derived from the another. Each row represents the original schema, and each column represents schemas that you can derive from the original schema.

See also

Learn how to resample trades data to other intervals, such as 5-minute intervals, from our bar aggregation tutorial.

You can also learn how to generate MBP-10 from MBO data using an order book, as seen in our limit order book construction tutorial.

Market by order (MBO)

Market by order (MBO) provides every order book event across every price level, keyed by its order ID. This allows you to determine the queue position of each order and provides the highest level of granularity available.

MBO data includes all trades, fills, adds, cancels, modifies (or replaces), book clear events, and, depending on the venue and dataset, other special order events. It is often called "L3 data".

Fields (mbo)

Snapshots

For the convenience of managing state and recovery, Databento provides a synthetic snapshot of the order book at the start of each UTC day in our historical MBO data and periodic book snapshots in our real-time MBO data. The mechanics of these snapshots is detailed here.

See also

Learn more about the different action types and how to manage order state with respect to each action from our State management of resting orders tutorial.

Also learn how to construct a limit order book from MBO data from our limit order book construction tutorial.

MBO data normalization differs slightly from one venue or dataset to another. Edge cases and differences are documented separately for each venue in the Venues and datasets section.

Market by price (MBP-10)

MBP-10 (market by price) provides every order book event across the top ten price levels, keyed by price. This includes every trade and changes to aggregate market depth, alongside total size and order count at the top ten price levels.

This is often called "L2 data".

Fields (mbp-10)

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

It is possible to construct MBP-10 yourself from MBO data if you want more price levels or prefer to reduce your bandwidth use. Learn how to construct a limit order book from MBO data from our limit order book construction tutorial.

Market by price (MBP-1)

MBP-1 (market by price) provides every order book event that updates the top price level, also known as the best bid and offer (BBO). This includes every trade and changes to book depth, alongside total size and order count at the BBO.

This is often called "L1 data".

Fields (mbp-1)

See also

Some venues or data feeds may publish two-sided BBO changes in a single update. In cases like this where the side is indeterminate, we'll also use the side code N. Exceptions and edge cases like this can be found in our Venues and datasets section.

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

MBP-1 has many similarities to the TBBO and BBO schemas. The main distinction is that MBP-1 is in book update space, while TBBO is in trade space, and BBO is in time space. Learn about the differences between each in our MBP-1 vs. TBBO vs. BBO schemas guide.

Consolidated market by price (CMBP-1)

CMBP-1 (consolidated market by price) provides every order book event that updates the top price level across all venues in the dataset, also known as the consolidated best bid and offer (CBBO). This includes every trade and changes to book depth, alongside total size and publisher attribution at the CBBO.

This is often called "L1 data".

Fields (cmbp-1)

See also

Some venues or data feeds may publish two-sided CBBO changes in a single update. In cases like this where the side is indeterminate, we'll also use the side code N. Exceptions and edge cases like this can be found in our Venues and datasets section.

CMBP-1 publisher

The value used to populate the publisher_id field will vary depending on the action. For Trades, publisher_id will correspond to the venue the trade executed on. For all other actions, publisher_id will be the consolidated publisher ID for the dataset.

In all scenarios, bid_pb_00 and ask_pb_00 will represent the venues showing the NBBO.

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

CMBP-1 has many similarities to the TCBBO and CBBO schemas. The main distinction is that CMBP-1 is in book update space, while TCBBO is in trade space, and CBBO is in time space. Learn about the differences between each in our CMBP-1 vs. TCBBO vs. CBBO schemas guide.

BBO on trade (TBBO)

BBO on trade (TBBO) provides every trade event alongside the BBO immediately before the effect of each trade. This is a subset of MBP-1.

Fields (tbbo)

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

TBBO has many similarities to the MBP-1 and BBO schemas. The main distinction is that TBBO is in trade space, while MBP-1 is in book update space, and BBO is in time space. In particular, the action type of TBBO is always Trade. Learn about the differences between each in our MBP-1 vs. TBBO vs. BBO schemas guide.

Consolidated BBO on trade (TCBBO)

Consolidated BBO on trade (TCBBO) provides every trade event alongside the consolidated BBO immediately before the effect of each trade. This is a subset of CMBP-1.

Fields (tcbbo)

TCBBO publisher

The publisher_id field will correspond to the venue the trade executed on.

bid_pb_00 and ask_pb_00 will represent the individual venues showing the NBBO.

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

TCBBO has many similarities to the CMBP-1 and CBBO schemas. The main distinction is that TCBBO is in trade space, while CMBP-1 is in book update space, and CBBO is in time space. In particular, the action type of TBBO is always Trade. Learn about the differences between each in our CMBP-1 vs. TCBBO vs. CBBO schemas guide.

BBO on interval (BBO)

BBO on interval (BBO) provides the last best bid, best offer, and sale at 1-second or 1-minute intervals. This is a subset of MBP-1.

Databento adopts the following convention for BBO:

• The time interval is indicated by the schema ID's suffix: -1s for 1-second and -1m for 1-minute.
• If no trade or BBO update occurs within the interval, no record is printed.
• If a BBO update occurs but no trade takes place within the interval, the last sale information is forward-filled from the previous interval.
• If no BBO update occurs but a trade takes place within the interval, the BBO information is forward-filled from the previous interval.

Fields (bbo-1s and bbo-1m)

Consolidated BBO on interval (CBBO)

Consolidated BBO on interval (CBBO) provides the consolidated last best bid, best offer, and sale at 1-second or 1-minute intervals. This is a subset of CMBP-1.

Databento adopts the following convention for CBBO:

• The time interval is indicated by the schema ID's suffix: -1s for 1-second and -1m for 1-minute.
• If no trade or CBBO update occurs within the interval, no record is printed.
• If a CBBO update occurs but no trade takes place within the interval, the last sale information is forward-filled from the previous interval.
• If no CBBO update occurs but a trade takes place within the interval, the CBBO information is forward-filled from the previous interval.

Fields (cbbo-1s and cbbo-1m)

CBBO publisher

The publisher_id field will always be the consolidated publisher ID for the dataset.

bid_pb_00 and ask_pb_00 will represent the individual venues showing the NBBO at the end of the interval.

Implementation differences between clients and encodings

Bid and ask depth messages (fields starting with bid_ and ask_) are structured differently in the C++ and Rust clients, the Python record interface, and JSON data. Instead of using the _N suffix, they're stored in an array of structures named levels, with the top-of-book at index 0.

For example, in C++, levels[5].bid_px corresponds to bid_px_05 in the Python DataFrame API and CSV format.

See also

CBBO has many similarities to the CMBP-1 and TCBBO schemas. The main distinction is that CBBO is in time space, while CMBP-1 is in book update space, and TCBBO is in trade space. Learn about the differences between each in our CMBP-1 vs. TCBBO vs. CBBO schemas guide.

Trades

Trades provides every trade event. This is a subset of MBO.

This is often referred to as "time and sales", "last sale," or "tick data."

Fields (trades)

Aggregate bars (OHLCV)

Aggregate bars (OHLCV) provide open, high, low, and close prices and total volume aggregated from trades at 1-second, 1-minute, 1-hour, or 1-day intervals.

Databento adopts the following convention for OHLCV:

• The time interval is indicated by the schema ID's suffix: -1s for 1-second, -1m for 1-minute, -1h for 1-hour, and -1d for 1-day.
• The ts_event timestamp marks the start of each interval.
• If no trade occurs within the interval, no record is printed.

Fields (ohlcv-1s, ohlcv-1m, ohlcv-1h, ohlcv-1d)

Other sampling intervals

If you need other sampling intervals, we recommend that you construct OHLCV aggregates from trades data or subsample the OHLCV schema with the nearest resolution on client side. For example, if you need 5-minute OHLCV aggregates, you can subsample this from ohlcv-1m.

Best practices

It's recommended that you construct OHLCV aggregates yourself from trades data if your application can handle the larger volume of data. There will always be slight implementation differences in how a vendor constructs their OHLCV aggregates. These differences include how trade conditions and retroactive trade breaks are handled, which timestamp is used for the event, and how precisely a vendor closes or publishes the aggregate bar after the end of the interval. Daily OHLCV aggregates may also differ by whether best bid or offer prices are used to compute the highs and lows, and whether the volumes should include block trades.

Constructing OHLCV aggregates yourself ensures additional transparency into how the aggregates are computed and consistency within your data.

Our ohlcv-1d schema is based on UTC dates. If you are interested in daily data based on exchange session hours, you may need to request data for a more granular OHLCV schema and aggregate the data yourself.

See also

ohlcv-1d is based entirely on prices and volumes from the electronic trading session and will be consistent with prices and volumes observed from aggregating our trades data. However, this may differ from official settlement prices and volumes published by the venue, which could include block trades, OTC transactions, and other effects. For official settlement data and more details about this, see our statistics schema.

Instrument definitions

Instrument definitions provide point-in-time reference information about each instrument, which includes properties like symbol, instrument name, expiration date, listing date, tick size, and strike price.

Fields (definition)

Instrument class

The instrument_class field groups instruments into different classes of securities and derivatives.

Security type

The security_type field classifies the type of instrument.

Matching algorithm

The matching_algorithm field defines the type of matching algorithm used for the instrument.

Point-in-time instrument definitions

An important distinction of Databento's instrument definition data is that we usually provide it in point-in-time format like the rest of our market data.

In other words, the instrument definitions are timestamped to the nanosecond and treated as a time series, allowing you to simulate the sequence in which they arrived in real time. This is useful as it avoids lookahead and survivorship bias, and also allows you to backtest any strategy that trades instruments immediately as they become available intraday, such as new options strikes and new listings from IPOs.

If you're switching to Databento from another data provider, you may find this behavior different, as many data providers only give you a cumulative list of all instruments or daily updates on instrument definitions.

Snapshots

For publishers that do not provide a full list of instrument definitions at least daily, the historical API provides a snapshot of all active definitions at UTC midnight Monday through Friday. Therefore, any request for definitions for a 24-hour period beginning at UTC midnight is guaranteed to include definitions for all instruments active during that time period.

Instrument definitions on time series endpoints vs. reference data endpoints

Databento also provides other forms of reference information and static data, such as corporate actions, cross-symbology mappings, and security master data. However, these additional forms of reference data may be found on API endpoints other than our time series endpoints.

The reason that these are separated into different endpoints is that instrument definitions on our time series endpoints are those included with the primary source of data, which is usually a direct feed from a trading venue. On the other hand, other forms of reference data may originate from different secondary sources or need to be licensed separately from such direct feeds.

Databento generally provides ways for you to join and merge instrument definitions and other forms of reference data.

Strategies

One definition record is created for each leg of a strategy or spread. The primary fields will be the same for each of these records, while the fields beginning with leg_ will have information about the particular leg.

Imbalance

Imbalance provides auction imbalance data such as paired quantity, total quantity, and auction status.

See also

Imbalance data normalization differs slightly from one venue or dataset to another. Edge cases and differences are documented separately for each venue in the Venues and datasets section.

Fields (imbalance)

Statistics

Statistics provides official summary statistics of each instrument that's published by the venue. This generally includes properties like daily volume, open interest, preliminary and final settlement prices, and official open, high, and low prices.

Fields (statistics)

Some fields are not applicable depending on the type of statistic. Null, invalid or inapplicable values are represented by the maximum value of the field's type. For example, null is represented by 2^64-1 for ts_event, which has an unsigned 64-bit integer type. In cases where the value is actually zero or null has the same meaning as zero, zero is used instead.

Types of statistics

Types of statistics by publisher

The table below shows which statistics are available depending on dataset and publisher.

Official vs. Databento summary statistics

The key distinction of the statistics schema is these are official summary statistics provided by the venue—Databento doesn't compute these statistics.

On most venues, Databento also provides separate OHLCV data that could be used in place of official open, high, low, and settlement prices, and volume from the statistics schema.

These schemas are different and may vary for a few intentional reasons:

• Often, official statistics have opaque methodology or are difficult to replicate because they're tallied by hand
• Some venues include volumes and open interest from open outcry, auction, block trades, RFQs, or other events that may not be disseminated in the electronic trading session or public feeds
• Some venues double count volumes

The main purpose of Databento's summary data is that it provides more consistency: we derive it systematically from the tick data or full order book data during the electronic trading session and ensures consistency with our tick data; we publish at more deterministic times, and we normalize across venues using a UTC midnight cutoff universally. For electronic trading and research applications, this consistency may be more important, whereas for back office and administrative applications, official statistics may be preferable.

Status

Status provides updates about the trading session, such as halts, pauses, short-selling restrictions, auction start, and other matching engine statuses. The granularity and frequency of these updates vary by publisher and dataset.

Fields (status)

Status action variants

Status reason variants

Trading event variants

Snapshots

As trading status can carry across UTC days, the historical API includes a snapshot of the last status record for each active instrument at UTC midnight.

Corporate actions

Corporate actions includes coverage from 215+ exchanges worldwide, 300,000+ listed and delisted securities from 100,000+ companies, categorized as 60+ event types. The dataset focuses on crucial events affecting global securities such as dividends, takeovers, mergers & acquisitions and rights issues.

See also

Reference data enums for a lookup table of possible enumeration field values.

Fields

• Timestamps are strings in ISO 8601 format, expressed in UTC
• Dates are strings in YYYY-MM-DD ISO 8601 format, expressed in the local time of the listing exchange
• Enums are strings, with each value corresponding to a predefined set of options listed in the associated lookup table

Identifiers

Identifiers for listing, security and issuer are grouped at the beginning of each record, which enables clearer comparisons of where a single security is associated with many individual listings across different exchanges. There is one primary_exchange for a security, and it may then be listed on multiple secondary exchanges.

Dynamic fields

The following dynamically-sized fields provide additional supplementary information as key-value pairs:

• date_info. Date specific information related to the event
• rate_info. Rate specific information related to the event
• event_info. Additional supplementary event information

Info

The content of these dynamic fields is contingent on the type of event and its event_subtype, allowing for flexible and detailed data representation.

Adjustment factors

Adjustment factors covers worldwide corporate actions capital events, spanning multiple currencies for the same event. Each record includes all necessary fields to identify the country, security, event, and adjustment factor.

See also

Reference data enums for a lookup table of possible enumeration field values.

Fields

• Timestamps are strings in ISO 8601 format, expressed in UTC
• Dates are strings in YYYY-MM-DD ISO 8601 format, expressed in the local time of the listing exchange
• Enums are strings, with each value corresponding to a predefined set of options listed in the associated lookup table

Security master

Security master provides up-to-date (latest) or point-in-time (PIT) static information on securities across international markets for 860,000+ listed and delisted securities. This provides a means for resolving securities attributes, identification, symbology and informational challenges efficiently. Some common use cases for the security master include research, backtesting, compliance and report generation, risk management, and keeping portfolios current.

See also

Reference data enums for a lookup table of possible enumeration field values.

Fields

• Timestamps are strings in ISO 8601 format, expressed in UTC
• Dates are strings in YYYY-MM-DD ISO 8601 format, expressed in the local time of the listing exchange
• Enums are strings, with each value corresponding to a predefined set of options listed in the associated lookup table

Identifiers

Identifiers for listing, security and issuer are grouped at the beginning of each record, which enables clearer comparisons of where a single security is associated with many individual listings across different exchanges. There is one primary_exchange for a security, and it may then be listed on multiple secondary exchanges.