Source: https://databento.com/docs/datasets # Venues and datasets Differences between venue protocols and how they're normalized by Databento. | Title | Subtitle | Image | Supported languages | | [Blue Ocean ATS MEMOIR Depth](https://databento.com/docs/datasets/ocea-memoir.md) | | ![Blue Ocean icon](https://databento.com/docs/ocea-purple.svg) | Equities, OCEA.MEMOIR | | [Cboe BYX Depth](https://databento.com/docs/datasets/baty-pitch.md) | | ![Cboe icon](https://databento.com/docs/cboe-purple.svg) | Equities, BATY.PITCH | | [Cboe BZX Depth](https://databento.com/docs/datasets/bats-pitch.md) | | ![Cboe icon](https://databento.com/docs/cboe-purple.svg) | Equities, BATS.PITCH | | [Cboe EDGA Depth](https://databento.com/docs/datasets/edga-pitch.md) | | ![Cboe icon](https://databento.com/docs/cboe-purple.svg) | Equities, EDGA.PITCH | | [Cboe EDGX Depth](https://databento.com/docs/datasets/edgx-pitch.md) | | ![Cboe icon](https://databento.com/docs/cboe-purple.svg) | Equities, EDGX.PITCH | | [CFE](https://databento.com/docs/datasets/xcbf-pitch.md) | | ![CFE icon](https://databento.com/docs/cboe-purple.svg) | Futures, Options, XCBF.PITCH | | [CME Globex MDP 3.0](https://databento.com/docs/datasets/glbx-mdp3.md) | | ![CME icon](https://databento.com/docs/cme-purple.svg) | Futures, Options, GLBX.MDP3 | | [Databento Core Indices - CGIF](https://databento.com/docs/datasets/dbix-cgif.md) | | ![Databento icon](https://databento.com/docs/dbeq-purple.svg) | Indices, DBIX.CORE | | [Databento US Equities Mini](https://databento.com/docs/datasets/equs-mini.md) | | ![DBEQ icon](https://databento.com/docs/dbeq-purple.svg) | Equities, EQUS.MINI | | [Databento US Equities Summary](https://databento.com/docs/datasets/equs-summary.md) | | ![DBEQ icon](https://databento.com/docs/dbeq-purple.svg) | Equities, EQUS.SUMMARY | | [Eurex](https://databento.com/docs/datasets/xeur-eobi.md) | | ![XEUR icon](https://databento.com/docs/xeur-purple.svg) | Futures, Option, XEUR.EOBI | | [European Energy Exchange](https://databento.com/docs/datasets/xeee-eobi.md) | | ![XEEE icon](https://databento.com/docs/xeee-purple.svg) | Futures, Option, XEEE.EOBI | | [ICE Endex](https://databento.com/docs/datasets/ndex-impact.md) | | ![ICE icon](https://databento.com/docs/ice-purple.svg) | Futures, Options, NDEX.IMPACT | | [ICE Europe Commodities](https://databento.com/docs/datasets/ifeu-impact.md) | | ![ICE icon](https://databento.com/docs/ice-purple.svg) | Futures, Options, IFEU.IMPACT | | [ICE Europe Financials](https://databento.com/docs/datasets/ifll-impact.md) | | ![ICE icon](https://databento.com/docs/ice-purple.svg) | Futures, Options, IFLL.IMPACT | | [ICE Futures US](https://databento.com/docs/datasets/ifus-impact.md) | | ![ICE icon](https://databento.com/docs/ice-purple.svg) | Futures, Options, IFUS.IMPACT | | [IEX TOPS](https://databento.com/docs/datasets/iexg-tops.md) | | ![IEX icon](https://databento.com/docs/iex-purple.svg) | Equities, IEXG.TOPS | | [MEMX MEMOIR Depth](https://databento.com/docs/datasets/memx-memoir.md) | | ![MEMX icon](https://databento.com/docs/memx-purple.svg) | Equities, MEMX.MEMOIR | | [MIAX Pearl Depth of Market](https://databento.com/docs/datasets/eprl-dom.md) | | ![MIAX icon](https://databento.com/docs/miax-purple.svg) | Equities, EPRL.DOM | | [Nasdaq Basic with NLS Plus](https://databento.com/docs/datasets/xnas-basic.md) | | ![NASDAQ icon](https://databento.com/docs/nasdaq-purple.svg) | Equities, XNAS.BASIC | | [Nasdaq TotalView-ITCH](https://databento.com/docs/datasets/xnas-itch.md) | | ![NASDAQ icon](https://databento.com/docs/nasdaq-purple.svg) | Equities, XNAS.ITCH | | [Nasdaq PSX TotalView-ITCH](https://databento.com/docs/datasets/xpsx-itch.md) | | ![NASDAQ icon](https://databento.com/docs/nasdaq-purple.svg) | Equities, XPSX.ITCH | | [Nasdaq Texas TotalView-ITCH](https://databento.com/docs/datasets/xbos-itch.md) | | ![NASDAQ icon](https://databento.com/docs/nasdaq-purple.svg) | Equities, XBOS.ITCH | | [NYSE Integrated](https://databento.com/docs/datasets/xnys-pillar.md) | | ![NYSE icon](https://databento.com/docs/nyse-purple.svg) | Equities, XNYS.PILLAR | | [NYSE American Integrated](https://databento.com/docs/datasets/xase-pillar.md) | | ![NYSE icon](https://databento.com/docs/nyse-purple.svg) | Equities, XASE.PILLAR | | [NYSE Arca Integrated](https://databento.com/docs/datasets/arcx-pillar.md) | | ![NYSE icon](https://databento.com/docs/nyse-purple.svg) | Equities, ARCX.PILLAR | | [NYSE National Trades and BBO](https://databento.com/docs/datasets/xcis-tradesbbo.md) | | ![NYSE icon](https://databento.com/docs/nyse-purple.svg) | Equities, XCIS.TRADESBBO | | [NYSE Texas Integrated](https://databento.com/docs/datasets/xchi-pillar.md) | | ![NYSE icon](https://databento.com/docs/nyse-purple.svg) | Equities, XCHI.PILLAR | | [OPRA](https://databento.com/docs/datasets/opra-pillar.md) | | ![OPRA icon](https://databento.com/docs/opra-purple.svg) | Options, OPRA.PILLAR | | [Adjustment factors](https://databento.com/docs/datasets/adjustment-factors.md) | | ![Adjustment factors icon](https://databento.com/docs/adjustment-factors-dataset.svg) | Equities, Reference | | [Corporate actions](https://databento.com/docs/datasets/corporate-actions.md) | | ![Corporate actions icon](https://databento.com/docs/corporate-actions-dataset.svg) | Equities, Reference | | [Security master](https://databento.com/docs/datasets/security-master.md) | | ![Security master icon](https://databento.com/docs/security-master-dataset.svg) | Equities, Reference | # Blue Ocean ATS MEMOIR Depth **Dataset ID**: [OCEA.MEMOIR]($PLATFORM_HOSTNAME/catalog/us-equities#OCEA.MEMOIR) Blue Ocean ATS disseminates full depth-of-book data for its venue through its Blue Ocean ATS MEMOIR Depth feed, which Databento receives over UDP multicast in our NY4 data center. This dataset captures Blue Ocean ATS trading activity in US equities during the Blue Ocean Session, which runs from 8:00 PM to 4:00 AM ET, Sunday through Thursday. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`); the source feed splits root and suffix across the `Symbol` and `SymbolSfx` fields, which Databento joins with a space (see [Definition normalization](https://databento.com/docs/datasets/ocea-memoir.md#definition-normalization)). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Blue Ocean messages received by Databento include one exchange timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. Blue Ocean does not include an exchange send timestamp, so `ts_in_delta` is set to `0` for normalized MBO and trades records. | Databento Field | MEMOIR Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | | `ts_in_delta` | N/A | Set to `0` because Blue Ocean does not include an exchange send timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization There are six MEMOIR message types that directly affect displayed book state or trade prints. Databento normalizes these messages to MBO records as follows: | Databento Message(s) | MEMOIR Message | Description | | **A**dd | *Order Added* | Adds a new displayed order with `order_id`, `side`, `price`, and `size`. | | **C**ancel | *Order Deleted* | Removes a displayed order. `size` is the displayed quantity removed from the book. | | **C**ancel | *Order Reduced* | Reduces a displayed order without changing time priority. `size` is the quantity removed from the order. | | **T**rade, **F**ill, **C**ancel | *Order Executed* | Executes a displayed order and updates displayed quantity by the executed amount. | | **T**rade | *Trade* | Non-displayed execution. Does not modify displayed book state. | | Clea**R** | *Clear Book* | Clears all displayed orders for the specified instrument. | For *Order Executed* messages, the aggressing order is not included in MEMOIR, so normalized **T**rade records have `order_id` set to `0`. Normalized records from a single native message share the same `sequence`. For *Order Executed* and *Trade*, Databento also publishes records in the [trades schema](https://databento.com/docs/schemas/trades.md). ### Trade side normalization For trade-derived records across the MBO, MBP-1, MBP-10, and trades schemas, the `side` field indicates the side that initiates the event. For `Order Executed`, Databento sets `side` to the aggressive order side, which is the opposite of the resting order side in the book. For `Trade`, the feed does not provide enough information to infer the aggressor, so the `side` is **N**one. ### Messages currently not normalized Databento does not emit MBO, trades, status, or definition output from these native message types: - *Broken Trade* - *Corrected Trade* ### Status normalization Status records are normalized from three MEMOIR messages: - *Security Trading Status* - *Trading Session Status* - *Reg SHO Restriction* **Security Trading Status** | Databento Action | Is trading | Is quoting | MEMOIR SecurityTradingStatus | | `8` (Halt) | No | No | `H` (Halted) | | `9` (Pause) | No | No | `P` (Paused) | | `3` (Quoting) | No | Yes | `Q` (Quoting) | | `7` (Trading) | Yes | Yes | `T` (Trading) | MEMOIR `SecurityTradingStatusReason` is normalized to status `reason`: | Databento Reason | MEMOIR SecurityTradingStatusReason | | None | `X` (None) | | Regulatory | `R` (Regulatory) | | Administrative | `A` (Administrative) | **Trading Session Status** | Databento Action | Is trading | Is quoting | MEMOIR TradingSession | | `1` (Pre-open) | No | Yes | `1` (Opening) | | `7` (Trading) | Yes | Yes | `2` (Trading) | | `13` (Post-close) | Yes | Yes | `3` (PostTrading) | | `12` (Close) | No | No | `4` (Closed) | **Reg SHO Restriction** `RegSHORestriction` is normalized to a status record for a short sell restriction change and updates `is_short_sell_restricted` from the source `ShortSaleRestriction` flag. Status publication is gated until both system-level state (`Trading Session Status`) and per-instrument state (`Security Trading Status`) are known. ### Definition normalization Definition records are sourced from *Instrument Directory* messages. | Databento Field | MEMOIR Field | Description | | `raw_instrument_id` | `SecurityID` | Native session-scoped identifier. | | `raw_symbol` | `Symbol` + `SymbolSfx` | CMS root and suffix combined with a space into Databento raw symbol. | | `min_lot_size_round_lot` | `RoundLot` | Round lot size. | | `min_price_increment_amount` | `MPV` | Minimum price variation converted to Databento's 1e-9 price units. | If a security is updated intraday, the corresponding definition is republished with updated values. Databento also sets `exchange` to `OCEA` and `match_algorithm` to `F` (FIFO). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Cboe BYX Depth **Dataset ID**: [BATY.PITCH]($PLATFORM_HOSTNAME/catalog/us-equities#BATY.PITCH) Cboe BYX disseminates full depth-of-book data for their exchange through their Pitch Feed, which Databento receives over UDP multicast in our NY4 data center. The [Multicast Depth of Book (Pitch) Specification](https://cdn.cboe.com/resources/membership/US_EQUITIES_OPTIONS_MULTICAST_PITCH_SPECIFICATION.pdf) is available as a PDF from Cboe's website. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`), even though the source venue is Cboe rather than Nasdaq. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Cboe messages received by Databento will generally include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pitch Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Time + TimeOffset | The matching-engine-received timestamp. | | `ts_in_delta` | None | The difference between `ts_recv` and `ts_event`. Redundant and can be safely ignored. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics Databento provides Opening Price and Closing Price statistics, normalized from the *Auction Summary* message. These statistics have the auction uncrossing price and quantity for the respective event. ### Imbalance normalization Cboe BYX provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pitch Field | Description | | `ref_price` | ReferencePrice | Collared price at which the periodic auction would trade. | | `paired_qty` | min(BuyShares, SellShares) | The number of shares that are eligible to match at the `ref_price`. | | `total_imbalance_qty` | abs(SellShares - BuyShares) | The number of shares not paired at the `ref_price`. | | `auction_type` | AuctionType | **P**: Periodic auction. | | `side` | BuyShares, SellShares | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Cboe BZX Depth **Dataset ID**: [BATS.PITCH]($PLATFORM_HOSTNAME/catalog/us-equities#BATS.PITCH) Cboe BZX disseminates full depth-of-book data for their exchange through their Pitch Feed, which Databento receives over UDP multicast in our NY4 data center. The [Multicast Depth of Book (Pitch) Specification](https://cdn.cboe.com/resources/membership/US_EQUITIES_OPTIONS_MULTICAST_PITCH_SPECIFICATION.pdf) is available as a PDF from Cboe's website. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`), even though the source venue is Cboe rather than Nasdaq. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Cboe messages received by Databento will generally include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pitch Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Time + TimeOffset | The matching-engine-received timestamp. | | `ts_in_delta` | None | The difference between `ts_recv` and `ts_event`. Redundant and can be safely ignored. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics normalization Cboe BZX provides information about the execution price and shares matched from an auction in their *Auction Summary* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following auction types: - Opening price: Opening - Closing price: Closing ### Imbalance normalization Cboe BZX provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pitch Field | Description | | `ref_price` | ReferencePrice | The BBO collared auction price. | | `cont_book_clr_price` | IndicativePrice | Price at which the auction book and the continuous book would match. | | `auct_interest_clr_price` | AuctionOnlyPrice | Price at which the auction book would match using only Eligible Auction Orders. | | `paired_qty` | min(BuyShares, SellShares) | The number of shares that are eligible to match at the `ref_price`. | | `total_imbalance_qty` | abs(SellShares - BuyShares) | The number of shares not paired at the `ref_price`. | | `auction_type` | AuctionType | **O**: Opening auction. **C**: Closing auction. **H**: Halt auction. **I**: IPO auction. **M**: Cboe Market Close. | | `side` | BuyShares, SellShares | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Cboe EDGA Depth **Dataset ID**: [EDGA.PITCH]($PLATFORM_HOSTNAME/catalog/us-equities#EDGA.PITCH) Cboe EDGA disseminates full depth-of-book data for their exchange through their Pitch Feed, which Databento receives over UDP multicast in our NY4 data center. The [Multicast Depth of Book (Pitch) Specification](https://cdn.cboe.com/resources/membership/US_EQUITIES_OPTIONS_MULTICAST_PITCH_SPECIFICATION.pdf) is available as a PDF from Cboe's website. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`), even though the source venue is Cboe rather than Nasdaq. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Cboe messages received by Databento will generally include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pitch Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Time + TimeOffset | The matching-engine-received timestamp. | | `ts_in_delta` | None | The difference between `ts_recv` and `ts_event`. Redundant and can be safely ignored. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Cboe EDGX Depth **Dataset ID**: [EDGX.PITCH]($PLATFORM_HOSTNAME/catalog/us-equities#EDGX.PITCH) Cboe EDGX disseminates full depth-of-book data for their exchange through their Pitch Feed, which Databento receives over UDP multicast in our NY4 data center. The [Multicast Depth of Book (Pitch) Specification](https://cdn.cboe.com/resources/membership/US_EQUITIES_OPTIONS_MULTICAST_PITCH_SPECIFICATION.pdf) is available as a PDF from Cboe's website. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`), even though the source venue is Cboe rather than Nasdaq. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Cboe messages received by Databento will generally include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pitch Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Time + TimeOffset | The matching-engine-received timestamp. | | `ts_in_delta` | None | The difference between `ts_recv` and `ts_event`. Redundant and can be safely ignored. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization For Cboe EDGX Depth, the `F_PUBLISHER_SPECIFIC` flag is used in the [MBO](https://databento.com/docs/schemas/mbo.md) schema to indicate Retail Priority Orders (RPOs). MBO messages with action `A` will have the `F_PUBLISHER_SPECIFIC` flag set if the order is an RPO. This flag will not repeat on subsequent modifications or cancellations. Retail Priority changes the order allocation priority model on EDGX to price-retail priority-time priority. Thus, Retail Priority Orders will have priority ahead of all other available interest at a price level. # CFE **Dataset ID**: [XCBF.PITCH]($PLATFORM_HOSTNAME/catalog/xcbf/XCBF.PITCH) Cboe disseminates full depth-of-book data for their Cboe Futures Exchange (CFE) through their Multicast Depth of Book (PITCH) feed, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available on [Cboe's website](https://www.cboe.com/us/futures/support/technical) for [Multicast PITCH](https://cdn.cboe.com/resources/membership/CBOE_FUTURES_EXCHANGE_MULTICAST_PITCH_SPECIFICATION.pdf) and [Options on Futures Multicast PITCH](https://cdn.cboe.com/resources/membership/CFE_OOF_Multicast_PITCH_Specification.pdf). ### Timestamps Cboe messages received by Databento will generally include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. As this protocol does not have a `Sending Time` field, the field `ts_in_delta` is set to 0 and should be ignored. | Databento Field | Pitch Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | EpochTime + TimeOffset | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Trading hours CFE operates from Sunday through Friday on a weekly schedule, with daily sessions. Trading sessions vary by product, refer to the Cboe [trading hours](https://www.cboe.com/about/hours/us-futures) for up to date information. * Regular trading runs each day from Monday through Friday * There is extended trading both before and after each regular trading session. * At the end of the post-session extended trading all orders are canceled * There is a pause between 4:00 PM and 5:00 PM CT | Description | Hours | | Regular Trading Hours | 8:30 AM to 3:00 PM (CT) | | Extended Trading Hours (pre-session) | 5:00 PM (previous day) to 08:30 AM (CT) | | Extended Trading Hours (post-session) | 3:00 PM to 4:00 PM (CT) | | Daily pause | 4:00 PM to 5:00 PM (CT) | Note: Not all products have extended trading. The [trading hours](https://www.cboe.com/about/hours/us-futures) for some commonly traded products are: * **[VX](https://www.cboe.com/tradable-products/vix/vix-futures/specifications) and [VXM](https://cdn.cboe.com/resources/futures/VXM_Contract_Specifications.pdf) futures**: Sunday 5:00 PM - Friday 4:00 PM CT (*extended trading hours* + *regular trading hours*), with a daily trading halt from 4:00 PM to 5:00 PM CT. * [**UX (Options on VX futures)**](https://cdn.cboe.com/resources/membership/VIX_OOF_Contract_Specification.pdf): Monday-Friday, 8:30 AM - 3:00 PM CT (*regular trading hours*). ### MBO normalization **Order lifecycle messages** There are seven message types in CFE PITCH that change the state of the order book. Databento normalizes these messages to MBO records as follows: | CFE Message | Databento Message(s) | Description | | Add Order (long) | **A**dd | | | Add Order (short) | **A**dd | | | Order Executed | **T**rade, **F**ill, **C**ancel | | | Reduce Size (long) | **C**ancel | The `size` field indicates the number of contracts to remove from the display size | | Reduce Size (short) | **C**ancel | The `size` field indicates the number of contracts to remove from the display size | | Modify (long) | **M**odify | The `price` and `size` fields indicate the new order price and quantity | | Modify (short) | **M**odify | The `price` and `size` fields indicate the new order price and quantity | | Delete | **C**ancel | The `size` field indicates the remaining display size of the deleted order | **T**rade records normalized from *Order Executed* will have `order_id` set to `0`, as Cboe does not provide information about the aggressor's order. **T**rade records normalized from *Order Executed* will have `side` set to the opposite side of the matched resting order. **Trade messages** CFE Multicast PITCH also has a `Trade` message for providing execution details for match events that occur off the CFE book, such as [ECRP or Block trades](https://cdn.cboe.com/resources/membership/cfe-block-ecrp-trades.pdf). Databento normalizes this *Trade* message as a single MBO **T**rade record with side **N**one and publisher set to `XCBF.PITCH.XOFF`. Cboe sends `Trade` messages with an obfuscated Order Id, this is passed through in the normalized **T**rade record. ### Statistics normalization For the `XCBF.PITCH` dataset, statistics records are normalized from the following CFE messages. | CFE Message | StatType(s) | | Price Limits | Upper price limit, lower price limit | | Settlement | Settlement price | | Open Interest | Open interest | | End of Day Summary | Highest bid, trading session high price, lowest offer, trading session low price, opening price, close price, cleared volume, block volume, venue-specific volume 1 | **Note**: Fields `Accrued Day Variance`, `Num Final Returns`, and `Num Elapsed Returns` from the Futures Variance Symbol Mapping Message are not currently normalized. Fields from these messages are normalized into records in the [statistics schema](https://databento.com/docs/schemas/statistics.md) with the appropriate `StatType`. **Price Limits** CFE publishes `Upper Price Limit` and `Lower Price Limit` in a `Price Limits` Message. These fields are normalized into [statistics](https://databento.com/docs/schemas/statistics.md) records with a `StatType` of upper price limit and lower price limit respectively, as prices. **Settlement** CFE publishes `Settlement Price` in a `Settlement` Message. This field is normalized into a [statistics](https://databento.com/docs/schemas/statistics.md) record with a `StatType` of settlement price, as a price. **Open Interest** CFE publishes `Open Interest` in an `Open Interest` Message. This field is normalized into a [statistics](https://databento.com/docs/schemas/statistics.md) record with a `StatType` of open interest, as a quantity. **End of Day Summary** The `End of Day Summary` message is sent after trading ends for a symbol. ***Quantity Fields*** `Total Volume` represents the total number of contracts traded for the day including [Block and ECRP trades](https://cdn.cboe.com/resources/membership/cfe-block-ecrp-trades.pdf). `Block Volume` represents the total number of Block and Derived contracts traded for the day. `ECRP Volume` represents the total number of ECRP contracts traded for the day. These fields are normalized as [statistics](https://databento.com/docs/schemas/statistics.md) records as quantities, with the following `StatType`s: | CFE Field | StatType | | Total Volume | Cleared volume | | Block Volume | Block volume | | ECRP Volume | Venue-specific volume 1 | ***Price Fields*** `High Price` is the higher of highest bid and highest trade price for the day. `Low Price` is the lower of lowest offer and lowest trade price for the day. `Open Price` is set by the first trade of the day (in any session). `Close Price` is set by the last trade of the day (in any session). [Block and ECRP trades](https://cdn.cboe.com/resources/membership/cfe-block-ecrp-trades.pdf) do not affect these price fields. | CFE Field | StatType | | High Price (if from bid) | Highest bid | | High Price (if from trade) | Trading session high price | | Low Price (if from offer) | Lowest offer | | Low Price (if from trade) | Trading session low price | | Open Price | Opening price | | Close Price | Close price | ### Status normalization `Trading Status` messages are normalized into records in the [status schema](https://databento.com/docs/schemas/status.md). | CFE Trading Status | `action` | `is_trading` | `is_quoting` | | Suspended | `SUSPEND` | No | No | | Queuing | `PRE_OPEN` | No | Yes | | Trading | `TRADING` | Yes | Yes | | Halted | `HALT` | No | No | ### Definition normalization Definition records are sourced from *Futures Instrument Definition*, *OOF Symbol Mapping*, and *Complex Instrument Definition Expanded* messages. The current lists of CFE products can be downloaded from the [CFE Contract Listings and Product Level System Parameters](https://www.cboe.com/markets/us/futures/product-level-system-parameters) on the Cboe page. * [Simple Futures](https://cdn.cboe.com/data/us/futures/market_statistics/symbol_reference/latest-simple.csv) * [Variance Futures](https://cdn.cboe.com/data/us/futures/market_statistics/symbol_reference/va-latest-simple.csv) * [Complex Futures](https://cdn.cboe.com/data/us/futures/market_statistics/symbol_reference/latest-complex.csv) * [Options on Futures](https://cdn.cboe.com/data/us/futures/market_statistics/symbol_reference/latest-simple-options.csv) * [Complex Options on Futures](https://cdn.cboe.com/data/us/futures/market_statistics/symbol_reference/latest-complex-options.csv) **Instrument IDs** CFE publishes instrument IDs as a six character base-62 ASCII value. Databento normalizes this to instrument ID by converting from base62 to decimal base 10. For example: | CFE Symbol | `instrument_id` | | "000000"b62 | 0 | | "000001"b62 | 1 | | "00000z"b62 | 61 | | "000100"b62 | 3844 (62*62) | Complex Options on Futures symbols start at values "z00000"b62, which is 55884102752. The maximum base-10 value that can be represented by a six-char base62 string is "zzzzzz" which is 56800235583. The Databento `instrument_id` field is 32-bit unsigned, which has a maximum value of 4294967295. Because of this, values above 4000000000 are used for normalization of Complex Option on Futures symbols. Values of "z00000"b62 and above are mapped to offset 4000000000. | CFE Symbol | `instrument_id` | | "z00000"b62 | 4000000000 | | "z00001"b62 | 4000000001 | | "z0000z"b62 | 4000000061 | **Futures Instrument Definition Message** | Databento Field | CFE Field | Description | | `instrument_id` | Symbol | The mapped value from base62 | | `raw_instrument_id` | Symbol | The original decoded base62 value | | `ts_event` | Unit Timestamp | Used only if non-zero, otherwise populated from TimeOffset | | `asset` | Report Symbol | | | `expiration` | Expiration Date | | | `original_contract_size` | Contract Size | | | `min_price_increment` | Price Increment | | | `leg_count` | Leg Count | | | `maturity_year` | Contract Date | Zero for spread instruments | | `maturity_month` | Contract Date | Zero for spread instruments | | `maturity_day` | Contract Date | Zero for spread instruments | | `leg_ratio_qty_numerator` | Leg Ratio | Absolute value of the leg ratio | | `leg_ratio_qty_denominator` | Leg Ratio | Always 1 | | `leg_side` | Leg Ratio | Positive ratio=`BID`, negative ratio=`ASK` | | `leg_instrument_id` | Leg Symbol | Converted from base62 | The following fields are not normalized: `Futures Flags`, `Listing State`, `Leg Offset`. The `raw_symbol` for Simple Futures attempts to match the Cboe "Product". This field is not published in the **Futures Instrument Definition** message. Instead it is constructed from the **Report Symbol** and **Contract Date** fields. The format is `[asset]/[maturity_month_code][maturity_year_code]`. Where these fields are derived from: Month Codes: | Month | Code | | January | F | | February | G | | March | H | | April | J | | May | K | | June | M | | July | N | | August | Q | | September | U | | October | V | | November | X | | December | Z | Maturity year uses the last digit of the year for maturities before 2030. Two digits are used for maturities from 2030 onward. Examples: | Asset | Maturity month | Maturity year | Symbol | | VX | January | 2026 | `VX/F6` | | PBT | December | 2035 | `PBT/Z35` | The raw symbol for future spreads is determined by the legs. The format is: `LegSymbol:Ratio:Side - LegSymbol:Ratio:Side`. For example, for a spread with ratio 1:1, with legs `VX/J6` and `VX/H6` the resulting spread `raw_symbol` will be `VX/H6:1:S - VX/J6:1:B` **VX Weekly Expirations** VX weekly futures use a different **Report Symbol** from the VX futures product. For these weekly contracts, CFE uses VX01 through VX53. The embedded suffix denotes the specific week of a calendar year during which a contract is settled. Each VX weekly expiration has a corresponding [parent contract](https://databento.com/docs/conventions/symbology.md#parent). Examples: | Asset | Maturity week | Maturity month | Maturity year | Symbol | | VX | Week 1 | January | 2026 | `VX01/F6` | | VX | Week 12 | March | 2052 | `VX12/Z52` | **OOF Symbol Mapping Message** | Databento Field | CFE Field | Description | | `ts_event` | Time Offset | | | `instrument_id` | Feed Symbol | The mapped value from base62 | | `raw_instrument_id` | Feed Symbol | The original decoded base62 value | | `group` | Futures Product | The underlying futures product root (e.g. `VX`) | | `underlying_id` | Futures Symbol | Converted from base62 | | `strike_price` | Strike Price | | | `instrument_class` | Call Put Indicator | | | `expiration` | Options Expiration | | | `raw_symbol` | Options on Futures Name | | The following fields are not normalized: `Futures Expiration`, `Symbol Condition`. The `asset` field is derived from the `raw_symbol` by extracting the root up to the first `/` character. The `security_type` field is set to `OOF`. The `raw_symbol` comes from the `Options on Futures Name` field, which follows the Cboe US Symbology definition. See section *CFE Options on Futures Symbology* of the [Cboe US Symbology Reference](https://cdn.cboe.com/resources/membership/US_Symbology_Reference.pdf). **Complex Instrument Definition Expanded Message** | Databento Field | CFE Field | Description | | `ts_event` | Time Offset | | | `instrument_id` | Complex Instrument Id | The mapped value from base62 | | `raw_instrument_id` | Complex Instrument Id | The original decoded base62 value | | `group` | Complex Instrument Underlying | The underlying futures product root (e.g. `VX`) | | `leg_count` | Leg Count | | | `leg_instrument_id` | Leg Symbol | Converted from base62 | | `leg_ratio_qty_numerator` | Leg Ratio | Absolute value of the leg ratio | | `leg_ratio_qty_denominator` | Leg Ratio | Always 1 | | `leg_side` | Leg Ratio | Positive ratio=`BID`, negative ratio=`ASK` | The following fields are not normalized: `Complex Instrument Type`. The `instrument_class` is set to `OPTION_SPREAD` and `security_type` is set to `OOF`. The `asset` field is determined from the leg definitions. The `raw_symbol` for complex options is determined by the legs, using the same format as complex futures: `LegSymbol:Ratio:Side - LegSymbol:Ratio:Side`. For example, for an option spread with ratio 1:1 and legs `UX4E/J6 C3000` and `UX4E/J6 C4000`, where both sides are `BID`, the resulting `raw_symbol` would be `UX4E/J6 C3000:1:B - UX4E/J6 C4000:1:B` ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Databento Core Indices - CGIF **Dataset ID**: [DBIX.CORE]($PLATFORM_HOSTNAME/catalog/dbix/DBIX.CORE) `DBIX.CORE` consolidates index data from multiple upstream feeds into a single dataset, organized by `venue`. It's an aggregate of index feeds rather than an order-driven market, so Databento does not offer an [MBO schema](https://databento.com/docs/schemas/mbo.md) for `DBIX.CORE`. This page covers the channels sourced from Cboe's Titanium Cboe Global Indices Feed (CGIF). ### Channels Cboe disseminates index values, contributor values, and index summaries from several index providers through CGIF, which Databento receives over UDP multicast in our NY4 data center and normalizes into `DBIX.CORE`. CGIF carries values from seven index providers, each its own channel, mapped to a distinct `Publisher` within `DBIX.CORE`: | Channel | Publisher (ID) | Provider | Source messages | | `MAIN` | `DBIX.CORE.MAIN` (146) | Cboe Global Indices Feed | `IndexValue` | | `MSCI` | `DBIX.CORE.MSCI` (147) | MSCI | `IndexValue` | | `FTSE` | `DBIX.CORE.FTSE` (148) | FTSE Russell | `IndexValue` | | `INAV` | `DBIX.CORE.INAV` (149) | Intraday Net Asset Values | `IndexValue` | | `MSTAR` | `DBIX.CORE.MSTAR` (150) | Morningstar | `ContributorValue`, `IndexSummary` | | `CCCY` | `DBIX.CORE.CCCY` (151) | Crypto Currency Indices | `IndexValue` | | `CGI` | `DBIX.CORE.CGI` (152) | Cboe Global Indices | `IndexValueWithStatus` | Index status is normalized to the [status](https://databento.com/docs/schemas/status.md) schema, and index definitions to [instrument definitions](https://databento.com/docs/schemas/instrument-definitions.md), the same way across all seven channels. ### Timestamps CGIF messages include a `SendingTime` in the packet header, shared by every message in that packet. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | CGIF Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | `SendingTime` (packet header) | The gateway-sending timestamp, shared by every message in the same packet. | | `ts_in_delta` | N/A | The delta between `ts_recv` and `ts_event`, or between `ts_recv` and `TransactTime` for `MSTAR`. | Note that `SendingTime` only carries millisecond resolution, and no additional precision is added when normalizing it to a nanosecond `ts_event`. For `MSTAR`, messages also include a `TransactTime`, a millisecond-resolution epoch timestamp present on both `ContributorValue` and `IndexSummary` messages. `TransactTime` is the source for `ts_in_delta` on `MSTAR`'s [MBP-1](https://databento.com/docs/schemas/mbp-1.md), [trades](https://databento.com/docs/schemas/trades.md), and [statistics](https://databento.com/docs/schemas/statistics.md) records; it's more accurate than the packet-level `SendingTime` used for the other channels' `ts_in_delta`. For `IndexSummary` messages, the [statistics schema](https://databento.com/docs/schemas/statistics.md)'s `ts_ref` field is set to the summary's effective `Date` (a `YYYYMMDD` value) decoded to UTC midnight, rather than a precise event time. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBP-1 and Trades normalization CGIF does not publish order book depth, so each source message is normalized to a single-level [MBP-1](https://databento.com/docs/schemas/mbp-1.md) record: the `Bid` and `Ask` values become `bid_px` and `ask_px` at the top book level, the index value becomes `price`, and `action` is fixed to Trade. Every MBP-1 record also produces a matching [trades](https://databento.com/docs/schemas/trades.md) record, copying over `ts_recv`, `ts_event`, `ts_in_delta`, `price`, `flags`, and `action`, with `depth` fixed to `0`. **MAIN, MSCI, FTSE, INAV, CCCY** The source message is `IndexValue`, and `flags` is fixed to `F_LAST`. **CGI** The source message is `IndexValueWithStatus`, but only entries without a `Close` or `Settlement` value are normalized this way; see Statistics normalization below for the rest. `F_LAST` is always set. `F_PUBLISHER_SPECIFIC` indicates an indicative index value: set when `IndexStatus` is `Indicative`, and unset when `Normal`. **MSTAR** The source message is `ContributorValue`, and `flags` is fixed to `F_LAST`. ### Statistics normalization Only `CGI` and `MSTAR` produce [statistics schema](https://databento.com/docs/schemas/statistics.md) records; the other CGIF channels have no source messages that normalize to statistics. **CGI** `CGI`'s `IndexValueWithStatus` entries that carry a `Close` or `Settlement` value are normalized to statistics instead of MBP-1/trades: | CGIF MDEntryType | `stat_type` | `stat_flags` | | `Close` | `CLOSE_PRICE`, or `INDICATIVE_CLOSE_PRICE` when `IndexStatus` is `Indicative` | Unset | | `Settlement` | `SETTLEMENT_PRICE` | `1` when `IndexStatus` is `Normal`, or `0` when `Indicative` | `CGI` only carries `IndexValueWithStatus` messages - it doesn't receive `IndexSummary` messages, so it never produces the start-of-day and end-of-day statistics that `MSTAR` does. **MSTAR** `MSTAR`'s `IndexSummary` messages are always normalized to statistics, one record per `MDEntryType` entry. `stat_type` is always `VENUE_SPECIFIC_PRICE_1`. `stat_flags` is a bit field that depends on the entry's `MDEntryType` and whether `SummaryType` is start-of-day (`SOD`) or end-of-day (`EOD`): | Value | Decimal | Description | | `1 << 0` | 1 | `IndexValue` entry, `SummaryType` is `SOD`. | | `1 << 1` | 2 | `Bid` entry, `SummaryType` is `SOD`. | | `1 << 2` | 4 | `Ask` entry, `SummaryType` is `SOD`. | | `1 << 3` | 8 | `IndexValue` entry, `SummaryType` is `EOD`. | | `1 << 4` | 16 | `Bid` entry, `SummaryType` is `EOD`. | | `1 << 5` | 32 | `Ask` entry, `SummaryType` is `EOD`. | `MSTAR` doesn't receive `IndexValueWithStatus` messages, so it never produces the Close and Settlement statistics that `CGI` does. ### Definition normalization Index definitions are normalized to [instrument definitions](https://databento.com/docs/schemas/instrument-definitions.md) the same way across all seven channels: | Databento Field | CGIF Field | Description | | `raw_symbol` | `Symbol` | Normalized verbatim. | | `group` | `Agent` | A 2-character classification code. | | `secsubtype` | `Channel` | The CGIF channel that index data is received on. | | `exchange` | N/A | Always set to the literal `CGIF`. | | `instrument_class` | N/A | Always `INDEX`. | | `match_algorithm` | N/A | Always `UNDEFINED`, since CGIF is an index feed with no matching engine. | ### Status normalization The corresponding [status schema](https://databento.com/docs/schemas/status.md) record is derived from the definition's `Status` field: | CGIF Trading Status | `action` | `is_trading` | `is_quoting` | | Active | `TRADING` | Yes | Yes | | Inactive | `CLOSE` | No | No | | Deleted | `SUSPEND` | No | No | # CME Globex MDP 3.0 **Dataset ID**: [GLBX.MDP3]($PLATFORM_HOSTNAME/catalog/cme/GLBX.MDP3) The CME Group disseminates full depth-of-book data for CME, CBOT, NYMEX, and COMEX through their MDP 3.0 feed, which Databento receives over UDP multicast in our DC3 datacenter. Databento provides full coverage of all futures and options on futures beginning in June 2010. In May 2017, CME introduced MDP 3.0, which provides full granularity for every order event (MBOFD) in addition to providing aggregated depth at a limited number of price levels (MBP). Before May 2017, data was based on the legacy FIX/FAST protocol. CME makes the specifications of their MDP 3.0 feed available [on their website here](https://www.cmegroup.com/confluence/display/EPICSANDBOX/CME+MDP+3.0+Market+Data). ### Timestamps CME MDP 3.0 messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | MDP 3.0 Tag | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | 60-TransactTime | The matching-engine-received timestamp. | | `ts_in_delta` | 52-SendingTime | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization **`F_LAST` flag in MBO** When interpreting MBO data, the `F_LAST` [flag](https://databento.com/docs/conventions/common-fields-enums-types.md#flags) (`0x80`, 128) is used to mark the last record in a single event for each `instrument_id`. This flag is based on CME's "End of event" flag in tag 5799-MatchEventIndicator. However, CME only sets this flag on one single message, even if the event spanned multiple instruments. Instead, we set the `F_LAST` flag on the last record for _each_ instrument, so that the data can be interpreted consistently with any subset of instruments. Records with `action` **N**one may carry `F_LAST`. Across all of Databento's feeds, it is important to use the `F_LAST` flag when calculating the best bid and offer, like our MBP-1 schema. Between records *without* the `F_LAST` flag, the book is in the process of updating, and the apparent best bid and offer may have already been traded. This is accounted for when we construct our other schemas, such as MBP-1 and Trades. Outside of MBO, this flag can be ignored. **Order events** Records in the [MBO schema](https://databento.com/docs/schemas/mbo.md) with **A**dd, **M**odify, and **C**ancel actions are normalized from [Market Data Incremental Refresh - MBOFD](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+MBOFD), and records with **T**rade and **F**ill actions are normalized from [Market Data Incremental Refresh - Trade Summary](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Trade+Summary) messages, and records with the clea**R** action are normalized from [Market Data Incremental Refresh - Channel Reset](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Channel+Reset) messages. **Trade events** The MBO schema includes detailed information about both sides of the trade by normalizing [Market Data Incremental Refresh - Trade Summary](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Trade+Summary) messages from CME into **T**rade and **F**ill actions. The [MBP-n and trades schemas](https://databento.com/docs/schemas/mbp-1.md) include **T**rade records, but do not include **F**ill detail records. Each CME MDP 3.0 [Market Data Incremental Refresh - Trade Summary](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Trade+Summary) message from CME contains two repeating groups: a set of *trade summaries*, followed by a set of *order ID entries*. Each trade summary in the message is normalized into one **T**rade record. If there is a defined aggressor for the trade, its corresponding order ID entry from the second set is used to populate the `order_id` field. This commonly happens when an incoming order partially executes, leaving the remaining quantity to rest on the book. Following each **T**rade record, the remaining (passive) order ID entries are normalized into **F**ill records. These records correspond to resting orders in the order book, but **do not** modify the orders. If the aggressing order existed on the book, then a **F**ill record will be emitted for it. This can happen if a resting order was modified to cross the book. If the aggressing order did not already exist on the book then its **F**ill record is suppressed, however its order ID is still reported in the **T**rade record as described above. **Side** [Implied trades](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Implied+Book) may not have an aggressing side set. These trades will be normalized with [side](https://databento.com/docs/conventions/common-fields-enums-types.md#side) set to **N**one. **Order Priority** Although Databento does not expose tag 37707-MDOrderPriority in the MBO schema, first in, first out priority (FIFO) can be determined from message order. Messages for a single `instrument_id` are never reordered, even if they have the same timestamps. Our daily MBO snapshots also preserve FIFO order priority. **CME MBO snapshot** At the beginning of the weekly trading session, CME publishes a MBO order snapshot. This snapshot contains any orders that persisted from the previous trading session, such as Good 'Till Cancelled orders. The event timestamp (`ts_event`) for these orders corresponds to the time the snapshot is generated by CME. It does not reflect the original time the order was entered or last modified. These orders are not published by CME in priority order. As Databento does not expose tag 37707-MDOrderPriority, if these MBO records were published by Databento in the same order they were sent from CME, it would not be possible to determine the correct order priority. In order to publish the records in priority order, Databento buffers the orders from the first CME event of the trading session, per instrument. Once the End of Event is reached, Databento sorts the records in priority order and then publishes them. Therefore, the order priority is correctly reflected by the order of publishing. The `ts_recv` for these records will be set to the `ts_recv` of the final record of the event. As the records have been slightly delayed and the `ts_recv` of the initial records may not match the original `ts_recv`, the flag `F_BAD_TS_RECV` is set on these records. The `F_SNAPSHOT` flag is also set on these records to indicate that they are from the CME snapshot. **Weekly Sessions** Although CME pauses trading daily, the market follows a weekly session structure with most instruments accepting orders from Sunday night to Friday night (local time). To make it easier to work with historical MBO data, Databento includes an [order book snapshot](https://databento.com/docs/conventions/mbo-snapshot.md) at 00:00:00 UTC each weekday (Monday-Friday). Before the trading session begins, or during the daily pause, it is normal for order books to be locked or crossed: CME will accept orders but won't execute any trades until the opening uncrossing. Details about the trading hours and sessions for each product group are available on the [Trading hours](https://www.cmegroup.com/trading-hours.html) page of CME's website. ### Implied book In futures and options markets, implied depth allows for extra liquidity beyond what's visible in the order book for a specific instrument. It is derived from the relationships between different instruments, such as between outrights, spreads, or multi-leg options. CME's matching engine uses implied depth to construct liquidity, ensuring that even if a direct order isn't placed, the system can infer potential liquidity from other related contracts. CME's MDP3 feed has support for implied orders over their MBP feed. The MBP feed provides up to 10 levels of real liquidity and 2 levels of implied liquidity. Databento combines the real and implied books, publishing the merged top-of-book on the [CMBP-1](https://databento.com/docs/schemas/mbp-1.md#consolidated-market-by-price-cmbp-1) schema. Databento also provides subsampled views of the merged top-of-book on the [CBBO-1s and CBBO-1m](https://databento.com/docs/schemas/bbo.md#consolidated-bbo-on-interval-cbbo) schemas. CME provides [documentation and examples](https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSANDBOX/pages/457095346/Implied+Orders) for the implied book. ### Statistics normalization Databento normalizes the following daily statistics from [Market Data Incremental Refresh - Daily Statistics](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Daily+Statistics): - Settlement price - Cleared volume - Open interest - Fixing price The exact times these statistics are published will vary. Cleared volume and open interest statistics will normally publish on the following UTC date, unless they are for Friday's session, in which case they will be published on the following Sunday. Settlement price statistics will first publish shortly after the [settlement window](https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSANDBOX/pages/457085528/Daily+Settlement+Time+Details) for an instrument. The CME does not publish settlement prices on their MDP feed for instruments without open interest or volume. Databento normalizes the following session statistics from [Market Data Incremental Refresh - Session Statistics](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Incremental+Refresh+-+Session+Statistics): - Opening and indicative opening price - Trading session high and low price - Trading session highest bid and lowest offer These statistics are published throughout the trading session as the values change. **Price limits** Databento normalizes the `HighLimitPrice` and `LowLimitPrice` fields of the Limits Banding message into upper price limit and lower price limit statistics. Both statistics are published on every limits banding update, using tag 60-TransactTime as `ts_event`. **Reference timestamp (`ts_ref`)** CME includes tag 5796-TradingReferenceDate, the trading session date, with daily statistics. This field is normalized to `ts_ref` as a nanosecond-precision UNIX timestamp for consistency with other timestamps. Because the source only has date precision, users should avoid localizing `ts_ref` to their time zone. **Multiple records for the same date** CME often publishes multiple messages for the same trading session date (indicated by `ts_ref`) and daily statistic. The initial messages indicate preliminary values, and the final message for the trading session date indicates the final value. The final message is always the most accurate and the one you should use, when available. **Stat flags** Settlement price statistics normalize the [bit field](https://en.wikipedia.org/wiki/Bit_field) [tag 731-SettlPriceType](https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Settlement+Price) to the `stat_flags` field. | Value | Decimal | Description | | `1 << 0` | 1 | Whether the settlement price is final, as opposed to preliminary. | | `1 << 1` | 2 | Whether the settlement price is actual, as opposed to theoretical. | | `1 << 2` | 4 | Some products have a different trading tick size than their clearing tick size. 1 if settling at the trading tick and 0 if at the clearing tick. | | `1 << 3` | 8 | Whether the price is an intraday settlement price and disseminated before the official end-of-day settlement calculation. | [The CME documentation on settlement prices](https://www.cmegroup.com/confluence/display/EPICSANDBOX/Settlement+Prices#SettlementPrices-SettlementatTradingTick/SettlementatClearingTick) expands on the meaning of each flag value. Settlement prices before 2015-11-20 follow different rules, described under [MDP 2 data](https://databento.com/docs/datasets/glbx-mdp3.md#mdp-2-data). ### Symbology Databento normalizes the `asset` field from tag 6937-Asset of the [Market Data Security Definition](https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSANDBOX/pages/457672532/MDP+3.0+-+Security+Definition) message. The `asset` field is used when requesting data with [parent symbology](https://databento.com/docs/conventions/symbology.md#parent). Databento normalizes the `raw_symbol` field from tag 55-Symbol of the Security Definition message. This symbol contains the product code, followed by the [month code](https://www.cmegroup.com/month-codes.html), followed by the year. Previously, this field would only contain a 1-digit year, e.g. `ESZ3` for the E-mini S&P 500 December 2023 contract. In some recently listed contracts, CME has begun using a 2-digit year, e.g. `NGN25` for the Henry Hub Natural Gas July 2025 contract. ### Definition normalization **FX spots** CME lists foreign exchange spot instruments alongside futures. Databento sets `instrument_class` to `X` (FX spot) for instruments whose CFI code begins with `IF`. The `cfi` and `security_type` fields are passed through from CME unchanged. **Instrument attributes** CME publishes a [bit field](https://en.wikipedia.org/wiki/Bit_field) of instrument attributes in tag 872-InstAttribValue, normalized to the `inst_attrib_value` field. | Value | Description | | `1 << 0` | Whether the instrument is eligible for electronic matching. | | `1 << 1` | Whether the instrument is eligible for order cross. | | `1 << 2` | Whether the instrument is eligible for block trades. | | `1 << 3` | Whether the instrument is EFP eligible. | | `1 << 4` | Whether the instrument is EBF eligible. | | `1 << 5` | Whether the instrument is EFS eligible. | | `1 << 6` | Whether the instrument is EFR eligible. | | `1 << 7` | Whether the instrument is OTC eligible. | | `1 << 8` | Whether the instrument is eligible for iLink indicative mass quoting. | | `1 << 9` | Whether the instrument may have a negative strike price. | | `1 << 10` | Whether the outright instrument may have a negative price. | | `1 << 11` | Whether prices for the instrument are displayed as fractions. | | `1 << 12` | Whether the option is quoted in volatility. | | `1 << 13` | Whether the instrument is eligible for RFQ cross. | | `1 << 14` | Whether the outright instrument may have a price of zero. | | `1 << 15` | Whether the product is a decaying product. | | `1 << 16` | Whether the product is a variable product. | | `1 << 17` | Whether the product is a daily product. | | `1 << 18` | Whether the instrument accepts good-til orders. | | `1 << 19` | Whether the instrument is eligible for implied matching. | | `1 << 20` | Whether the instrument is eligible for triangulation. | | `1 << 21` | Whether the instrument is eligible for a variable cabinet price. | | `1 << 22` | Whether the instrument's book is inverted. | | `1 << 23` | Whether the instrument is all-or-none. | | `1 << 24` | Whether the instrument is SEF regulated. | | `1 << 25` | Whether the instrument is MTF regulated. | | `1 << 26` | Whether the instrument is an EFIX instrument. | | `1 << 27` | Whether the instrument is a hedge instrument. | [The CME MDP 3.0 documentation](https://www.cmegroup.com/confluence/display/EPICSANDBOX/CME+MDP+3.0+Market+Data) expands on the meaning of each attribute. `1 << 11` pertains to how a price is displayed and marks instruments CME quotes in the exchange's [fractional convention](https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSANDBOX/pages/457092591/Fractional+Pricing), and pairs with the `main_fraction`, `sub_fraction`, and `price_display_format` fields. The [Displaying fractional prices](https://databento.com/docs/examples/instrument-definitions/fractional-prices.md) example renders prices using these fields. ### Spreads and combos In addition to futures and options on futures, this dataset also contains data for both exchange-listed spreads and [user-defined spreads (UDS)](https://www.cmegroup.com/confluence/display/EPICSANDBOX/User+Defined+Spread+-+UDS). Spreads and combinations consisting only of futures (such as exchange-listed calendar spreads) are considered part of the futures [parent symbols](https://databento.com/docs/conventions/symbology.md#parent). For example, `ESH4-ESZ4` is a calendar spread contained in `ES.FUT`. These spreads can have negative prices. Spreads and combinations containing options (such as butterfly or covered spreads) are considered part of the option [parent symbols](https://databento.com/docs/conventions/symbology.md#parent). For example, `UD:1V: VT 2533938` is a vertical option spread contained in `ES.OPT`. [The CME documentation on spreads and combinations](https://www.cmegroup.com/confluence/display/EPICSANDBOX/Spreads+and+Combinations+Available+on+CME+Globex) contains an exhaustive list of the different spread types. The `instrument_class` field in the definition schema can be used to identify spread instruments generically. The `SecuritySubType` field in the exchange documentation is normalized to the `secsubtype` field in the `definition` schema and can be used to identify the exchange-specific spread type. To identify the parent symbol of strategies and user-defined spreads, the `asset` field in the definition schema should be used. **Leg definitions** Databento publishes one definition record per leg of a spread or combination. The records for an instrument are identical apart from the fields beginning with `leg_`, which describe a single leg. `leg_count` is the number of legs and `leg_index` is the 0-based index of the leg the record describes. | Databento field | MDP 3.0 tag | Description | | `leg_count` | 555-NoLegs | The number of legs in the spread. | | `leg_index` | N/A | The 0-based index of the leg within the spread. | | `leg_instrument_id` | 602-LegSecurityID | The instrument ID of the leg instrument. | | `leg_side` | 624-LegSide | The side taken for the leg when buying the spread. | | `leg_ratio_qty_numerator` | 623-LegRatioQty | The quantity of the leg instrument per unit of the spread. | | `leg_ratio_qty_denominator` | N/A | Always 1 for this dataset. | | `leg_price` | 566-LegPrice | The tied price of the leg, used for hedged strategies. | | `leg_delta` | 1017-LegOptionDelta | The delta of the leg, used for hedged strategies. | | `leg_raw_symbol` | N/A | Taken from the leg instrument's own definition. | | `leg_instrument_class` | N/A | Taken from the leg instrument's own definition. | | `leg_underlying_id` | N/A | Taken from the leg instrument's own definition. | The `leg_ratio_price_numerator` and `leg_ratio_price_denominator` fields are not populated for this dataset. CME can publish the definition of a spread before the definitions of its legs. Until the leg instrument's own definition arrives, the fields taken from it are unset. Databento then republishes the affected leg records with `security_update_action` set to **M**odify. ### Status normalization Status records are normalized from [Market Data Security Status](https://cmegroupclientsite.atlassian.net/wiki/x/JbtAGw) messages. A single status message from CME may update the status of all instruments in a `group`, all instruments in a `group` and `asset`, or an individual instrument. To support requesting data for individual instruments, Databento normalizes all these messages into instrument-level status messages. When requesting status schema data for multiple instruments in the same `group` or `asset`, there will be status records that are identical minus the `instrument_id`. The state fields of the status schema: `is_trading`, `is_quoting`, and `is_short_sell_restricted` are based solely on the status updates from CME. Group-level status updates generate a status record for each instrument with a definition, regardless of activation or expiration. As a result, the status state fields (`is_trading` and `is_quoting`) may be incorrect before activation or after expiration. ### Matching Algorithms CME implements various [matching algorithms](https://databento.com/docs/schemas/instrument-definitions.md#matching-algorithm) across its universe of products, with **F**IFO being the most common. Specific details on the different algorithms can be found on the [Supported Matching Algorithms page](https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSANDBOX/pages/457218479/Supported+Matching+Algorithms) on the CME website. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. ### MDP 2 data On 2017-05-21, CME introduced full granularity for every order event (MBOFD). Data prior to this date is based on the MDP 2, level-aggregated FIX protocol. This leads to some differences in the data. **Timestamps** MDP 2 data is sourced from FIX flat files. Because of this, capture timestamps (commonly available as `ts_recv`) are not available. The `ts_recv` field on all MDP 2 data is set to the same value as `ts_event`. Because of this, all records on MDP 2 data have the `F_BAD_TS_RECV` flag set. This is normal and expected. Nanosecond-resolution timestamps have been introduced by CME on 2015-11-20. Timestamps prior to that date are limited to millisecond resolution. | Databento Field | CME Tag | Description | | `ts_recv` | 52-SendingTime | The matching-engine-sending timestamp. | | `ts_event` | 52-SendingTime | The matching-engine-sending timestamp. | **MBO format** MDP 2 data predates the introduction of full-depth MBO by CME. Because of this, the MBO schema is not available. The highest granularity schema available for MDP 2 data is MBP-10. **CMBP-1 format** The CMBP-1, CBBO-1s, and CBBO-1m schemas are not available for MDP 2 data. **Settlement prices** Before 2015-11-20, CME sent the settlement price type in tag 286-OpenCloseSettlFlag instead of tag 731-SettlPriceType. For products not subject to settlement price rounding, a `stat_flags` value of 4 marks an unrounded preliminary price rather than settlement at the trading tick. Final settlement prices before 2015-11-20 come from the weekly security definition replay CME sent each Sunday, which covers the last trading session of the week. From 2015-11-20, CME sent tag 731-SettlPriceType on daily updates, which include final settlement prices. **Status schema** There's significantly less status data prior to 2015; this is expected. Before November 2015, status changes from the normal trading schedule did not result in a status message. # Databento US Equities Mini **Dataset ID**: [EQUS.MINI]($PLATFORM_HOSTNAME/catalog/us-equities#EQUS.MINI) Databento US Equities Mini is a derived top-of-book dataset, sourced from the proprietary market data feeds of both alternative trading systems (ATS) and RegNMS venues. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`); in this dataset, `raw_symbol` and the `nasdaq` stype are equivalent. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Schemas The following schemas are available for this dataset: [MBP-1](https://databento.com/docs/schemas/mbp-1.md), [BBO](https://databento.com/docs/schemas/bbo.md), [TBBO](https://databento.com/docs/schemas/tbbo.md), [Trades](https://databento.com/docs/schemas/trades.md), [OHLCV-1s](https://databento.com/docs/schemas/ohlcv.md), [OHLCV-1m](https://databento.com/docs/schemas/ohlcv.md), [OHLCV-1h](https://databento.com/docs/schemas/ohlcv.md), [OHLCV-1d](https://databento.com/docs/schemas/ohlcv.md), and [Definitions](https://databento.com/docs/schemas/instrument-definitions.md). ### Aggregated BBO Databento US Equities Mini is an aggregated dataset, quotes are not provided for the individual component venues. A single aggregated BBO is printed for each instrument. To provide this aggregated BBO, Databento calculates the best bid and best ask prices across the component venues of the feed, adding together all quantity (including odd lots) at the price of the BBO regardless of venue. We do not aggregate the order counts across the component venues, so `bid_ct_00` and `ask_ct_00` will always be 0. ### Venues To qualify as a derived dataset, trade and BBO data are anonymized from the original venues' data. Due to this, the `publisher_id` of a trade does not identify the original venue of the trade, and will always have the value `EQUS.MINI.EQUS`. The `sequence` field is set to 0 on all messages. ### Aggregated OHLCV The OHLCV schemas for Databento US Equities Mini aggregate prices and volume across all the component venues. A single aggregated OHLCV message is printed per instrument, per interval. ### Timestamps Messages received by Databento from component venues will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The `ts_in_delta` field is not populated for this dataset and will always be set to 0. | Databento Field | Description | | `ts_recv` | The capture-server-received timestamp. | | `ts_event` | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Databento US Equities Summary **Dataset ID**: [EQUS.SUMMARY]($PLATFORM_HOSTNAME/catalog/us-equities#EQUS.SUMMARY) Nasdaq disseminates consolidated volume and end-of-day statistics via their Nasdaq NLS+ feed. Databento receives these over UDP multicast in our NY4 data center. PDF specifications are available on Nasdaq's website for the [NLS+](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf) feed. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`); in this dataset, `raw_symbol` and the `nasdaq` stype are equivalent. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Schemas Only the `ohlcv-1d`, `statistics`, and `definition` schemas are available. Databento normalizes the consolidated volume provided by the Nasdaq NLS+ feed on every trade into the `statistics` schema. This volume is consolidated across all US exchanges. Databento normalizes the end-of-day summary provided by the Nasdaq NLS+ feed into the `ohlcv-1d` schema. Databento normalizes *Stock Directory* messages into the `definition` schema. ### Definition normalization Definition records are sourced from NLS+ *Stock Directory* messages. | Databento Field | NLS+ Field | Description | | `exchange` | Market Category | Normalized to the MIC code of the listing venue. All Nasdaq-listed instruments will normalize to `XNAS`. | | `security_type` | Issue Classification | Identifies the security class for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix E [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlusSpecification3.0.pdf). | | `secsubtype` | Issue Sub-Type | Identifies the security sub-type for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix F [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlusSpecification3.0.pdf). | ### Timestamps Nasdaq NLS+ messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | TotalView Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### End-of-day Summary Nasdaq NLS+ disseminates an end-of-day summary via their API at three different times: 1. Once at approximately 16:15 ET 2. Once at approximately 17:00 ET 3. Once at approximately 20:15 ET Each of those summaries reflects the summary at the publishing time. Databento only provides the last summary (from 20:15 ET) on the `ohlcv-1d` schema, which includes the total post-market volume, published as a single OHLCV-1d message per date. This can lead to discrepancies when comparing against other data sources that use the other earlier summaries. The first two preliminary summaries are disseminated by Databento under the `statistics` schema. The `stat_flags` field is set to 1 for the first summary and 2 for the second summary. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Eurex **Dataset ID**: [XEUR.EOBI]($PLATFORM_HOSTNAME/catalog/xeur/XEUR.EOBI) Eurex Exchange disseminates full depth-of-book data for their exchange through their EOBI (Enhanced Order Book Interface) Feed, which Databento receives over UDP multicast in our FR2 data center. The latest [EOBI Specification](https://www.eurex.com/resource/blob/4417682/52249af896465e10295f4203d25fef91/data/T7_R.13.1_%20EOBI_Manual_Version_3.pdf) is available from Eurex website. ### Trading hours The [regular trading hours](https://www.eurex.com/ex-en/trade/trading-hours) for Eurex are: * 08:00 CET to 17:30 CET (European hours) * 17:30 CET to 22:00 CET (U.S. hours) The [extended trading hours](https://www.eurex.com/ex-en/trade/trading-hours) for Eurex are: * 01:00 CET to 08:00 CET (Asian hours) Extended (Asian) trading hours only include a [subset of instruments](https://www.eurex.com/ex-en/trade/trading-hours/thx), including European indices, interest rate futures, and cryptocurrency futures. ### Timestamps Eurex EOBI messages received by Databento include multiple timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). | Databento Field | Eurex EOBI Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | RequestTime, TransactTime | RequestTime (Gateway request in timestamp) when available. TransactTime otherwise. | | `ts_in_delta` | TransactTime from PacketHeader | The matching-engine-sending timestamp. | EOBI `RequestTime` field is not available for the off-market trades (EOBI message `TES Trade Report`), manual and auction trades (EOBI message `Trade Report`), and cross request (EOBI message `Cross Request`). Thus `ts_event` is populated with `TransactTime` field, which corresponds to the time when the transaction was processed by the exchange's matching engine. The `ts_event` for passive trade messages (EOBI messages `Partial Order Execution` and `Full Order Execution`) is populated with the `RequestTime` from the aggressive order. ### Definition normalization Definition records are sourced daily from [Reference Data Interface](https://www.eurex.com/resource/blob/4417678/b4ce595ef44d7959f2aca306eaab85a8/data/T7_R.13.1_%20EMDI_MDI_RDI_Manual_Version_3.pdf) snapshot. Definition records for intra-day instruments are sourced from EOBI incremental feed. | Databento Field | Eurex RDI Field | Description | | `appl_id` | PartitionId | Identifies the instrument's partition. | | `asset` | MarketSegment | Identifies the instrument's market segment. This field is used by parent and continuous [symbology](https://databento.com/docs/conventions/symbology.md). | | `group` | MarketId | Market identifier code specified in ISO 10383. | | `raw_symbol` | SecurityDesc | Instrument identifier (See **Symbology** section below). | | `secsubtype` | ProductComplex | Identifies the type of instrument (See **Symbology** section below). | | `underlying` | MarketSegment | The market segment of the underlying (if there is an underlying). | ### Symbology Raw symbols are sourced from `SecurityDesc` string field. The semantics varying according to the type of instrument (`secsubtype` in the normalized message). The table below describes each instrument type, based on the venue's [documentation](https://www.eurex.com/resource/blob/4417678/b4ce595ef44d7959f2aca306eaab85a8/data/T7_R.13.1_%20EMDI_MDI_RDI_Manual_Version_3.pdf). | Value | Instrument type | Format | | `1` | Simple Instrument | `PROD SI YYYYMMDD SM ES C/P StrikePrice VN` | | `2` | Standard option strategy | `PROD.O.YYMMDD.IST.SEQ-NO` | | `3` | Non-standard option strategy | `PROD.O.YYMMDD.IST.SEQ-NO` | | `4` | Volatility strategy | `PROD.V.YYMMDD.IST.SEQ-NO` | | `5` | Futures spread | `PROD.S.MONYY.MONYY.SPD` | | `6` | Inter-product spread | `PROD.I.YYMMDD.IST.SEQ-NO` | | `7` | Standard futures strategy | `PROD.F.YYMMDD.IST.SEQ-NO` | | `8` | Pack and bundle | `PROD.B.MONYY.IST` | | `9` | Strip | `PROD.T.YYMMDD.SEQ-NO` | | `10` | Flexible instrument | `PROD FI YYYYMMDD SM ES C/P StrikePrice VN` for options. `PROD FI YYYYMMDD SM` for futures. | | `11` | Commodity strip | `PROD.C.YYMMDD.IST.SEQ-NO` | | `12` | Scaled simple instrument | `PROD SI YYYYMMDD SM ES C/P StrikePrice VN QSF` | | `13` | Non-standard volatility strategy | `PROD.D.YYYYMMDD.SEQ-NO` | | `14` | Total return future strategy | `PROD.R.YYYYMMDD.IST.SEQ-NO` | The acronyms from the table above correspond to: * `SM`: Settlement method (`CS` for cash, `PS` for Physical). * `ES`: Exercise style (`EU` for European, `AM` for American). * `VN`: Version number of an option. The version changes as a result of corporate events. * `IST`: Standard strategy type for complex instrument, as detailed in the [Trading parameters document](https://www.eurex.com/ex-en/data/trading-files/product-information). * `SEQ-NO`: Unique sequence number of 6 characters per product. * `QSF`: Quantity scaling factor of scaled simple instruments. * `YYYYMMDD`: Contract date. * `MONYY`: Contract month-year. * `YYMMDD`: Creation date. * `O`, `N`, `V`, `D`, `F`: Identify different strategy types. * `I`: Identify inter-product spreads instruments. * `T`: Identify strip instruments. * `C`: Identify commodity strip instruments. ### Expirations Eurex only provides date-granularity expiration data in `MaturityDate` field from the reference data. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the instrument expires during or after the trading day. Eurex also provides the `MaturityMonthYear` field in the reference data, which corresponds to the instrument expiration month, and may be in a different month than the `MaturityDate` field. The `MaturityMonthYear` field is mapped to the fields `maturity_year` and `maturity_month` in the definition schema (`maturity_day` is not populated). ### Statistics normalization Databento provides the following statistics: * Opening price: First trade price of the trading session. * Trading session high price: Highest trade price of the trading session. * Trading session low price: Lowest trade price of the trading session. * Close price: Last trade price of the trading session. * Cleared volume: Total on-exchange trading volume. * Settlement price: Published during the day, as soon as they are available. Different products have different schedules. * [Adjusted Open Interest](https://www.eurex.com/resource/blob/3974454/ff664ad26b898adf33e78a310bcce01e/data/C7%20Derivatives%20Clearing%20Functional%20Reference%20Guide.pdf): Published daily around 13:30 CE(S)T, it considers the previous day's open interest and adjusts it by subtracting the current day's closed contracts. Trading session high price, low price, close price, and cleared volume are published daily around 22:00:00 UTC, and only account for on-exchange trades. On-exchange trades manually entered by [T7 Market Supervision](https://www.eurex.com/resource/blob/4417682/52249af896465e10295f4203d25fef91/data/T7_R.13.1_%20EOBI_Manual_Version_3.pdf) do not affect price statistics. **Stat flags** Settlement price statistics normalize the field [tag 731-SettlPriceType](https://www.eurex.com/resource/blob/4332354/8a3ed0147bbbb5e5f7452fad509289b5/data/T7_R.13.1_EMDS_-_Trade_Prices-_Settlement_Prices_and_Open_Interest_Data_Manual_Version_1.pdf) to the `stat_flags` field. | Value | Decimal | Description | | `1 << 0` | 1 | Final settlement | | `1 << 1` | 2 | Theoretical (daily) settlement | ### Off-market trades Trades reported in `TES Trade Report` EOBI message are considered off-market. Off-market trades with the price field set to `UNDEF_PRICE` contain the non-disclosed trade volume during the day. Off-market trades have their `publisher_id` set to `XEUR.EOBI.XOFF` to distinguish them from on-exchange trades, which have their `publisher_id` set to `XEUR.EOBI.XEUR`. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `XEUR.EOBI.XEUR` as the `publisher_id` and one with `XEUR.EOBI.XOFF`. ### Order Identifier EOBI does not provide an order identifier (`order_id`) field. Instead, it relies on two fields to uniquely identify an order of a given instrument: `TrdRegTSTimePriority` (priority timestamp) and `Side`. Databento normalizes the `order_id` field in [MBO schema](https://databento.com/docs/schemas/mbo.md) by combining both priority timestamp and side. To extract the priority timestamp from the `order_id`, the most-significant bit can be masked, as below: ```python priority_timestamp = order_id & 0x7FFFFFFFFFFFFFFF ``` ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # European Energy Exchange **Dataset ID**: [XEEE.EOBI]($PLATFORM_HOSTNAME/catalog/xeee/XEEE.EOBI) European Energy Exchange (EEX) disseminates full depth-of-book data for their exchange through their EOBI (Enhanced Order Book Interface) feed, which Databento receives over UDP multicast in our FR2 data center. The `XEEE.EOBI` dataset does not disseminate data for EEX spot market, only for derivatives market. The latest [EOBI Specification](https://www.eurex.com/resource/blob/4417682/52249af896465e10295f4203d25fef91/data/T7_R.13.1_%20EOBI_Manual_Version_3.pdf) is available from Eurex website. ### Trading hours The base [trading hours](https://www.eex.com/fileadmin/EEX/Downloads/Trading/Trading_Hours/20250701_Trading_Hours_on_EEX_Derivatives_Markets_.pdf) on the EEX derivatives take place on weekdays from 08:00:00 CET to 17:00:00 CET. These trading hours can vary depending on the type of market. EEX [Japanese power futures](https://www.eex.com/en/markets/power/japanese-power-market) trading hours are from 16:00 JST to 18:00 JST (08:00 CET to 10:00 CET). [EEX](https://www.eex.com/en/trading-resources/trading-information/calendar) provides holiday calendars across different market types. ### Timestamps EEX EOBI messages received by Databento include multiple timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). | Databento Field | EEX EOBI Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | RequestTime, TransactTime | RequestTime (Gateway request in timestamp) when available. TransactTime otherwise. | | `ts_in_delta` | TransactTime from PacketHeader | The matching-engine-sending timestamp. | EOBI `RequestTime` field is not available for the off-market trades (EOBI message `TES Trade Report`), manual and auction trades (EOBI message `Trade Report`), and cross request (EOBI message `Cross Request`). Thus `ts_event` is populated with `TransactTime` field, which corresponds to the time when the transaction was processed by the exchange's matching engine. The `ts_event` for passive trade messages (EOBI messages `Partial Order Execution` and `Full Order Execution`) is populated with the `RequestTime` from the aggressive order. ### Definition normalization Definition records are sourced daily from [Reference Data Interface](https://www.eurex.com/resource/blob/4417678/b4ce595ef44d7959f2aca306eaab85a8/data/T7_R.13.1_%20EMDI_MDI_RDI_Manual_Version_3.pdf) snapshot. Definition records for intra-day instruments are sourced from EOBI incremental feed. | Databento Field | EEX RDI Field | Description | | `appl_id` | PartitionId | Identifies the instrument's partition. | | `asset` | MarketSegment | Identifies the instrument's market segment. This field is used by parent and continuous [symbology](https://databento.com/docs/conventions/symbology.md). | | `group` | MarketId | Market identifier code specified in ISO 10383. | | `raw_symbol` | SecurityDesc | Instrument identifier (See **Symbology** section below). | | `secsubtype` | ProductComplex | Identifies the type of instrument (See **Symbology** section below). | | `underlying` | MarketSegment | The market segment of the underlying (if there is an underlying). | The `MarketSegment` field from EEX message is truncated to fit into `asset` field in `InstrumentDefMsg` (size of `ASSET_CSTR_LEN`). ### Symbology Raw symbols are sourced from `SecurityDesc` string field. The semantics varying according to the type of instrument (`secsubtype` in the normalized message). The table below describes each instrument type, based on the venue's [documentation](https://www.eurex.com/resource/blob/4417678/b4ce595ef44d7959f2aca306eaab85a8/data/T7_R.13.1_%20EMDI_MDI_RDI_Manual_Version_3.pdf). | Value | Instrument type | Format | | `1` | Simple Instrument | `PROD SI YYYYMMDD SM ES C/P StrikePrice VN` | | `2` | Standard option strategy | `PROD.O.YYMMDD.IST.SEQ-NO` | | `3` | Non-standard option strategy | `PROD.O.YYMMDD.IST.SEQ-NO` | | `4` | Volatility strategy | `PROD.V.YYMMDD.IST.SEQ-NO` | | `5` | Futures spread | `PROD.S.MONYY.MONYY.SPD` | | `6` | Inter-product spread | `PROD.I.YYMMDD.IST.SEQ-NO` | | `7` | Standard futures strategy | `PROD.F.YYMMDD.IST.SEQ-NO` | | `8` | Pack and bundle | `PROD.B.MONYY.IST` | | `9` | Strip | `PROD.T.YYMMDD.SEQ-NO` | | `10` | Flexible instrument | `PROD FI YYYYMMDD SM ES C/P StrikePrice VN` for options. `PROD FI YYYYMMDD SM` for futures. | | `11` | Commodity strip | `PROD.C.YYMMDD.IST.SEQ-NO` | | `12` | Scaled simple instrument | `PROD SI YYYYMMDD SM ES C/P StrikePrice VN QSF` | | `13` | Non-standard volatility strategy | `PROD.D.YYYYMMDD.SEQ-NO` | | `14` | Total return future strategy | `PROD.R.YYYYMMDD.IST.SEQ-NO` | The acronyms from the table above correspond to: * `SM`: Settlement method (`CS` for cash, `PS` for Physical). * `ES`: Exercise style (`EU` for European, `AM` for American). * `VN`: Version number of an option. The version changes as a result of corporate events. * `IST`: Standard strategy type for complex instrument, as detailed in the [Trading parameters document](https://www.eurex.com/ex-en/data/trading-files/product-information). * `SEQ-NO`: Unique sequence number of 6 characters per product. * `QSF`: Quantity scaling factor of scaled simple instruments. * `YYYYMMDD`: Contract date. * `MONYY`: Contract month-year. * `YYMMDD`: Creation date. * `O`, `N`, `V`, `D`, `F`: Identify different strategy types. * `I`: Identify inter-product spreads instruments. * `T`: Identify strip instruments. * `C`: Identify commodity strip instruments. ### Expirations EEX only provides date-granularity expiration data in `MaturityDate` field from the reference data. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the instrument expires during or after the trading day. EEX also provides the `MaturityMonthYear` field in the reference data, which corresponds to the instrument expiration month, and may be in a different month than the `MaturityDate` field. The `MaturityMonthYear` field is mapped to the fields `maturity_year` and `maturity_month` in the definition schema (`maturity_day` is not populated). ### Statistics normalization Databento provides the following statistics: * Opening price: First trade price of the trading session. * Trading session high price: Highest trade price of the trading session. * Trading session low price: Lowest trade price of the trading session. * Close price: Last trade price of the trading session. * Cleared volume: Total on-exchange trading volume. * Final Settlement price: Published daily around 19:25:00 CET. * Open Interest: Published daily around 21:10:00 CET. Settlement price and open interest are only provided for options and futures outrights that are currently active and tradable in EEX's trading system. Settlement price and open interest are not provided for Freight Futures and Options. Trading session high price, low price, close price, and cleared volume are published daily around 22:00:00 UTC, and only account for on-exchange trades. On-exchange trades manually entered by [T7 Market Supervision](https://www.eurex.com/resource/blob/4417682/52249af896465e10295f4203d25fef91/data/T7_R.13.1_%20EOBI_Manual_Version_3.pdf) do not affect price statistics. ### Off-market trades Trades reported in `TES Trade Report` EOBI message are considered off-market. Off-market trades with the price field set to `UNDEF_PRICE` contain the non-disclosed trade volume during the day. Off-market trades have their `publisher_id` set to `XEEE.EOBI.XOFF` to distinguish them from on-exchange trades, which have their `publisher_id` set to `XEEE.EOBI.XEEE`. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `XEEE.EOBI.XEEE` as the `publisher_id` and one with `XEEE.EOBI.XOFF`. ### Order Identifier EOBI does not provide an order identifier (`order_id`) field. Instead, it relies on two fields to uniquely identify an order of a given instrument: `TrdRegTSTimePriority` (priority timestamp) and `Side`. Databento normalizes the `order_id` field in [MBO schema](https://databento.com/docs/schemas/mbo.md) by combining both priority timestamp and side. To extract the priority timestamp from the `order_id`, the most-significant bit can be masked, as below: ```python priority_timestamp = order_id & 0x7FFFFFFFFFFFFFFF ``` ### Data prior to 2025-03-09 Data prior to 2025-03-09 was not normalized from EEX full depth-of-book captures, but from [EEX Market Data API](https://www.eex.com/fileadmin/EEX/Downloads/Market_Data/EEX_Group_DataSource/API/EEX_Market_Data_-_API_User_Guide_v021.pdf). The `definition`, `statistics`, `ohlcv-1d`, and `trades` schemas are available prior to this date for options and futures. The reported volume (in `statistics` and `ohlcv-1d`) and `size` (in `trades`) correspond to the number of traded contracts/lots reported by [EEX Market Data API](https://www.eex.com/fileadmin/EEX/Downloads/Market_Data/EEX_Group_DataSource/API/EEX_Market_Data_-_API_User_Guide_v021.pdf). The `raw_instrument_id` field from `definition` schema is not populated, as it is not provided by the venue. The `group` field is populated according to the [EEX instrument's commodity class](https://www.eex.com/fileadmin/EEX/Downloads/Market_Data/EEX_Group_DataSource/API/EEX_Market_Data_-_API_User_Guide_v020.pdf): `POWER`, `NATGAS`, `AGRI`, `EMISSIONS`, and `FREIGHT`. The following `statistics` are published: Opening price, trading session high price, trading session low price, close price, cleared volume, open interest, and settlement price. Cleared volume accounts for on and off-exchange traded volume. Implied trades have no price reported by the venue, thus price will be `UNDEF_PRICE` (`0x7fffffffffffffff`, `9223372036854775808`). Due to not being generated from packet captures, both `ts_event` and `ts_recv` fields are set to midnight UTC, except: * For `trades`, where both `ts_event` and `ts_recv` report the trade timestamp. * For `ohlcv-1d`, where `ts_event` marks the start of the bar. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # ICE Endex **Dataset ID**: [NDEX.IMPACT]($PLATFORM_HOSTNAME/catalog/ndex/NDEX.IMPACT) ICE disseminates FOD (full order depth), MBP (market by price), TOB (top-of-book) and market statistics data for futures, spot contracts and options on futures via their iMpact feed, which Databento receives over UDP multicast in our Aurora DC3 data center. For futures and spot, ICE publishes both FOD and MBP-5. Databento only normalizes the FOD messages. For options, ICE publishes MBP-10, MBP-5 and top-of-book. Databento only normalizes the MBP-10 messages. This means that for options, only the top 10 levels of the book are available even in the MBO schema. For futures the full market is shown. ### Timestamps ICE iMpact messages received by Databento will generally include two timestamps with microsecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | ICE Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | TransactDateTime and SequenceWithinMillis | The matching-engine-processed timestamp with sub-microsecond sequence number. | | `ts_in_delta` | SentDateTime | The matching-engine-sending timestamp. | ICE provides microsecond granularity timestamps along with sub-microsecond sequence numbers for ordering that are normalized as nanosecond timestamps for consistency in ordering with other datasets. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Matching Engine Location ICE Endex has matching engines both in Chicago and Basildon. Databento captures the data for both matching engines in our Aurora DC3 location. ### MBO normalization For options, the ICE iMpact feed only includes market-by-price, which matches Databento's [MBP-10 schema](https://databento.com/docs/schemas/mbp-10.md). We do not offer these messages in our [MBO schema](https://databento.com/docs/schemas/mbo.md). They are only available on MBP-10 and other downstream schemas. Each ICE "Trade Message" is normalized into a single MBO record with the **T**rade action, and the message's "TradeID" field is used as the `order_id`. ICE does not disseminate information about the passive side of the trade. The corresponding **F**ill record will have `side` equal to `N` and the same `order_id` value as the **T**rade record. When ICE does not specify the aggressing side, **T**rade records will have `side` equal to `N`. ### System priced legs A system priced leg is the result of a trade in the outright spread. These trades, as published by the exchange, can have prices that are 10% or more off from the best bid and offer. ICE recommends that the price of these trades isn't counted towards market statistics. Databento normalizes system priced legs as a **T**rade record with `side` equal to `N` and an `UNDEF_PRICE` (`0x7fffffffffffffff`, 9223372036854775808) value for the price. These trades are counted towards OHLCV volume, but not towards the open, high, low or close values. This means that OHLCV messages can have some (or even all) price values as `UNDEF_PRICE`. ### Off-market trades Trades marked by the exchange as off-market have their `publisher_id` set to `NDEX.IMPACT.XOFF` to distinguish them from on-exchange trades. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `NDEX.IMPACT.NDEX` as the `publisher_id` and one with `NDEX.IMPACT.XOFF`. ### Statistics **Stat flags** The `IsOfficial` field in the ICE iMpact Settlement Price Message and `IsSettlePriceOfficial` in the Market Snapshot Message are normalized to the first (least-significant) bit of `stat_flags` where the bit is set if `IsOfficial` is `Y`. This is consistent with the behavior for [CME Globex MDP 3.0](https://databento.com/docs/datasets/glbx-mdp3.md#statistics). ### Symbology ICE uses separate symbology for outrights and strategies. **Outrights** For outrights -- individual futures and options -- raw symbols are sourced from `ContractSymbol` field. If `ContractSymbolExtra` is present, it's appended to the `ContractSymbol`. Outright contract symbols are composed of several parts indicating different aspects of the instrument. For example, the Dutch TTF Natural Gas August 2024 futures contract's symbol is `TFM FMQ0024!`. This can be broken down as follows: | Component | Name | Description | | `TFM ` | Contract code | The right space-padded four-character [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | `F` | Contract type | Indicates whether the instrument is a future (`F`), swap, or other type of contract. | | `M` | Term | The contract term, usually month (`M`). | | `Q` | Month | The expiration month, e.g. `Q` for August. | | `00` | Day | The effective date of the contract. `00` for monthly contracts. | | `24` | Year | The two-digit expiration year. | | `!` | Switch character | Indicates additional distinguishing information about the contract. `!` indicates no additional data. | Options begin with their underlying future's symbol followed by an underscore (`_`) and the option block. Using the Dutch TTF Natural Gas August 2024 87.00 call `TFO FMQ0024_OMCE0000045002072624` as an example, the option block breaks down as: | Component | Name | Description | | `O` | Option indicator | Will always be `O` for options. | | `M` | Option type | The option term, e.g. `M`onthly or `W`eekly. | | `C` | Put or call | Whether the option is a `C`all or `P`ut. | | `E` | Option style | The exercise style of the option, e.g. `E` for European. | | `000004500` | Strike | The zero-padded nine-digit strike price. | | `2` | Strike decimal | The number of decimal places in the strike price. Applying two decimal places to 4500 gives a strike price of 45.00. | | `072624` | Expiration date | The MMDDYY expiration date of the option. `TFO` options expire before the contract month, in this case the August options expire July 26, 2024. | Refer to ICE's [Instrument Naming Convention document](https://www.ice.com/publicdocs/technology/Instrument_Naming_Convention.pdf) for a full explanation of the format of contract symbols. Because each dataset has a separate symbology namespace, Databento omits the MIC code from the fully-qualified symbol in ICE's examples. **Spots** Endex spots have neither a `ContractSymbol` nor a `StrategySymbol`, instead raw symbols are sourced from `MarketDesc`. **Strategies** For spreads and strategies where ICE does not define a `ContractSymbol`, raw symbols are sourced from `StrategySymbol`. Strategy symbols are composed of several parts, however, due to the use of numeric identifiers, strategy raw symbols are harder to interpret than those for outrights. The Dutch TTF Natural Gas year calendar strip `TFO 800 108281469` can be broken down as follows: | Component | Name | Description | | `TFO ` | Contract code | The [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | `800` | Strategy code | A left space-padded three-digit number identifying the type of strategy, e.g. `900` for year calendar strip. Refer to the table in [this document](https://www.ice.com/publicdocs/technology/ICE_Strategy_Code_Reference_Manual.pdf) for a complete list of strategy codes. | | ` 108281469` | Market ID | A left space-padded 10-digit unique identifier. The same as `instrument_id`. | To inspect spread legs, including intercommodity futures spreads, use the leg fields in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) instead of parsing the raw strategy symbol. See [Finding ICE intercommodity spreads](https://databento.com/docs/examples/instrument-definitions/ice-crack-spreads.md). ### Definitions - `MarketTypeId` is normalized to `market_segment_id` - `MICCode` is normalized to `exchange` when it is provided: only for futures and future strategies **Expirations** ICE only provides date-granularity expiration data in `OptionsExpirationDate` and `ScreenLastTrade` fields. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the option expires during or after the trading day. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # ICE Europe Commodities **Dataset ID**: [IFEU.IMPACT]($PLATFORM_HOSTNAME/catalog/ifeu/IFEU.IMPACT) ICE disseminates FOD (full order depth), MBP (market by price), TOB (top-of-book) and market statistics data for futures and options on futures via their iMpact feed, which Databento receives over UDP multicast in our Aurora DC3 data center. For futures, ICE publishes both FOD and MBP-5. Databento only normalizes the FOD messages. For options, ICE publishes MBP-10, MBP-5 and top-of-book. Databento only normalizes the MBP-10 messages. This means that for options, only the top 10 levels of the book are available even in the MBO schema. For futures the full market is shown. ### Timestamps ICE iMpact messages received by Databento will generally include two timestamps with microsecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | ICE Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | TransactDateTime and SequenceWithinMillis | The matching-engine-processed timestamp with sub-microsecond sequence number. | | `ts_in_delta` | SentDateTime | The matching-engine-sending timestamp. | ICE provides microsecond granularity timestamps along with sub-microsecond sequence numbers for ordering that are normalized as nanosecond timestamps for consistency in ordering with other datasets. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Matching Engine Location ICE Europe Commodities has matching engines both in Chicago and Basildon. Currently, the products offered by Databento all trade in the ICE silo, located in 350 E Cermak, Chicago. Databento captures the data in our Aurora DC3 location. ### MBO normalization For options, the ICE iMpact feed only includes market-by-price, which matches Databento's [MBP-10 schema](https://databento.com/docs/schemas/mbp-10.md). We do not offer these messages in our [MBO schema](https://databento.com/docs/schemas/mbo.md). They are only available on MBP-10 and other downstream schemas. Each ICE "Trade Message" is normalized into a single MBO record with the **T**rade action, and the message's "TradeID" field is used as the `order_id`. ICE does not disseminate information about the passive side of the trade. The corresponding **F**ill record will have `side` equal to `N` and the same `order_id` value as the **T**rade record. When ICE does not specify the aggressing side, **T**rade records will have `side` equal to `N`. ### System priced legs A system priced leg is the result of a trade in the outright spread. These trades, as published by the exchange, can have prices that are 10% or more off from the best bid and offer. ICE recommends that the price of these trades isn't counted towards market statistics. Databento normalizes system priced legs as a **T**rade record with `side` equal to `N` and an `UNDEF_PRICE` (`0x7fffffffffffffff`, 9223372036854775808) value for the price. These trades are counted towards OHLCV volume, but not towards the open, high, low or close values. This means that OHLCV messages can have some (or even all) price values as `UNDEF_PRICE`. ### Off-market trades Trades marked by the exchange as off-market have their `publisher_id` set to `IFEU.IMPACT.XOFF` to distinguish them from on-exchange trades. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `IFEU.IMPACT.IFEU` as the `publisher_id` and one with `IFEU.IMPACT.XOFF`. ### Statistics **Stat flags** The `IsOfficial` field in the ICE iMpact Settlement Price Message and `IsSettlePriceOfficial` in the Market Snapshot Message are normalized to the first (least-significant) bit of `stat_flags` where the bit is set if `IsOfficial` is `Y`. This is consistent with the behavior for [CME Globex MDP 3.0](https://databento.com/docs/datasets/glbx-mdp3.md#statistics). ### Symbology ICE uses separate symbology for outrights and strategies. **Outrights** For outrights -- individual futures and options -- raw symbols are sourced from `ContractSymbol` field. If `ContractSymbolExtra` is present, it's appended to the `ContractSymbol`. Outright contract symbols are composed of several parts indicating different aspects of the instrument. For example, the Brent August 2024 futures contract's symbol is `BRN FMQ0024!`. This can be broken down as follows: | Component | Name | Description | | `BRN ` | Contract code | The right space-padded four-character [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | `F` | Contract type | Indicates whether the instrument is a future (`F`), swap, or other type of contract. | | `M` | Term | The contract term, usually month (`M`). | | `Q` | Month | The expiration month, e.g. `Q` for August. | | `00` | Day | The effective date of the contract. `00` for monthly contracts. | | `24` | Year | The two-digit expiration year. | | `!` | Switch character | Indicates additional distinguishing information about the contract. `!` indicates no additional data. | Options begin with their underlying future's symbol followed by an underscore (`_`) and the option block. Using the Brent August 2024 87.00 call `BRN FMQ0024_OMCA0000087002062524` as an example, the option block breaks down as: | Component | Name | Description | | `O` | Option indicator | Will always be `O` for options. | | `M` | Option type | The option term, e.g. `M`onthly or `W`eekly. | | `C` | Put or call | Whether the option is a `C`all or `P`ut. | | `A` | Option style | The exercise style of the option, e.g. `A` for American. | | `000008700` | Strike | The zero-padded nine-digit strike price. | | `2` | Strike decimal | The number of decimal places in the strike price. Applying two decimal places to 8700 gives a strike price of 87.00. | | `062524` | Expiration date | The MMDDYY expiration date of the option. `BRN` options expire before the contract month, in this case the August options expire June 25, 2024. | Refer to ICE's [Instrument Naming Convention document](https://www.ice.com/publicdocs/technology/Instrument_Naming_Convention.pdf) for a full explanation of the format of contract symbols. Because each dataset has a separate symbology namespace, Databento omits the MIC code from the fully-qualified symbol in ICE's examples. **Strategies** For spreads and strategies where ICE does not define a `ContractSymbol`, raw symbols are sourced from `StrategySymbol`. Strategy symbols are composed of several parts, however, due to the use of numeric identifiers, strategy raw symbols are harder to interpret than those for outrights. The Brent options iron condor `BRN 40 30775617` can be broken down as follows: | Component | Name | Description | | `BRN ` | Contract code | The [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | ` 40` | Strategy code | A left space-padded three-digit number identifying the type of strategy, e.g. `40` for iron condor. Refer to the table in [this document](https://www.ice.com/publicdocs/technology/ICE_Strategy_Code_Reference_Manual.pdf) for a complete list of strategy codes. | | ` 30775617` | Market ID | A left space-padded 10-digit unique identifier. The same as `instrument_id`. | To inspect spread legs, including intercommodity futures spreads, use the leg fields in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) instead of parsing the raw strategy symbol. See [Finding ICE intercommodity spreads](https://databento.com/docs/examples/instrument-definitions/ice-crack-spreads.md). **Databento MBO snapshot** ICE Europe Commodities iMpact daily [trading hours](https://www.ice.com/trading-hours) cross 00:00:00 UTC. To make it easier to work with historical MBO data, Databento includes an [order book snapshot](https://databento.com/docs/conventions/mbo-snapshot.md) at 00:00:00 UTC each weekday (Monday-Friday). ### Definitions - `MarketTypeId` is normalized to `market_segment_id` - `MICCode` is normalized to `exchange` when it is provided: only for futures and future strategies **Expirations** ICE only provides date-granularity expiration data in `OptionsExpirationDate` and `ScreenLastTrade` fields. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the option expires during or after the trading day. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # ICE Europe Financials **Dataset ID**: [IFLL.IMPACT]($PLATFORM_HOSTNAME/catalog/ifll/IFLL.IMPACT) ICE disseminates FOD (full order depth), MBP (market by price), TOB (top-of-book) and market statistics data for futures and options on futures via their iMpact feed, which Databento receives over UDP multicast in our Aurora DC3 data center. For futures, ICE publishes both FOD and MBP-5. Databento only normalizes the FOD messages. For options, ICE publishes MBP-10, MBP-5 and top-of-book. Databento only normalizes the MBP-10 messages. This means that for options, only the top 10 levels of the book are available even in the MBO schema. For futures the full market is shown. ### Timestamps ICE iMpact messages received by Databento will generally include two timestamps with microsecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | ICE Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | TransactDateTime and SequenceWithinMillis | The matching-engine-processed timestamp with sub-microsecond sequence number. | | `ts_in_delta` | SentDateTime | The matching-engine-sending timestamp. | ICE provides microsecond granularity timestamps along with sub-microsecond sequence numbers for ordering that are normalized as nanosecond timestamps for consistency in ordering with other datasets. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Matching Engine Location ICE Europe Financials has matching engines both in Chicago and Basildon. Currently, the products offered by Databento all trade in the ICE silo, located in 350 E Cermak, Chicago. Databento captures the data in our Aurora DC3 location. ### MBO normalization For options, the ICE iMpact feed only includes market-by-price, which matches Databento's [MBP-10 schema](https://databento.com/docs/schemas/mbp-10.md). We do not offer these messages in our [MBO schema](https://databento.com/docs/schemas/mbo.md). They are only available on MBP-10 and other downstream schemas. Each ICE "Trade Message" is normalized into a single MBO record with the **T**rade action, and the message's "TradeID" field is used as the `order_id`. ICE does not disseminate information about the passive side of the trade. The corresponding **F**ill record will have `side` equal to `N` and the same `order_id` value as the **T**rade record. When ICE does not specify the aggressing side, **T**rade records will have `side` equal to `N`. ### System priced legs A system priced leg is the result of a trade in the outright spread. These trades, as published by the exchange, can have prices that are 10% or more off from the best bid and offer. ICE recommends that the price of these trades isn't counted towards market statistics. Databento normalizes system priced legs as a **T**rade record with `side` equal to `N` and an `UNDEF_PRICE` (`0x7fffffffffffffff`, 9223372036854775808) value for the price. These trades are counted towards OHLCV volume, but not towards the open, high, low or close values. This means that OHLCV messages can have some (or even all) price values as `UNDEF_PRICE`. ### Off-market trades Trades marked by the exchange as off-market have their `publisher_id` set to `IFLL.IMPACT.XOFF` to distinguish them from on-exchange trades. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `IFLL.IMPACT.IFLL` as the `publisher_id` and one with `IFLL.IMPACT.XOFF`. ### Statistics **Stat flags** The `IsOfficial` field in the ICE iMpact Settlement Price Message and `IsSettlePriceOfficial` in the Market Snapshot Message are normalized to the first (least-significant) bit of `stat_flags` where the bit is set if `IsOfficial` is `Y`. This is consistent with the behavior for [CME Globex MDP 3.0](https://databento.com/docs/datasets/glbx-mdp3.md#statistics). ### Symbology ICE uses separate symbology for outrights and strategies. **Outrights** For outrights -- individual futures and options -- raw symbols are sourced from `ContractSymbol` field. If `ContractSymbolExtra` is present, it's appended to the `ContractSymbol`. Outright contract symbols are composed of several parts indicating different aspects of the instrument. For example, the Three Month Euribor June 2025 futures contract's symbol is `I FMM0025!`. This can be broken down as follows: | Component | Name | Description | | `I ` | Contract code | The right space-padded four-character [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | `F` | Contract type | Indicates whether the instrument is a future (`F`), swap, or other type of contract. | | `M` | Term | The contract term, usually month (`M`). | | `M` | Month | The expiration month, e.g. `M` for June. | | `00` | Day | The effective date of the contract. `00` for monthly contracts. | | `25` | Year | The two-digit expiration year. | | `!` | Switch character | Indicates additional distinguishing information about the contract. `!` indicates no additional data. | Options begin with their underlying future's symbol followed by an underscore (`_`) and the option block. Using the Three Month Euribor June 2025 87.00 call `I FMM0025_OMCA0009793754061625` as an example, the option block breaks down as: | Component | Name | Description | | `O` | Option indicator | Will always be `O` for options. | | `M` | Option type | The option term, e.g. `M`onthly or `W`eekly. | | `C` | Put or call | Whether the option is a `C`all or `P`ut. | | `A` | Option style | The exercise style of the option, e.g. `A` for American. | | `000979375` | Strike | The zero-padded nine-digit strike price. | | `4` | Strike decimal | The number of decimal places in the strike price. Applying four decimal places to 979375 gives a strike price of 97.9375. | | `061625` | Expiration date | The MMDDYY expiration date of the option. | Refer to ICE's [Instrument Naming Convention document](https://www.ice.com/publicdocs/technology/Instrument_Naming_Convention.pdf) for a full explanation of the format of contract symbols. Because each dataset has a separate symbology namespace, Databento omits the MIC code from the fully-qualified symbol in ICE's examples. **Strategies** For spreads and strategies where ICE does not define a `ContractSymbol`, raw symbols are sourced from `StrategySymbol`. Strategy symbols are composed of several parts, however, due to the use of numeric identifiers, strategy raw symbols are harder to interpret than those for outrights. The Three Month Euribor options call spread `I 6 20284553` can be broken down as follows: | Component | Name | Description | | `I ` | Contract code | The [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | ` 6` | Strategy code | A left space-padded three-digit number identifying the type of strategy, e.g. `6` for call spread. Refer to the table in [this document](https://www.ice.com/publicdocs/technology/ICE_Strategy_Code_Reference_Manual.pdf) for a complete list of strategy codes. | | ` 20284553` | Market ID | A left space-padded 10-digit unique identifier. The same as `instrument_id`. | To inspect spread legs, including intercommodity futures spreads, use the leg fields in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) instead of parsing the raw strategy symbol. See [Finding ICE intercommodity spreads](https://databento.com/docs/examples/instrument-definitions/ice-crack-spreads.md). **Databento MBO snapshot** ICE Europe Financials iMpact daily [trading hours](https://www.ice.com/trading-hours) cross 00:00:00 UTC. To make it easier to work with historical MBO data, Databento includes an [order book snapshot](https://databento.com/docs/conventions/mbo-snapshot.md) at 00:00:00 UTC each weekday (Monday-Friday). ### Definitions - `MarketTypeId` is normalized to `market_segment_id` - `MICCode` is normalized to `exchange` when it is provided: only for futures and future strategies **Expirations** ICE only provides date-granularity expiration data in `OptionsExpirationDate` and `ScreenLastTrade` fields. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the option expires during or after the trading day. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # ICE Futures US **Dataset ID**: [IFUS.IMPACT]($PLATFORM_HOSTNAME/catalog/ifus/IFUS.IMPACT) ICE disseminates FOD (full order depth), MBP (market by price), TOB (top-of-book) and market statistics data for futures and options on futures via their iMpact feed, which Databento receives over UDP multicast in our Aurora DC3 data center. For futures, ICE publishes both FOD and MBP-5. Databento only normalizes the FOD messages. For options, ICE publishes MBP-10, MBP-5 and top-of-book. Databento only normalizes the MBP-10 messages. This means that for options, only the top 10 levels of the book are available even in the MBO schema. For futures the full market is shown. ### Timestamps ICE iMpact messages received by Databento will generally include two timestamps with microsecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | ICE Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | TransactDateTime and SequenceWithinMillis | The matching-engine-processed timestamp with sub-microsecond sequence number. | | `ts_in_delta` | SentDateTime | The matching-engine-sending timestamp. | ICE provides microsecond granularity timestamps along with sub-microsecond sequence numbers for ordering that are normalized as nanosecond timestamps for consistency in ordering with other datasets. More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization For options, the ICE iMpact feed only includes market-by-price, which matches Databento's [MBP-10 schema](https://databento.com/docs/schemas/mbp-10.md). We do not offer these messages in our [MBO schema](https://databento.com/docs/schemas/mbo.md). They are only available on MBP-10 and other downstream schemas. Each ICE "Trade Message" is normalized into a single MBO record with the **T**rade action, and the message's "TradeID" field is used as the `order_id`. ICE does not disseminate information about the passive side of the trade. The corresponding **F**ill record will have `side` equal to `N` and the same `order_id` value as the **T**rade record. When ICE does not specify the aggressing side, **T**rade records will have `side` equal to `N`. ### System priced legs A system priced leg is the result of a trade in the outright spread. These trades, as published by the exchange, can have prices that are 10% or more off from the best bid and offer. ICE recommends that the price of these trades isn't counted towards market statistics. Databento normalizes system priced legs as a **T**rade record with `side` equal to `N` and an `UNDEF_PRICE` (`0x7fffffffffffffff`, 9223372036854775808) value for the price. These trades are counted towards OHLCV volume, but not towards the open, high, low or close values. This means that OHLCV messages can have some (or even all) price values as `UNDEF_PRICE`. ### Off-market trades Trades marked by the exchange as off-market have their `publisher_id` set to `IFUS.IMPACT.XOFF` to distinguish them from on-exchange trades. This distinction is also reflected on the OHLCV schemas - if an interval contains both on-market and off-market trades for an instrument, two OHLCV messages will be published for that instrument on that interval, one with `IFUS.IMPACT.IFUS` as the `publisher_id` and one with `IFUS.IMPACT.XOFF`. ### Statistics **Stat flags** The `IsOfficial` field in the ICE iMpact Settlement Price Message and `IsSettlePriceOfficial` in the Market Snapshot Message are normalized to the first (least-significant) bit of `stat_flags` where the bit is set if `IsOfficial` is `Y`. This is consistent with the behavior for [CME Globex MDP 3.0](https://databento.com/docs/datasets/glbx-mdp3.md#statistics). ### Symbology ICE uses separate symbology for outrights and strategies. **Outrights** For outrights -- individual futures and options -- raw symbols are sourced from `ContractSymbol` field. If `ContractSymbolExtra` is present, it's appended to the `ContractSymbol`. Outright contract symbols are composed of several parts indicating different aspects of the instrument. For example, the Sugar No. 11 July 2025 futures contract's symbol is `SB FMN0025!`. This can be broken down as follows: | Component | Name | Description | | `SB ` | Contract code | The right space-padded four-character [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | `F` | Contract type | Indicates whether the instrument is a future (`F`), swap, or other type of contract. | | `M` | Term | The contract term, usually month (`M`). | | `N` | Month | The expiration month, e.g. `N` for July. | | `00` | Day | The effective date of the contract. `00` for monthly contracts. | | `25` | Year | The two-digit expiration year. | | `!` | Switch character | Indicates additional distinguishing information about the contract. `!` indicates no additional data. | Options begin with their underlying future's symbol followed by an underscore (`_`) and the option block. Using the Sugar No. 11 October 2025 _____ put `SB FMV0025_OMPA0000011002091525` as an example, the option block breaks down as: | Component | Name | Description | | `O` | Option indicator | Will always be `O` for options. | | `M` | Option type | The option term, e.g. `M`onthly or `W`eekly. | | `P` | Put or call | Whether the option is a `C`all or `P`ut. | | `A` | Option style | The exercise style of the option, e.g. `A` for American. | | `000001100` | Strike | The zero-padded nine-digit strike price. | | `2` | Strike decimal | The number of decimal places in the strike price. Applying two decimal places to 1100 gives a strike price of 11.00. | | `091525` | Expiration date | The MMDDYY expiration date of the option. `SB` options expire before the contract month, in this case the October options expire September 15, 2025. | Refer to ICE's [Instrument Naming Convention document](https://www.ice.com/publicdocs/technology/Instrument_Naming_Convention.pdf) for a full explanation of the format of contract symbols. Because each dataset has a separate symbology namespace, Databento omits the MIC code from the fully-qualified symbol in ICE's examples. **Strategies** For spreads and strategies where ICE does not define a `ContractSymbol`, raw symbols are sourced from `StrategySymbol`. Strategy symbols are composed of several parts, however, due to the use of numeric identifiers, strategy raw symbols are harder to interpret than those for outrights. The Sugar No. 11 options diagonal put spread `SB 10 30951651` can be broken down as follows: | Component | Name | Description | | `SB ` | Contract code | The [parent symbol](https://databento.com/docs/conventions/symbology.md#parent) or product this instrument belongs to. | | ` 10` | Strategy code | A left space-padded three-digit number identifying the type of strategy, e.g. `10` for diagonal put spread. Refer to the table in [this document](https://www.ice.com/publicdocs/technology/ICE_Strategy_Code_Reference_Manual.pdf) for a complete list of strategy codes. | | ` 30951651` | Market ID | A left space-padded 10-digit unique identifier. The same as `instrument_id`. | To inspect spread legs, including intercommodity futures spreads, use the leg fields in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) instead of parsing the raw strategy symbol. See [Finding ICE intercommodity spreads](https://databento.com/docs/examples/instrument-definitions/ice-crack-spreads.md). **Databento MBO snapshot** ICE Futures US iMpact daily [trading hours](https://www.ice.com/trading-hours) cross 00:00:00 UTC. To make it easier to work with historical MBO data, Databento includes an [order book snapshot](https://databento.com/docs/conventions/mbo-snapshot.md) at 00:00:00 UTC each weekday (Monday-Friday). ### Definitions - `MarketTypeId` is normalized to `market_segment_id` - `MICCode` is normalized to `exchange` when it is provided: only for futures and future strategies **Expirations** ICE only provides date-granularity expiration data in `OptionsExpirationDate` and `ScreenLastTrade` fields. As a result, the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight, even when the option expires during or after the trading day. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # IEX TOPS **Dataset ID**: [IEXG.TOPS]($PLATFORM_HOSTNAME/catalog/us-equities#IEXG.TOPS) IEX disseminates top-of-book and trade information for its US equities exchange through their IEX TOPS feed over the IEX-TP v1 protocol, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available through IEX's website for [IEX TOPS](https://www.iexexchange.io/documents/tops-v1-66) and [IEX-TP v1](https://www.iexexchange.io/documents/iex-tp-v1). > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps IEX TOPS messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | IEX Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp (TOPS) | The matching-engine-received timestamp. | | `ts_in_delta` | Send Time (TP) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # MEMX MEMOIR Depth **Dataset ID**: [MEMX.MEMOIR]($PLATFORM_HOSTNAME/catalog/us-equities#MEMX.MEMOIR) MEMX disseminates full depth-of-book data for their exchange through their MEMOIR Depth Feed, which Databento receives over UDP multicast in our NY4 data center. The [MEMX Depth Feed Client Specifications](https://www.memxtrading.com/specs) are available from MEMX's website. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps MEMX messages received by Databento include one timestamp with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. MEMX does not include an exchange send timestamp, thus `ts_in_delta` field is set to `0` for all messages. | Databento Field | MEMX Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | | `ts_in_delta` | N/A | Set to 0. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # MIAX Pearl Depth of Market **Dataset ID**: [EPRL.DOM]($PLATFORM_HOSTNAME/catalog/us-equities#EPRL.DOM) MIAX disseminates full depth-of-book information for MIAX Pearl Equities through the Miax Pearl Equities Depth of Market Feed (DoM) over the MACH protocol, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available through MIAX's website for [Depth of Market](https://www.miaxglobal.com/sites/default/files/2022-05/MIAX_PEARL_Equities_DoM_Feed_v1.3a.pdf) and [MACH](https://www.miaxglobal.com/sites/default/files/page-files/MIAX_PEARL_MIAX_MACH_Protocol_v1_2c.pdf). > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps MIAX DoM messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | MIAX DoM Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Nasdaq Basic with NLS Plus **Dataset ID**: [XNAS.BASIC]($PLATFORM_HOSTNAME/catalog/us-equities#XNAS.BASIC) Nasdaq disseminates top-of-book data via their QBBO feed, and last sale data via their NLS and NLS+ feeds. Databento receives these over UDP multicast in our NY4 data center. Because they come from separate feeds, BBO and trade events may be out-of-order, even within a single instrument. PDF specifications are available on Nasdaq's website for the [QBBO](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/QBBOSpecification2.1.pdf), [NLS](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSSpecification3.0.pdf) and [NLS+](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf) feeds. ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Nasdaq Basic messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | TotalView Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Source feeds Nasdaq Basic is composed of data from the Nasdaq QBBO and Nasdaq NLS+ feeds. ### Venues Nasdaq Basic contains trade data for the following venues: - Nasdaq (XNAS) - Nasdaq PSX (XPSX) - Nasdaq Texas (XBOS) - FINRA/Nasdaq TRF Carteret (FINN) - FINRA/Nasdaq TRF Chicago (FINC) However, only quotes for Nasdaq (XNAS) are provided. The `publisher_id` field on every record from Databento can be used to identify the venue associated with the event. Because Nasdaq PSX, Nasdaq Texas, and the TRF facilities do not have a BBO disseminated by Nasdaq QBBO, the [MBP-1](https://databento.com/docs/schemas/mbp-1.md), [BBO](https://databento.com/docs/schemas/bbo.md) and [TBBO](https://databento.com/docs/schemas/tbbo.md) schemas are not available. Instead, the equivalent consolidated schemas are provided: [CMBP-1](https://databento.com/docs/schemas/mbp-1.md#consolidated-market-by-price-cmbp-1), [CBBO](https://databento.com/docs/schemas/bbo.md#consolidated-bbo-on-interval-cbbo) and [TCBBO](https://databento.com/docs/schemas/tbbo.md#consolidated-bbo-on-trade-tcbbo). ### Normalization There are only **A**dd and **T**rade actions for Nasdaq Basic. **T**rade records are normalized from NLS+ *Trade Report for Non-NextShares* or *Long form Trade Report for Non-NextShares* messages. The `publisher_id` field will identify the venue associated with the trade. There is no information about the side of the trade, so `side` will always be `N`. **A**dd records are normalized from Nasdaq QBBO *Quotation Message*. An **A** record will be published anytime the best bid or offer quote is updated. The `publisher_id` field will always be set to 93 (`XNAS_BASIC_EQUS`) ### Definition normalization Definition records are sourced from NLS+ *Stock Directory* messages. | Databento Field | NLS+ Field | Description | | `exchange` | Market Category | Normalized to the MIC code of the listing venue. All Nasdaq-listed instruments will normalize to `XNAS`. | | `security_type` | Issue Classification | Identifies the security class for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix E [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlusSpecification3.0.pdf). | | `secsubtype` | Issue Sub-Type | Identifies the security sub-type for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix F [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlusSpecification3.0.pdf). | ### Statistics normalization Statistics records are sourced from NLS+ *Trade Report for Non-NextShares* messages. Databento provides the opening price and closing price statistics based on the sale condition field of these trade messages. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Nasdaq TotalView-ITCH **Dataset ID**: [XNAS.ITCH]($PLATFORM_HOSTNAME/catalog/us-equities#XNAS.ITCH) Nasdaq disseminates full depth-of-book data for their exchange through their TotalView-ITCH feed over the MoldUDP64 protocol, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available on Nasdaq's website for [Nasdaq TotalView-ITCH](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf) and [MoldUDP64](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/moldudp64.pdf). ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Nasdaq TotalView-ITCH messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | TotalView Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization **Order lifecycle messages** There are seven message types in TotalView that change the state of the order book. Databento normalizes these messages to MBO records as follows: - *Add Order - No MPID Attribution*: **A**dd - *Add Order with MPID Attribution*: **A**dd, but the attribution field is not included - *Order Executed*: **T**rade, **F**ill, then **C**ancel - *Order Executed with Price*: **T**rade, **F**ill, then **C**ancel, but the initial **T**rade is omitted if the message is marked as non-printable - *Order Cancel*: **C**ancel, with the `quantity` indicating the number of shares to remove from the display size - *Order Delete*: **C**ancel, with the `quantity` indicating the current display size of the order - *Order Replace*: **C**ancel with the previous `order_id`, followed by **A**dd with the new `order_id` All **T**rade records will have `order_id` set to `0`, because Nasdaq does not provide information about the aggressor's order. Normalized records from the same TotalView message will have the same `sequence`. **Trade messages** TotalView also has two messages for providing execution details for match events against non-displayed orders or cross events. Databento normalizes each *Trade (Non-Cross)* message into a single MBO **T**rade record with side **N**one. *Cross Trade* messages are also normalized into MBO **T**rade records with side **N**one when shares are matched on the cross event. Nasdaq will always send a *Cross Trade* message following an opening, closing, or EMC cross event for an instrument. If no shares are matched during a cross event, the *Cross Trade* message is omitted from MBO and represented only in the [statistics schema](https://databento.com/docs/schemas/statistics.md). ### Statistics normalization TotalView provides information about the execution price and shares matched on completion of a cross event in their *Cross Trade* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following cross events: - Opening price: Nasdaq Opening Cross - Closing price: Nasdaq Closing Cross - Uncrossing price: EMC Cross (Halt/IPO) Nasdaq will always send a *Cross Trade* message following an opening, closing, or EMC cross event for an instrument. If no shares are matched during a cross event, `quantity` will be `0` and `price` will be `UNDEF_PRICE` (`0x7fffffffffffffff`, `9223372036854775808`). ### Imbalance normalization TotalView provides information about cross imbalances in their *Net Order Imbalance Indicator (NOII)* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | TotalView Field | Description | | `paired_qty` | Paired Shares | The number of shares that are eligible to match at the `ref_price`. | | `total_imbalance_qty` | Imbalance Shares | The number of shares not paired at the `ref_price`. | | `side` | Imbalance Direction | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | | `auct_interest_clr_price` | Far Price | Hypothetical auction-clearing price for cross orders only. | | `cont_book_clr_price` | Near Price | Hypothetical auction-clearing price for cross and continuous orders. | | `ref_price` | Current Reference Price | The price at which imbalance shares are being calculated. | | `auction_type` | Cross Type | The type of cross: **O**pening, **C**losing, un-**H**alt or IPO, **A**: Extended Trading Close. | | `significant_imbalance` | Price Variation Indicator | Indicates the deviation of `cont_book_clr_price` to the `ref_price`. Refer to the Nasdaq TotalView specification for values. | #### **Opening/Closing Cross** Nasdaq begins disseminating imbalance data at 9:25 ET for the Opening Cross and 15:50 ET for the Closing Cross. Between 9:25 and 9:28, and 15:50 and 15:55, imbalance data is sent every 10 seconds. The `auct_interest_clr_price` and `cont_book_clr_price` fields will not be populated during these times. After 9:28 and 15:55, imbalance data is sent every 1 second and will include `auct_interest_clr_price` and `cont_book_clr_price` data. At the conclusion of the Opening Cross at 9:30 and the Closing Cross at 16:00, a [statistics record](https://databento.com/docs/datasets/xnas-itch.md#statistics-normalization) will be published with the official opening/closing price. > **Info** > > For more information on the Nasdaq Opening and Closing Cross, see this [Nasdaq FAQ](https://nasdaqtrader.com/content/productsservices/trading/crosses/openclose_faqs.pdf). ### Definition normalization Definition records are sourced from Stock Directory messages. | Databento Field | TotalView Field | Description | | `exchange` | Market Category | Normalized to the MIC code of the listing venue. All Nasdaq-listed instruments will normalize to `XNAS`. | | `security_type` | Issue Classification | Identifies the security class for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix D [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | | `secsubtype` | Issue Sub-Type | Identifies the security sub-type for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix E [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Nasdaq Texas TotalView-ITCH **Dataset ID**: [XBOS.ITCH]($PLATFORM_HOSTNAME/catalog/us-equities#XBOS.ITCH) Nasdaq Texas disseminates full depth-of-book data for their exchange through their TotalView-ITCH feed over the MoldUDP64 protocol, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available on Nasdaq's website for [Nasdaq TotalView-ITCH](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf) and [MoldUDP64](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/moldudp64.pdf). ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Nasdaq Texas TotalView-ITCH messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | TotalView Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization **Order lifecycle messages** There are seven message types in TotalView that change the state of the order book. Databento normalizes these messages to MBO records as follows: - *Add Order - No MPID Attribution*: **A**dd - *Add Order with MPID Attribution*: **A**dd, but the attribution field is not included - *Order Executed*: **T**rade, **F**ill, then **C**ancel - *Order Executed with Price*: **T**rade, **F**ill, then **C**ancel, but the initial **T**rade is omitted if the message is marked as non-printable - *Order Cancel*: **C**ancel, with the `quantity` indicating the number of shares to remove from the display size - *Order Delete*: **C**ancel, with the `quantity` indicating the current display size of the order - *Order Replace*: **C**ancel with the previous `order_id`, followed by **A**dd with the new `order_id` All **T**rade records will have `order_id` set to `0`, because Nasdaq does not provide information about the aggressor's order. Normalized records from the same TotalView message will have the same `sequence`. **Trade messages** TotalView also has two messages for providing execution details for match events against non-displayed orders or cross events. Databento normalizes both *Trade (Non-Cross)* and *Cross Trade* messages into a single MBO **T**rade record with side **N**one. ### Definition normalization Definition records are sourced from Stock Directory messages. | Databento Field | TotalView Field | Description | | `exchange` | Market Category | Normalized to the MIC code of the listing venue. All Nasdaq-listed instruments will normalize to `XNAS`. | | `security_type` | Issue Classification | Identifies the security class for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix D [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | | `secsubtype` | Issue Sub-Type | Identifies the security sub-type for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix E [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Nasdaq PSX TotalView-ITCH **Dataset ID**: [XPSX.ITCH]($PLATFORM_HOSTNAME/catalog/us-equities#XPSX.ITCH) Nasdaq PSX disseminates full depth-of-book data for their exchange through their TotalView-ITCH feed over the MoldUDP64 protocol, which Databento receives over UDP multicast in our NY4 data center. PDF specifications are available on Nasdaq's website for [Nasdaq TotalView-ITCH](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf) and [MoldUDP64](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/moldudp64.pdf). ### Symbol convention Raw symbols in this dataset use the **Nasdaq** convention (e.g. `BRK.B`, `AAC+`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps Nasdaq PSX TotalView-ITCH messages received by Databento have at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The extra field `ts_in_delta` does not convey any additional information for this dataset, and is equivalent to `ts_recv - ts_event`. | Databento Field | TotalView Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Timestamp | The matching-engine-received timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### MBO normalization **Order lifecycle messages** There are seven message types in TotalView that change the state of the order book. Databento normalizes these messages to MBO records as follows: - *Add Order - No MPID Attribution*: **A**dd - *Add Order with MPID Attribution*: **A**dd, but the attribution field is not included - *Order Executed*: **T**rade, **F**ill, then **C**ancel - *Order Executed with Price*: **T**rade, **F**ill, then **C**ancel, but the initial **T**rade is omitted if the message is marked as non-printable - *Order Cancel*: **C**ancel, with the `quantity` indicating the number of shares to remove from the display size - *Order Delete*: **C**ancel, with the `quantity` indicating the current display size of the order - *Order Replace*: **C**ancel with the previous `order_id`, followed by **A**dd with the new `order_id` All **T**rade records will have `order_id` set to `0`, because Nasdaq does not provide information about the aggressor's order. Normalized records from the same TotalView message will have the same `sequence`. **Trade messages** TotalView also has two messages for providing execution details for match events against non-displayed orders or cross events. Databento normalizes both *Trade (Non-Cross)* and *Cross Trade* messages into a single MBO **T**rade record with side **N**one. ### Definition normalization Definition records are sourced from Stock Directory messages. | Databento Field | TotalView Field | Description | | `exchange` | Market Category | Normalized to the MIC code of the listing venue. All Nasdaq-listed instruments will normalize to `XNAS`. | | `security_type` | Issue Classification | Identifies the security class for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix D [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | | `secsubtype` | Issue Sub-Type | Identifies the security sub-type for the issue as assigned by Nasdaq. A list of the different values and their meanings can be found in Appendix E [here](https://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf). | ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # NYSE Integrated **Dataset ID**: [XNYS.PILLAR]($PLATFORM_HOSTNAME/catalog/us-equities#XNYS.PILLAR) NYSE disseminates full depth-of-book data for their exchange through their Integrated Feed, which Databento receives over UDP multicast in our NY4 data center. The [Integrated Feed Client Specification](https://www.nyse.com/market-data/real-time/integrated-feed) is available as a PDF from NYSE's website. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps NYSE messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pillar Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | SourceTime | The matching-engine-received timestamp. | | `ts_in_delta` | SendTime (Common) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics normalization NYSE provides information about the execution price and shares matched on completion of a crossing auction in their *Cross Trade* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following auction types: - Opening price: Core Opening auction - Closing price: Closing auction - Uncrossing price: Halt/Re-opening auction ### Imbalance normalization NYSE provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pillar Field | Description | | `ref_price` | ReferencePrice | The price at which imbalance shares are being calculated. | | `auction_time` | AuctionTime | The projected auction time, with second precision. | | `cont_book_clr_price` | ContinuousBookClearingPrice | The price closest to the reference price where the imbalance is 0. For significant imbalances, or if a continuous book clearing price is not reached, this field is defaulted to 0. | | `auct_interest_clr_price` | AuctionInterestClearingPrice | For the closing auction, the price closest to the reference price where the imbalance of closing-only interest is 0. If a closing-only clearing price is not reached, this field is defaulted to 0. | | `ssr_filing_price` | SSRFilingPrice | For non-Significant imbalances, if a Sell Short Restriction is in effect, the price at which Sell Short interest will be filed. | | `paired_qty` | PairedQty | The number of shares that are eligible to match at the `ref_price`. | | `total_imbalance_qty` | TotalImbalanceQty | The number of shares not paired at the `ref_price`. | | `unpaired_qty` | UnpairedQty | During the Closing Auction, the number of unpaired shares priced at or better than the `ref_price`. | | `auction_type` | AuctionType | The type of cross: **M**: Core opening, **H**: Halt resume, **C**: Closing, **P**: Extreme closing imbalance, **R**: Significant closing imbalance. | | `side` | ImbalanceSide | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | | `auction_status` | AuctionStatus | **0**: Will run as always for Open and Close. **1**: Will run, interest exists inside or at the collars or is fully paired off. **2**: Will not run because there is an imbalance through the collars. **3**: Will not run, will transition to the Closing Auction instead. | | `freeze_status` | FreezeStatus | **0**: Imbalance freeze not in effect. **1**: Imbalance freeze is in effect. | | `significant_imbalance` | SignificantImbalance | Indicates whether the current imbalance is significant. **Y** when significant, **' '** (space) otherwise. **~** prior to March 2019 when the Significant Imbalance field was added to the protocol. | #### **Auctions** During auction periods, an imbalance message will be published every 1 second, if changed from the previous second. Imbalance data will only be published for NYSE listed securities during these auction periods. At the conclusion of an auction, a [statistics record](https://databento.com/docs/datasets/xnys-pillar.md#statistics-normalization) will be published. NYSE conducts two scheduled auctions per trading day at the following times: - Core Open: 8:00 to 9:30 ET - Closing: 15:50 to 16:00 ET Auctions can also occur as the result of a trading halt. > **See also** > > See this [NYSE Pillar documentation](https://www.nyse.com/publicdocs/nyse/markets/nyse/Functional_Differences_NYSE_Pillar.pdf) for more information on auctions. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # NYSE American Integrated **Dataset ID**: [XASE.PILLAR]($PLATFORM_HOSTNAME/catalog/us-equities#XASE.PILLAR) NYSE American disseminates full depth-of-book data for their exchange through their Integrated Feed, which Databento receives over UDP multicast in our NY4 data center. The [Integrated Feed Client Specification](https://www.nyse.com/market-data/real-time/integrated-feed) is available as a PDF from NYSE's website. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps NYSE American messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pillar Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | SourceTime | The matching-engine-received timestamp. | | `ts_in_delta` | SendTime (Common) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics normalization NYSE American provides information about the execution price and shares matched on completion of a crossing auction in their *Cross Trade* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following auction types: - Opening price: Core Opening auction - Closing price: Closing auction - Uncrossing price: Early Opening and Halt/Re-opening auction ### Imbalance normalization NYSE American provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pillar Field | Description | | `ref_price` | ReferencePrice | The price at which imbalance shares are being calculated. | | `auction_time` | AuctionTime | The projected auction time, with second precision. | | `cont_book_clr_price` | ContinuousBookClearingPrice | Hypothetical auction-clearing price for cross and continuous orders. | | `auct_interest_clr_price` | AuctionInterestClearingPrice | The price at which all eligible auction-only interest would trade, subject to auction collars. | | `ind_match_price` | IndicativeMatchPrice | The price at which the highest number of shares would trade, subject to auction collars. It includes the non-displayed quantity of Reserve Orders. | | `upper_collar` | UpperCollar | Upper auction collar. | | `lower_collar` | LowerCollar | Lower auction collar. | | `paired_qty` | PairedQty | The number of shares paired off at the `ref_price`. Before 2025-11-10, the number of shares paired off at the `ind_match_price`. | | `total_imbalance_qty` | TotalImbalanceQty | The total imbalance quantity at the `ref_price`. Before 2025-11-10, the total imbalance quantity at the `ind_match_price`. | | `market_imbalance_qty` | MarketImbalanceQty | The total market order imbalance quantity at the `ref_price`. Before 2025-11-10, the total market order imbalance quantity at the `ind_match_price`. | | `auction_type` | AuctionType | The type of cross: **O**: Early opening, **M**: Core opening, **H**: Halt resume, **C**: Closing. | | `side` | ImbalanceSide | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | | `auction_status` | AuctionStatus | **0**: Will run as always for Open and Close. **1**: Will run, interest exists inside or at the collars or is fully paired off. **2**: Will not run because there is an imbalance through the collars. **3**: Will not run, will transition to the Closing Auction instead. | | `freeze_status` | FreezeStatus | **0**: Imbalance freeze not in effect. **1**: Imbalance freeze is in effect. | | `num_extensions` | NumExtensions | Number of times the halt period has been extended. | | `significant_imbalance` | SignificantImbalance | Indicates whether the current imbalance is significant. **Y** when significant, **' '** (space) otherwise. **~** prior to March 2019 when the Significant Imbalance field was added to the protocol. | #### **Auctions** During auction periods, an imbalance message will be published every 1 second, if changed from the previous second. Imbalance data will only be published for NYSE American listed securities during these auction periods. At the conclusion of an auction, a [statistics record](https://databento.com/docs/datasets/xase-pillar.md#statistics-normalization) will be published. NYSE American conducts three scheduled auctions per trading day at the following times: - Early Opening: 6:30 to 7:00 ET - Core Opening: 8:00 to 9:30 ET - Closing: 15:00 to 16:00 ET Auctions can also occur as the result of a trading halt. > **See also** > > See the [NYSE Pillar documentation](https://www.nyse.com/publicdocs/nyse/markets/nyse/Functional_Differences_NYSE_Pillar.pdf) for more information on auctions. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # NYSE Arca Integrated **Dataset ID**: [ARCX.PILLAR]($PLATFORM_HOSTNAME/catalog/us-equities#ARCX.PILLAR) NYSE Arca disseminates full depth-of-book data for their exchange through their Integrated Feed, which Databento receives over UDP multicast in our NY4 data center. The [Integrated Feed Client Specification](https://www.nyse.com/market-data/real-time/integrated-feed) is available as a PDF from NYSE's website. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps NYSE Arca messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pillar Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | SourceTime | The matching-engine-received timestamp. | | `ts_in_delta` | SendTime (Common) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics normalization NYSE Arca provides information about the execution price and shares matched on completion of a crossing auction in their *Cross Trade* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following auction types: - Opening price: Core Opening auction - Closing price: Closing auction - Uncrossing price: Early Opening and Halt/Re-opening auction ### Imbalance normalization NYSE Arca provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pillar Field | Description | | `ref_price` | ReferencePrice | The price at which imbalance shares are being calculated. | | `auction_time` | AuctionTime | The projected auction time, with second precision. | | `cont_book_clr_price` | ContinuousBookClearingPrice | Hypothetical auction-clearing price for cross and continuous orders. | | `auct_interest_clr_price` | AuctionInterestClearingPrice | The price at which all eligible auction-only interest would trade, subject to auction collars. | | `ind_match_price` | IndicativeMatchPrice | The price at which the highest number of shares would trade, subject to auction collars. It includes the non-displayed quantity of Reserve Orders. | | `upper_collar` | UpperCollar | Upper auction collar. | | `lower_collar` | LowerCollar | Lower auction collar. | | `paired_qty` | PairedQty | The number of shares paired off at the `ref_price`. Before 2025-11-13, the number of shares paired off at the `ind_match_price`. | | `total_imbalance_qty` | TotalImbalanceQty | The total imbalance quantity at the `ref_price`. Before 2025-11-13, the total imbalance quantity at the `ind_match_price`. | | `market_imbalance_qty` | MarketImbalanceQty | The total market order imbalance quantity at the `ref_price`. Before 2025-11-13, the total market order imbalance quantity at the `ind_match_price`. | | `auction_type` | AuctionType | The type of cross: **O**: Early opening, **M**: Core opening, **H**: Halt resume, **C**: Closing. | | `side` | ImbalanceSide | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | | `auction_status` | AuctionStatus | **0**: Will run as always for Open and Close. **1**: Will run, interest exists inside or at the collars or is fully paired off. **2**: Will not run because there is an imbalance through the collars. **3**: Will not run, will transition to the Closing Auction instead. | | `freeze_status` | FreezeStatus | **0**: Imbalance freeze not in effect. **1**: Imbalance freeze is in effect. | | `num_extensions` | NumExtensions | Number of times the halt period has been extended. | | `significant_imbalance` | SignificantImbalance | Indicates whether the current imbalance is significant. **Y** when significant, **' '** (space) otherwise. **~** prior to March 2019 when the Significant Imbalance field was added to the protocol. | #### **Auctions** During auction periods, an imbalance message will be published every 1 second, if changed from the previous second. At the conclusion of an auction, a [statistics record](https://databento.com/docs/datasets/arcx-pillar.md#statistics-normalization) will be published. NYSE Arca conducts three scheduled auctions per trading day at the following times: - Early Open: 2:30 to 4:00 ET for all UTP securities - Core Open: 8:00 to 9:30 ET for only NYSE Arca listed securities - Closing: 15:00 to 16:00 ET for all UTP securities Auctions can also occur as the result of a trading halt. Imbalance data will only be published for NYSE Arca listed securities during these trading halts. > **See also** > > See this [NYSE Pillar documentation](https://www.nyse.com/publicdocs/nyse/markets/nyse/Functional_Differences_NYSE_Pillar.pdf) for more information on auctions. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # NYSE National Trades and BBO **Dataset ID**: [XCIS.TRADESBBO]($PLATFORM_HOSTNAME/catalog/us-equities#XCIS.TRADESBBO) NYSE National disseminates top-of-book and last sale data through their BBO and Trades feeds, respectively. Databento receives these over UDP multicast in our NY4 data center. Because they come from separate feeds, BBO and trade events may be out-of-order, even within a single instrument. PDF specifications are available on NYSE's website for the [BBO](https://www.nyse.com/market-data/real-time/bbo) and [Trades](https://www.nyse.com/market-data/real-time/trades) feeds. > **Info** > > The Databento team is currently working on this article. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps NYSE National messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | BBO/Trades Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | SourceTime | The matching-engine-received timestamp. | | `ts_in_delta` | SendTime (Common) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Normalization NYSE National Trades and BBO are sourced from two separate feeds: one for the BBO and one for trades. **A**dd records are normalized from BBO *Quote Message (140)* records in the BBO feed. An **A** record will be published anytime the best bid or offer quote is updated. **T**rade records are normalized from *Trade Message (220)* records in the Trades feed. When combined, this level of data matches Databento's [MBP-1 schema](https://databento.com/docs/schemas/mbp-1.md); however, unlike most other datasets, there is not a strict ordering between the events on the BBO feed and the Trades feed. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # NYSE Texas Integrated **Dataset ID**: [XCHI.PILLAR]($PLATFORM_HOSTNAME/catalog/us-equities#XCHI.PILLAR) NYSE Texas disseminates full depth-of-book data for their exchange through their Integrated Feed, which Databento receives over UDP multicast in our NY4 data center. The [Integrated Feed Client Specification](https://www.nyse.com/market-data/real-time/integrated-feed) is available as a PDF from NYSE's website. ### Symbol convention Raw symbols in this dataset use the **CMS** convention (e.g. `BRK B`, `AAC WS`). See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the per-dataset convention map. ### Timestamps NYSE Texas messages received by Databento will generally include two timestamps with nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. | Databento Field | Pillar Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | SourceTime | The matching-engine-received timestamp. | | `ts_in_delta` | SendTime (Common) | The matching-engine-sending timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Statistics normalization NYSE Texas provides information about the execution price and shares matched on completion of a crossing auction in their *Cross Trade* message. Databento normalizes each message into a record in the [statistics schema](https://databento.com/docs/schemas/statistics.md). Statistics are categorized by the following auction types: - Opening price: Core Opening auction - Closing price: Closing auction - Uncrossing price: Early Opening and Halt/Re-opening auction ### Imbalance normalization NYSE Texas provides information about opening and closing cross imbalances in their *Imbalance* message. Databento normalizes each message into a record in the [imbalance schema](https://databento.com/docs/schemas/imbalance.md). | Databento Field | Pillar Field | Description | | `ref_price` | ReferencePrice | The price at which imbalance shares are being calculated. | | `auction_time` | AuctionTime | The projected auction time, with second precision. | | `cont_book_clr_price` | ContinuousBookClearingPrice | Hypothetical auction-clearing price for cross and continuous orders. | | `auct_interest_clr_price` | AuctionInterestClearingPrice | The price at which all eligible auction-only interest would trade, subject to auction collars. | | `ind_match_price` | IndicativeMatchPrice | The price at which the highest number of shares would trade, subject to auction collars. It includes the non-displayed quantity of Reserve Orders. | | `upper_collar` | UpperCollar | Upper auction collar. | | `lower_collar` | LowerCollar | Lower auction collar. | | `paired_qty` | PairedQty | The number of shares paired off at the `ref_price`. Before 2025-11-20, the number of shares paired off at the `ind_match_price`. | | `total_imbalance_qty` | TotalImbalanceQty | The total imbalance quantity at the `ref_price`. Before 2025-11-20, the total imbalance quantity at the `ind_match_price`. | | `market_imbalance_qty` | MarketImbalanceQty | The total market order imbalance quantity at the `ref_price`. Before 2025-11-20, the total market order imbalance quantity at the `ind_match_price`. | | `auction_type` | AuctionType | The type of cross: **O**: Early opening, **M**: Core opening, **H**: Halt resume, **C**: Closing. | | `side` | ImbalanceSide | The market side of the order imbalance: **B**id (buy) imbalance, **A**sk (sell) imbalance, **N**o imbalance. | | `auction_status` | AuctionStatus | **0**: Will run as always for Open and Close. **1**: Will run, interest exists inside or at the collars or is fully paired off. **2**: Will not run because there is an imbalance through the collars. **3**: Will not run, will transition to the Closing Auction instead. | | `freeze_status` | FreezeStatus | **0**: Imbalance freeze not in effect. **1**: Imbalance freeze is in effect. | | `num_extensions` | NumExtensions | Number of times the halt period has been extended. | | `significant_imbalance` | SignificantImbalance | Indicates whether the current imbalance is significant. **Y** when significant, **' '** (space) otherwise. **~** prior to March 2019 when the Significant Imbalance field was added to the protocol. | #### **Auctions** During auction periods, an imbalance message will be published every 1 second, if changed from the previous second. Imbalance data will only be published for NYSE Texas listed securities during these auction periods. At the conclusion of an auction, a [statistics record](https://databento.com/docs/datasets/xchi-pillar.md#statistics-normalization) will be published. NYSE Texas conducts three scheduled auctions per trading day at the following times: - Early Open: 6:30 to 7:00 ET - Core Open: 8:00 to 9:30 ET - Closing: 15:00 to 16:00 ET Auctions can also occur as the result of a trading halt. > **See also** > > See this [NYSE Pillar documentation](https://www.nyse.com/publicdocs/nyse/markets/nyse/Functional_Differences_NYSE_Pillar.pdf) for more information on auctions. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # OPRA **Dataset ID**: [OPRA.PILLAR]($PLATFORM_HOSTNAME/catalog/opra/OPRA.PILLAR) OPRA disseminates top-of-book, last sale, and daily statistics data for all US equity options exchanges through their Binary Output feed, which Databento receives over UDP multicast in our NY4 data center. The [OPRA Pillar Output Specification](https://cdn.opraplan.com/documents/OPRA_Pillar_Output_Specification.pdf) is available as a PDF from the OPRA website. ### Timestamps OPRA messages received by Databento include at most one timestamp, which has nanosecond precision. As with all of our datasets, we also collect a `ts_recv` timestamp when the packet was received by our capture server. The unused `ts_in_delta` field is set to `0` for all messages. | Databento Field | OPRA Field | Description | | `ts_recv` | N/A | The capture-server-received timestamp. | | `ts_event` | Block Timestamp | The consolidator-processed timestamp. | More details about our timestamps are available in our [timestamping guide](https://databento.com/docs/architecture/timestamping-guide.md). ### Venues OPRA includes consolidated data for 20 options venues (as of September 2026). The `publisher_id` field on every record from Databento can be used to identify the venue associated with a trade event. | Value | MIC | Venue | | 20 | AMXO | NYSE American | | 21 | XBOX | BOX Options | | 22 | XCBO | Cboe Options | | 23 | EMLD | MIAX Emerald | | 24 | EDGO | Cboe EDGX Options | | 25 | GMNI | Nasdaq GEMX | | 26 | XISX | Nasdaq ISE | | 27 | MCRY | Nasdaq MRX | | 28 | XMIO | MIAX Options | | 29 | ARCO | NYSE Arca Options | | 30 | OPRA | Options Price Reporting Authority | | 31 | MPRL | MIAX Pearl | | 32 | XNDQ | Nasdaq Options | | 33 | XBXO | Nasdaq Texas Options | | 34 | C2OX | Cboe C2 Options | | 35 | XPHL | Nasdaq PHLX | | 36 | BATO | Cboe BZX Options | | 37 | MXOP | MEMX Options | | 61 | SPHR | MIAX Sapphire | | 108 | MXTO | MEMX MX2 Options | | 109 | IEXO | IEX Options | > **Info** > > These mappings are also available programmatically through the > [metadata.list_publishers](https://databento.com/docs/api-historical/metadata/metadata-list-publishers.md) endpoint. ### Trades Trade records are published with the `publisher_id` corresponding to the venue where the trade occurred. OPRA does not disseminate the side of a trade, so `side` will always be `N`. The [TCBBO schema](https://databento.com/docs/schemas/tbbo.md#consolidated-bbo-on-trade-tcbbo) can be used to see the NBBO at the time of the trade. ### CMBP-1 Along with the 20 options venues, OPRA also publishes the OPRA consolidated BBO. This consolidated BBO is available through Databento as the [CMBP-1 schema](https://databento.com/docs/schemas/mbp-1.md#consolidated-market-by-price-cmbp-1). OPRA generates this consolidated BBO based on the following criteria in this order: - **Price**: Venues with the highest bid and lowest offer have overall priority. Minimum price increments must be at least 1 cent higher or lower than the previous price. - **Size**: Largest size takes precedence when multiple venues submit the same bid and/or offer price. There are no initial minimum size requirements. Size increments must be 10 contracts or better than the previous size. - **Time**: Earliest time takes precedence when multiple venues submit the same bid and/or offer price with the same size. > **See also** > > More information on the OPRA consolidated BBO can be found in the [OPRA BBO > Guidelines FAQ](https://cdn.opraplan.com/documents/OPRA_BBO_Guidelines.pdf) OPRA consolidated BBO records are published with a `publisher_id` value of 30 (`OPRA_PILLAR_OPRA`). These records indicate the venues containing the best bid and best offer by including their respective `publisher_id` in the `bid_pb_00` and `ask_pb_00` fields. Trade records are published with the `publisher_id` corresponding to the venue where the trade happened. Databento does not publish BBO records for regional quotes. > **See also** > > [Examples - US equity options volume by venue](https://databento.com/docs/examples/options/options-venues.md) ### Historical CMBP-1 requests Due to the large quantity of data for the CMBP-1 schema, we recommend users request a batch download via [batch.submit_job](https://databento.com/docs/api-historical/batch.md) or the [User portal](https://databento.com/docs/portal/batch-download.md). ### Live CMBP-1 subscriptions Due to the large quantity of data for the CMBP-1 schema, the [slow reader behavior](https://databento.com/docs/api-live/basics/slow-reader-behavior.md) of CMBP-1 subscriptions will always be set to `skip` regardless of client settings. ### Symbology OPRA Pillar raw symbols are based on OCC symbology, also known as Options Symbology Initiative (OSI) symbology. These two terms are interchangeable. In OCC symbology, every raw symbol has exactly 21 characters and is composed of four parts. Using the SPY November 17, 2023 $486.00 put option (`SPY 231117P00486000`) as an example, these are: | Component | Name | Description | | `SPY ` | Parent symbol | The right space-padded six-character [parent symbol](https://databento.com/docs/conventions/symbology.md#parent). Often the symbol of the underlying, but not always such as with `SPXW` (SPX weekly options). | | `211117` | Expiration date | The YYMMDD expiration date of the option. | | `P` | Put or call | Whether the option is a `C`all or `P`ut. | | `00486000` | Strike | The zero-padded eight-digit strike price with three decimal places. | ### Adjusted symbol conventions Under OSI symbology, the parent symbol in most cases will be the same as the stock symbol. For example, `MSFT` will be the parent symbol for the underlying security `MSFT`. However, the parent symbol may change based on corporate actions. If a parent symbol change is necessary in a contract adjustment (e.g. in a 3-for-2 split), the symbol `MSFT` will change to `MSFT1`, with the numeric suffix identifying this option as an adjusted, "non-standard" contract. If the standard 100 share contract (`MSFT`) is reintroduced after the adjustment and another adjustment is subsequently necessary to this contract, then the adjusted symbol would use the next numeric suffix not already in use. For example, if `MSFT` and `MSFT1` are active contracts, `MSFT` could change to `MSFT2` if another adjustment is necessary. Parent symbol changes are necessary if the adjusted contract becomes "non-standard", i.e., containing a deliverable different from the "standard" option contract originally listed by an exchange. However, in some contract adjustments, the adjusted contracts remain identical to standard contracts (e.g. in a 2-for-1 split where the resulting contracts remain 100 share options). In these cases, the parent symbol would not change. See [OCC's memo](https://infomemo.theocc.com/infomemos?number=26853) on contract adjustments for more details. In addition, OCC provides a [searchable list of symbol adjustments](https://infomemo.theocc.com/infomemo/search-memo). ### Expirations OPRA only provides date-granularity expiration data so the `expiration` field in the [definition schema](https://databento.com/docs/schemas/instrument-definitions.md) will always be at UTC midnight even when the option expires during or after the trading day. ### Differences between live and historical data The `instrument_id` to `raw_symbol` mappings are not the same between live and historical data. The live `instrument_id` mappings are stable within a week, while the historical mappings can change. ### Data prior to 2023-03-28 Data prior to 2023-03-28 was not generated from OPRA packet captures, but from subsampled data. This leads to the following differences on the data for those dates: * The `cmbp-1`, `cbbo-1s` and `tcbbo` schemas are not available. The highest granularity available for BBO data is the `cbbo-1m` schema. * Due to not being generated from packet captures, the `ts_recv` field has the event timestamp, and `F_BAD_TS_RECV` is always set. * The `bid_pb_00` and `ask_pb_00` fields do not contain the venue which is showing the NBBO. * The `statistics` schema only has information for Open Interest, which is published on an aggregated basis. ### Flags normalization The `F_PUBLISHER_SPECIFIC` flag provides no additional meaning in this dataset and can be safely ignored. # Adjustment factors The adjustment factors dataset covers global corporate actions capital events, spanning multiple currencies for the same event. Each record includes all necessary fields to identify the country, security, event, and adjustment factor. Each record represents a unique capital event at the [listing](https://databento.com/docs/faqs/instruments-and-products.md#listings) level. - **History.** From 2018-05-01 to present. - **Coverage.** Global. - **Update frequency.** Daily. The primary use case for this dataset is to back-adjust end-of-day (EOD) per-share price data, allowing the price series to be graphed with the effects of various events (e.g., bonus issues, rights issues, consolidations) factored into earlier prices. Individual events can receive multiple updates both prior to and after the ex-date, seamlessly accounting for potentially messy corporate actions, changes, and cancellations. All per-share series, including prices, earnings-per-share, dividends-per-share, assets-per-share, cash-flow-per-share, etc., need to be adjusted before meaningful conclusions can be drawn about growth rates and trends. > **See also** > > Adjustment factors [field descriptions](https://databento.com/docs/schemas/adjustment-factors.md#fields) for further details. ### Symbol convention The `nasdaq_symbol` field is normalized to the Nasdaq convention (e.g. `BF.B`). The `local_code` field keeps the listing exchange's native form, as published by the upstream reference-data vendor. For reliable linkage to time series data, identify the security by `isin`, `us_code` (CUSIP), or `figi`. See [Symbology](https://databento.com/docs/schemas/security-master.md#symbology) on the security master schema page for the full reference. ### Exchange coverage The adjustment factors dataset has coverage across 215 exchanges worldwide. - `primary_exchange` specifies the primary exchange for the security associated with the adjustment factor event. - `operating_mic` field specifies the [ISO 10383 MIC](https://www.iso20022.org/market-identifier-codes) of the listing exchange associated with the event. See the [EXCHANGE](https://databento.com/docs/conventions/reference-data-enums.md#exchange) enum lookup table for possible values. > **Tip** > > The `operating_mic` field will match the `venue` from the listed [publishers](https://databento.com/docs/api-historical/metadata/metadata-list-publishers.md) and the Definition schema `exchange` field, which is an ISO 10383 Market Identifier Code (MIC). ### Security continuity The `security_id` maintains continuity in the following scenarios: - **Name changes.** Issuer name changes which result in a ticker symbol change (e.g. Facebook: FB -> Meta Platforms: META). - **Mergers.** Either where the company operates under a new name, or one of the original company names. - **Demergers.** Either where the companies operate under new names, or one keeps the original company name. The `security_id` does **not** have continuity in the following scenarios: - **Spin-offs.** When the parent company continues operating (the spun on companies are new securities). - **Delistings.** When the security is never relisted. ### Timing The first generation of pending factors for a particular ex-date will happen close of market the day before the ex-date. Adjustment factors are provided on the ex-date of the event, except: - When the corporate action is incomplete (missing core fields such as "Ratio New", etc.) in which case the factor will be provided as soon as the corporate action becomes complete after the ex-date. - Or, the factor calculation requires the open price of a security that has not traded yet in which case a factor will be generated after the security trades. ### Reason codes Each record has a `reason` code which distinguishes between different event types. | Value | Description | | `1` | Rights (in same class) | | `2` | Rights (in different class) | | `3` | Entitlement (in same class) | | `4` | Entitlement (in different class) | | `5` | Subdivision | | `6` | Consolidation | | `7` | Reclassification | | `8` | Security Swap | | `9` | Demerger | | `10` | Capital Return | | `11` | Distribution | | `12` | Liquidation | | `13` | Bonus (same class) | | `14` | Bonus (different class) | | `16` | Capital Reduction | | `17` | Cash Dividend | | `18` | Scrip dividend (same class) | | `19` | Both Cash and Scrip dividend | | `20` | Scrip dividend (different class) | | `21` | Capital Call | ### Status codes Included with each record is a single character `status` code which indicates how the adjustment should be handled. | Value | Description | | `'A'` | Apply | | `'P'` | Pending, indicates that this adjustment will be finalized when additional information becomes available | | `'R'` | Rescind (remove/reverse), indicates that this adjustment has been cancelled or re-issued after an adjustment record has already been published | > **Info** > > Note that rescinded adjustments can happen years after the `ex_date` for various reasons. ### Option field Often the shareholder is given the choice of taking the event in a few different ways, for example a dividend might be paid in USD or GBP and the shareholder needs to choose how they wish to receive it, alternatively it might be paid as cash or script or a combination of both, again the shareholder needs to choose how they wish to receive the dividend. Also de-mergers and distributions might have different options for the shareholder to consider. In all cases, the `option` field will contain an integer value ranging from 1 to 9, representing the available choices. You must select one of these options to proceed. If you're unsure which option to choose, default to 1. > **Warning** > > Never apply multiple adjustments with the same `ex_date`, `event_id` and different options. ### Corporate actions relationship Generally, there is a one-to-one relationship to the underlying corporate action - however in a few circumstances this is not true. - When a dividend delivers both cash and script, the corporate action specifies this as a dividend type of "Both". This is split into two separate adjustment records, one for the cash component and the other for the script component, which can be recombined later as desired. - When an event is linked to a previous event they will be combined into a single adjustment record. For example in Stockholm, they often return cash to shareholders in the form of free redemption rights (RDR) followed by a return of capital on those redemption rights. When this happens only one adjustment record will be generated for both corporate actions. > **Tip** > > Where you see both cash and script dividend adjustment records with the same `event_id`, `ex_date` and `option` you can optionally recombine the two by multiplying the factors together. ### Applying adjustment factors > **See also** > > [Applying adjustment factors](https://databento.com/docs/examples/adjustment-factors/applying-adjustment-factors.md) tutorial. Decide which type of adjustments you want to apply (`DIV`, `BON`, etc) then use the `reason` field to access only those adjustments and ignore the others. Prior to using any adjustment factor check that no other adjustment record has the same `event_id` and `ex_date`, if it has then check the `option` - you should only use one of the options for the same `event_id` and `ex_date`. This is common when a shareholder has the choice of taking a dividend in multiple currencies and cash or script selection. Refer to the description of the [option](https://databento.com/docs/datasets/adjustment-factors.md#option-field) field above for further information. Compare the factor to the sentiment value, if available, they should be roughly the same if they are significantly different then the factor should be used with caution or even modified to bring it closer to sentiment (this is purely your prerogative). Multiply all historical prices, up to but not including the `ex_date`, by the `factor`. Optionally divide all historical volumes, up to but not including the `ex_date`, by the `factor` (for consolidations and subdivisions only). > **Info** > > Be prepared for corrections to come through, at a later date, in the form of rescinded adjustment records. In these cases you need to reverse or remove the original adjustment. ### Negative factors These are rare but can happen. For example, when a `DIV` (dividend) is paid that is greater than the share price, on the `ex_date`, or a `RCAP` (return of capital) that is also greater than the share price, on the `ex_date`. This type of event can happen when a fund is being closed down and final distributions made in the form of a dividend or return of capital. ### Factors of 1 A factor of "1" can either be a valid factor or default factor where one could not be calculated. Default factors are 1.0 because this is mathematically neutral and can be used as a placeholder when a factor cannot be calculated. Reasons for not being able to calculate factors include: - No stock close price when a stock close price required. - No issue open price when an issue open price required. - Unable to perform currency conversion when required. Factors of "1" are valid in following circumstances: - When a `RTS` (Rights) or `ENT` (Entitlement) issue makes the issue available at a price that is equal to or above the current stock price (issue price > stock price). - When a company does a `RTS` (Rights) or `ENT` (Entitlement) issue of options with an exercise date far in the future the impact may be of little significance or difficult to determine due to uncertainty. - `CAPRD` (Capital reduction) of 1 for 1. ### Reissued adjustments Adjustments will be re-issued and the original adjustment record rescinded, if any of the corporate action key fields change after the date the adjustment was first published. For example if the `ratio_new` or `ratio_old` change, after the date of first publication, then the original adjustment will be rescinded (after March 2010) and a new adjustment will be issued. ### Rescinded adjustments Adjustments are rescinded if the original adjustment is cancelled or re-issued after the publication date (usually the `ex-date`). Where an adjustment has been rescinded this will need to be removed/reversed out of your system. If you are loading the adjustment records into a database then you will need to remove the rescinded adjustment record from your database. If you are back-adjusting price series data then you will need to reverse the rescinded adjustment record from your price series data using the inverted factor to reverse the original adjustment. The inverted factor will be present on the rescinded record, or you can manually apply the inverse factor. It is recommended to process rescinded adjustment records before processing any re-issued adjustment records. > **Info** > > During the day of publication, if an adjustment is cancelled or re-issued it is not rescinded; the change is simply reflected in the next update. > For example, if a dividend is cancelled on the day of publication then the interim pending record before cancellation will contain an adjustment record > for that dividend. A pending record after the cancellation will not contain an adjustment record (it is not rescinded, it is simply gone). ### Linking to Corporate Actions In most cases you can use the `event_id` field to directly link back to the corporate actions. There are instances where there is additional logic needed to link back to the source corporate action. If the corporate action event associated with the adjustment factor record is either an `FSPLT` or `RSPLT`, you will instead need to check the `event_type` against the `event_subtype` + check the `event_id` numerical component matches for direct linkage. If the adjustment factor record has an event_type of `BON`, `RTS`, `BR`, or `ENT` then the numerical component of the `event_id` will need to be matched against the corporate action's `record_date_id`. # Corporate actions The corporate actions dataset includes coverage from 215+ exchanges worldwide, 310,000+ listed and delisted securities from 100,000+ companies, categorized as 60+ event types. The dataset focuses on crucial events affecting global securities such as dividends, takeovers, mergers & acquisitions and rights issues. Each record represents a unique corporate actions event at the [listing](https://databento.com/docs/faqs/instruments-and-products.md#listings) level. - **History.** From 2018-05-01 to present. - **Coverage.** Global. - **Update frequency.** 4x per day. - **Listing level.** All records are maintained at the listing level, which means each individual listing for a security has its own set of records. - **Delisting and relisting.** The system continues to track securities even after they have been delisted. This is crucial because securities are occasionally relisted, so maintaining a continuous record prevents gaps in the time series thus avoiding ambiguity. > **See also** > > Corporate actions [field descriptions](https://databento.com/docs/schemas/corporate-actions.md#fields) for further details. ### Symbol convention The `nasdaq_symbol` field is normalized to the Nasdaq convention (e.g. `BF.B`). The `local_code` field keeps the listing exchange's native form, as published by the upstream reference-data vendor. For reliable linkage to time series data, identify the security by `isin`, `us_code` (CUSIP), or `figi`. See [Symbology](https://databento.com/docs/schemas/security-master.md#symbology) on the security master schema page for the full reference. ### Securities and listings A security is a fungible, tradable financial asset listed on regulated exchanges. Each security has a primary exchange listing and may be listed on multiple secondary exchanges. The corporate actions dataset utilizes a `security_id` to associate these various listings. Each public listing is identified by a unique `listing_id` that links together a series of point-in-time (PIT) records. The records are ordered by the `event_date` and `ts_record` timestamp. A `listing_group_id` groups all listings for the same security on a specific exchange, including secondary listings such as those in different trading currencies. ### Events Corporate actions can be broadly categorized into three types of events at both the **security** and **listing** levels: - **General events.** Announcements, company meetings, lawsuits/class actions, listing status changes, incorporation change, security description change, etc. - **Capital change events.** Mergers, stock splits / reverse stock splits, return of capital, franking, liquidation, divestments, dividends, currency redenomination, etc. - **Exchange events.** New listing, security reclassification, security swap, preferential offer, etc. The `event` field categorizes the type of corporate action and is the primary field for filtering and categorizing events at a granular level. Event records can be paired with an `event_id`, which links associated events, such as all payment rows. The **top ten** event types are listed in descending order based on their frequency of occurrence, see the [EVENT](https://databento.com/docs/conventions/reference-data-enums.md#event) enum lookup table for possible values, and [Corporate actions events](https://databento.com/docs/conventions/corporate-actions-events.md) for details about each event type. | Value | Description | | `SHOCH` | **Shares Outstanding Change**: The number of shares outstanding has been adjusted due to a capital event. | | `DIV` | **Capital Distribution**: A distribution of assets to shareholders, usually in the form of dividends. | | `AGM` | **Annual General Meeting**: A yearly gathering of a company's interested shareholders. | | `NLIST` | **New Listing**: The company's securities are now listed on an exchange. | | `ANN` | **Announcement**: Public communication regarding significant company events. | | `LSTAT` | **Listing Status**: Changes in the status of the company's listing on the exchange. | | `MKCHG` | **Market Segment Change**: The company has moved to a different market segment classification. | | `SDCHG` | **Sedol Change**: The company's SEDOL (**S**tock **E**xchange **D**aily **O**fficial **L**ist) identifier has been changed. | | `BBCC` | **Bloomberg Composite ID Change**: The Bloomberg Composite Identifier for the company has been changed. | | `BBEC` | **Bloomberg Exchange ID Change**: The Bloomberg Exchange Identifier for the company has been changed. | | `DIVRC` | **Dividend Reclassification**: The classification of dividend payments has been changed. | | `RD` | **Record Date**: The date on which a shareholder must own shares to be entitled to a dividend or other distribution. | | `ICC` | **International Code Change**: The international identification code for the company's securities has been changed. | | `DRIP` | **Dividend Reinvestment Plan**: A plan that allows shareholders to reinvest their cash dividends into additional shares. | | `LCC` | **Local Code Change**: The ticker symbol for the company's securities has been changed. | ### Event subtypes The `event_subtype` field is only used for a limited number of events which fall into distinct sub-groupings, see the [EVENTSUBTYPE](https://databento.com/docs/conventions/reference-data-enums.md#eventsubtype) enum lookup table for all possible values, and [Corporate actions events](https://databento.com/docs/conventions/corporate-actions-events.md) for a list of subtypes for each `event` code. | Value | Description | | `AGM` | **Annual General Meeting**: The table name implies Annual General Meeting, but there are several other possible meeting types categorized under this event subtype. | | `BB` | **Buyback**: Buyback events can also be used to transmit Dutch auction events. | | `BON` | **Bonus**: Bonus events can also be used to transmit depository receipt dividends, supplied by a depository bank in lieu of a bonus issue on the underlying security. | | `CONV` | **Conversion**: Conversion events are occasionally used to bundle data for different event types, e.g., offers. These need to be clearly distinguished from actual conversions. | | `DIV` | **Dividend**: Dividend events can also be used to transmit capital distribution and capital gain events. Certain interest payments are also mapped into dividends. | | `DIVIF` | **Dividend Income or Accumulation**: The subtype transmits accumulation or income, and also dividend, interest, or property-related events. | | `ENT` | **Entitlement**: Entitlement events can contain either non-renounceable rights or an offer event. | | `RCAP` | **Return of Capital**: Return of capital events have several distinct types, each of which can significantly impact client-side processing. | | `RTS` | **Rights**: Rights events can also be used to transmit depository receipt dividends, supplied by a depository bank in lieu of a rights issue on the underlying security. | | `REDEM` | **Redemption**: Redemption events are occasionally used to bundle data for different event types, e.g., offers. These need to be clearly distinguished from actual redemptions. | | `TKOVR` | **Takeover**: Takeover events are used as an umbrella term for several underlying event scenarios. | | `MRGR` | **Merger**: If a close date but no open date is populated, it indicates a mandatory event. | | `TKOVRMINI` | **Mini-Takeover**: If flagged as a Mini-Takeover, it is considered a voluntary event. | | `TENDMRGR` | **Tender Merger**: If close and open dates are populated and the event is still ongoing, or has lapsed, it is considered a voluntary event. | | `TEND` | **Tender Offer**: If close and open dates are populated and the event is still ongoing, or has lapsed, it is considered a voluntary event. | ### Security types The **top ten** security types covered are listed in descending order based on their frequency of occurrence, see the [SECTYPE](https://databento.com/docs/conventions/reference-data-enums.md#sectype) enum lookup table for possible values. | Value | Description | | `ETF` | Exchange Traded Fund | | `EQS` | Equity Shares | | `DR` | Depository Receipts | | `PRF` | Preference Share | | `UNT` | Units | | `WAR` | Warrants | | `ETC` | Exchange Traded Commodities | | `TRT` | Tradeable Rights | | `STP` | Stapled Security | | `BND` | Bond | ### Exchange coverage The corporate actions dataset has coverage across 220+ exchanges worldwide. - `primary_exchange` specifies the primary exchange for the security associated with the event. - `exchange` field specifies the exchange for the listing and can be filtered by specifying a `dataset`. - `operating_mic` field specifies the [ISO 10383 MIC](https://www.iso20022.org/market-identifier-codes) of the listing exchange associated with the event. See the [EXCHANGE](https://databento.com/docs/conventions/reference-data-enums.md#exchange) enum lookup table for possible values. > **Tip** > > The `operating_mic` field will match the `venue` from the listed [publishers](https://databento.com/docs/api-historical/metadata/metadata-list-publishers.md) and the Definition schema `exchange` field, which is an ISO 10383 Market Identifier Code (MIC). ### Listing continuity The `listing_id` and `security_id` have continuity with changes to the underlying security in the following scenarios: - **Name changes.** Issuer name changes which result in a ticker symbol change (e.g. Facebook: FB -> Meta Platforms: META). - **Mergers.** Either where the company operates under a new name, or one of the original company names. - **Demergers.** Either where the companies operate under new names, or one keeps the original company name. The `listing_id` and `security_id` do **NOT** have continuity in the following scenarios: - **Spin-offs.** When the parent company continues operating, the spun-off companies become new listings. - **Delistings.** When the security is never relisted. ### Point-in-time (PIT) corporate actions The `corporate_actions.get_range` endpoint provides a time series of point-in-time (PIT) records for the queried symbols and events. This is ideal for research or backtesting where it's crucial to include records as they were known at specific points in time. Note that, like other APIs, the time ranges have an exclusive end. The Python client library offers a `pit` parameter to control record retention: - If True: All historical records for each `event_unique_id` will be retained, preserving the complete point-in-time history. - If False (default): The DataFrame will include only the most recent record for each `event_unique_id`, based on the `ts_record` timestamp. ### Split and restructuring events The corporate actions dataset includes normalization of several stock restructuring events, such as splits, spin-offs, and demergers, as shown in the following table: | Event type | Event subtype | Description | | `FSPLT` (Forward Split) | `DIV` (Dividend) | Stock dividend (represented as a forward split) | | `FSPLT` (Forward Split) | `BON` (Bonus) | Bonus (represented as a forward split) | | `RSPLT` (Reverse Split) | `CONSD` (Consolidation) | Consolidation (represented as a reverse split) | | `SOFF` (Spin-Off) | `DIST` (Distribution) | Distribution resulting from a spin-off | | `SOFF` (Spin-Off) | `DMRGR` (Demerger) | Demerger resulting in a spin-off | | `FSPLT` (Forward Split) | `SD` (Subdivision) | Subdivision (represented as a forward split) | ### Payment ratios For `FSPLT`, `DIV`, and `BON` events, the `ratio_new` includes the existing holdings. As a result, some reported ratios may differ from the actual figures in official announcements. # Security master > **Info** > > The Databento team is currently working on this article. The security master provides point-in-time (PIT) static information on securities across international markets for 860,000+ listed and delisted securities. This provides a means for resolving securities attributes, identification, symbology and informational challenges efficiently. Some common use cases for the security master include research, backtesting, compliance and report generation, risk management, and keeping portfolios current. - **History.** From 2005-01-01 to present. - **Coverage.** Global. - **Update frequency.** Once per week. - **Listing level.** All records are maintained at the listing level, which means each individual listing for a security has its own set of records. - **Delisting and relisting.** The system continues to track securities even after they have been delisted. This is crucial because securities are occasionally relisted, so maintaining a continuous record prevents gaps in the time series thus avoiding ambiguity. > **See also** > > Security master [field descriptions](https://databento.com/docs/schemas/security-master.md#fields) for further details. ### Symbol convention The `nasdaq_symbol` field is normalized to the Nasdaq convention (e.g. `BF.B`). The `local_code` field keeps the listing exchange's native form, as published by the upstream reference-data vendor. For reliable linkage to time series data, identify the security by `isin`, `us_code` (CUSIP), or `figi`. See [CMS vs. Nasdaq symbology](https://databento.com/docs/conventions/symbology.md#raw-symbol) for the convention reference and per-dataset map. > **Warning** > > For symbols with suffix codes (class shares, warrants, preferreds, etc.) listed on `exchange=USNYSE`, the upstream reference-data vendor delivers the symbol in a no-delimiter form that the normalization step cannot recover. > In these cases, `nasdaq_symbol` is left in the no-delimiter form (e.g. `BRKB` for Berkshire Hathaway Class B, `HEIA` for Heico Class A) and does not match the time series `raw_symbol` for the same security. > Cross-listings of the same security on other US exchanges (e.g. `USCIN`, `USAMEX`) are unaffected. > Matching by `isin`, `us_code`, or `figi` is the reliable workaround. ### Securities and listings A security is a fungible, tradable financial asset listed on regulated exchanges. Each security has a primary exchange [listing](https://databento.com/docs/faqs/instruments-and-products.md#listings) and may be listed on multiple secondary exchanges. The security master utilizes a `security_id` to associate these various listings. Each public listing is identified by a unique `listing_id` that links together a series of point-in-time (PIT) records. The records are ordered by the `ts_effective` timestamp, which is when the information was valid from. A `listing_group_id` groups all listings for the same security on a specific exchange, including secondary listings such as those in different trading currencies. ### Security types The **top ten** security types covered are listed in descending order based on their frequency of occurrence: | Value | Description | | `EQS` | Equity Shares | | `ETF` | Exchange Traded Fund | | `MF` | Mutual Fund | | `DR` | Depository Receipts | | `TRT` | Tradeable Rights | | `WAR` | Warrants | | `BND` | Bond | | `PRF` | Preference Share | | `UNT` | Units | | `ETC` | Exchange Traded Commodities | | `STP` | Stapled Security | See the [SECTYPE](https://databento.com/docs/conventions/reference-data-enums.md#sectype) enum lookup table for possible values. ### Exchange coverage The security master has coverage across 220+ exchanges worldwide. - `primary_exchange` field specifies the primary exchange for the security. - `exchange` field specifies the exchange for the listing and can be filtered by specifying a `dataset`. - `operating_mic` field specifies the [ISO 10383 MIC](https://www.iso20022.org/market-identifier-codes) of the listing exchange. See the [EXCHANGE](https://databento.com/docs/conventions/reference-data-enums.md#exchange) enum lookup table for possible values. > **Tip** > > The `operating_mic` field will match the `venue` from the listed [publishers](https://databento.com/docs/api-historical/metadata/metadata-list-publishers.md) and the Definition schema `exchange` field, which is an ISO 10383 Market Identifier Code (MIC). ### Listing continuity The `listing_id` and `security_id` have continuity with changes to the underlying security in the following scenarios: - **Name changes.** Issuer name changes which result in a ticker symbol change (e.g. Facebook: `FB` -> Meta Platforms: `META`). - **Mergers.** Either where the company operates under a new name, or one of the original company names. - **Demergers.** Either where the companies operate under new names, or one keeps the original company name. The `listing_id` and `security_id` do **NOT** have continuity in the following scenarios: - **Spin-offs.** When the parent company continues operating, the spun-off companies become new listings. - **Delistings.** When the security is never relisted. ### Enriching the definition schema The security master data can be used to enrich instrument definitions by merging two pandas dataframes by an inner join, using the `definition` schema 'symbol' field and the security master 'symbol' field (where `stype_in` is `raw_symbol`). ```{python} df_combined = pd.merge( df_definition, df_security_master, left_on="symbol", right_on="symbol", how="inner", ) ``` This assumes Nasdaq Integrated Platform Suffix is used for the definition schema instrument (such as those obtained from the `XNAS.ITCH` dataset), otherwise joining on the definitions `raw_symbol` may be necessary. > **See also** > > [Enriching instrument definitions](https://databento.com/docs/examples/security-master/enrich-instrument-definitions.md) tutorial.