We're making improvements to MBO publishing

In our ongoing mission to enhance your user experience, we're making improvements to the way we publish MBO for our datasets, effective February 10, 2024.

Currently, each day of historical data starts from 00:00 UTC. For datasets with sessions crossing over UTC boundaries like CME, on the MBO schema, we publish the current state of the order book as a number of order snapshots with this timestamp.

For MBO data timestamped on or after February 10, 2024, at 00:00 UTC, we'll publish a clear record with action=R. This will be sent at the start of every session or order book snapshot for every instrument, and for book recovery, if needed. This clear message will have the F_SNAPSHOT flag set to make it explicit that it's part of the snapshot process.

For datasets with sessions contained within UTC date boundaries that don't have order snapshots (such as Databento Equities Basic), a clear message will be published at 00:00 UTC as well. This clear message will not have the F_SNAPSHOT flag set.

For live data, a clear message will be published for every instrument before the first message received for that instrument in the session (weekly session in CME's case, daily session for other datasets).

There was recurring confusion regarding how to process the snapshot records when reading multiple files sequentially. This update explicitly notifies the user when they should clear their order book and simplifies the processing of snapshot records.

Live data starting on February 10, 2024, at 00:00 UTC, will begin with a clear record with action=R. Historical data for February 10, 2024, and onward will also include these clear records.

For historical data prior to February 10, 2024, clear records are expected to be available by early March 2024.

Users of the MBO schema should make sure that they correctly process action=R and clear the order book for the respective instrument upon receiving that message. Users of other schemas won't be affected.

Check out our docs site for a code example here.