Evaluating market data APIs: point-in-time definitions

Many APIs provide instrument definitions as either a static endpoint where you can get the data for a specific day or as a flat file. On the other hand, point-in-time definitions give the user a stream of messages as they get published by the venue.

1. Venues use point-in-time definitions While flat files were at some point the standard way in which venues published their instrument data, most venues today have switched to point-in-time definitions, with the flat file as a fallback.

Futures venues such as CME and Eurex use point-in-time definitions to create new complex instruments in the middle of the trading day, which don't get reflected on the flat file. CME specifically discourages users from using the flat file since it doesn't account for intraday updates to instruments.

2. Without them, you risk missing out on an IPO Providers that don't offer point-in-time definitions usually populate their endpoint by polling the venues' flat files a few times per day to load new instruments.

The issue with this is when you load the venue's flat file for the day, the new instruments haven't been made available yet, so you won't have the data for the newcomers and aren't able to trade them. This is a recurring issue that's often seen in the industry: traders want to get in on an IPO at the open, but their system doesn't recognize the new instrument yet.

Point-in-time instrument definitions solve this problem by ensuring that as soon as the venue publishes data for the instrument, you'll receive it, and your system will be able to trade it.

3. Point-in-time instrument definitions behave the same for historical and real-time data When you're backtesting a model against an API using a reference data flat file, it's much harder to account for intraday updates since you usually don't know when the file was updated on the source and when your system pulled the update.

This can lead to look-ahead bias because your model only has access to data that was published later in the day. Point-in-time instrument definitions, on the other hand, behave in the same way for historical and real-time data.

At Databento, much of our team comes from a trading background, so we've seen all of these problems and more in our previous experiences. We've made sure that Databento natively supports point-in-time definitions for both our historical and real-time APIs, to address these issues commonly seen in other API providers.

To learn more about our point-in-time definitions and view examples of how they're used, you can check out our docs.