Estimate Databento feed latency

Overview

In this example we will use the Live client to calculate latency of incoming records. We will use the ts_recv field to make a measurement of the time between Databento's servers receiving a message from the exchange, to the message being received by the client. To do this, we will request MBP-1 data for a parent symbol and calculate the latency for each message.

You can read our timestamping guide to learn more about our architecture.

Latency results will depend upon the quality of your system clock and internet connection. Your individual results might be very large or even negative given these factors. To obtain the most accurate results, we suggest you sync your clocks to our NTP service.

Example

PythonC++Rust

      
    
import time
import databento as db
import matplotlib.pyplot as plt
import numpy as np

# Set parameters
dataset = "GLBX.MDP3"
product = "ES"

num_samples = 5000
percentiles = [50, 90, 95, 99]
num_bins = 100

# Create a live client
live_client = db.Live(key="$YOUR_API_KEY")

# Subscribe to MBP-1 data
live_client.subscribe(
    dataset="GLBX.MDP3",
    schema="mbp-1",
    symbols=f"{product}.FUT",
    stype_in="parent",
)

# Iterate over the live client and calculate latency
latency_ms = []
for record in live_client:
    if isinstance(record, db.MBP1Msg):
        delta = time.time_ns() - record.ts_recv
        latency_ms.append(delta / 1e6)
    if len(latency_ms) >= num_samples:
        break

fig, ax = plt.subplots(figsize=(10, 6))

x_min = min(latency_ms) * 0.95
x_max = max(latency_ms) * 1.05

# Create histogram with log bins
log_bins = np.logspace(np.log10(x_min), np.log10(x_max), num_bins)
n, bins, patches = ax.hist(latency_ms, bins=log_bins, rwidth=0.85, color="C3")
y_max = max(n) * 1.25

percentile_handles = []
percentile_labels = []

# Create percentile lines
for p in percentiles:
    p_value = np.percentile(latency_ms, p)
    line = ax.axvline(p_value, color="C0", linestyle="--", linewidth=1)
    ax.text(p_value, y_max * 0.9, f"{p}th", color="C0", rotation=-90)
    percentile_handles.append(line)
    percentile_labels.append(f"p{p} - {round(p_value, 2)} ms")

# Set log scale and labels
tick_locations = np.logspace(np.log10(x_min), np.log10(x_max), 10)
ax.set_xscale("log")
ax.set_xticks(tick_locations)
ax.set_xticklabels([str(int(round(x))) for x in tick_locations])
ax.tick_params(axis="x", which="minor", bottom=False, labelbottom=False)

# Set limits and labels
ax.set_xlim(x_min * 0.95, x_max * 1.05)
ax.set_ylim(0, y_max)
ax.set_xlabel("Latency (ms)", fontsize=12)
ax.set_ylabel("Frequency", fontsize=12)
ax.set_title("Latency Distribution", fontsize=14)

# Add legend and display
ax.legend(handles=percentile_handles, labels=percentile_labels)
plt.tight_layout()
plt.show()

Latency plot