Client

Live

To access Databento's live API, first create an instance of the Live client. The entire API for a streaming session is exposed through instance methods of the client.

Databento's Live client is built with Python's asyncio module and can be easily integrated into asynchronous applications.

Note that the API key can be passed as a parameter, which is not recommended for production applications. Instead, you can leave out this parameter to pass your API key via the DATABENTO_API_KEY environment variable:

Parameters

Parameter
Type
Required
Description
key
str
optional
32-character API key. Found on your API keys page. If None then DATABENTO_API_KEY environment variable is used.
gateway
str
optional
The url of the remote gateway to connect to. This is for advanced use.
port
int
optional
The port to connect to the remote gateway on. This is for advanced use.
ts_out
bool, default False
optional
Whether the gateway should append the send timestamp (ts_out) to each record.
heartbeat_interval_s
int, default None
optional
The interval in seconds at which the gateway will send heartbeat records if no other data records are sent. By default heartbeats will be sent at the gateway's default interval. Must be between 5 and 1800 seconds.
reconnect_policy
ReconnectPolicy or str, default None
optional
The reconnection policy for automatically resuming a live session when an unexpected disconnection occurs.
slow_reader_behavior
SlowReaderBehavior or str, default None
optional
How the gateway should handle a connection unable to keep up with realtime.
compression
Compression or str, default None
optional
Sets the compression mode for the read stream. Uncompressed by default.
loop
asyncio.AbstractEventLoop, default None
optional
The event loop to run the client connection in. If None, a shared event loop running on a background thread will be used.
API method
1
2
3
4
5
6
7
8
9
10
11
class Live(
    key: str | None = None,
    gateway: str | None = None,
    port: int | None = None,
    ts_out: bool = False,
    heartbeat_interval_s: int | None = None,
    reconnect_policy: ReconnectPolicy | str = ReconnectPolicy.NONE,
    slow_reader_behavior: SlowReaderBehavior | str | None = None,
    compression: Compression | str = Compression.NONE,
    loop: asyncio.AbstractEventLoop | None = None,
)
Example usage
1
2
3
4
5
6
7
import databento as db

# Pass as parameter
client = db.Live(key="$YOUR_API_KEY")

# Or, pass as `DATABENTO_API_KEY` environment variable
client = db.Live()

Live.add_callback

Add a callback to the live client. This callback must take a single record argument. Refer to the What's a schema article for documentation on the fields contained with each record type.

A callback will receive error messages from the gateway with the ErrorMsg record type. This indicates a problem occurred with the session. Exceptions raised in a callback can be handled explicitly by specifying an exception_callback.

The callbacks are executed in the order they are added.

Info

It is recommended that callback functions be non-blocking to ensure the networking event loop remains running.

Parameters

Parameter
Type
Required
Description
record_callback
Callable[[DBNRecord], None]
required
A callback to dispatch records to. The callback must take a single record argument. If no exception callback is provided, any exceptions encountered will be logged and raised as warnings for visibility.
exception_callback
Callable[[Exception], None]
optional
A callback to dispatch exceptions to that are raised when executing record_callback. The callback must take a single exception argument.
API method
1
2
3
4
Live.add_callback(
    record_callback: Callable[[DBNRecord], None],
    exception_callback: Callable[[Exception], None] | None = None,
) -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import databento as db

# Create a callback to handle DBN records
def user_callback(record: db.DBNRecord) -> None:
    print(f"callback run for {record}")


# Create a callback to handle exceptions from `user_callback`
def error_handler(exception: Exception) -> None:
    print(f"an error occurred {exception}")


# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Add the user_callback to the client
client.add_callback(
    record_callback=user_callback,
    exception_callback=error_handler,  # optional error handler
)

Live.add_stream

Add an output stream to the live client. The client will write binary DBN records to the stream. This stream must be writable in bytes mode, or a path to a file that is writable.

A stream will receive error messages from the gateway with the ErrMsg record type. This indicates a problem occurred with the session. Exceptions raised in a callback can be handled explicitly by specifying an exception_callback.

The writes are performed in the order the streams were added.

See also

DBN records are optimized for stream-like processing.

While pandas DataFrames are not well-suited for this due to their column-oriented format, they can still be used by first streaming DBN data to a file, then converting to a DataFrame with DBNStore.from_file().to_df(). See this example for more information.

Parameters

Parameter
Type
Required
Description
stream
IO[bytes] or PathLike[str] or str
required
An IO stream to write records to, or a writable file path. The stream must support writing bytes.
exception_callback
Callable[[Exception], None]
optional
An error handling callback to process exceptions that are raised when writing to the stream. The callback must take a single exception argument. If no exception callback is provided, any exceptions encountered will be logged and raised as warnings for visibility.
API method
1
2
3
4
Live.add_stream(
    stream: IO[bytes] | PathLike[str] | str,
    exception_callback: Callable[[Exception], None] | None = None,
) -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import databento as db

# Open a file for writing
output = open("output.dbn", mode="wb")


# Create a callback to handle exceptions when writing to `output`
def error_handler(exception: Exception) -> None:
    print(f"an error occurred {exception}")


# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Add the output file stream to the client
client.add_stream(
    stream=output,
    exception_callback=error_handler,  # optional error handler
)

Live.add_reconnect_callback

Add a reconnect callback to the live client. This callback must take two arguments. When a reconnection policy is set on the Live client, this callback will be executed when a reconnection occurs with two arguments:

  • The last received record's ts_event timestamp, or start timestamp from the session Metadata if no records were received yet.
  • The start timestamp from the reconnected session's Metadata.

These values can be used to record any gaps in the received data due to a disconnection. Exceptions raised in a callback can be handled explicitly by specifying an exception_callback.

The callbacks are executed in the order they are added.

Info

It is recommended that callback functions be non-blocking to ensure the networking event loop remains running.

Parameters

Parameter
Type
Required
Description
reconnect_callback
Callable[[pd.Timestamp, pd.Timestamp], None]
required
A callback for handling reconnections. The callback must take two arguments.
exception_callback
Callable[[Exception], None]
optional
A callback to dispatch exceptions to that are raised when executing reconnect_callback. The callback must take a single exception argument.
API method
1
2
3
4
Live.add_reconnect_callback(
    reconnect_callback: Callable[[pd.Timestamp, pd.Timestamp], None]
    exception_callback: Callable[[Exception], None] | None = None,
) -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import databento as db

# Create a callback to handle reconnections
def reconnect_callback(start, end) -> None:
    print(f"reconnection gap from {start} to {end}")


# Create a callback to handle exceptions from `reconnect_callback`
def error_handler(exception: Exception) -> None:
    print(f"an error occurred {exception}")


# Create a live client with a reconnect policy
client = db.Live(
    key="$YOUR_API_KEY",
    reconnect_policy="reconnect",
)

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Add the reconnect callback to the client
client.add_reconnect_callback(
    reconnect_callback=reconnect_callback,
    exception_callback=error_handler,  # optional error handler
)

Live.block_for_close

Block until the session closes or a timeout is reached. A session will close when the remote gateway disconnects or after Live.stop or Live.terminate are called. In the event the connection is closed unexpectedly, a BentoError will be raised.

If a timeout is specified, Live.terminate will be called when the timeout is reached.

When this method unblocks, the session is guaranteed to be closed.

Parameters

Parameter
Type
Required
Description
timeout
float
optional
A duration in seconds to wait for the session to close. If None, wait forever.
API method
1
2
3
Live.block_for_close(
    timeout: float | None = None,
) -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start the session
client.start()

# Block for the session to close
client.block_for_close()

Live.start

Start the session.

A client can only be started once, and after a successful connection is made by calling Live.subscribe.

A client can be gracefully stopped by calling Live.stop or forcefully using Live.terminate

When iterating records with Live.__iter__ or Live.__aiter__ it is not necessary to call Live.start and doing so before iterating will raise a ValueError.

Info

A session cannot be started more than once.

API method
1
Live.start() -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Print records as they arrive
client.add_callback(print)

# Start the streaming session
client.start()

# Block until disconnection
client.block_for_close()

Live.stop

Stop the session and finish processing received records.

A client can only be stopped after a successful connection is made by calling Live.subscribe.

This method does not block waiting for the connection to close. If this is desired, use Live.block_for_close or Live.wait_for_close to wait for the client to disconnect.

The connection will eventually close after calling this method. Once the connection has closed, the Live client can be reused but the session state is not preserved.

API method
1
Live.stop() -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start the streaming session
client.start()

# Stop the streaming session
client.stop()

Live.subscribe

Add a new subscription to the session. All subscriptions must be for the same dataset. The returned integer is the subscription_id which will match the id used by the gateway when acknowledging the subscription.

Supports multiple subscriptions for different schemas, which enables rich data streams containing mixed record types.

Specify an optional start time for intraday replay with subscriptions made before starting the session.

Please note there is no unsubscribe method. Subscriptions end when the client disconnects or by calling Live.stop or Live.terminate

Parameters

Parameter
Type
Required
Description
dataset
Dataset or str
required
The dataset code (string identifier). Must be one of the values from list_datasets.
schema
Schema or str
required
The data record schema. Must be one of the values from list_schemas.
symbols
Iterable[str | int] or str or int
optional
The product symbols to subscribe to. If 'ALL_SYMBOLS' or None then will select all symbols.
stype_in
SType or str, default 'raw_symbol'
optional
The symbology type of input symbols. Must be one of 'raw_symbol', 'instrument_id', 'parent', or 'continuous'.
start
pd.Timestamp, datetime, date, str or int
optional
The inclusive start of subscription replay. Filters on ts_event except for the CBBO and BBO schemas, which filter on ts_recv. Takes pd.Timestamp, Python datetime, Python date, ISO 8601 string, or UNIX timestamp in nanoseconds. Assumes UTC as timezone unless otherwise specified. Must be within the last 24 hours. Pass 0 to request all available data. Cannot be specified after the session is started.
snapshot
bool, default False
optional
Request subscription with snapshot. Only supported with mbo schema. Defaults to False. Conflicts with the start parameter.
API method
1
2
3
4
5
6
7
8
Live.subscribe(
    dataset: Dataset | str,
    schema: Schema | str,
    symbols: Iterable[str | int] | str | int = "ALL_SYMBOLS",
    stype_in: SType | str = "raw_symbol",
    start: pd.Timestamp | datetime | date | str | int | None = None,
    snapshot: bool = False,
) -> int
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

Live.terminate

Terminate the session and stop processing records immediately.

A client can only be terminated after a successful connection is made by calling Live.subscribe.

Unlike Live.stop, the session will end immediately and received records will no longer be processed.

Once terminated, the Live client can be reused but the session state is not preserved.

API method
1
Live.terminate() -> None
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start the streaming session
client.start()

# Terminate the streaming session
client.terminate()

Live.wait_for_close

Wait until the session closes or a timeout is reached. A session will close when the remote gateway disconnects, or after Live.stop or Live.terminate are called. In the event the connection is closed unexpectedly, a BentoError will be raised.

If a timeout is specified, Live.terminate will be called when the timeout is reached.

When this method unblocks, the session is guaranteed to be closed.

Parameters

Parameter
Type
Required
Description
timeout
float
optional
A duration in seconds to wait for the session to close. If None, wait forever.
API method
1
2
3
Live.wait_for_close(
    timeout: float | None = None,
) -> None
Info

This method is a coroutine and must be used with an await expression.

Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to the trades schema for all ES futures
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start the session
client.start()

# Wait for the session to end
await client.wait_for_close()

Live.__aiter__

Using async for; records will be yielded as they arrive. Iteration will stop when the connection is closed and all records are processed. This is best for integration into asynchronous applications.

Asynchronous iteration is only supported inside a coroutine.

Asynchronous iteration will automatically call Live.start if the session is connected but it has not been started. Starting iteration after the session has started will cause a ValueError.

Asynchronous iteration will automatically call Live.stop if the iterator is destroyed, such as when an async for loop is escaped with an exception or break statement. To prevent this behavior, an iterator can be created explicitly using aiter().

API method
1
Live.__aiter__() -> AsyncIterator[DBNRecord]
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to a data stream
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start streaming and asynchronously iterate the records
async for record in client:
    print(record)

Live.__iter__

Using for; records will be yielded as they arrive. Iteration will stop when the connection is closed and all records are processed. This is best for integration into simple synchronous applications.

Iteration will automatically call Live.start if the session is connected but it has not been started. Starting iteration after the session has started will cause a ValueError.

Iteration will automatically call Live.stop if the iterator is destroyed, such as when a for loop is escaped with an exception or break statement. To prevent this behavior, an iterator can be created explicitly using iter().

API method
1
Live.__iter__() -> Iterator[DBNRecord]
Example usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import databento as db

# Create a live client
client = db.Live(key="$YOUR_API_KEY")

# Subscribe to a data stream
client.subscribe(
    dataset="GLBX.MDP3",
    schema="trades",
    stype_in="parent",
    symbols="ES.FUT",
)

# Start streaming and synchronously iterate the records
for record in client:
    print(record)