Stream live data to a file

Overview

In this example we will use the Live client to save live data to a file using an output stream. We will then open this file as a DBNStore. This allows for live data to be archived for replay and testing purposes.

Databento Binary Encoding (DBN)

The binary data will be written to the stream in the DBN format. This is a zero-copy format that is extremely fast and efficient on bandwidth. The easiest way to read data encoded in DBN is through our client libraries or the dbn-cli tool.

Example 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import databento as db

# First, create a live client and connect
live_client = db.Live(key="$YOUR_API_KEY")

# Next, we will subscribe to the OHLCV-1s schema for a few symbols
live_client.subscribe(
    dataset="GLBX.MDP3",
    schema="ohlcv-1s",
    stype_in="continuous",
    symbols=["ES.c.0", "ES.c.1", "CL.c.0", "CL.c.1"],
)

# Now, we will open a file for writing and start streaming
live_client.add_stream("example.dbn")
live_client.start()

# We will listen for 10 seconds before closing the connection
# Any ohlcv-1s bars received during this time will be saved
live_client.block_for_close(timeout=10)

# Finally, we will open the DBN file
dbn_store = db.DBNStore.from_file("example.dbn")
print(dbn_store.to_df(schema="ohlcv-1s"))

Result 

1
2
3
4
5
6
7
8
                           rtype  publisher_id  instrument_id     open     high      low    close  volume symbol
ts_event
2023-12-18 14:40:19+00:00     32             1          17077  4785.75  4785.75  4785.75  4785.75     143   ESH4
2023-12-18 14:40:19+00:00     32             1         686071    74.19    74.19    74.19    74.19       6   CLG4
2023-12-18 14:40:20+00:00     32             1          17077  4785.75  4785.75  4785.50  4785.75       8   ESH4
2023-12-18 14:40:20+00:00     32             1         686071    74.19    74.20    74.19    74.20       6   CLG4
2023-12-18 14:40:21+00:00     32             1          17077  4785.75  4785.75  4785.75  4785.75     105   ESH4
2023-12-18 14:40:21+00:00     32             1         686071    74.20    74.21    74.20    74.21       8   CLG4