Helpers

DBNStore

The DBNStore object is an I/O helper class for working with DBN-encoded data. Typically, this object is created when performing historical requests. However, it can be created directly using DBN data on disk or in memory using provided factory methods:

Attributes

Parameter
Type
Description
nbytes
int
The size of the data in bytes.
raw
bytes
The raw data from the I/O stream.
metadata
Metadata
The metadata header for the DBNStore.
dataset
str
The dataset ID.
schema
Schema or None
The data record schema. If None, the DBNStore may contain multiple schemas.
symbols
list[str]
The query symbols for the data.
stype_in
SType or None
The query input symbology type for the data. If None, the DBNStore may contain mixed STypes.
stype_out
SType
The query output symbology type for the data.
start
pd.Timestamp
The query start for the data as a pd.Timestamp.
end
pd.Timestamp or None
The query end for the data as a pd.Timestamp. If None, the DBNStore data was created without a known end time.
limit
int or None
The query limit for the data.
encoding
Encoding
The data encoding.
compression
Compression
Return the data compression format (if any).
mappings
dict[str, list[dict[str, Any]]]
Return the symbology mappings for the data.
symbology
dict[str, Any]
Return the symbology resolution information for the data.

DBNStore.from_bytes

Read data from a DBN byte stream.

Parameters

Parameter
Type
Required
Description
data
BytesIO or bytes or IO[bytes]
required
The bytes to read from.

Returns

A DBNStore object.

API method 
1
2
3
DBNStore.from_bytes(
    data: BytesIO | bytes | IO[bytes],
) -> DBNStore
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .dbn.zst
path = "GLBX-ESM2-20220606.trades.dbn.zst"
data.to_file(path)

# Open saved data as a byte stream.
with open(path, "rb") as saved:
    stored_data = db.DBNStore.from_bytes(saved)

# Convert to dataframe
df = stored_data.to_df()
print(df.head())
Example response 
1
2
3
4
5
6
7
8
9
                                                               ts_event  rtype  publisher_id  instrument_id action  ... size  flags  ts_in_delta  sequence  symbol
ts_recv                                                                                                             ...
2022-06-06 00:00:00.070314216+00:00 2022-06-06 00:00:00.070033767+00:00      0             1           3403      T  ...    1      0        18681    157862    ESM2
2022-06-06 00:00:00.090544076+00:00 2022-06-06 00:00:00.089830441+00:00      0             1           3403      T  ...    1      0        18604    157922    ESM2
2022-06-06 00:00:00.807324169+00:00 2022-06-06 00:00:00.807018955+00:00      0             1           3403      T  ...    4      0        18396    158072    ESM2
2022-06-06 00:00:01.317722490+00:00 2022-06-06 00:00:01.317385867+00:00      0             1           3403      T  ...    1      0        22043    158111    ESM2
2022-06-06 00:00:01.317736158+00:00 2022-06-06 00:00:01.317385867+00:00      0             1           3403      T  ...    7      0        17280    158112    ESM2

[5 rows x 13 columns]

DBNStore.from_file

Read data from a DBN file.

See also

databento.read_dbn is an alias for DBNStore.from_file.

Parameters

Parameter
Type
Required
Description
path
PathLike[str] or str
required
The file path to read from.

Returns

A DBNStore object.

API method 
1
2
3
DBNStore.from_file(
    path: PathLike[str] | str,
) -> DBNStore
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .dbn.zst
path = "GLBX-ESM2-20220606.trades.dbn.zst"
data.to_file(path)

# Read saved .dbn.zst
stored_data = db.DBNStore.from_file(path)

# Convert to dataframe
df = stored_data.to_df()
print(df.head())
Example response 
1
2
3
4
5
6
7
8
9
                                                               ts_event  rtype  publisher_id  instrument_id action  ... size  flags  ts_in_delta  sequence  symbol
ts_recv                                                                                                             ...
2022-06-06 00:00:00.070314216+00:00 2022-06-06 00:00:00.070033767+00:00      0             1           3403      T  ...    1      0        18681    157862    ESM2
2022-06-06 00:00:00.090544076+00:00 2022-06-06 00:00:00.089830441+00:00      0             1           3403      T  ...    1      0        18604    157922    ESM2
2022-06-06 00:00:00.807324169+00:00 2022-06-06 00:00:00.807018955+00:00      0             1           3403      T  ...    4      0        18396    158072    ESM2
2022-06-06 00:00:01.317722490+00:00 2022-06-06 00:00:01.317385867+00:00      0             1           3403      T  ...    1      0        22043    158111    ESM2
2022-06-06 00:00:01.317736158+00:00 2022-06-06 00:00:01.317385867+00:00      0             1           3403      T  ...    7      0        17280    158112    ESM2

[5 rows x 13 columns]

DBNStore.reader

Return an I/O reader for the data.

Returns

A raw IO stream for reading the DBNStore data.

API method 
1
DBNStore.reader() -> IO[bytes]
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
import databento as db

client = db.Historical("$YOUR_API_KEY")

bento = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols="ZWZ3",
    start="2022-06-06",
)

# Create an IO stream reader
reader = bento.reader

DBNStore.replay

Replay data by passing records sequentially to the given callback.

Refer to the List of fields by schema article for documentation on the fields contained with each record type.

Parameters

Parameter
Type
Required
Description
callback
Callable
required
The callback function or method to be dispatched on every event.

Returns

None

API method 
1
2
3
DBNStore.replay(
    callback: Callable[[DBNRecord], None],
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    start="2022-06-06",
)


def print_large_trades(trade):
    size = getattr(trade, "size", 0)
    if size >= 200:
        print(trade)


data.replay(print_large_trades)
Example response 
1
2
3
TradeMsg(ts_recv=1654524078342408839, pretty_ts_recv='2022-06-06T14:01:18.342408839Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654524078339857609, pretty_ts_event='2022-06-06T14:01:18.339857609Z', action='T', side='B', depth=0, price=4164000000000, pretty_price=4164, size=291, flags=0, ts_in_delta=20352, sequence=3605032)
TradeMsg(ts_recv=1654524133737794739, pretty_ts_recv='2022-06-06T14:02:13.737794739Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654524133736900455, pretty_ts_event='2022-06-06T14:02:13.736900455Z', action='T', side='B', depth=0, price=4160000000000, pretty_price=4160, size=216, flags=0, ts_in_delta=28024, sequence=3659203)
TradeMsg(ts_recv=1654538295589900967, pretty_ts_recv='2022-06-06T17:58:15.589900967Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654538295588752739, pretty_ts_event='2022-06-06T17:58:15.588752739Z', action='T', side='B', depth=0, price=4140000000000, pretty_price=4140, size=200, flags=0, ts_in_delta=21708, sequence=10031624)

DBNStore.request_full_definitions

Request for full instrument Definition(s) for all symbols based on the metadata properties. This is useful for retrieving the instrument definitions for saved DBN data.

A timeseries.get_range request is made to obtain the definitions data which will incur a cost.

Parameters

Parameter
Type
Required
Description
client
Historical
required
The historical client to use for the request (contains the API key).
path
PathLike[str] or str
optional
The path to stream the data to on disk (will then return a DBNStore).

Returns

A DBNStore object.

A full list of fields for each schema is available through Historical.metadata.list_fields.

API method 
1
2
3
4
DBNStore.request_full_definitions(
    client: Historical,
    path: PathLike[str] | str | None = None,
) -> DBNStore
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import databento as db

client = db.Historical(
    key="$YOUR_API_KEY",
)

trades = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ES.FUT"],
    stype_in="parent",
    schema="trades",
    start="2022-06-06",
)

definitions = trades.request_full_definitions(client).to_df()
definitions = definitions.sort_values(["expiration", "symbol"]).set_index("expiration")

print(definitions[["symbol"]])
Example response 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
                              symbol
expiration
2022-06-17 13:30:00+00:00       ESM2
2022-06-17 13:30:00+00:00  ESM2-ESH3
2022-06-17 13:30:00+00:00  ESM2-ESM3
2022-06-17 13:30:00+00:00  ESM2-ESU2
2022-06-17 13:30:00+00:00  ESM2-ESZ2
2022-09-16 13:30:00+00:00       ESU2
2022-09-16 13:30:00+00:00  ESU2-ESH3
2022-09-16 13:30:00+00:00  ESU2-ESM3
2022-09-16 13:30:00+00:00  ESU2-ESU3
2022-09-16 13:30:00+00:00  ESU2-ESZ2
2022-12-16 14:30:00+00:00       ESZ2
2022-12-16 14:30:00+00:00  ESZ2-ESH3
2022-12-16 14:30:00+00:00  ESZ2-ESM3
2022-12-16 14:30:00+00:00  ESZ2-ESU3
2023-03-17 13:30:00+00:00       ESH3
2023-03-17 13:30:00+00:00  ESH3-ESM3
2023-03-17 13:30:00+00:00  ESH3-ESU3
2023-03-17 13:30:00+00:00  ESH3-ESZ3
2023-06-16 13:30:00+00:00       ESM3
2023-06-16 13:30:00+00:00  ESM3-ESU3
2023-06-16 13:30:00+00:00  ESM3-ESZ3
2023-09-15 13:30:00+00:00       ESU3
2023-09-15 13:30:00+00:00  ESU3-ESH4
2023-09-15 13:30:00+00:00  ESU3-ESZ3
2023-12-15 14:30:00+00:00       ESZ3
2023-12-15 14:30:00+00:00  ESZ3-ESH4
2024-03-15 13:30:00+00:00       ESH4
2024-03-15 13:30:00+00:00  ESH4-ESM4
2024-06-21 13:30:00+00:00       ESM4
2024-09-20 13:30:00+00:00       ESU4
2024-12-20 14:30:00+00:00       ESZ4
2025-12-19 14:30:00+00:00       ESZ5
2026-12-18 14:30:00+00:00       ESZ6

DBNStore.request_symbology

Request to resolve symbology mappings based on the metadata properties.

Parameters

Parameter
Type
Required
Description
client
Historical
required
The historical client to use for the request (contains the API key).

Returns

dict[str, Any]

A result including a map of input symbol to output symbol across a date range.

API method 
1
2
3
DBNStore.request_symbology(
    client: Historical,
) -> dict[str, Any]
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .dbn.zst
data.to_file("GLBX-ESM2-20201229.trades.dbn.zst")

# Read saved .dbn.zst
stored_data = db.DBNStore.from_file("GLBX-ESM2-20201229.trades.dbn.zst")

# Request symbology from .dbn.zst metadata
symbology = stored_data.request_symbology(client=client)
print(symbology)
Example response 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
    "result": {
        "ESM2": [
            {
                "d0": "2022-06-06",
                "d1": "2022-06-07",
                "s": "3403"
            }
        ]
    },
    "symbols": [
        "ESM2"
    ],
    "stype_in": "raw_symbol",
    "stype_out": "instrument_id",
    "start_date": "2022-06-06",
    "end_date": "2022-06-07",
    "partial": [],
    "not_found": [],
    "message": "OK",
    "status": 0
}

DBNStore.to_csv

Write data to a file in CSV format.

Parameters

Parameter
Type
Required
Description
path
PathLike[str] or str
required
The file path to write to.
pretty_ts
bool, default True
optional
Whether timestamp columns are converted to tz-aware pandas.Timestamp (UTC).
pretty_px
bool, default True
optional
Whether price columns are correctly scaled as display prices.
map_symbols
bool, default True
optional
If symbology mappings from the metadata should be used to create a 'symbol' column, mapping the instrument ID to its native symbol for every record.
compression
Compression or str
optional
The output compression for writing. Must be either 'zstd' or 'none'.
schema
Schema or str
optional
The data record schema for the output CSV. Must be one of the values from list_schemas. This is only required when reading a DBNStore with mixed record types.
mode
str
optional
The file write mode to use, either "x" or "w". Defaults to "w".

Returns

None

API method 
1
2
3
4
5
6
7
8
9
DBNStore.to_csv(
    path: PathLike[str] | str,
    pretty_ts: bool = True,
    pretty_px: bool = True,
    map_symbols: bool = True,
    compression: Compression | str = Compression.NONE,
    schema: Schema | str | None = None,
    mode: Literal["w", "x"] = "w",
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .csv
data.to_csv("GLBX-ESM2-20220606-trades.csv")

DBNStore.to_df

Converts data to a pandas DataFrame.

Info

The DataFrame index will be set to ts_recv if it exists in the schema, otherwise it will be set to ts_event.

See also

While not optimized for use with live data due to their column-oriented format, pandas DataFrames can still be used with live data by first streaming DBN data to a file, then converting to a DataFrame with DBNStore.from_file().to_df(). See this example for more information.

Parameters

Parameter
Type
Required
Description
price_type
PriceType or str, default "float"
optional
The price type to use for price fields. If "fixed", prices will have a type of int in fixed decimal format; each unit representing 1e-9 or 0.000000001. If "float", prices will have a type of float. If "decimal", prices will be instances of decimal.Decimal.
pretty_ts
bool, default True
optional
Whether timestamp columns are converted to tz-aware pandas.Timestamp. The timezone can be specified using the tz parameter.
map_symbols
bool, default True
optional
If symbology mappings from the metadata should be used to create a 'symbol' column, mapping the instrument ID to its raw symbol for every record.
schema
Schema or str
optional
The data record schema for the output DataFrame. Must be one of the values from list_schemas. This is only required when reading a DBNStore with mixed record types.
tz
datetime.tzinfo or str, default UTC
optional
If pretty_ts is True, all timestamps will be converted to the specified timezone.
count
int
optional
If set, instead of returning a single DataFrame a DataFrameIterator instance will be returned. When iterated, this object will yield a DataFrame with at most count elements until the entire contents of the DBNStore are exhausted.

Returns

A pandas DataFrame object.

API method 
1
2
3
4
5
6
7
8
DBNStore.to_df(
    price_type: PriceType | str = "float",
    pretty_ts: bool = True,
    map_symbols: bool = True,
    schema: Schema | str | None = None,
    tz: datetime.tzinfo | str = zoneinfo.ZoneInfo("UTC"),
    count: int | None = None,
) -> pd.DataFrame | DataFrameIterator
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    schema="trades",
    symbols=["ESM2"],
    start="2022-03-06",
)

df = data.to_df()
print(df.head())
Example response 
1
2
3
4
5
6
7
8
9
                                                               ts_event  rtype  publisher_id  instrument_id action  ... size  flags  ts_in_delta  sequence  symbol
ts_recv                                                                                                             ...
2022-03-06 23:00:00.039463300+00:00 2022-03-06 23:00:00.036436177+00:00      0             1           3403      T  ...    1      0        18828      5178    ESM2
2022-03-06 23:00:01.098111252+00:00 2022-03-06 23:00:01.097477845+00:00      0             1           3403      T  ...    1      0        19122      6816    ESM2
2022-03-06 23:00:04.612334175+00:00 2022-03-06 23:00:04.611714663+00:00      0             1           3403      T  ...    1      0        18687     10038    ESM2
2022-03-06 23:00:04.613776789+00:00 2022-03-06 23:00:04.613240435+00:00      0             1           3403      T  ...    1      0        18452     10045    ESM2
2022-03-06 23:00:06.881864467+00:00 2022-03-06 23:00:06.880575603+00:00      0             1           3403      T  ...    1      0        18478     11343    ESM2

[5 rows x 13 columns]

DBNStore.to_file

Write data to a DBN file.

Parameters

Parameter
Type
Required
Description
path
PathLike[str] or str
required
The file path to write to.
mode
str
optional
The file write mode to use, either "x" or "w". Defaults to "w".
compression
Compression or str
optional
The compression format to write. If None, uses the same compression as the underlying data.

Returns

A DBNStore object.

API method 
1
2
3
4
5
DBNStore.to_file(
    path: PathLike[str] | str,
    mode: Literal["w", "x"] = "w",
    compression: Compression | str | None = None,
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .dbn.zst
data.to_file("GLBX-ESM2-20220606.trades.dbn.zst")

DBNStore.to_json

Write data to a file in JSON format.

Parameters

Parameter
Type
Required
Description
path
PathLike[str] or str
required
The file path to write to.
pretty_ts
bool, default True
optional
Whether timestamp columns are converted to tz-aware pandas.Timestamp (UTC).
pretty_px
bool, default True
optional
Whether price columns are correctly scaled as display prices.
map_symbols
bool, default True
optional
If symbology mappings from the metadata should be used to create a 'symbol' column, mapping the instrument ID to its raw symbol for every record.
compression
Compression or str
optional
The output compression for writing. Must be either 'zstd' or 'none'.
schema
Schema or str
optional
The data record schema for the output JSON. Must be one of the values from list_schemas. This is only required when reading a DBNStore with mixed record types.
mode
str
optional
The file write mode to use, either "x" or "w". Defaults to "w".

Returns

None

API method 
1
2
3
4
5
6
7
8
9
DBNStore.to_json(
    path: PathLike[str] | str,
    pretty_ts: bool = True,
    pretty_px: bool = True,
    map_symbols: bool = True,
    compression: Compression | str = Compression.NONE,
    schema: Schema | str | None = None,
    mode: Literal["w", "x"] = "w",
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .json
data.to_json("GLBX-ESM2-20220606-trades.json")

DBNStore.to_ndarray

Converts data to a numpy N-dimensional array. Each element will contain a Python representation of the binary fields as a Tuple.

Parameters

Parameter
Type
Required
Description
schema
Schema or str
optional
The data record schema for the output array. Must be one of the values from list_schemas. This is only required when reading a DBNStore with mixed record types.
count
int
optional
If set, instead of returning a single np.ndarray a NDArrayIterator instance will be returned. When iterated, this object will yield a np.ndarray with at most count elements until the entire contents of the DBNStore are exhausted.

Returns

A numpy.ndarray object.

API method 
1
2
3
4
DBNStore.to_ndarray(
    schema: Schema | str | None = None,
    count: int | None = None,
) -> np.ndarray | NDArrayIterator
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    start="2022-06-06",
)

array = data.to_ndarray()
print(array[0].item())
Example response 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
(12,
0,
1,
3403,
1654473600070033767,
4108500000000,
1,
b'T',
b'A',
0,
0,
1654473600070314216,
18681,
157862)

DBNStore.to_parquet

Write data to a file in Apache parquet format.

Parameters

Parameter
Type
Required
Description
path
PathLike[str] or str
required
The file path to write the data to.
price_type
PriceType or str, default "float"
optional
The price type to use for price fields. If "fixed", prices will have a type of int in fixed decimal format; each unit representing 1e-9 or 0.000000001. If "float", prices will have a type of float.
pretty_ts
bool, default True
optional
Whether timestamp columns are converted to tz-aware pyarrow.TimestampType (UTC).
map_symbols
bool, default True
optional
If symbology mappings from the metadata should be used to create a 'symbol' column, mapping the instrument ID to its raw symbol for every record.
schema
Schema or str
optional
The data record schema for the output parquet file. Must be one of the values from list_schemas. This is only required when reading a DBNStore with mixed record types.
mode
str
optional
The file write mode to use, either "x" or "w". Defaults to "w".
**kwargs
Any
optional
Keyword arguments to pass to pyarrow.parquet.ParquetWriter. These can be used to override the default behavior of the writer.
API method 
1
2
3
4
5
6
7
8
9
DBNStore.to_parquet(
    path: PathLike[str] | str,
    price_type: PriceType | str = "float",
    pretty_ts: bool = True,
    map_symbols: bool = True,
    schema: Schema | str | None = None,
    mode: Literal["w", "x"] = "w",
    **kwargs: Any,
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    schema="trades",
    start="2022-06-06",
)

# Save streamed data to .parquet
data.to_parquet("GLBX-ESM2-20220606-trades.parquet")

DBNStore.__iter__

Using for; records will be iterated one at a time. Iteration will stop when there are no more records in the DBNStore instance.

Refer to the List of fields by schema article for documentation on the fields contained with each record type.

API method 
1
DBNStore.__iter__() -> Iterator[DBNRecord]
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="GLBX.MDP3",
    symbols=["ESM2"],
    start="2022-06-06",
)

for trade in data:
    size = getattr(trade, "size", 0)
    if size >= 200:
        print(trade)
Example response 
1
2
3
TradeMsg(ts_recv=1654524078342408839, pretty_ts_recv='2022-06-06T14:01:18.342408839Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654524078339857609, pretty_ts_event='2022-06-06T14:01:18.339857609Z', action='T', side='B', depth=0, price=4164000000000, pretty_price=4164, size=291, flags=0, ts_in_delta=20352, sequence=3605032)
TradeMsg(ts_recv=1654524133737794739, pretty_ts_recv='2022-06-06T14:02:13.737794739Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654524133736900455, pretty_ts_event='2022-06-06T14:02:13.736900455Z', action='T', side='B', depth=0, price=4160000000000, pretty_price=4160, size=216, flags=0, ts_in_delta=28024, sequence=3659203)
TradeMsg(ts_recv=1654538295589900967, pretty_ts_recv='2022-06-06T17:58:15.589900967Z', rtype=<RType.MBP_0: 0>, publisher_id=GLBX.MDP3.GLBX (1), instrument_id=3403, ts_event=1654538295588752739, pretty_ts_event='2022-06-06T17:58:15.588752739Z', action='T', side='B', depth=0, price=4140000000000, pretty_price=4140, size=200, flags=0, ts_in_delta=21708, sequence=10031624)

DBNStore.insert_symbology_json

Insert JSON symbology data which may be obtained from a symbology request or loaded from a file.

Parameters

Parameter
Type
Required
Description
json_data
str or Mapping[str, Any] or TextIO
required
The JSON data to insert.
clear_existing
bool
optional
If existing symbology data should be cleared from the internal mappings.
API method 
1
2
3
4
DBNStore.insert_symbology_json(
    json_data: str | Mapping[str, Any] | TextIO,
    clear_existing: bool = True,
) -> None
Example usage 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import databento as db

client = db.Historical("$YOUR_API_KEY")

data = client.timeseries.get_range(
    dataset="XNAS.ITCH",
    symbols=["ALL_SYMBOLS"],
    schema="trades",
    start="2022-06-06",
    end="2022-06-07",
)

# Request symbology for all symbols and then insert this data
symbology_json = data.request_symbology(client)
data.insert_symbology_json(symbology_json, clear_existing=True)

map_symbols_csv

Use a symbology.json file to map a symbols column onto an existing CSV file. The result is written to out_file.

Parameters

Parameter
Type
Required
Description
symbology_file
PathLike[str] or str
required
Path to a symbology.json file to use as a symbology source.
csv_file
PathLike[str] or str
required
Path to a CSV file that contains encoded DBN data; must contain a ts_recv or ts_event and instrument_id column.
out_file
PathLike[str] or str
optional
Path to a file to write results to. If unspecified, _mapped will be appended to the csv_file name.

Returns

Path to the written file.

API method 
1
2
3
4
5
map_symbols_csv(
    symbology_file: PathLike[str] | str,
    csv_file: PathLike[str] | str,
    out_file: PathLike[str] | str | None = None,
) -> Path:
Example usage 
1
2
3
4
5
6
7
8
import databento as db

result = db.map_symbols_csv(
    "symbology.json",
    "xnas-itch-20230821-20230825.ohlcv-1d.csv",
)

print(result.read_text())
Example response 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
ts_event,rtype,publisher_id,instrument_id,open,high,low,close,volume,symbol
1692576000000000000,35,2,523,133550000000,135200000000,132710000000,134360000000,11015261,AMZN
1692576000000000000,35,2,7290,439090000000,472900000000,437260000000,470990000000,11098972,NVDA
1692576000000000000,35,2,10157,217000000000,233500000000,217000000000,233380000000,21336884,TSLA
1692576000000000000,35,2,7130,430170000000,434700000000,0,433000000000,68661,NOC
1692662400000000000,35,2,7132,431610000000,438480000000,0,437740000000,86950,NOC
1692662400000000000,35,2,10161,236710000000,241550000000,229560000000,232870000000,17069349,TSLA
1692662400000000000,35,2,523,135320000000,135900000000,133740000000,134200000000,8133698,AMZN
1692662400000000000,35,2,7293,475120000000,483440000000,453340000000,457910000000,11700447,NVDA
1692748800000000000,35,2,523,135120000000,137860000000,133220000000,137060000000,11430081,AMZN
1692748800000000000,35,2,7296,463190000000,518870000000,452080000000,502150000000,13361964,NVDA
1692748800000000000,35,2,7135,0,440000000000,0,434030000000,68411,NOC
1692748800000000000,35,2,10163,236590000000,243750000000,226500000000,240810000000,13759234,TSLA
1692835200000000000,35,2,7287,508190000000,512600000000,466720000000,468700000000,21611800,NVDA
1692835200000000000,35,2,10154,242000000000,244140000000,228180000000,229980000000,13724054,TSLA
1692835200000000000,35,2,7126,433650000000,437730000000,0,429880000000,54570,NOC
1692835200000000000,35,2,521,137250000000,137820000000,131410000000,131880000000,11990573,AMZN

map_symbols_json

Use a symbology.json file to insert a symbols key into records of an existing JSON file. The result is written to out_file.

Parameters

Parameter
Type
Required
Description
symbology_file
PathLike[str] or str
required
Path to a symbology.json file to use as a symbology source.
json_file
PathLike[str] or str
required
Path to a JSON file that contains encoded DBN data.
out_file
PathLike[str] or str
optional
Path to a file to write results to. If unspecified, _mapped will be appended to the csv_file name.

Returns

Path to the written file.

API method 
1
2
3
4
5
map_symbols_json(
    symbology_file: PathLike[str] | str,
    json_file: PathLike[str] | str,
    out_file: PathLike[str] | str | None = None,
) -> Path:
Example usage 
1
2
3
4
5
6
7
8
import databento as db

result = db.map_symbols_json(
    "symbology.json",
    "xnas-itch-20230821-20230825.ohlcv-1d.json",
)

print(result.read_text())
Example response 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{"hd":{"ts_event":"1692576000000000000","rtype":35,"publisher_id":2,"instrument_id":523},"open":"133550000000","high":"135200000000","low":"132710000000","close":"134360000000","volume":"11015261","symbol":"AMZN"}
{"hd":{"ts_event":"1692576000000000000","rtype":35,"publisher_id":2,"instrument_id":7290},"open":"439090000000","high":"472900000000","low":"437260000000","close":"470990000000","volume":"11098972","symbol":"NVDA"}
{"hd":{"ts_event":"1692576000000000000","rtype":35,"publisher_id":2,"instrument_id":10157},"open":"217000000000","high":"233500000000","low":"217000000000","close":"233380000000","volume":"21336884","symbol":"TSLA"}
{"hd":{"ts_event":"1692576000000000000","rtype":35,"publisher_id":2,"instrument_id":7130},"open":"430170000000","high":"434700000000","low":"0","close":"433000000000","volume":"68661","symbol":"NOC"}
{"hd":{"ts_event":"1692662400000000000","rtype":35,"publisher_id":2,"instrument_id":10161},"open":"236710000000","high":"241550000000","low":"229560000000","close":"232870000000","volume":"17069349","symbol":"TSLA"}
{"hd":{"ts_event":"1692662400000000000","rtype":35,"publisher_id":2,"instrument_id":523},"open":"135320000000","high":"135900000000","low":"133740000000","close":"134200000000","volume":"8133698","symbol":"AMZN"}
{"hd":{"ts_event":"1692662400000000000","rtype":35,"publisher_id":2,"instrument_id":7293},"open":"475120000000","high":"483440000000","low":"453340000000","close":"457910000000","volume":"11700447","symbol":"NVDA"}
{"hd":{"ts_event":"1692662400000000000","rtype":35,"publisher_id":2,"instrument_id":7132},"open":"431610000000","high":"438480000000","low":"0","close":"437740000000","volume":"86950","symbol":"NOC"}
{"hd":{"ts_event":"1692748800000000000","rtype":35,"publisher_id":2,"instrument_id":523},"open":"135120000000","high":"137860000000","low":"133220000000","close":"137060000000","volume":"11430081","symbol":"AMZN"}
{"hd":{"ts_event":"1692748800000000000","rtype":35,"publisher_id":2,"instrument_id":7296},"open":"463190000000","high":"518870000000","low":"452080000000","close":"502150000000","volume":"13361964","symbol":"NVDA"}
{"hd":{"ts_event":"1692748800000000000","rtype":35,"publisher_id":2,"instrument_id":7135},"open":"0","high":"440000000000","low":"0","close":"434030000000","volume":"68411","symbol":"NOC"}
{"hd":{"ts_event":"1692748800000000000","rtype":35,"publisher_id":2,"instrument_id":10163},"open":"236590000000","high":"243750000000","low":"226500000000","close":"240810000000","volume":"13759234","symbol":"TSLA"}
{"hd":{"ts_event":"1692835200000000000","rtype":35,"publisher_id":2,"instrument_id":7287},"open":"508190000000","high":"512600000000","low":"466720000000","close":"468700000000","volume":"21611800","symbol":"NVDA"}
{"hd":{"ts_event":"1692835200000000000","rtype":35,"publisher_id":2,"instrument_id":10154},"open":"242000000000","high":"244140000000","low":"228180000000","close":"229980000000","volume":"13724054","symbol":"TSLA"}
{"hd":{"ts_event":"1692835200000000000","rtype":35,"publisher_id":2,"instrument_id":7126},"open":"433650000000","high":"437730000000","low":"0","close":"429880000000","volume":"54570","symbol":"NOC"}
{"hd":{"ts_event":"1692835200000000000","rtype":35,"publisher_id":2,"instrument_id":521},"open":"137250000000","high":"137820000000","low":"131410000000","close":"131880000000","volume":"11990573","symbol":"AMZN"}