PandasPairUniverse#

tradingstrategy.pair.PandasPairUniverse Python class in Trading Strategy framework.

class PandasPairUniverse[source]#

Bases: object

A pair universe implementation that is created from Pandas dataset.

This is a helper class, as pandas.DataFrame is somewhat more difficult to interact with. This class will read the raw data frame and convert it to DEXPair objects with a lookup index. Because the DEXPair conversion is expensive for 10,000s of Python objects, it is recommended that you filter the raw pandas.DataFrame by using filtering functions in tradingstrategy.pair first, before initializing PandasPairUniverse.

About the usage:

Single trading pairs can be looked up using PandasPairUniverse.get_pair_by_smart_contract() and PandasPairUniverse.get_pair_by_id()
Multiple pairs can be looked up by directly reading PandasPairUniverse.df Pandas dataframe

Example how to use:

# Get dataset from the server as Apache Pyarrow table
columnar_pair_table = client.fetch_pair_universe()

# Convert Pyarrow -> Pandas -> in-memory DEXPair index
pair_universe = PandasPairUniverse(columnar_pair_table.to_pandas())

# Lookup SUSHI-WETH trading pair from DEXPair index
# https://tradingstrategy.ai/trading-view/ethereum/sushi/sushi-eth
pair: DEXPair = pair_universe.get_pair_by_smart_contract("0x795065dcc9f64b5614c407a6efdc400da6221fb0")

If the pair index is too slow to build, or you want to keep it lean, you can disable the indexing with build_index. In this case, some of the methods won’t work:

# Get dataset from the server as Apache Pyarrow table
columnar_pair_table = client.fetch_pair_universe()

# Convert Pyarrow -> Pandas -> in-memory DEXPair index
pair_universe = PandasPairUniverse(columnar_pair_table.to_pandas(), build_index=False)

__init__(df, build_index=True, exchange_universe=None)[source]#

Parameters:

df (DataFrame) – The source DataFrame that contains all DEXPair entries
build_index – Build quick lookup index for pairs
exchange_universe (Optional[ExchangeUniverse]) –
Optional exchange universe needed for human-readable pair lookup.

We cannot properly resolve pairs unless we can map exchange names to their ids. Currently optional, only needed by get_pair().

Methods

`__init__`(df[, build_index, exchange_universe])	param df:
`build_index`()	Create pair_id -> data mapping.
`create_limited_pair_universe`(df, exchange, pairs)	Create a trading pair universe that contains only few trading pairs.
`create_pair_universe`(df, pairs)	Create a PandasPairUniverse instance based on loaded raw pairs data.
`create_parquet_load_filter`([count_limit])	Returns a Parquet loading filter that contains pairs in this universe.
`create_single_pair_universe`(df, exchange, ...)	Create a trading pair universe that contains only a single trading pair.
`get_all_pair_ids`()	Get all pair ids in the data frame.
`get_all_tokens`()	Get all base and quote tokens in trading pairs.
`get_by_symbols`(base_token_symbol, ...)	For strategies that trade only a few trading pairs, get the only pair in the universe.
`get_by_symbols_safe`(base_token_symbol, ...)	Get a trading pair by its ticker symbols.
`get_count`()	How many trading pairs there are.
`get_one_pair_from_pandas_universe`(...[, ...])	Get a trading pair by its ticker symbols.
`get_pair`(chain_id, exchange_slug, ...[, ...])	Get a pair by its description.
`get_pair_by_human_description`(...)	Get pair by its human readable description.
`get_pair_by_id`(pair_id)	Look up pair information and return its data.
`get_pair_by_smart_contract`(address)	Resolve a trading pair by its pool smart contract address.
`get_pair_ids_by_exchange`(exchange_id)	Get all pair ids on a specific exchange.
`get_single`()	For strategies that trade only a single trading pair, get the only pair in the universe.
`get_token`(address)	Get a token that is part of any trade pair.
`iterate_pairs`()	Iterate over all pairs in this universe.

Attributes

`pair_map`	pair_id -> raw dict data mappings
`dex_pair_obj_cache`	pair_id -> constructed DEXPair cache

__init__(df, build_index=True, exchange_universe=None)[source]#

Parameters:

df (DataFrame) – The source DataFrame that contains all DEXPair entries
build_index – Build quick lookup index for pairs
exchange_universe (Optional[ExchangeUniverse]) –
Optional exchange universe needed for human-readable pair lookup.

We cannot properly resolve pairs unless we can map exchange names to their ids. Currently optional, only needed by get_pair().

pair_map: Dict[int, dict]#

pair_id -> raw dict data mappings

Constructed in one pass from Pandas DataFrame.

Don’t access directly, use iterate_pairs().

dex_pair_obj_cache: Dict[int, DEXPair]#

pair_id -> constructed DEXPair cache

Don’t access directly, use iterate_pairs().

iterate_pairs()[source]#

Iterate over all pairs in this universe.

Return type:: Iterable[DEXPair]

build_index()[source]#

Create pair_id -> data mapping.

Allows fast lookup of individual pairs.

Warning

This function assumes the universe contains data for only one blockchain. The same address can exist across multiple EVM chains. The created smart contract address index does not index chain id and thus is invalid.

get_all_pair_ids()[source]#

Get all pair ids in the data frame.

Return type:: Collection[int]

get_pair_ids_by_exchange(exchange_id)[source]#

Get all pair ids on a specific exchange.

Returns:: Raw slide of DataFrame
Parameters:: exchange_id (int) –
Return type:: DataFrame

get_count()[source]#

How many trading pairs there are.

Return type:: int

get_pair_by_id(pair_id)[source]#

Look up pair information and return its data.

Uses a cached path. Constructing DEXPair objects is a bit slow, so this is a preferred method if you need to access multiple pairs in a hot loop.

Raises:: PairNotFoundError – If pair is not found
Returns:: Nicely presented DEXPair.
Parameters:: pair_id (int) –
Return type:: Optional[DEXPair]

get_pair_by_smart_contract(address)[source]#

Resolve a trading pair by its pool smart contract address.

Warning

This function assumes the universe contains data for only one blockchain. The same address can exist across multiple EVM chains.

Parameters:: address (str) – Ethereum smart contract address of the Uniswap pair contract
Return type:: Optional[DEXPair]

get_token(address)[source]#

Get a token that is part of any trade pair.

Get a token details for a token that is base or quotetoken of any trading pair.

..note

TODO: Not a final implementation subject to chage.

Returns:: Tuple (name, symbol, address, decimals) or None if not found.
Parameters:: address (str) –
Return type:: Optional[Token]

get_all_tokens()[source]#

Get all base and quote tokens in trading pairs.

Warning

This method is useful for only test/limited pair count universes. It is very slow and mainly purported for debugging and diagnostics.

Return type:: Set[Token]

get_single()[source]#

For strategies that trade only a single trading pair, get the only pair in the universe.

Raises:

AssertionError –

If our pair universe does not have an exact single pair.

If the target pair could not be decoded.

Return type:

DEXPair

get_by_symbols(base_token_symbol, quote_token_symbol)[source]#

For strategies that trade only a few trading pairs, get the only pair in the universe.

Warning

Currently, this method is only safe for prefiltered universe. There are no safety checks if the returned trading pair is legit. In the case of multiple matching pairs, a random pair is returned.g

Raises:

PairNotFoundError – If we do not have a pair with the given symbols

Parameters:

base_token_symbol (str) –
quote_token_symbol (str) –

Return type:

Optional[DEXPair]

get_by_symbols_safe(base_token_symbol, quote_token_symbol)[source]#

Get a trading pair by its ticker symbols. In the case of multiple matching pairs, an exception is raised.

Raises:

DuplicatePair – If multiple pairs are found for the given symbols
PairNotFoundError – If we do not have a pair with the given symbols

Return DEXPair:

The trading pair

Parameters:

base_token_symbol (str) –
quote_token_symbol (str) –

Return type:

Optional[DEXPair]

get_one_pair_from_pandas_universe(exchange_id, base_token, quote_token, fee_tier=None, pick_by_highest_vol=False)[source]#

Get a trading pair by its ticker symbols.

Note that this method works only very simple universes, as any given pair is poised to have multiple tokens and multiple trading pairs on different exchanges.

Example:

# Get PancakeSwap exchange,
# for the full exchange list see https://tradingstrategy.ai/trading-view/exchanges
pancake = exchange_universe.get_by_chain_and_slug(ChainId.bsc, "pancakeswap-v2")

# Because there can be multiple trading pairs with same tickers,
# we pick the genuine among the scams based on its trading volume
wbnb_busd_pair = pair_universe.get_one_pair_from_pandas_universe(
    pancake.exchange_id,
    "WBNB",
    "BUSD",
    pick_by_highest_vol=True,
    )

print("WBNB address is", wbnb_busd_pair.base_token_address)
print("BUSD address is", wbnb_busd_pair.quote_token_address)
print("WBNB-BUSD pair contract address is", wbnb_busd_pair.address)

Parameters:

fee_tier (Optional[float]) –
Uniswap v3 and likes provide the same ticker in multiple fee tiers.

You need to use fee_tier parameter to separate the Uniswap pools. Fee tier is not needed for Uniswap v2 like exchanges as all of their trading pairs have the same fee structure.

The fee tier is 0…1 e.g. 0.0030 for 3 BPS or 0.3% fee tier.

If fee tier is not provided, then the lowest fee tier pair is returned. However the lowest fee tier might not have the best liquidity or volume.
pick_by_highest_vol – If multiple trading pairs with the same symbols are found, pick one with the highest volume. This is because often malicious trading pairs are create to attract novice users.
exchange_id (int) –
base_token (str) –
quote_token (str) –

Raises:

DuplicatePair – If the universe contains more than single entry for the pair.
PairNotFoundError – If the pair is not found in the universe.

Returns:

DEXPairs with the given symbols

Return type:

Optional[DEXPair]

get_pair(chain_id, exchange_slug, base_token, quote_token, fee_tier=None)[source]#

Get a pair by its description.

The simplest way to access pairs in the pair universe.

To use this method, we must include exchange_universe in the __init__() as otherwise we do not have required look up tables.

Returns:

The trading pair on the exchange.

Highest volume trading pair if multiple matches.

Raises:

PairNotFoundError – In the case input data cannot be resolved.

Parameters:

chain_id (ChainId) –
exchange_slug (str) –
base_token (str) –
quote_token (str) –
fee_tier (Optional[float]) –

Return type:

DEXPair

get_pair_by_human_description(exchange_universe, desc)[source]#

Get pair by its human readable description.

Look up a trading pair by chain, exchange, base, quote token tuple.

See HumanReadableTradingPairDescription for more information.

Example:

# Get BNB-BUSD pair on PancakeSwap v2
desc = (ChainId.bsc, "pancakeswap-v2", "WBNB", "BUSD")
bnb_busd = pair_universe.get_pair_by_human_description(exchange_universe, desc)
assert bnb_busd.base_token_symbol == "WBNB"
assert bnb_busd.quote_token_symbol == "BUSD"
assert bnb_busd.buy_volume_30d > 1_000_000

Another example:

pair_human_descriptions = (
    (ChainId.ethereum, "uniswap-v2", "WETH", "USDC"),  # ETH
    (ChainId.ethereum, "uniswap-v2", "EUL", "WETH", 0.0030),  # Euler 30 bps fee
    (ChainId.ethereum, "uniswap-v3", "EUL", "WETH", 0.0100),  # Euler 100 bps fee
    (ChainId.ethereum, "uniswap-v2", "MKR", "WETH"),  # MakerDAO
    (ChainId.ethereum, "uniswap-v2", "HEX", "WETH"),  # MakerDAO
    (ChainId.ethereum, "uniswap-v2", "FNK", "USDT"),  # Finiko
    (ChainId.ethereum, "sushi", "AAVE", "WETH"),  # AAVE
    (ChainId.ethereum, "sushi", "COMP", "WETH"),  # Compound
    (ChainId.ethereum, "sushi", "WETH", "WBTC"),  # BTC
    (ChainId.ethereum, "sushi", "ILV", "WETH"),  # Illivium
    (ChainId.ethereum, "sushi", "DELTA", "WETH"),  # Delta
    (ChainId.ethereum, "sushi", "UWU", "WETH"),  # UwU lend
    (ChainId.ethereum, "uniswap-v2", "UNI", "WETH"),  # UNI
    (ChainId.ethereum, "uniswap-v2", "CRV", "WETH"),  # Curve
    (ChainId.ethereum, "sushi", "SUSHI", "WETH"),  # Sushi
    (ChainId.bsc, "pancakeswap-v2", "WBNB", "BUSD"),  # BNB
    (ChainId.bsc, "pancakeswap-v2", "Cake", "BUSD"),  # Cake
    (ChainId.bsc, "pancakeswap-v2", "MBOX", "BUSD"),  # Mobox
    (ChainId.bsc, "pancakeswap-v2", "RDNT", "WBNB"),  # Radiant
    (ChainId.polygon, "quickswap", "WMATIC", "USDC"),  # Matic
    (ChainId.polygon, "quickswap", "QI", "WMATIC"),  # QiDao
    (ChainId.polygon, "sushi", "STG", "USDC"),  # Stargate
    (ChainId.avalanche, "trader-joe", "WAVAX", "USDC"),  # Avax
    (ChainId.avalanche, "trader-joe", "JOE", "WAVAX"),  # TraderJoe
    (ChainId.avalanche, "trader-joe", "GMX", "WAVAX"),  # GMX
    (ChainId.arbitrum, "camelot", "ARB", "WETH"),  # ARB
    # (ChainId.arbitrum, "sushi", "MAGIC", "WETH"),  # Magic
)

client = persistent_test_client
exchange_universe = client.fetch_exchange_universe()
pairs_df = client.fetch_pair_universe().to_pandas()
pair_universe = PandasPairUniverse(pairs_df, exchange_universe=exchange_universe)

pairs: List[DEXPair]
pairs = [pair_universe.get_pair_by_human_description(exchange_universe, d) for d in pair_human_descriptions]

assert len(pairs) == 26
assert pairs[0].exchange_slug == "uniswap-v2"
assert pairs[0].get_ticker() == "WETH-USDC"

assert pairs[1].exchange_slug == "uniswap-v2"
assert pairs[1].get_ticker() == "EUL-WETH"

Parameters:

exchange_universe (ExchangeUniverse) – The current database used to decode exchanges.
desc (Union[Tuple[ChainId, str, str, str, float], Tuple[ChainId, str, str, str]]) –

Returns:

The trading pair on the exchange.

Highest volume trading pair if multiple matches.

Raises:

PairNotFoundError – In the case input data cannot be resolved.

Return type:

DEXPair

create_parquet_load_filter(count_limit=10000)[source]#

Returns a Parquet loading filter that contains pairs in this universe.

When candle or liquidity file is read to the memory, only read pairs that are within this pair universe. This severely reduces the memory usage and speed ups loading.

See tradingstrategy.reader.read_parquet().

Parameters:: count_limit – Sanity check assert limit how many pairs we can cram into the filter.
Returns:: Filter to be passed to read_table
Return type:: List[Tuple]

static create_single_pair_universe(df, exchange, base_token_symbol, quote_token_symbol, pick_by_highest_vol=True, fee_tier=None)[source]#

Create a trading pair universe that contains only a single trading pair.

Warning

Deprecated

This is useful for trading strategies that to technical analysis trading on a single trading pair like BTC-USD.

Parameters:

df (DataFrame) – Unfiltered DataFrame for all pairs
exchange (Exchange) – Exchange instance on the pair is trading
base_token_symbol (str) – Base token symbol of the trading pair
quote_token_symbol (str) – Quote token symbol of the trading pair
pick_by_highest_vol – In the case of multiple match per token symbol, or scam tokens, pick one with the highest trade volume
fee_tier (Optional[float]) –
Pick a pair for a specific fee tier.

Uniswap v3 has

Raises:

DuplicatePair – Multiple pairs matching the criteria
PairNotFoundError – No pairs matching the criteria

Return type:

PandasPairUniverse

static create_limited_pair_universe(df, exchange, pairs, pick_by_highest_vol=True)[source]#

Create a trading pair universe that contains only few trading pairs.

Warning

Deprecated

This is useful for trading strategies that to technical analysis trading on a few trading pairs, or single pair three-way trades like Cake-WBNB-BUSD.

Parameters:

df (DataFrame) – Unfiltered DataFrame for all pairs
exchange (Exchange) – Exchange instance on the pair is trading
pairs (List[Tuple[str, str]]) – List of trading pairs as ticket tuples. E.g. [ (“WBNB, “BUSD”), (“Cake”, “WBNB”) ]
pick_by_highest_vol – In the case of multiple match per token symbol, or scam tokens, pick one with the highest trade volume

Raises:

DuplicatePair – Multiple pairs matching the criteria
PairNotFoundError – No pairs matching the criteria

Return type:

PandasPairUniverse

static create_pair_universe(df, pairs)[source]#

Create a PandasPairUniverse instance based on loaded raw pairs data.

A shortcut method to create a pair universe for a single or few trading pairs, from DataFrame of all possible trading pairs.

Example for a single pair:

pairs_df = client.fetch_pair_universe().to_pandas()
pair_universe = PandasPairUniverse.create_pair_universe(
        pairs_df,
        [(ChainId.polygon, "uniswap-v3", "WMATIC", "USDC", 0.0005)],
    )
assert pair_universe.get_count() == 1
pair = pair_universe.get_single()
assert pair.base_token_symbol == "WMATIC"
assert pair.quote_token_symbol == "USDC"
assert pair.fee_tier == 0.0005  # BPS

Example for multiple trading pairs.:

pairs_df = client.fetch_pair_universe().to_pandas()

# Create a trading pair universe for a single trading pair
#
# WMATIC-USD on Uniswap v3 on Polygon, 5 BPS fee tier and 30 BPS fee tier
#
pair_universe = PandasPairUniverse.create_pair_universe(
        pairs_df,
        [
            (ChainId.polygon, "uniswap-v3", "WMATIC", "USDC", 0.0005),
            (ChainId.polygon, "uniswap-v3", "WMATIC", "USDC", 0.0030)
        ],
    )
assert pair_universe.get_count() == 2

Parameters:

df (DataFrame) –
Pandas DataFrame of all pair data.

See tradingstrategy.client.Client.fetch_pair_universe() for more information.
pairs (Collection[Union[Tuple[ChainId, str, str, str, float], Tuple[ChainId, str, str, str]]]) –

Returns:

A trading pair universe that contains only the listed trading pairs.

Return type:

PandasPairUniverse