Tags: synthetic-data, ema, trend-analysis

Osmosis synthetic data backtesting example#

This is an example notebook how to backtest trading strategies on Osmosis Cosmos DEX. It is based on work done in HackAtom Seoul 2022 hackathon.

Some highlights of this notebook:

Set up#

Set up strategy paramets that will decide its behavior

[1]:
import datetime
import logging

import pandas as pd

from tradingstrategy.chain import ChainId
from tradingstrategy.timebucket import TimeBucket
from tradeexecutor.strategy.cycle import CycleDuration
from tradeexecutor.strategy.strategy_module import TradeRouting, ReserveCurrency

# Rebalance every 8h
trading_strategy_cycle = CycleDuration.cycle_8h

# How much of the cash to put on a single trade
position_size = 0.90

candle_time_bucket = TimeBucket.h1

chain_id = ChainId.osmosis

#
# Strategy thinking specific parameter
#

# 14 days
slow_ema_candle_count = 14*24

# 5 days
fast_ema_candle_count = 5*24

# How many candles to extract from the dataset once
batch_size = slow_ema_candle_count * 2

# Range of backtesting and synthetic data generation.
# Because we are using synthetic data actual dates do not really matter -
# only the duration

# Osmosis launched
# generate a few months of data before strategy start

start_at_data = datetime.datetime(2021, 12, 25)
start_at_strategy = datetime.datetime(2022, 4, 25)

# When our data and strategy ends
end_at = datetime.datetime(2022, 7, 25)

Create our fake exchange and pair#

This will be needed to generate the candles with the same pair_id, and also later, when we generate our synthetic universe

[2]:
import random
from tradeexecutor.testing.synthetic_pair_data import generate_pair
from tradeexecutor.testing.synthetic_ethereum_data import generate_random_ethereum_address
from tradeexecutor.testing.synthetic_exchange_data import generate_exchange

pair_id = 1

exchange = generate_exchange(
        exchange_id=random.randint(1, 1000),
        chain_id=chain_id,
        address=generate_random_ethereum_address(),
    )

pair = generate_pair(exchange, symbol0="ATOM", symbol1="OSMO", internal_id=pair_id)

Create our candles#

Bullish data#

For the purposes of this notebook, we have created bullish data, this was achieved by slightly skewing the daily_drift argument to the right of 1. Notice how it is 2% above 1 but 1.95% below 1.

Bearish data#

Try skewing to the left for bearish data. I.e:

daily_drift = (0.98, 1.0195)

Ranging#

No skew for sideways data! I.e.:

daily_drift = (0.98, 1.02)

Volatility#

Experiment with the high_drift and low_drift parameters to adjust the volatility

[3]:
# Create our candles
from tradeexecutor.testing.synthetic_price_data import generate_ohlcv_candles
from tradingstrategy.charting.candle_chart import visualise_ohlcv
import pandas as pd

candles = generate_ohlcv_candles(
    start=start_at_data,
    end=end_at,
    bucket=candle_time_bucket,
    pair_id = pair.internal_id,
    exchange_id=exchange.exchange_id,
    daily_drift=(0.9805, 1.02),  # bullish
    # daily_drift = (0.98, 1.0195),  # bearish
    # daily_drift = (0.98, 1.02),  # sideways
    high_drift=1.01,
    low_drift=0.99,
)

visualise_ohlcv(candles, chart_name="Bullish synthetic data for ATOM/OSMO", y_axis_name="Price (USD)")

Strategy logic and trade decisions#

decide_trades function decide what trades to take. In this example, we calculate two exponential moving averages (EMAs) and make decisions based on those.

[4]:
from typing import List, Dict

from pandas_ta.overlap import ema

from tradingstrategy.universe import Universe

from tradeexecutor.state.visualisation import PlotKind
from tradeexecutor.state.trade import TradeExecution
from tradeexecutor.strategy.pricing_model import PricingModel
from tradeexecutor.strategy.pandas_trader.position_manager import PositionManager
from tradeexecutor.state.state import State

def decide_trades(
        timestamp: pd.Timestamp,
        universe: Universe,
        state: State,
        pricing_model: PricingModel,
        cycle_debug_data: Dict) -> List[TradeExecution]:
    """The brain function to decide the trades on each trading strategy cycle."""

    # The pair we are trading
    pair = universe.pairs.get_single()

    assert pair.token0_symbol == "ATOM", f"Got pair {pair}"
    assert pair.token1_symbol == "OSMO", f"Got pair {pair}"

    # How much cash we have in the hand
    cash = state.portfolio.get_current_cash()

    # Get OHLCV candles for our trading pair as Pandas Dataframe.
    # We could have candles for multiple trading pairs in a different strategy,
    # but this strategy only operates on single pair candle.
    # We also limit our sample size to N latest candles to speed up calculations.
    candles: pd.DataFrame = universe.candles.get_single_pair_data(timestamp, sample_count=batch_size)

    # We have data for open, high, close, etc.
    # We only operate using candle close values in this strategy.
    close = candles["close"]

    # Calculate exponential moving averages based on slow and fast sample numbers.
    # https://github.com/twopirllc/pandas-ta
    # https://github.com/twopirllc/pandas-ta/blob/bc3b292bf1cc1d5f2aba50bb750a75209d655b37/pandas_ta/overlap/ema.py#L7
    slow_ema_series = ema(close, length=slow_ema_candle_count)
    fast_ema_series = ema(close, length=fast_ema_candle_count)

    if slow_ema_series is None or fast_ema_series is None:
        # Cannot calculate EMA, because
        # not enough samples in backtesting
        return []

    slow_ema = slow_ema_series.iloc[-1]
    fast_ema = fast_ema_series.iloc[-1]

    # Get the last close price from close time series
    # that's Pandas's Series object
    # https://pandas.pydata.org/docs/reference/api/pandas.Series.iat.html
    current_price = close.iloc[-1]

    # List of any trades we decide on this cycle.
    # Because the strategy is simple, there can be
    # only zero (do nothing) or 1 (open or close) trades
    # decides
    trades = []

    # Create a position manager helper class that allows us easily to create
    # opening/closing trades for different positions
    position_manager = PositionManager(timestamp, universe, state, pricing_model)

    if not position_manager.is_any_open():
        if current_price >= slow_ema:
        # Entry condition:
        # Close price is higher than the slow EMA
            buy_amount = cash * position_size
            trades += position_manager.open_1x_long(pair, buy_amount)
    else:
        if slow_ema >= fast_ema:
        # Exit condition:
        # Fast EMA crosses slow EMA
            trades += position_manager.close_all()

    # Visualize strategy
    # See available Plotly colours here
    # https://community.plotly.com/t/plotly-colours-list/11730/3?u=miohtama
    visualisation = state.visualisation
    visualisation.plot_indicator(timestamp, "Slow EMA", PlotKind.technical_indicator_on_price, slow_ema, colour="darkblue")
    visualisation.plot_indicator(timestamp, "Fast EMA", PlotKind.technical_indicator_on_price, fast_ema, colour="#003300")

    return trades

Defining trading universe#

We create a trading universe with a single blockchain, exchange and trading pair. For the sake of easier understanding the code, we name this “Uniswap v2” like exchange with a single ETH-USDC trading pair.

The trading pair contains generated noise-like OHLCV trading data.

[5]:
from tradeexecutor.strategy.trading_strategy_universe import TradingStrategyUniverse
from tradeexecutor.testing.synthetic_universe_data import create_synthetic_single_pair_universe

def create_trading_universe() -> TradingStrategyUniverse:

    trading_strategy_universe = create_synthetic_single_pair_universe(
        candles,
        chain_id,
        exchange,
        candle_time_bucket,
        pair,
        lending_candles=None
    )

    return trading_strategy_universe


Running the backtest#

Run backtest using giving trading universe and strategy function.

Running the backtest outputs state object that contains all the information on the backtesting position and trades.

[6]:
from tradeexecutor.testing.synthetic_exchange_data import generate_simple_routing_model
from tradeexecutor.backtest.backtest_runner import run_backtest_inline

universe = create_trading_universe()

start_candle, end_candle = universe.universe.candles.get_timestamp_range()
print(f"Our universe has synthetic candle data for the period {start_candle} - {end_candle}")

# This function set ups trade routing for our synthetic trading universe.
# Because we have only one trading pair, there is no complicated
# routing needed
routing_model = generate_simple_routing_model(universe)

state, universe,    debug_dump = run_backtest_inline(
    name="ATOM/OSMO backtest",
    start_at=start_at_strategy,
    end_at=end_at,
    client=None,  # None of downloads needed, because we are using synthetic data
    cycle_duration=trading_strategy_cycle,  # Override to use 24h cycles despite what strategy file says
    decide_trades=decide_trades,
    universe=universe,
    initial_deposit=10_000,
    reserve_currency=ReserveCurrency.busd,
    trade_routing=TradeRouting.user_supplied_routing_model,
    routing_model=routing_model,
    log_level=logging.WARNING,
)

Our universe has synthetic candle data for the period 2021-12-25 00:00:00 - 2022-07-24 23:00:00

Examine backtest results#

Examine state that contains all actions the trade executor took.

We plot out a chart that shows - The price action - When the strategy made buys or sells

[7]:
print(f"Positions taken: {len(list(state.portfolio.get_all_positions()))}")
print(f"Trades made: {len(list(state.portfolio.get_all_trades()))}")
print(f"Visualisation entries: {len(list(state.visualisation.plots))}")
Positions taken: 15
Trades made: 30
Visualisation entries: 2
[8]:
from tradeexecutor.visual.single_pair import visualise_single_pair

figure = visualise_single_pair(state, universe.universe.candles, start_at=start_at_strategy)
figure.update_layout(template="plotly_dark")  # Dark color theme https://plotly.com/python/templates/
figure.show()

Equity curve and drawdown#

Visualise equity curve and related performnace over time.

  • Returns

  • Drawdown

  • Daily returns

[9]:
# Set Jupyter Notebook output mode parameters
# Used to avoid warnings
from tradeexecutor.backtest.notebook import setup_charting_and_output
setup_charting_and_output()

# Needed to improve the resolution of matplotlib chart used here
%config InlineBackend.figure_format = 'svg'

from tradeexecutor.visual.equity_curve import calculate_equity_curve, calculate_returns
from tradeexecutor.visual.equity_curve import visualise_equity_curve

curve = calculate_equity_curve(state)
returns = calculate_returns(curve)
visualise_equity_curve(returns)
[9]:
../../_images/programming_strategy-examples_hackatom-2022-osmosis-backtest_18_0.svg

Returns monthly breakdown#

  • Monthly returns

  • Best day/week/month/year

[10]:
from tradeexecutor.visual.equity_curve import visualise_returns_over_time

visualise_returns_over_time(returns)
[10]:
../../_images/programming_strategy-examples_hackatom-2022-osmosis-backtest_20_0.svg

Benchmarking the strategy performance#

Here we benchmark the strategy performance against some baseline scenarios.

  • Buy and hold US dollar

  • Buy and hold the underlying trading pair base asset

[11]:
from tradeexecutor.visual.benchmark import visualise_benchmark

traded_pair = universe.universe.pairs.get_single()

fig = visualise_benchmark(
    state.name,
    portfolio_statistics=state.stats.portfolio,
    all_cash=state.portfolio.get_initial_deposit(),
    buy_and_hold_asset_name=traded_pair.base_token_symbol,
    buy_and_hold_price_series=universe.universe.candles.get_single_pair_data()["close"],
)

fig.update_layout(template="plotly_dark")  # Dark color theme https://plotly.com/python/templates/
fig.show()

Analysing the strategy success#

Here we calculate statistics on how well the strategy performed.

  • Won/lost trades

  • Timeline of taken positions with color coding of trade performance

[12]:
from tradeexecutor.analysis.trade_analyser import build_trade_analysis

analysis = build_trade_analysis(state.portfolio)

Strategy summary#

Overview of strategy performance

[13]:
from IPython.core.display_functions import display

summary = analysis.calculate_summary_statistics(candle_time_bucket, state)

with pd.option_context("display.max_row", None):
    summary.display()
Returns
Annualised return % 211.88%
Lifetime return % 49.15%
Realised PnL $4,914.74
Trade period 84 days 16 hours
Holdings
Total assets $14,914.74
Cash left $14,914.74
Open position value $0.00
Open positions 0
Winning Losing Total
Closed Positions
Number of positions 7 8 15
% of total 46.67% 53.33% 100.00%
Average PnL % 12.68% -3.23% 4.19%
Median PnL % 2.26% -1.70% -0.48%
Biggest PnL % 74.53% -8.64% -
Average duration 196 bars 10 bars 97 bars
Max consecutive streak 3 3 -
Max runup / drawdown 136.18% -34.23% -
Stop losses Take profits
Position Exits
Triggered exits 0 0
Percent winning - -
Percent losing - -
Percent of total 0.00% 0.00%
Risk Analysis
Biggest realized risk 90.00%
Average realized risk -2.91%
Max pullback of capital -12.74%
Sharpe Ratio 233.84%
Sortino Ratio 391.02%
Profit Factor 124.27%

Performance metrics#

Here is an example how to use Quantstats library to calculate the tearsheet metrics for the strategy with advanced metrics. The metrics include popular risk-adjusted return comparison metrics.

This includes metrics like:

  • Sharpe

  • Sortino

  • Max drawdown

Note: These metrics are based on equity curve and returns. Analysis here does not go down to the level of an individual trade or a position. Any consecutive wins and losses are measured in days, not in trade or candle counts.

[14]:
from tradeexecutor.visual.equity_curve import calculate_equity_curve, calculate_returns
from tradeexecutor.analysis.advanced_metrics import visualise_advanced_metrics, AdvancedMetricsMode

equity = calculate_equity_curve(state)
returns = calculate_returns(equity)
metrics = visualise_advanced_metrics(returns, mode=AdvancedMetricsMode.full)

with pd.option_context("display.max_row", None):
    display(metrics)
Strategy
Start Period 2022-04-25
End Period 2022-07-24
Risk-Free Rate 0.0%
Time in Market 73.0%
Cumulative Return 49.15%
CAGR﹪ 405.95%
Sharpe 1.35
Prob. Sharpe Ratio 88.12%
Smart Sharpe 1.28
Sortino 2.17
Smart Sortino 2.06
Sortino/√2 1.53
Smart Sortino/√2 1.46
Omega 1.24
Max Drawdown -34.23%
Longest DD Days 38
Volatility (ann.) 48.27%
Calmar 11.86
Skew 0.46
Kurtosis 1.93
Expected Daily 0.15%
Expected Monthly 10.51%
Expected Yearly 49.15%
Kelly Criterion 8.72%
Risk of Ruin 0.0%
Daily Value-at-Risk -3.98%
Expected Shortfall (cVaR) -3.98%
Max Consecutive Wins 5
Max Consecutive Losses 8
Gain/Pain Ratio 0.44
Gain/Pain (1M) 3.63
Payoff Ratio 1.54
Profit Factor 1.24
Common Sense Ratio 1.76
CPC Index 0.85
Tail Ratio 1.42
Outlier Win Ratio 4.89
Outlier Loss Ratio 3.46
MTD -12.66%
3M 49.15%
6M 49.15%
YTD 49.15%
1Y 49.15%
3Y (ann.) 405.95%
5Y (ann.) 405.95%
10Y (ann.) 405.95%
All-time (ann.) 405.95%
Best Day 9.63%
Worst Day -7.69%
Best Month 59.2%
Worst Month -12.66%
Best Year 49.15%
Worst Year 49.15%
Avg. Drawdown -5.69%
Avg. Drawdown Days 3
Recovery Factor 1.44
Ulcer Index 0.19
Serenity Index 0.19
Avg. Up Month 34.85%
Avg. Down Month -7.79%
Win Days 44.67%
Win Month 50.0%
Win Quarter 50.0%
Win Year 100.0%

Position and trade timeline#

Display all positions and how much profit they made.

[15]:
from tradeexecutor.analysis.trade_analyser import expand_timeline

timeline = analysis.create_timeline()

expanded_timeline, apply_styles = expand_timeline(
        universe.universe.exchanges,
        universe.universe.pairs,
        timeline)

# Do not truncate the row output
with pd.option_context("display.max_row", None):
    display(apply_styles(expanded_timeline))

Remarks Type Opened at Duration Exchange Base asset Quote asset Position max value PnL USD PnL % Open mid price USD Close mid price USD Trade count LP fees
Long 2022-04-25 8 hours ATOM OSMO $9,000.00 $203.46 2.26% $3,132.490478 $3,203.305485 2 $9.10
Long 2022-04-26 8 hours ATOM OSMO $9,183.11 $379.17 4.13% $3,250.620899 $3,384.838097 2 $9.38
Long 2022-04-27 55 days 8 hours ATOM OSMO $9,524.36 $7,098.07 74.53% $3,503.302555 $6,114.150891 2 $13.08
Long 2022-06-29 8 hours ATOM OSMO $15,912.62 $-603.07 -3.79% $5,784.851555 $5,565.612410 2 $15.61
Long 2022-07-01 8 hours ATOM OSMO $15,369.86 $-73.40 -0.48% $5,895.645722 $5,867.491641 2 $15.34
Long 2022-07-02 8 hours ATOM OSMO $15,303.80 $-1,322.08 -8.64% $5,841.496198 $5,336.855353 2 $14.65
Long 2022-07-06 8 hours ATOM OSMO $14,113.93 $51.99 0.37% $5,705.125314 $5,726.139261 2 $14.14
Long 2022-07-07 8 hours ATOM OSMO $14,160.72 $-67.54 -0.48% $5,967.578273 $5,939.115722 2 $14.13
Long 2022-07-08 8 hours ATOM OSMO $14,099.93 $-156.65 -1.11% $5,739.033604 $5,675.271910 2 $14.03
Long 2022-07-08 8 hours ATOM OSMO $13,958.95 $467.19 3.35% $5,750.644720 $5,943.112219 2 $14.20
Long 2022-07-09 8 hours ATOM OSMO $14,379.42 $-118.06 -0.82% $5,907.742542 $5,859.236703 2 $14.32
Long 2022-07-10 1 days ATOM OSMO $14,273.16 $-1,177.63 -8.25% $5,965.922750 $5,473.694626 2 $13.69
Long 2022-07-17 8 hours ATOM OSMO $13,213.29 $280.19 2.12% $5,342.759564 $5,456.054046 2 $13.36
Long 2022-07-18 8 hours ATOM OSMO $13,465.47 $-307.58 -2.28% $5,548.820986 $5,422.072700 2 $13.31
Long 2022-07-19 8 hours ATOM OSMO $13,188.64 $260.70 1.98% $5,385.013756 $5,491.458809 2 $13.32

Finishing notes#

Print out a line to signal the notebook finished the execution successfully.

[16]:
print("All ok")
All ok