Tags: synthetic-data, ema, trend-analysis

Osmosis synthetic data backtesting example

This is an example notebook how to backtest trading strategies on Osmosis Cosmos DEX. It is based on work done in HackAtom Seoul 2022 hackathon.

Some highlights of this notebook:

• Backtest Osmosis manually extracted data

• ATOM/OSMO pair

• Hourly OHCLV candles

• Uses simple overfitted EMA strategy (not realistic profits)

Set up

Set up strategy paramets that will decide its behavior

import datetime
import logging

import pandas as pd

from tradingstrategy.chain import ChainId
from tradingstrategy.timebucket import TimeBucket
from tradeexecutor.strategy.cycle import CycleDuration
from tradeexecutor.strategy.strategy_module import TradeRouting, ReserveCurrency

# Rebalance every 8h
trading_strategy_cycle = CycleDuration.cycle_8h

# How much of the cash to put on a single trade
position_size = 0.90

candle_time_bucket = TimeBucket.h1

chain_id = ChainId.osmosis

#
# Strategy thinking specific parameter
#

# 14 days
slow_ema_candle_count = 14*24

# 5 days
fast_ema_candle_count = 5*24

# How many candles to extract from the dataset once
batch_size = slow_ema_candle_count * 2

# Range of backtesting and synthetic data generation.
# Because we are using synthetic data actual dates do not really matter -
# only the duration

# Osmosis launched
# generate a few months of data before strategy start

start_at_data = datetime.datetime(2021, 12, 25)
start_at_strategy = datetime.datetime(2022, 4, 25)

# When our data and strategy ends
end_at = datetime.datetime(2022, 7, 25)

Create our fake exchange and pair

This will be needed to generate the candles with the same pair_id, and also later, when we generate our synthetic universe

import random
from tradeexecutor.testing.synthetic_pair_data import generate_pair
from tradeexecutor.testing.synthetic_ethereum_data import generate_random_ethereum_address
from tradeexecutor.testing.synthetic_exchange_data import generate_exchange

pair_id = 1

exchange = generate_exchange(
        exchange_id=random.randint(1, 1000),
        chain_id=chain_id,
        address=generate_random_ethereum_address(),
    )

pair = generate_pair(exchange, symbol0="ATOM", symbol1="OSMO", internal_id=pair_id)

Create our candles

Bullish data

For the purposes of this notebook, we have created bullish data, this was achieved by slightly skewing the daily_drift argument to the right of 1. Notice how it is 2% above 1 but 1.95% below 1.

Bearish data

Try skewing to the left for bearish data. I.e:

daily_drift = (0.98, 1.0195)

Ranging

No skew for sideways data! I.e.:

daily_drift = (0.98, 1.02)

Volatility

Experiment with the high_drift and low_drift parameters to adjust the volatility

# Create our candles
from tradeexecutor.testing.synthetic_price_data import generate_ohlcv_candles
from tradingstrategy.charting.candle_chart import visualise_ohlcv
import pandas as pd

candles = generate_ohlcv_candles(
    start=start_at_data,
    end=end_at,
    bucket=candle_time_bucket,
    pair_id = pair.internal_id,
    exchange_id=exchange.exchange_id,
    daily_drift=(0.9805, 1.02),  # bullish
    # daily_drift = (0.98, 1.0195),  # bearish
    # daily_drift = (0.98, 1.02),  # sideways
    high_drift=1.01,
    low_drift=0.99,
)

visualise_ohlcv(candles, chart_name="Bullish synthetic data for ATOM/OSMO", y_axis_name="Price (USD)")

Strategy logic and trade decisions

decide_trades function decide what trades to take. In this example, we calculate two exponential moving averages (EMAs) and make decisions based on those.

from typing import List, Dict

from pandas_ta.overlap import ema

from tradingstrategy.universe import Universe

from tradeexecutor.state.visualisation import PlotKind
from tradeexecutor.state.trade import TradeExecution
from tradeexecutor.strategy.pricing_model import PricingModel
from tradeexecutor.strategy.pandas_trader.position_manager import PositionManager
from tradeexecutor.state.state import State

def decide_trades(
        timestamp: pd.Timestamp,
        universe: Universe,
        state: State,
        pricing_model: PricingModel,
        cycle_debug_data: Dict) -> List[TradeExecution]:
    """The brain function to decide the trades on each trading strategy cycle."""

    # The pair we are trading
    pair = universe.pairs.get_single()

    assert pair.token0_symbol == "ATOM", f"Got pair {pair}"
    assert pair.token1_symbol == "OSMO", f"Got pair {pair}"

    # How much cash we have in the hand
    cash = state.portfolio.get_current_cash()

    # Get OHLCV candles for our trading pair as Pandas Dataframe.
    # We could have candles for multiple trading pairs in a different strategy,
    # but this strategy only operates on single pair candle.
    # We also limit our sample size to N latest candles to speed up calculations.
    candles: pd.DataFrame = universe.candles.get_single_pair_data(timestamp, sample_count=batch_size)

    # We have data for open, high, close, etc.
    # We only operate using candle close values in this strategy.
    close = candles["close"]

    # Calculate exponential moving averages based on slow and fast sample numbers.
    # https://github.com/twopirllc/pandas-ta
    # https://github.com/twopirllc/pandas-ta/blob/bc3b292bf1cc1d5f2aba50bb750a75209d655b37/pandas_ta/overlap/ema.py#L7
    slow_ema_series = ema(close, length=slow_ema_candle_count)
    fast_ema_series = ema(close, length=fast_ema_candle_count)

    if slow_ema_series is None or fast_ema_series is None:
        # Cannot calculate EMA, because
        # not enough samples in backtesting
        return []

    slow_ema = slow_ema_series.iloc[-1]
    fast_ema = fast_ema_series.iloc[-1]

    # Get the last close price from close time series
    # that's Pandas's Series object
    # https://pandas.pydata.org/docs/reference/api/pandas.Series.iat.html
    current_price = close.iloc[-1]

    # List of any trades we decide on this cycle.
    # Because the strategy is simple, there can be
    # only zero (do nothing) or 1 (open or close) trades
    # decides
    trades = []

    # Create a position manager helper class that allows us easily to create
    # opening/closing trades for different positions
    position_manager = PositionManager(timestamp, universe, state, pricing_model)

    if not position_manager.is_any_open():
        if current_price >= slow_ema:
        # Entry condition:
        # Close price is higher than the slow EMA
            buy_amount = cash * position_size
            trades += position_manager.open_1x_long(pair, buy_amount)
    else:
        if slow_ema >= fast_ema:
        # Exit condition:
        # Fast EMA crosses slow EMA
            trades += position_manager.close_all()

    # Visualize strategy
    # See available Plotly colours here
    # https://community.plotly.com/t/plotly-colours-list/11730/3?u=miohtama
    visualisation = state.visualisation
    visualisation.plot_indicator(timestamp, "Slow EMA", PlotKind.technical_indicator_on_price, slow_ema, colour="darkblue")
    visualisation.plot_indicator(timestamp, "Fast EMA", PlotKind.technical_indicator_on_price, fast_ema, colour="#003300")

    return trades

Defining trading universe

We create a trading universe with a single blockchain, exchange and trading pair. For the sake of easier understanding the code, we name this “Uniswap v2” like exchange with a single ETH-USDC trading pair.

The trading pair contains generated noise-like OHLCV trading data.

from tradeexecutor.strategy.trading_strategy_universe import TradingStrategyUniverse
from tradeexecutor.testing.synthetic_universe_data import create_synthetic_single_pair_universe

def create_trading_universe() -> TradingStrategyUniverse:

    trading_strategy_universe = create_synthetic_single_pair_universe(
        candles,
        chain_id,
        exchange,
        candle_time_bucket,
        pair,
        lending_candles=None
    )

    return trading_strategy_universe

Running the backtest

Run backtest using giving trading universe and strategy function.

Running the backtest outputs state object that contains all the information on the backtesting position and trades.

from tradeexecutor.testing.synthetic_exchange_data import generate_simple_routing_model
from tradeexecutor.backtest.backtest_runner import run_backtest_inline

universe = create_trading_universe()

start_candle, end_candle = universe.universe.candles.get_timestamp_range()
print(f"Our universe has synthetic candle data for the period {start_candle} - {end_candle}")

# This function set ups trade routing for our synthetic trading universe.
# Because we have only one trading pair, there is no complicated
# routing needed
routing_model = generate_simple_routing_model(universe)

state, universe,    debug_dump = run_backtest_inline(
    name="ATOM/OSMO backtest",
    start_at=start_at_strategy,
    end_at=end_at,
    client=None,  # None of downloads needed, because we are using synthetic data
    cycle_duration=trading_strategy_cycle,  # Override to use 24h cycles despite what strategy file says
    decide_trades=decide_trades,
    universe=universe,
    initial_deposit=10_000,
    reserve_currency=ReserveCurrency.busd,
    trade_routing=TradeRouting.user_supplied_routing_model,
    routing_model=routing_model,
    log_level=logging.WARNING,
)

Our universe has synthetic candle data for the period 2021-12-25 00:00:00 - 2022-07-24 23:00:00

Examine backtest results

Examine state that contains all actions the trade executor took.

We plot out a chart that shows - The price action - When the strategy made buys or sells

print(f"Positions taken: {len(list(state.portfolio.get_all_positions()))}")
print(f"Trades made: {len(list(state.portfolio.get_all_trades()))}")
print(f"Visualisation entries: {len(list(state.visualisation.plots))}")

Positions taken: 15
Trades made: 30
Visualisation entries: 2

from tradeexecutor.visual.single_pair import visualise_single_pair

figure = visualise_single_pair(state, universe.universe.candles, start_at=start_at_strategy)
figure.update_layout(template="plotly_dark")  # Dark color theme https://plotly.com/python/templates/
figure.show()

Equity curve and drawdown

Visualise equity curve and related performnace over time.

• Returns

• Drawdown

• Daily returns

# Set Jupyter Notebook output mode parameters
# Used to avoid warnings
from tradeexecutor.backtest.notebook import setup_charting_and_output
setup_charting_and_output()

# Needed to improve the resolution of matplotlib chart used here
%config InlineBackend.figure_format = 'svg'

from tradeexecutor.visual.equity_curve import calculate_equity_curve, calculate_returns
from tradeexecutor.visual.equity_curve import visualise_equity_curve

curve = calculate_equity_curve(state)
returns = calculate_returns(curve)
visualise_equity_curve(returns)

Returns monthly breakdown

• Monthly returns

• Best day/week/month/year

from tradeexecutor.visual.equity_curve import visualise_returns_over_time

visualise_returns_over_time(returns)

Benchmarking the strategy performance

Here we benchmark the strategy performance against some baseline scenarios.

• Buy and hold US dollar

• Buy and hold the underlying trading pair base asset

from tradeexecutor.visual.benchmark import visualise_benchmark

traded_pair = universe.universe.pairs.get_single()

fig = visualise_benchmark(
    state.name,
    portfolio_statistics=state.stats.portfolio,
    all_cash=state.portfolio.get_initial_deposit(),
    buy_and_hold_asset_name=traded_pair.base_token_symbol,
    buy_and_hold_price_series=universe.universe.candles.get_single_pair_data()["close"],
)

fig.update_layout(template="plotly_dark")  # Dark color theme https://plotly.com/python/templates/
fig.show()

Analysing the strategy success

Here we calculate statistics on how well the strategy performed.

• Won/lost trades

• Timeline of taken positions with color coding of trade performance

from tradeexecutor.analysis.trade_analyser import build_trade_analysis

analysis = build_trade_analysis(state.portfolio)

Strategy summary

Overview of strategy performance

from IPython.core.display_functions import display

summary = analysis.calculate_summary_statistics(candle_time_bucket, state)

with pd.option_context("display.max_row", None):
    summary.display()

Returns
Annualised return %	211.88%
Lifetime return %	49.15%
Realised PnL	$4,914.74
Trade period	84 days 16 hours

Holdings
Total assets	$14,914.74
Cash left	$14,914.74
Open position value	$0.00
Open positions	0

	Winning	Losing	Total
Closed Positions
Number of positions	7	8	15
% of total	46.67%	53.33%	100.00%
Average PnL %	12.68%	-3.23%	4.19%
Median PnL %	2.26%	-1.70%	-0.48%
Biggest PnL %	74.53%	-8.64%	-
Average duration	196 bars	10 bars	97 bars
Max consecutive streak	3	3	-
Max runup / drawdown	136.18%	-34.23%	-

	Stop losses	Take profits
Position Exits
Triggered exits	0	0
Percent winning	-	-
Percent losing	-	-
Percent of total	0.00%	0.00%

Risk Analysis
Biggest realized risk	90.00%
Average realized risk	-2.91%
Max pullback of capital	-12.74%
Sharpe Ratio	233.84%
Sortino Ratio	391.02%
Profit Factor	124.27%

Performance metrics

Here is an example how to use Quantstats library to calculate the tearsheet metrics for the strategy with advanced metrics. The metrics include popular risk-adjusted return comparison metrics.

This includes metrics like:

• Sharpe

• Sortino

• Max drawdown

Note: These metrics are based on equity curve and returns. Analysis here does not go down to the level of an individual trade or a position. Any consecutive wins and losses are measured in days, not in trade or candle counts.

from tradeexecutor.visual.equity_curve import calculate_equity_curve, calculate_returns
from tradeexecutor.analysis.advanced_metrics import visualise_advanced_metrics, AdvancedMetricsMode

equity = calculate_equity_curve(state)
returns = calculate_returns(equity)
metrics = visualise_advanced_metrics(returns, mode=AdvancedMetricsMode.full)

with pd.option_context("display.max_row", None):
    display(metrics)

	Strategy
Start Period	2022-04-25
End Period	2022-07-24
Risk-Free Rate	0.0%
Time in Market	73.0%
Cumulative Return	49.15%
CAGR﹪	405.95%
Sharpe	1.35
Prob. Sharpe Ratio	88.12%
Smart Sharpe	1.28
Sortino	2.17
Smart Sortino	2.06
Sortino/√2	1.53
Smart Sortino/√2	1.46
Omega	1.24
Max Drawdown	-34.23%
Longest DD Days	38
Volatility (ann.)	48.27%
Calmar	11.86
Skew	0.46
Kurtosis	1.93
Expected Daily	0.15%
Expected Monthly	10.51%
Expected Yearly	49.15%
Kelly Criterion	8.72%
Risk of Ruin	0.0%
Daily Value-at-Risk	-3.98%
Expected Shortfall (cVaR)	-3.98%
Max Consecutive Wins	5
Max Consecutive Losses	8
Gain/Pain Ratio	0.44
Gain/Pain (1M)	3.63
Payoff Ratio	1.54
Profit Factor	1.24
Common Sense Ratio	1.76
CPC Index	0.85
Tail Ratio	1.42
Outlier Win Ratio	4.89
Outlier Loss Ratio	3.46
MTD	-12.66%
3M	49.15%
6M	49.15%
YTD	49.15%
1Y	49.15%
3Y (ann.)	405.95%
5Y (ann.)	405.95%
10Y (ann.)	405.95%
All-time (ann.)	405.95%
Best Day	9.63%
Worst Day	-7.69%
Best Month	59.2%
Worst Month	-12.66%
Best Year	49.15%
Worst Year	49.15%
Avg. Drawdown	-5.69%
Avg. Drawdown Days	3
Recovery Factor	1.44
Ulcer Index	0.19
Serenity Index	0.19
Avg. Up Month	34.85%
Avg. Down Month	-7.79%
Win Days	44.67%
Win Month	50.0%
Win Quarter	50.0%
Win Year	100.0%

Position and trade timeline

Display all positions and how much profit they made.

from tradeexecutor.analysis.trade_analyser import expand_timeline

timeline = analysis.create_timeline()

expanded_timeline, apply_styles = expand_timeline(
        universe.universe.exchanges,
        universe.universe.pairs,
        timeline)

# Do not truncate the row output
with pd.option_context("display.max_row", None):
    display(apply_styles(expanded_timeline))

Remarks	Type	Opened at	Duration	Exchange	Base asset	Quote asset	Position max value	PnL USD	PnL %	Open mid price USD	Close mid price USD	Trade count	LP fees
	Long	2022-04-25	8 hours		ATOM	OSMO	$9,000.00	$203.46	2.26%	$3,132.490478	$3,203.305485	2	$9.10
	Long	2022-04-26	8 hours		ATOM	OSMO	$9,183.11	$379.17	4.13%	$3,250.620899	$3,384.838097	2	$9.38
	Long	2022-04-27	55 days 8 hours		ATOM	OSMO	$9,524.36	$7,098.07	74.53%	$3,503.302555	$6,114.150891	2	$13.08
	Long	2022-06-29	8 hours		ATOM	OSMO	$15,912.62	$-603.07	-3.79%	$5,784.851555	$5,565.612410	2	$15.61
	Long	2022-07-01	8 hours		ATOM	OSMO	$15,369.86	$-73.40	-0.48%	$5,895.645722	$5,867.491641	2	$15.34
	Long	2022-07-02	8 hours		ATOM	OSMO	$15,303.80	$-1,322.08	-8.64%	$5,841.496198	$5,336.855353	2	$14.65
	Long	2022-07-06	8 hours		ATOM	OSMO	$14,113.93	$51.99	0.37%	$5,705.125314	$5,726.139261	2	$14.14
	Long	2022-07-07	8 hours		ATOM	OSMO	$14,160.72	$-67.54	-0.48%	$5,967.578273	$5,939.115722	2	$14.13
	Long	2022-07-08	8 hours		ATOM	OSMO	$14,099.93	$-156.65	-1.11%	$5,739.033604	$5,675.271910	2	$14.03
	Long	2022-07-08	8 hours		ATOM	OSMO	$13,958.95	$467.19	3.35%	$5,750.644720	$5,943.112219	2	$14.20
	Long	2022-07-09	8 hours		ATOM	OSMO	$14,379.42	$-118.06	-0.82%	$5,907.742542	$5,859.236703	2	$14.32
	Long	2022-07-10	1 days		ATOM	OSMO	$14,273.16	$-1,177.63	-8.25%	$5,965.922750	$5,473.694626	2	$13.69
	Long	2022-07-17	8 hours		ATOM	OSMO	$13,213.29	$280.19	2.12%	$5,342.759564	$5,456.054046	2	$13.36
	Long	2022-07-18	8 hours		ATOM	OSMO	$13,465.47	$-307.58	-2.28%	$5,548.820986	$5,422.072700	2	$13.31
	Long	2022-07-19	8 hours		ATOM	OSMO	$13,188.64	$260.70	1.98%	$5,385.013756	$5,491.458809	2	$13.32

Finishing notes

Print out a line to signal the notebook finished the execution successfully.

print("All ok")

All ok