Streaming JSONL data#

In this example, we demonstrate how to stream JSONL data from a URL and parse it into a dataframe and visualise the data. We will use the XSGD-USDC pair on Uniswap V3 token as an example.

JSON lines#

JSON Lines is a convenient format for storing structured data that may be processed one record at a time. This is preferred method for getting large amounts of OHLCV data due to the ease of incremental processing and lower memory footprint

JSON Lines is like JSON, but each new line (record) is a new JSON object. The basic structure of JSON Lines is very simple:

{"name": "John", "age": 30, "city": "New York"}
{"name": "Jane", "age": 25, "city": "Chicago"}
{"name": "Jim", "age": 35, "city": "Los Angeles"}

In the above example, each line is a valid JSON object, and each line can be parsed independently of the other lines. This property makes JSON Lines ideal for data storage where new records are appended over time, such as log files. It’s also good for data processing tasks that can be carried out in a line-by-line manner, such as Unix ‘grep’, ‘sed’, or ‘awk’.

In comparison, a regular JSON data would look like:

  {"name": "John", "age": 30, "city": "New York"},
  {"name": "Jane", "age": 25, "city": "Chicago"},
  {"name": "Jim", "age": 35, "city": "Los Angeles"}

In the above, you can see that the entire string has to be parsed as a whole to get the data, unlike JSON Lines where each line is independently a JSON object.

Data Streaming#

import requests
import pandas as pd
import json

pair_url = ""

def get_candles(url: str) -> pd.DataFrame:
    x = requests.get(url)

    data = x.text.split('\n')  # split by newline character

    json_objects = []
    for line in data:

    candles = pd.DataFrame(json_objects)
    candles.rename(columns = {'ts':'date','o':'open', 'h':'high','l':'low','c':'close','v':'volume'}, inplace = True)
    candles['timestamp'] = pd.to_datetime(candles['date'])
    candles = candles.set_index('date')
    return candles

pair_data = get_candles(pair_url)

open high low close volume xr b s tc bv sv p sb eb timestamp
1623196800 0.751273 0.761408 0.751273 0.753983 46404.942246 1.0 None None None None None 2699634 12599483 12602480 1970-01-01 00:00:01.623196800
1623283200 0.753983 0.754360 0.749023 0.754360 27170.926637 1.0 None None None None None 2699634 12607138 12608081 1970-01-01 00:00:01.623283200
1623369600 0.754511 0.754511 0.754511 0.754511 556.739452 1.0 None None None None None 2699634 12611519 12611519 1970-01-01 00:00:01.623369600
1623456000 0.749398 0.753380 0.733605 0.753380 126186.194150 1.0 None None None None None 2699634 12618259 12619487 1970-01-01 00:00:01.623456000
1623542400 0.744022 0.751574 0.744022 0.751574 44663.119939 1.0 None None None None None 2699634 12628856 12628860 1970-01-01 00:00:01.623542400

Data Visualisation#

from tradingstrategy.charting.candle_chart import visualise_ohlcv

def get_figure(candles: pd.DataFrame, chart_name: str):
    return visualise_ohlcv(

fig = get_figure(pair_data, "XSGD/USDC")