LOBSTER logo

academic
data.


output.

LOBSTER generates a 'message' and an 'orderbook' file for each active trading day of a selected ticker. The 'orderbook' file contains the evolution of the limit order book up to the requested number of levels. The ’message’ file contains indicators for the type of event causing an update of the limit order book in the requested price range. All events are timestamped to seconds after midnight, with decimal precision of at least milliseconds and up to nanoseconds depending on the requested period.

Both the 'message' and 'orderbook' files are provided in the .CSV format and can easily be read with any statistical software package.

Below the structure of the message and orderbook file are described in detail.

message file.

Time (sec)
Event Type
Order ID
Size
Price
Direction






34713.685155243
1
206833312
100
118600
-1
34714.133632201
3
206833312
100
118600
-1






variable explanation.

  • Time: Seconds after midnight with decimal precision of at least milliseconds and up to nanoseconds depending on the period requested
  • Event Type:
    • 1: Submission of a new limit order
    • 2: Cancellation (partial deletion of a limit order)
    • 3: Deletion (total deletion of a limit order)
    • 4: Execution of a visible limit order
    • 5: Execution of a hidden limit order
    • 7: Trading halt indicator (detailed information below)
  • Order ID: Unique order reference number
  • Size: Number of shares
  • Price: Dollar price times 10000 (i.e. a stock price of $91.14 is given by 911400)
  • Direction:
    • -1: Sell limit order
    • 1: Buy limit order
    • Note: Execution of a sell (buy) limit order corresponds to a buyer (seller) initiated trade, i.e. buy (sell) trade.

order book file.

Ask
Price 1
Ask
Size 1
Bid
Price 1
Bid
Size 1
Ask
Price 2
Ask
Size 2
Bid
Price 2
Bid
Size 2

...









1186600
9484
118500
8800
118700
22700
118400
14930
...
1186600
9384
118500
8800
118700
22700
118400
14930
...









variable explanation.

  • Ask Price 1: Level 1 ask price (best ask price)
  • Ask Size 1: Level 1 ask volume (best ask volume)
  • Bid Price 1: Level 1 bid price (best bid price)
  • Bid Size 1: Level 1 bid volume (best bid volume)
  • Ask Price 2: Level 2 ask price (second best ask price)
  • Ask Size 2: Level 2 ask volume (second best ask volume)
  • ...

further details.

level.

The term 'level' refers to occupied price levels. The difference between two levels in the LOBSTER output is not necessarily the minimum tick size.

message and orderbook file.

The 'message' and 'orderbook' files can be viewed as matrices of size (Nx6) and (Nx(4xNumLevel)), respectively, where N is the number of events in the requested price range and NumLevel is the number of levels requested.

The k-th row in the 'message' file describes the limit order event causing the change in the limit order book from line k-1 to line k in the 'orderbook' file.

Consider the 'message' and 'orderbook' figures above. The limit order deletion (event type 3) in the second line of the 'message' file removes 100 shares from the ask side at price 118600. The change in the 'orderbook' file from line one to two corresponds to this removal of liquidity. The volume available at the best ask price of 118600 drops from 9484 to 9384 shares.

period available.

The initial period for which limit order books are available covers all trading days on NASDAQ from January 6th 2009 to the day before yesterday. In the future, we aim to increase the available period further into the past.

unoccupied price levels.

When the selected number of levels exceeds the number of levels available, the empty order book positions are filled with dummy information to guarantee a symmetric output. The extra bid and/or ask prices are set to -9999999999 and 9999999999, respectively. The corresponding volumes are set to 0.

trading halts.

When trading halts, a message of type '7' is written into the 'message' file. The corresponding price and trade direction are set to '-1' and all other properties are set to '0'. Should the resume of quoting be indicated by an additional message in NASDAQ's Historical TotalView-ITCH files, another message of type '7' with price '0' is added to the 'message' file. Again, the trade direction is set to '-1' and all other fields are set to '0'. When trading resumes a message of type '7' and price '1' (Trade direction '-1' and all other entries '0') is written to the 'message' file. For messages of type '7', the corresponding order book rows contain a duplication of the preceding order book state.

The figure below illustrates the trading halt messages as they appear in the message file.

Time (sec)
Event Type
Order ID
Size
Price
Direction






34713.685155243
7
0
0
-1
-1






34719.193632201
7
0
0
0
-1






34801.333638703
7
0
0
1
-1






To keep the output structure as clean as possible, the reason for the trading halt is not included in the output.

other events.

Beyond the usual limit order activities - submissions, cancellations, deletions and executions - NASDAQ's Historical TotalView-ITCH files contain other messages such as net order imbalances indicators. These messages are ignored and skipped by the limit order book reconstruction algorithm, since they do not affect the evolution of the limit order book.

more information.

You can familiarize yourself with LOBSTER's data by reviewing our sample files or running our small demo codes.