test section title.
news.
LOBSTER's data services will be offline from Friday 28-Feb-2014 to Friday 07-Mar-2014.
The Humboldt Universität zu Berlin department hosting LOBSTER's servers has just informed us that due to unexpected maintenance work LOBSTER's servers will not be available in the period from 28-Feb-2014 to 07-Mar-2014.
All customers will be personally informed via email about details of the downtime. We apologize to all users for the inconvenience caused.
Thank you, database update and other news. - 20th Feb 14
LOBSTER | academic data. has now been online for almost 4 months. The reception has been great and we would like to thank all customers for their trust in our young project.
We are currently working on an extension of the available data period of over 3 years into the past.
Soon the available period will cover all trading days from the 27-th of July 2007 to the day before yesterday. Those are more than 6 years of data in total and include the build up to the financial crisis.
With LOBSTER's data you will be able to do truly long term studies on the evolution of market microstructure or the effect of the financial crisis on 'your favorite limit order book topic'.
Further, the team is preparing for a change in the underlying data. In the upcoming weeks NASDAQ will switch to an updated version of their ITCH protocol which forms the basis for LOBSTER's limit order book data.
The system will be updated with the user experience in mind. The limit order book output will not change and our users will not be affected by the update.
Interested in the data? Check out the information section of the website covering all details from LOBSTER's output structure to how to get access.
Let us know if you are interested in the data. For your convenience, we are now also offering a call back service. Select 'request call back' in the contact form and let us know at which time you are available.
View more entries...
All documents required for joining LOBSTER are available in the documents section. The process of joining is detailed in the 'how to apply' guide (direct [pdf]).
updated - faq reconstruction details - entries.
- how is the limit order book initialized?
LOBSTER initializes the reconstruction of the limit order book with an empty order book each morning when the first limit order is submitted. This process guarantees that the order pool, on which LOBSTER's output is based, contains all information provided throughout the day. There are, for example, no executions of orders that were not submitted earlier the same day.
- how are messages of type 'XYZ' handled?
Provided below are comments on the implementations details of a small selection of messages in NASDAQ's TotalViewITCH. Comments are only given if the implementation is not straight forward. NASDAQ's TotalViewITCH 4.1 specification is used as reference.
- Type: 'U' (Order Replacement Message)
Description from NASDAQ's ITCH specification:
"This message is sent whenever an order on the book has been cancel-replaced. All remaining shares from the original order are no longer accessible, and must be removed. The new order details are provided for the replacement, along with a new order reference number which will be used henceforth. Since the side, stock symbol and attribution (if any) cannot be changed by an Order Replace event, these fields are not included in the message. Firms should retain the side, stock symbol and MPID from the original Add Order message."
LOBSTER's reconstruction algorithm handles messages of type 'U' by splitting the replacement into two steps: A deletion and a submission.
First, the existing order is deleted from the order pool. Second, a new (visible) limit order is submitted. Both events are assigned the time stamp of the order replacement message.
The new limit order is assigned the information in the replacement message, i.e. price, order ID and number of shares.
The deletion and submission are represented by a deletion (type '3') message and a submission (type '1') message, respectively in LOBSTER's output. Both messages appear next to each other in the output, but are only inlcuded if the events causes changes in the requested price range.
- Type: 'C' (Order Executed With Price Message)
Description from NASDAQ's ITCH specification:
"This message is sent whenever an order on the book is executed in whole or in part at a price
different from the initial display price. Since the execution price is different than the display price of
the original Add Order, NASDAQ includes a price field within this execution message.
It is possible to receive multiple Order Executed and Order Executed With Price messages for the
same order if that order is executed in several parts. The multiple Order Executed messages on the
same order are cumulative.
These executions may be marked as non-printable. If the execution is marked as non-printed, it
means that the shares will be included into a later bulk print (e.g., in the case of cross executions).
If a firm is looking to use the data in time-and-sales displays or volume calculations, NASDAQ
recommends that firms ignore messages marked as non-printable to prevent double counting."
LOBSTER's reconstruction algorithm handles messages of type 'C' like a message of Type 'E' (Order Executed Message). The time stamp, order ID and executed shares, included in a type 'C' message are used to create a message of type '4' in LOBSTER's output (execution of a visible limit order). The fields printable and execution price are ignored.
- how is the limit order book initialized?
LOBSTER initializes the reconstruction of the limit order book with an empty order book each morning when the first limit order is submitted. This process guarantees that the order pool, on which LOBSTER's output is based, contains all information provided throughout the day. There are, for example, no executions of orders that were not submitted earlier the same day.
user experience on LOBSTER | academic data.
We would like to show case the user experience on LOBSTER | academic data.
Registered users have access to a request interface and a data management system.
The request interface allows an intuitive specification of several requests at the same time and the data management system allows users to check the current status of their requests, download data and delete data from their account.
request interface.
Requests are entered into an intuitive interface, which is depicted below.
To specify requests users only have to enter the ticker, start and end date as well as the number of levels required. Setting up a request takes less than 5 seconds!
LOBSTER's request interface.
my data.
The 'my data' section, depicted below, gives an overview over the user's latest request activity.
At the top the progress of requests currently being processed is displayed.
The middle section lists the 10 latest requests available for download and an overview over the storage space occupied by the user's data is shown at the bottom.
LOBSTER's data management system - my data.
archive.
The 'archive' lists the user's files available for download under his or her account and allows the removal of data from LOBSTER's storage.
LOBSTER's data management system - archive.
more information.
Please contact us if you have any question regarding the data offered on LOBSTER or how to get access.
new name and improved website.
Welcome to the new and improved LOBSTER. We have updated the website and extended the name to
LOBSTER | academic data. now available at lobsterdata.com.
With the name extension to LOBSTER | academic data. we hope to make our mission more evident:
LOBSTER | academic data. has the goal of providing researchers with easy-to-use, high-quality limit order book data.
The new website allows a much more user friendly experience: We have updated the data request interface and introduced a personal data management system. The data management system allows users to check the current status of their requests, download data and delete data from their account. Further, we have revised the information section and introduced a new help section. Below the changes are described in more detail.
request interface.
Requests are entered into a new and intuitive interface, which is depicted below.
To specify their requests users only have to enter the ticker, start and end date as well as the number of levels required. Setting up a request takes less than 5 seconds!
LOBSTER's request interface.
my data.
The 'my data' section depicted below gives an overview over the user's latest request actvity. At the top the current progress of the latest requests is displayed.
The middle section lists the 10 lastest requests available for download and an overview over the storage space occupied by the user's data is depicted at the bottom.
LOBSTER's data management system - my data.
archive.
The 'archive' lists the user's files available for download under his or her account and allows the removal of data from LOBSTER's storage.
LOBSTER's data management system - archive.
information and help.
The redesigned website features a revised information section and a new help section with faq and sample Matlab and R code.
The new information section introduces LOBSTER | academic data., presents the output structure and offers sample files for you to try our data yourself. The two available access options - a flat rate access
for large institutes and a pay-per-use option for individual researchers - are described in detail. The three simple steps necessary to join LOBSTER | academic data. are outlined in how to join?. Background information on the reconstruction mechanism is introduced in how does it work?.
The new faq tries to answer potential questions regarding joining LOBSTER, the data and its processing as well as the requesting of data and its management. Questions regarding the reconstruction mechanism are collected in the reconstruction details. Code demos for Matlab and R are available in the code help.
Let us know if you have questions and/or suggestions for LOBSTER | academic data.
