https://uk.mathworks.com/help/finance/machine-learning-with-financial-data.html?s_tid=CRUX_lftnav
Updated queueing system.
As our customer base increases, so are the usage patterns of our users. We saw an increase in very big data queries the last weeks, which our queuing system handled efficiently but not very fair-minded with respect to users that wanted to see their small queries fulfilled in a reasonable time frame. To accommodate all of our users, we had to tweak the queuing system of LOBSTER.
From now on we have a fixed amount of threads (currently three) that run in parallel associated with each user. Of course you can still enter all your queries at once, and three of them will always start immediately while all others get queued until the first one finishes.
LOBSTER data now available back to 2007.
We updated the LOBSTER database to include data up to the 27th of June 2007.
LOBSTER update complete.
With the new LOBSTER framework in place all of NASDAQs limit order book data from 6th January 2009 up until yesterday is now available. More historical data will follow swiftly.
UPDATE new source data format.
Nasdaq has introduced a new ITCH format last Monday. We are testing our updated platform currently and plan to roll out the new LOBSTER version next Monday (Oct 13th). The service will not be available for about two hours on that day. Until then you will still be able to access data up to Sep 26th.
With the new version we also update the whole LOBSTER platform and with it the possibility to add even more historical data. As a first step, data from Jan 6th 2009 on up until yesterday will be available starting next Monday. More historical data will follow swift.
Thank you, database update and other news.
LOBSTER | academic data. has has now been online for almost 4 months. The reception has been great and we would like to thank all customers for their trust in our young project.
We are currently working on an extension of the available data period of over 3 years into the past. Soon the available period will cover all trading days from the 27-th of July 2007 to the day before yesterday. Those are more than 6 years of data in total and include the build up to the financial crisis. With LOBSTER’s data you will be able to do truly long term studies on the evolution of market microstructure or the effect of the financial crisis on ‘your favorite limit order book topic’.
Further, the team is preparing for a change in the underlying data. In the upcoming weeks NASDAQ will switch to an updated version of their ITCH protocol which forms the basis for LOBSTER’s limit order book data. The system will be updated with the user experience in mind. The limit order book output will not change and our users will not be affected by the update.
Interested in the data? Check out the information section of the website covering all details from LOBSTER’s output structure to how to get access.
Let us know if you are interested in the data. For your convenience, we are now also offering a call back service. Select ‘request call back’ in the contact form and let us know at which time you are available.
output update.
We have updated LOBSTER’s reconstruction algorithm. Besides speed improvements in the reconstruction process the output has been augmented. It now includes trading halt information under the new event type ‘7’.
The beginning and end of a trading halt are indicated by messages of type ‘7’ with prices set to ‘-1’ and ‘1’, respectively. A potential resume of quoting is indicated by a type ‘7’ message with the price set to ‘0’. For these event messages the order book contains a duplication of the preceding order book state to keep the symmetry of the files. More information can be found in the updated output structure section.
The incorporation of trading halt information has great advantages for researchers: Non trading periods are precisely identified thus improving data cleaning procedures and making the arbitrary rules of thumb, previously used for identification, obsolete. Further, trading halts are indicators for order book behaviors that are potentially of great interest to researchers.
Additionally, we have defined the intraday time period covered by LOBSTER’s standard limit order book output. For the time being, the output will be restricted to NASDAQ’s official trading hours from 09:30:00 to 16:00:00. The decision is based on feedback from potential users highlighting a particular focus on the order book dynamics during regular trading hours and the implied reductions in download times as well as the reduced hard disk requirements.
Be assured this definition, of course, does not prohibit a future extension to allow, for example, an optional inclusion of the pre-trading session in the output.