Historical Stock Data

historical-stock-data

This is a while I’m trying to find historical stock data to build an AI system. Sadly it’s not as easy as it appears. I’m putting together my findings here:

  • Option 1: Using Yahoo’s CSV approach above you can also get historical data!
    You can reverse engineer the following example:

http://ichart.finance.yahoo.com/table.csv?s=AMZN&d=0&e=28&f=2016&g=d&a=3&b=12&c=1996&ignore=.csv

Essentially:

sn = TICKER
a = fromMonth-1
b = fromDay (two digits)
c = fromYear
d = toMonth-1
e = toDay (two digits)
f = toYear
g = d for day, m for month, y for yearly

The complete list of parameters:

a Ask
a2 Average Daily Volume
a5 Ask Size
b Bid
b2 Ask (Real-time)
b3 Bid (Real-time)
b4 Book Value
b6 Bid Size
c Change & Percent Change
c1 Change
c3 Commission
c6 Change (Real-time)
c8 After Hours Change (Real-time)
d Dividend/Share
d1 Last Trade Date
d2 Trade Date
e Earnings/Share
e1 Error Indication (returned for symbol changed / invalid)
e7 EPS Estimate Current Year
e8 EPS Estimate Next Year
e9 EPS Estimate Next Quarter
f6 Float Shares
g Day’s Low
h Day’s High
j 52-week Low
k 52-week High
g1 Holdings Gain Percent
g3 Annualized Gain
g4 Holdings Gain
g5 Holdings Gain Percent (Real-time)
g6 Holdings Gain (Real-time)
i More Info
i5 Order Book (Real-time)
j1 Market Capitalization
j3 Market Cap (Real-time)
j4 EBITDA
j5 Change From 52-week Low
j6 Percent Change From 52-week Low
k1 Last Trade (Real-time) With Time
k2 Change Percent (Real-time)
k3 Last Trade Size
k4 Change From 52-week High
k5 Percent Change From 52-week High
l Last Trade (With Time)
l1 Last Trade (Price Only)
l2 High Limit
l3 Low Limit
m Day’s Range
m2 Day’s Range (Real-time)
m3 50-day Moving Average
m4 200-day Moving Average
m5 Change From 200-day Moving Average
m6 Percent Change From 200-day Moving Average
m7 Change From 50-day Moving Average
m8 Percent Change From 50-day Moving Average
n Name
n4 Notes
o Open
p Previous Close
p1 Price Paid
p2 Change in Percent
p5 Price/Sales
p6 Price/Book
q Ex-Dividend Date
r P/E Ratio
r1 Dividend Pay Date
r2 P/E Ratio (Real-time)
r5 PEG Ratio
r6 Price/EPS Estimate Current Year
r7 Price/EPS Estimate Next Year
s Symbol
s1 Shares Owned
s7 Short Ratio
t1 Last Trade Time
t6 Trade Links
t7 Ticker Trend
t8 1 yr Target Price
v Volume
v1 Holdings Value
v7 Holdings Value (Real-time)
w 52-week Range
w1 Day’s Value Change
w4 Day’s Value Change (Real-time)
x Stock Exchange
y Dividend Yield

  • Option 2: With less than 100$ one can get the end of day data straight from CSI data, this is where Yahoo gets their EOD data as well AFAIK. They have an API where you can extract the data to whatever format you want.
  • Option 3 : For playing around with some intraday data I’d look into IQFeed, they provide several APIs to extract historical data, although they are mainly an outfit for real-time feeds. But here there are quite a few options, some brokers even provide historical data downloads via their APIs, so just pick your poison.
  • Option 4 : Then, if you need the real stuff (level II order book, all ticks as they have happened at all exchanges) one “affordable”, yet excellent option is Nanex. They’ll actually ship you a drive with terabytes of data. Its about $3k-4K per year of data. But trust me, once you understand how hard it is to get good intraday data, you won’t think this is very much money at all.

 

ATS (Automated Trading System):

  1. Quantpian –  a free, community-centered, hosted platform for building and executing trading strategies.
  2. Zipline – Zipline is a Pythonic algorithmic trading library. It is an event-driven system that supports both backtesting and live-trading. Zipline is currently used in production as the backtesting and live-trading engine powering Quantopian — a free, community-centered, hosted platform for building and executing trading strategies.
  3. Quantconnect: https://www.quantconnect.com/lean/
  4. tradelink
    http://code.google.com/p/tradelink/
    Write automated trading systems, connect with 17+ broker APIsAIOTrade
    http://sourceforge.net/projects/humaitrader
    AIOTrade (formerly Humai Trader Platform) is a free, open source stock technical analysis platform built on pure java.manticore-trader
    http://www.manticore-projects.com/
    manticore-trader is a free and open java software for day trading warants on stocks, currencies and comodities. It includes modules for charting, position and risk management, automatic ordering and system trading. Instruments and quotes of the main financial markets are provided dailyG-BOT
    http://www.datatime.eu/public/gbot/
    G-BOT is a public academic project, headed by Prof. Tom Gastaldi (first University of Rome “La Sapienza”). The project is about the study of trading algorithms and fully automated strategies for systematic profitability.

    Marketcetera

    http://trac.marketcetera.org/
    http://www.marketcetera.com/
    Marketcetera focuses on building the key trading functions that are common to all organizations, thus freeing our clients to concentrate on proprietary trading algorithms and other specialized software that provide a competitive advantage.

    Merchant of Venice

    http://sourceforge.net/projects/mov
    http://mov.sourceforge.net/
    MOV is a stock market trading programme that supports portfolio management, charting, technical analysis, paper trading and genetic programming. Venice runs in a graphical user interface with online help and has full documentation.EclipseTrader
    http://sourceforge.net/projects/eclipsetrader/
    http://eclipsetrader.sourceforge.net/
    Eclipse Rich Client Platform (RCP) application featuring shares pricing watch, intraday and history charts with technical analysis indicators, level II/market depth view, news watching, and integrated trading.

    JBookTrader
    http://code.google.com/p/jbooktrader/
    All aspects of trading, such as obtaining market prices, analyzing price patterns, making trading decisions, placing orders, monitoring order executions, and controlling the risk are automated according to the user preferences. JBookTrader is a “sister” project to JSystemTrader.

    Matrex

    http://sourceforge.net/projects/matrex/
    http://matrex.sourceforge.net/
    The perfect desktop tool for mathematical, statistical models and complex calculations. Adapters to matlab, scilab, octave, R.

    OpenGamma
    http://www.opengamma.com/
    OpenGamma provides technology for financial institutions to improve analytics calculation and delivery to front-office and risk users.

    Open Java Trading System (Last Update 2010-08-14)
    http://sourceforge.net/projects/ojts/
    http://ojts.sourceforge.net/
    The Open Java Trading System (OJTS) is meant to be a common infrastructure to develop (stock) trading systems. There are four parts: gathering of raw data over the internet, recognition of trading signals, a visualization module and trading with banks.

    Joone
    http://sourceforge.net/projects/joone/
    Joone is a neural net framework written in Java(tm). It’s composed by a core engine, a GUI editor and a distributed training environment and can be extended by writing new modules to implement new algorithms or architectures starting from base component

    Data Visualizer (Last Update 2009-07-17)
    http://sourceforge.net/projects/dataviews
    http://dataviews.sourceforge.net/
    Modular environment for graphical visualization of stock market type data

    SFL Java Trading System Enviroment
    (Last Update 2009-07-17)
    http://sourceforge.net/projects/sfljtse
    http://www.sflweb.org/index.php?blog=sfljtse
    Java application built on KISS principle ( Keep It Simple,Stupid ) and its aim is to provide a fast and platform indipendent infrastructure to develop and execute trading systems.

    ActiveQuant (Last Update 2009)
    http://www.activequant.org/
    Somewhat heavy stuff, for proficient programmers only

    JSystemTrader (Last Update 2009)
    Developed to work with Interactive Broker’s API, fully automated trading system (ATS) that can trade various types of market securities during the trading day without user monitoring.

    Market Analysis System (Last Update 2009-07-17)
    http://sourceforge.net/projects/eiffel-mas
    http://eiffel-mas.sourceforge.net/
    System for analysis of financial markets using technical analysis. Includes facilities for stock charting and futures charting, as well as automated generation of trading signals based on user-selected criteria. Operates on both daily and intraday data.

    Oropuro trading system (Last Update 2009)
    http://sourceforge.net/projects/oropuro
    http://www.oropuro.org
    Complete technical analysis & trading system, full set of features: retrieve, analyze EOD stocks data; manage multiple portfolios; technical analysis & graphical rendering; neural networks for generation of trading signals; support trader community,

    AlgoTrader
    http://code.google.com/p/algo-trader/
    AlgoTrader is an automated trading system (ATS) that can trade any type of security on any market available through InteractiveBrokers or FIX. All aspects of trading like getting market data, analyzing prices, taking trade decisions, placing orders & tracking executions can be automated.

    Encog
    Encog Java and DotNet Neural Network Framework | Heaton Research
    Encog Ninjatrader Getting Started – Encog Machine Learning Framework

    Encog is an advanced neural network and machine learning framework. Encog contains classes to create a wide variety of networks, as well as support classes to normalize and process data for these neural networks. Encog trains using multithreaded resilient propagation. Encog can also make use of a GPU to further speed processing time. A GUI based workbench is also provided to help model and train neural networks.

 

ats

 

Useful links:

  1. Barchart : Stock data provider , Stocks with most daily changes
  2. Quantquote: https://quantquote.com/products_second-data.php
  3. Andy Pavlo data base 2006 http://cs.brown.edu/~pavlo/pennystocks/
  4. Quantqoute free S&P 500 data : https://quantquote.com/historical-stock-data
  5. Do the Executive Program in Algorithmic Trading offered by QuantInsti.com
  6. Ernie P. Chan’s book titled: Quantitative Trading: How to build your own algorithmic trading business. This book is the basics.

Leave a Reply