INTERACTIVE BROKERS DATA PLUGIN

AmiBroker Now supports Real-Time streaming quotes from Interactive Brokers TWS

IMPORTANT: YOU DON'T NEED TO INSTALL the plugin if you installed AmiBroker 5.70 or later. It is ALREADY installed by AmiBroker setup.

FEATURES SUMMARY:

• supports upto 100 streaming symbols in real time (equal to IB TWS limit)
• supports all base time intervals: 15-,5-,1-minute, 15-,5-second, tick
• automatic connection (no need to manually "accept incoming connection" in TWS)
• supports upto 30 (180*) DAYS intraday data BACKFILL in 1-minute bar interval
• upto 2000 bars backfill using 1-sec/5-sec/15-second bar intervals
• works on all 32-bit Windows versions
  
  * 180 days backfill may be slow due to IB throttling the backfill requests

NOTE ABOUT IB/TWS PERFORMANCE:

IB / TWS backfill is very limited (1 symbol at a time) and VERY VERY slow. IB throttles backfills and you can not send more than 60 requests within 5 minutes (it equals to one month of 1-minute data for 12 symbols only).

For much faster backfills we recommend eSignal or IQFeed.

IMPORTANT SYSTEM REQUIREMENTS NOTE:

while AmiBroker itself does not have big requirements (see Getting Started page ), Interactive Brokers' TWS is Java-based application that is memory and CPU hungry. You would need to have at least 800MHz CPU to use TWS, see official TWS system requirements page: http://www.interactivebrokers.com/en/software/requirements.php

IMPORTANT NOTE ON IB BACKFILL IN TICK MODE:

The finest resolution of BACKFILLS that Interactive Brokers TWS offers is 1-SECOND bars (see TWS API docs here). It means that although you can collect streaming real-time data in tick format, the backfill will always have resolution limited to 1-second bars.
Also IB TWS streaming data are NOT tick-by-tick, but rather 0.2-0.3 second snapshots, read this for details: http://www.interactivebrokers.com/cgi-bin/discus/board-auth.pl?file=/2/37364.html

For this reason we recommend using higher intervals like 5-sec, 15-sec or better yet 1 minute.

HISTORY:

• 2.0.4 - newest version, released July 12, 2010
  
  For Windows XP, Vista, Windows 7, Windows NT, 98
  
  + includes a compatiblity fix for TWS 907+
  
  
• 2.0.3, released April 20, 2010
  
  For Windows XP, Vista, Windows 7, Windows NT, 98
  
  Upgraded for AmiBroker 5.30 (it is already included in the setup of AmiBroker 5.30)
  
• 2.0.2, released October 2, 2009
  
  for Windows XP, Vista, Win 7, Win 2000, Win NT, Win98
  
  + supports 64 bit date/time and float volume/openint (AmiBroker 4.27 and above)
  + on Vista and Windows 7 previous versions of IB plugin crashed when requesting invalid symbol because AfxIsValidAddress function is broken on Vista/Win7. Fixed by rewriting code not to use AfxIsValidAddress anymore.
  
  
• 1.8.4, released October 26, 2008
  
  + fix for non-US indices traded with different currencies than USD
  
  
• 1.8.2 released October 23, 2008
  + the plugin now accepts single-letter security type codes in addition to 3 letter codes. So now security type can be:
  S or STK for stocks
  F or FUT for futures
  O or OPT for options
  P or FOP for futures options
  C or CASH for currencies (Forex)
  I or IND for indices
  
  This is provided to shorten symbols for instruments quoted in non-US currencies so they fit within maximum 25 character length. For example "FSMI DEC 08-SOFFEX-F-CHF" (Swiss index futures quoted in CHF)
  
• 1.8.1 released October 22, 2008
  + due to possible ambiguity of SMART routing when same symbol is traded with multiple currencies, IB plugin now allows to specify CURRENCY in a symbol. The symbol format is now: SYMBOL-EXCHANGE-TYPE-CURRENCY.
  + all symbols that do not have explicit currency specification use USD now (except forex).
  The default currency is USD and it is used when nothing is specified as 4th part of symbol. So for example MSFT gets internally resolved to MSFT-SMART-STK-USD
  
  
• 1.8.0 released October 15, 2008 (obsolete - please use 1.8.1)
  + fix to "duplicate ticker ID" error occuring with newest TWS when more than 5 day backfill was requested
  
• 1.7.1 released June 10, 2008
  + Extended backfill upto 180 days (experimental / may be slow / subject to IB throttling) (one-year backfill removed because of instability)
  
• 1.7.0 released May 8, 2008
  + Open price available in RT quote window (for stocks)
  + Extended backfill 180+ days (experimental / may be slow / subject to IB throttling)
  + Better automatic reconnection handling on network disconnection
  + Updated error code handling (code 162 - data pacing violation)
  + Updated to use new TWS API 9.41
  
  
  
• 1.6.8 released October 10, 2006
  
  + decreased CPU load during backfill
  + decreased CPU load during high activity periods
  + upgraded to newest TWSAPI 9.0 and tested with newest TWS 863 as well as 862 and 861.
  
  
• 1.6.6 released July 6, 2006
  
  + PrimaryExchange is now set to empty string and only local symbol is used when requesting data. This solves "Invalid symbol" problem occuring for some accounts during last 2 weeks following apparent changes in IB.
  
• 1.6.7 released March 31, 2006
  
  + now streaming connection uses small buffer, while backfill connection uses large buffer
  + added protection against stopping backfill when user scrolls through symbol list and invalid ticker is encountered
  
• 1.6.0 released March 30, 2006
  
  + added handling of TWS error message = 300 (can't find Eid)
  + added handling of TWS error message = 165 (HDMS disconnect) - allows to workaround TWS throttling
  + added "Cancel Backfill" menu option
  + added "Minimum" backfill length that backfills in less than a day if you have already some data from today
  + most recently selected backfill length is stored between session
  + added separate connection for backfill to workaround TWS problems with hanging backfills plus automatic reconnection if backfill times out or HDMS disconnects
  + added some runtime checks for valid pointers to workaround TWS bug of sending incorrect IDs for certain messages
  + quotes after backfill are flushed from plugin memory
  + added get_quotes notification for use in future versions of AmiBroker
  
  
• 1.5.1 released January 31, 2006
  
  + fixed problem with -IND symbols (bug appeared in 1.5.0)
  + 1.4.4 socket code re-used (newer one was working fine on all but one machine)
  
  
• 1.5.0 released January 27, 2006
  
  
  + 1-minute backfill length is now extended to upto 30 days,
  user selectable from RIGHT mouse button menu over plugin status area.
  (Note that due to TWS limitation backfills longer than 5 days are splitted into chunks of 5 days and downloaded sequentially)

  + Implemented workaround to IB weird way of sending tickSize events
  
  Problem was that IB sometimes repeats many times the same tick and sometimes skips some ticks and cumulate them into one tickSize LAST_SIZE event.
  This is because IB feed was NOT designed as tick-by-tick and never intended to create time&sales series, but rather solely to update TWS display grid.
  Now the plugin tries to workaround this weirdness by ignoring duplicate tickSize LAST_SIZE events (with the same size) and correcting missing ticks usign CUMULATIVE volume sent with tickSize VOLUME event. Correction is needed because without it we would end up having incorrect total volume (sometimes real trades may have same events so multiple REAL ticks may have same price/size, unfortunatelly there is no way to detect whenever it is real trade or duplicate generated by IB, so correction according to cumulative volume is the only way to go).

  + plugin now accepts "O" and "OP" as type specification and treats it as "OPT"
  
  This allows to get quotes for some options that have very long symbols (exceeding 26 characters allowed by AB).
  For example to get DAX options use this symbols:
  
  C ODAX MAR 06 5500-DTB-O
  P ODAX MAR 06 5500-DTB-O

  (note that there are TWO spaces between 06 (year code) and 5500 (price).)

  + Fixed some problems with Microsoft's CSocket buggy implementation

  + IB API error message 165 is used now to detect whenever backfill is available or not (demo account for example does not offer backfill)

• 1.4.4 released October 19, 2005
  
  + core API parts rewritten to use fast buffered sockets - gives upto 10x performance improvement
  (previous versions used EClientSocket code supplied as a part of TWS API. Unfortunatelly this code was using single-byte-at-a-time read from socket and it was terrible performance hog when backfills were longer than one day. New buffered socket code is able to read upto 4096 bytes at once)
  
  
• 1.4.3 released October 14, 2005
  
  + full 5-day backfill for all symbols (1-minute bar interval)
  + IDEALPRO now uses MIDPRICE instead of BID to get cleaner backfill, but user can switch back to BID in config screen.
  
  
• 1.4.2 released June 16th, 2005

  This version offers slightly improved performance (less CPU usage)

• 1.4.1 released June 13th, 2005
  
  This version fixes real-time update for IDEALPRO (forex) market, i.e. symbols like
  EUR.USD-IDEALPRO-CASH are updated correctly in real time.
  (IB does not send last trade price/size updates for Forex so BID price is used instead)
  
  
• 1.4.0 released May 1st, 2005
  
  supports backfill (read below for the details), does not require any API installation. Already included with full setup of AmiBroker 4.70.5
  
  Known issues:
  
  a) backfill is not available on DEMO accounts (this is the limitation of TWS)
  b) backfill data for some symbols may not be present at IB servers
  c) UseRTH (backfill extended hours on/off switch) does not seem to be working (bug in TWS reported to IB already)
  d) TWS supports only ONE backfill at a time, so plugin prevents from triggering more.
  e) TWS only allows certain amounts of data to be downloaded depending on base time interval
  
  For more details please read TWS API BETA Release Notes
  

INSTRUCTIONS:

NOTE: Interactive Brokers TWS is CPU-hungry application, therefore for best results we recommend using machine with 1GHz processor or faster.

1. YOU DON'T NEED TO INSTALL the plugin if you installed AmiBroker 5.70 or later. It is ALREADY included.
   download IB plugin from:
   http://www.amibroker.com/bin/ib204/IB.dll
   Current STABLE version of IB.DLL: 2.0.4
   
   and copy it to PLUGINS subfolder of AmiBroker directory.
   
2. run web-based TWS or download standalone TWS
3. In TWS, select Configure -> API -> Enable Active X and Socket clients
   Also enter 127.0.0.1 in TWS, Configure->API->Trusted IP addresses menu to prevent "Allow incoming connection?" dialog.
4. Run AmiBroker and create new database with Interactive Brokers plugin as a data source, following these steps:

• Run AmiBroker

• Choose File->New database

• Type a new folder name (for example: C:\Program Files\AmiBroker\IB ) and click Create as shown in the picture below:
   
  
  
  

• Choose InteractiveBrokers(r) data Plug-in from Data source combo and "Enable" from Local data storage

• Enter 30000 or more into "Number of bars to load" field

• Now choose Base time interval. Supported intervals are: EOD, hourly, 15-minute, 5-minute, 1-minute. Professional Edition of AmiBroker allows also to select Tick, 5-second, 15-second intervals.
  
  Note that backfill is in bar interval of 1-minute or less (TWS limitation).
  
  If you want to have long daily histories AND intraday charts you should consider running TWO instances of AmiBroker. One for EOD charts and second for intraday charting. Both instances may use IB as a data source.

• Click OK.

From now on your AmiBroker reads quotes directly from the Interactive Brokers.

SYMBOLOGY

Symbol format now uses the symbol mode of TWS, not the underlying mode. The symbol mode in TWS can be seen in the 'View->Symbol Mode' menu option in TWS.

The format is: SYMBOL-EXCHANGE-TYPE-CURRENCY

where

SYMBOL is the same as the symbol column as displayed in TWS while under symbol mode

EXCHANGE (optional) is the exchange d in TWS while under symbol mode

TYPE (optional) is one the following:

STK or S - stocks, FUT or F- futures, FOP or P - options on futures, OPT or O - options, IND or I - indexes, CASH or C -cash (ideal FX)
Note that for stocks only the EXCHANGE and TYPE fields are optional. The exchange will be set to BEST (SMART) and the TYPE will be set to STK. Note that SINGLE LETTER type codes are allowed ONLY in version 1.8.2 and above.

CURRENCY (optional - ONLY plugin version 1.8.1 and above ) - is the currency that given symbol is traded with. The default for STK, FUT, FOP, OPT, IND types is USD (US dollar). The default currency for CASH (forex) is empty.

Please take special care when typing symbols as some of them (futures) have MULTIPLE SPACES in the symbol name. You have to type EXACTLY THE SAME number of spaces as provided in the examples below
(see the dashes below symbol name that make it easier to see the number of characters)

Examples:

CSCO

GE

VOD-LSE

ESM1-GLOBEX-FUT

QQQFJ-CBOE-OPT

INDU-NYSE-IND

YM   JUN 11-ECBOT-FUT
  ---   -  

QMN5-NYMEX-FUT

EUR.USD-IDEAL-CASH
EUR.USD-IDEALPRO-CASH

Again:

ECBOT futures symbols have length of 21 characters with 3 spaces between contract symbol and month name and one space between month and 2 digit year

NOTES ON IB API LIMITATIONS:

1. Backfill is available for REAL IB accounts only (not on demo)

2. Open price is NOT provided by IB. For that reason Open field is empty in real time quote window

3. The data from IB does not include a timestamp on the trades. The current system time is used to timestamp each tick.

4. IB TWS streaming data are NOT tick-by-tick, but rather 0.2-0.3 second snapshots, read this for details: http://www.interactivebrokers.com/cgi-bin/discus/board-auth.pl?file=/2/37364.html

HOW TO USE BACKFILL FEATURE

Backfill feature in plugin 1.3.7 allows to download 24 intraday historical data to fill-in the gaps that may have occurred when AmiBroker / TWS is not running.

IB Backfill feature is configurable from File->Database Settings, Configure :

Two main backfill-related settings are:

1. request length
2. automatic backfill

When request length is considered, as explained in TWS API Release Notes at: http://www.interactivebrokers.com/en/software/apiReleaseNotes/apiBetanotes.php currently IB backfill feature is limited to some fixed duration / bar interval ranges. For example you can get maximum 2000 1-second ticks, maximum 10000 seconds in 5-second interval (2000 bars), maximum 30000 seconds in 15-second interval (also 2000 bars) and maximum of 5 DAYS of 1-minute bars.

By default AmiBroker uses maximum allowable amounts.

As for "automatic backfill on first data access" - when it is checked AmiBroker attempts to backfill symbol when you display a chart for given symbol (or perform backtest or scan). Please note that TWS API currently allows only one backfill at a time so when there is a backfill already running in the background, automatic backfill request for next symbol will be ignored, until previous backfill is complete.

It is convenient to have this option turned on, however it can cause additional load on your internet connection because of data needed to be downloaded during backfill process.

If you switch "automatic backfill on first data access" option off, you will still be able to backfill data for current symbol or all symbols in real-time quote window list usign appropriate menu options from plugin status menu.

Backfill Current option allows to force backfill of currently selected symbol, while Backfill All RTQ symbols allow to force backfill of all symbols listed in Real-Time Quote window. Backfill of multiple symbols is performed sequentially (one at a time) due to limitations of TWS.

During backfilling a tooltip pops up informing the user about symbol being currently backfilled and plugin status color changes to light blue (turquoise) as shown below: