Issue 2/2001 (No. 10)
Issue 2/2001. No 10.
14 January 2001.
Copyright (C)2001 Tomasz Janeczko.
All back issues available from:
http://www.amibroker.com/newsletter/
|
Issue 2/2001 (No. 10) |
AmiBroker Tips weekly newsletter. Issue 2/2001. No 10. 14 January 2001. Copyright (C)2001 Tomasz Janeczko. All back issues available from: http://www.amibroker.com/newsletter/ |
Welcome to the 2nd issue of AmiBroker Tips newsletter in the 2001. In this
issue I will describe my new little utility called "URLGet" and its
use in automatic download and update of AmiBroker's quotation database.
Just a reminder: if you have any comments/suggestions or article ideas, please don't hesitate to drop a line to newsletter@amibroker.com
In my previous articles on scripting (see 1/2000 and 4/2000 issues of the newsletter) I often used FileSystemObject to access local files. FileSystemObject is provided by Microsoft as a part of Windows Scripting Host platform. Unfortunatelly you can not access remote files using this object - therefore you can not use it to dowload the quotes from FTP or HTTP (Web) server.
In order to fix this situation I wrote a little command line program called URLGet - a simple utility that allows you to download any text file from HTTP (Web) or anonymous FTP server and store it to the local file. The program accepts two arguments: remote file URL and local file name (colours added only for clarity):
URLGet <URL_to_remote_file> <local file name>
For example, to download free Australian Stock Exchange (ASX) data from www.australian-stockmarket.com site you need to type following command:
URLGet http://www.australian-stockmarket.com/updates/20001201.prn c:\mydata\20001201.PRN
This will store the ASX data file from December 1st, 2000 to the local file 20001201.PRN located in C:\MyData directory (this file could be then imported to AmiBroker using "PRN with Ticker" importer). The destination directory must exist, of course, otherwise the download will fail. If download was successful URLGet program return value is 0, otherwise 1. One important note must be made: due to the way HTTP servers work if remote file does not exist you will often get "Error 404" file downloaded instead of empty one or just an error code. Some sites go even further, replacing standard "Error 404" page with something completely different. For example www.australian-stockmarket.com site sends you back home page if requested document was not found. This could be very annoying when you want to write a script with some error handling.
So we are now able to download the quotes using URLGet program... but how to call it from the script? Well, it is very simple:
/* first, create Shell object that allows us to run external programs */
WshShell = new ActiveXObject( "WScript.Shell" );
/* this handy function takes just URL and local file name and calls URLGet program to do the work */
function Download( URL, filename )
{
if( WshShell.Run( "URLGet " + URL + " " + filename, 0, true ) == 0 ) return true;
WScript.echo("Download of " + URL + " failed." );
return false;
}
OK, so we know how to download the file from the web and how to store it locally in predefined folder. Could we then tell AmiBroker to import that data without manual intervention? Of course we can, and it is again pretty simple:
/* Create AmiBroker application object */
AmiBroker = new ActiveXObject( "Broker.Application" );
function Import( filename )
{
/* import the data using appropriate format definition file - in our case (PRN files with Ticker) this is prnn.format file */
AmiBroker.Import( 0, filename, "prnn.format" );
/* refresh ticker list and windows */
AmiBroker.RefreshAll();
}
These are two the most important building blocks of our automatic downloader/updater script. I guess you will find them useful also for your own scripts. Now I am going to present a complete solution for automatic download of quotes from free ASX data source (www.australian-stockmarket.com site) and another one for free WSE (Warsaw Stock Exchange) quotes provided by bossa.pl . Both sites use PRN format with Ticker so most parts of the script will be common.
Our real-world example will take care about following additional issues:
The first issue is important because you can work with different databases - for example I work with NYSE, Nasdaq and WSE databases - and don't want to import WSE stocks into NYSE database. To avoid such cases the script would check for one ticker name that identifies the database. In case of ASX this will be "XAO" - ASX All Ordinaries index, in case of WSE this will be "WIG20" index. If this ticker could not be found - the script should not attempt to update currently loaded database.
The second issue is that we may forget to update our database everyday, so we want the script to check the last date stored in AmiBroker database and to download all missing quotes from that date until today.
As you can guess the whole script is a little bit complicated so I won't include complete listing here but will focus on just the user definable stuff, so let's take a look at constants that could be changed by the user:
/* The ticker to check */
ChkTicker = "XAO";
/* The folder where the files will be downloaded */
DestDir = "C:\\ASX\\";
/* The name and the path to downloader program */
DownloaderPrg = "URLGet.exe";
/* Force download - if true causes downloading data file */
/* even if it exists on the local drive */
ForceDownloadFlag = false;
/* URL from where data are downloaded */
/* Note that australian-stockmarket site changed directory for 2001 quotes */
URLPrefix = "http://www.australian-stockmarket.com/updates/";
URLPrefix2001 = "http://www.australian-stockmarket.com/updates2001/";
/* extension of file name, YYYYMMDD will be prepended */
FileExt = ".prn";
/* max number of days to download when history is empty */
nMaxHistoryLen = 365;
One note about URLPrefix2001. This constant was added because guys at www.australian-stockmarket.com site decided to put 2001 quotes in a new directory called updates2001. The old quotes are available from updates folder. So the script checks the year and switches the URLs.
The script uses C:\ASX folder by default for downloaded data. If you don't want to modify the script you should create such folder and copy the URLGet program there. Other data and program locations could be defined by changing DestDir and DownloaderPrg constants. ForceDownloadFlag could be changed to true if you want to re-download the files that were downloaded wrong. This may happen quite often with www.australian-stockmarket.com site because of unavailable data and other delays.
As for Warsaw Stock Exchange data, a service called bossa.pl does a great job in providing free accurate data available everyday at 7:00PM Warsaw time. In order to enable the script to work with their service following changes are needed:
/* The ticker to check */
ChkTicker = "WIG20";
/* The folder where the files will be downloaded */
DestDir = "C:\\WSE\\";
/* The name and the path to downloader program */
DownloaderPrg = "URLGet.exe";
/* URL from where data are downloaded */
URLPrefix = "http://bossa.pl/pub/metastock/";
URLPrefix2001 = "http://bossa.pl/pub/metastock/";
Now I suggest you to take a look at complete scripts available here: ASX downloader script and WSE downloader script. You will need also URLGet program available here.
.... and that's all for this week - hope you enjoyed reading
AmiBroker Tips weekly newsletter. Issue 2/2001. Copyright (C)2001 Tomasz Janeczko. All back issues available from: http://www.amibroker.com/newsletter/