AmiBroker 6.02.0 BETA Read Me

October 30, 2015 12:55

THIS IS A BETA VERSION OF THE SOFTWARE. EXPECT BUGS !!!

Backup your data files and entire AmiBroker folder first!

INSTALLATION INSTRUCTIONS

First you need to have full version of AmiBroker 6.00 installed. Then just run the BETA installer and follow the instructions.

Then run AmiBroker. You should see "AmiBroker 6.02.0 BETA" written in the About box.

See CHANGE LOG below for detailed list of changes. Note that only changes that affect end-user directly are listed here. Internal code changes/refactoring is usually not mentioned.

CHANGE LOG

CHANGES FOR VERSION 6.02.0 (as compared to 6.01.0)

• AFL Editor: Implemented Line comment/uncomment feature (Edit->Line Comment, Ctrl+Q) to automatically add/remove // line comments from single or multiple lines. If multiple lines are selected, the content of the first selected line is deciding whenever block of lines is to be commented or uncommented
• AFL: If any matrix cell has Null value then matrix product operator @ produces Null respective row/column of the result matrix.
• AFL: if user called (a) GetPerfomanceCounter( 1 ) then (b) GetPerformanceCounter( 0 ) then subsequent call to GetPerformanceCounter(0) returned cumulated time not from call (a) but from system boot. Fixed.
• AFL: matrix identifier can now be used in if-else statement. Such condition checks whenever very first element of matrix [0][0] is NOT NULL. This is useful for checking output of functions like MxInverse/MxSolve that would return NULL in all cells if matrix is singular
  So

  m = Matrix( 10, 10, 0 );

// do something with matrix
if( m )
{
// some code
}

is equivalent to

if( NOT IsNull( m[ 0 ][ 0 ] ) )
{
// some code
}

• AFL: MxSolve/MxInverse now return a matrix filled with Nulls as a result if source matrix can not be inverted and produce warning level 2 instead of an error.
• AFL: On Windows Vista and higher static variables use slim read-write (SRW) lock instead of critical section. This gives 5% performance increase in multithreading scenarios.
• AFL: StaticVarAdd( "name", value, keepAll = True, persistent = False ) - an atomic addition (interlocked read-add-write) operation for static variables
  
  It is multithreading safe addition for static variables that are shared by multiple threads. This function is atomic with respect to calls to other static variable functions.

  KeepAll flag when it is set to true emulates the behavior of AddToComposite. It keeps all values that are already present, so if data holes exists in current symbol,
  the bars that are present in static variable but not present in current symbol remain untouched.
  When KeepAll is set to false then only bars that are present in current symbol are kept. Any other bars that were present in static variable but not present in currently
  processed symbols are removed. That is what normally happens with StaticVarSet().

  In fact when KeepAll is set to False, StaticVarAdd can be seen as the following pseudo code:
  EnterCriticalSection
x = Nz( StaticVarGet( "name" ) ); // read exisiting value (and convert Nulls to zero)
x += Nz( value ); // add value to existing
StaticVarSet( "name", x ); // store updated value
LeaveCriticalSection

  The function can be used to create composites like this:

  if( status("stocknum") == 0 )
{
    // remove any earier composite values
   StaticVarRemove("~Composite");
}

StaticVarAdd( "~Composite", MACD() > Signal() );
Buy = 0;

  NOTES:
  1. StaticVarAdd automatically converts all Nulls to zeros (as AddToComposite does).
  2. If you want to replace AddToComposite with StaticVarAdd, keep in mind that by default AddToComposite skips symbols in group 253. This is done so composite symbols
  are not added to themselves. If you have composite symbols in your database and want to skip symbols in group 253 you can use
  if( GroupID() != 253 ) StaticVarAdd("~Composite", values );
  3. Thanks to extensive code tuning, StaticVarAdd generally offers better performance than AddToComposite which was already blazing fast. Single threaded StaticVarAdd may be twice as fast as ATC. With 8 threads running StaticVarAdd may be 4x as fast (it does not scale as much as naive person may think, because critical section limits performance due to lock contention). To illustrate the amount of fine tuning applied it can be said that first 'straightforward' version of StaticVarAdd was actually 20 times slower than ATC.
  4. Be careful when using "quickafl" as StaticVarAdd would not increase 'required bars' (as ATC does), so if you want to actually add all bars and quick afl is turned on in analysis, it is better to add SetBarsRequired(sbrAll, sbrAll)

• AFL: Study() returned NULL array when line's start date was greater than end date. Fixed (now it works for lines drawn from right to left too).
• Docs: Example polynomial fit formula shows how to gracefully handle singular matrix and overflow/not-a-numbers in polynomial calcs
  
  order = Param( "n-th Order", 10, 1, 16, 1 );
length = 60;

lvb = BarCount - 1;
fvb = lvb - length;

yy = Matrix( length + 1, 1, 0 );
xx = Matrix( length + 1, order + 1, 1 );

yy = MxSetBlock( yy, 0, length, 0, 0, Ref( C, fvb ) );

x = BarIndex() - length/2;

for( j = 1; j <= order; j++ )
{
     xx = MxSetBlock( xx, 0, length, j, j, x ^ j );
}

xxt = MxTranspose( xx );
aa = MxSolve( xxt @ xx, xxt ) @ yy;
//aa = MxInverse( xxt @ xx ) @ xxt @ yy; // alternative way
   
if( aa ) // check if matrix is not null (so solution exists)
{
    rr = Null; // store the fit in rr
   for( i = fvb; i <= lvb; i++ )
    {
       rr[i] = aa[0][0];

       for( j = 1; j <= order; j++ )
       {
          rr[i] += aa[j][0] * x[ i - fvb ] ^ j;
       }
    }

     if( IsNan( rr[ fvb ] ) )
     {
       // our polynomial yields infinite or not-a-number result due to overflow/underflow
      Title = "Polyfit failed. The order of polynomial is too High";
     }
     else
     {
       SetChartOptions( 0, chartShowDates );
       SetBarFillColor( IIf( C > O, ColorRGB( 0, 75, 0 ), IIf( C <= O, ColorRGB( 75, 0, 0 ), colorLightGrey ) ) );
       Plot( rr, "rr", colorWhite, styleLine | styleThick);
    }
}
else
{
   Title = "Matrix is singular. The order of polynomial is too high";
}

Plot( C, "", IIf( C > O, ColorRGB( 0, 255, 0 ), IIf( C <= O, ColorRGB( 255, 0, 0 ), colorLightGrey ) ), styleDots | styleNoLine );
  
• Persistent static variables are now saved to PersistVars.temp and once write is successful the file is renamed to PersistVars.bin. This is to prevent data loss when writing very large sets of persistent variables.
• When more than one error is detected in single line of the formula then the first error message is displayed instead of last one in chart/commentary/analysis.
  

CHANGES FOR VERSION 6.01.0 (as compared to 6.00.0)

• AFL: MxDet( mx, method = 0 ) - calculates determinant of the matrix
  method = 0 - auto (use slow method for matrices of upto and including 5x5, fast for larger matrices)
  method = 1 - slow (slow, more accurate)
  method = 2 - fast (LU decomposition, less accurate )

  "slow" method uses Laplace expansion
  " fast" method uses LU decomposition
  " Slow" method for small matrices (1x1, 2x2, 3x3, 4x4) is actually faster than "fast", equally fast for matrix 5x5 and
  slower than "fast" method for matrices larger than 5x5

  For this reason "auto" method uses "fast" LU method only for matrices larger than 5x5

  LU decomposition is fast but subject to higher numerical errors. "Slow" method is slower yet produces much more reliable results.
  For example Octave/MatLab that use LU decomposition would say that determinant of singular matrix like this
  { {16, 2, 3, 13}, { 5, 11, 10, 8}, {9, 7, 6, 12}, {4, 14, 15, 1 } }
  is -1.4495e-012 due to roundoff errors of LU method.

  If you want to calculate determinant using fast (LU decomposition) method, call MxDet with fast parameter set to 2.

  CAVEAT: Laplace method has complexity of O(N!) and for this reason, even if you use method = 1, the maximum dimension for this method is limited to 10x10.
  Matrices larger than that are always calculated using LU method

  
  
• AFL: MxFromString() - creates a new matrix out of string in Mathematica/Wolfram list-style: "{ { 1, 2, 3 }, { 4, 5, 6 } }" or Matlab/Maple style "[ [ 1, 2, 3 ], [ 4, 5, 6 ] ]" or GNU Octave comma-semicolon style [ 1, 2, 3; 4, 5, 6 ]
• AFL: MxGetBlock( matrix, startrow, endrow, startcol, endcol, asArray = False )
  
  Retrieves items from rectangular submatrix (block) and returns either smaller matrix (when asArray is set to False)
  or "normal" AFL array (when asArray is set to True). If array has different number of bars, unused elements are filled with Null.

  z = Matrix( 2, 20, 0 );
  // first row
  z = MxSetBlock( z, 0, 0, 0, 19, Close );
  // second row
  z = MxSetBlock( z, 1, 1, 0, 19, RSI( 5 ) );
  printf("Matrix z\n");
  printf( MxToString( z ) );

  x = MxGetBlock( z, 0, 1, 0, 19, True );

  printf("Items are now in regular array (data series):\n" );
  for( i = 0; i < 20; i++ )
  printf( NumToStr( x[ i ] ) + "\n" );

  z = MxGetBlock( z, 0, 1, 0, 1 ); // retrieve upper 2x2 submatrix
  printf("Upper submatrix z\n");
  printf( MxToString( z ) );
  

• AFL: MxInverse( mx ) - calculates inverse of the matrix (see comments to MxSolve for more info)
  
  
• AFL: MxSetBlock( matrix, startrow, endrow, startcol, endcol, values = 0 )
  Sets values in the rectangular block of cells (rows in the range startrow..endrow and columns in the range startcol..endcol inclusive).
  This allows to fill entire or partial rows, columns and all other kind of rectangular areas in the matrix with user specified data
  Row and column numbers are zero based.
  If values parameter is scalar, all cells in specified block are filled with that value.
  If values parameter is an array, cells in the block are filled from left to right and from top to bottom with consecutive values taken from that array.
  If there are more cells in the block than values in the array, the array item counter wraps around to zero and starts taking values from the beginning

  Note: the function creates new matrix as a result (so source matrix is unaffected unless you do the assignment of the result back to the original variable)

  Example 1:
  // Create a matrix 6x6
  // and fill 4x4 interior (except edges with consecutively increasing numbers)

  y = Matrix( 6, 6, 0 );
  y = MxSetBlock( y, 1, 4, 1, 4, Cum(1));
  printf("Matrix y\n");
  printf( MxToString( y ) );

  Example 2:
  // Create a matrix 2 rows x 20 columns and fill rows 0, 1 with first 20 values of Close and RSI(5) arrays respectively

  z = Matrix( 2, 20, 0 );
  // first row
  z = MxSetBlock( z, 0, 0, 0, 19, Close );
  // second row
  z = MxSetBlock( z, 1, 1, 0, 19, RSI( 5 ) );
  printf("Matrix z\n");
  printf( MxToString( z ) );

• AFL: MxSolve( A, B ) - solves linear equation system A@X = B
  
  A needs to be square matrix NxN
  B has to have N rows and at least one column (vertical vector).
  Then calling
  X = MxSolve( A, B ) would give vertical vector holding solution of the system of equations A @ X = B

  B can also be a matrix,with each of its column representing different vector B. This way single call to MxSolve can solve several systems with same matrix A but different right hand vectors.
  If B is a matrix NxM then MxSolve will produce result also having NxM cells with each column representing single solution.

  Example 1:

  A = MxFromString("[ 1, 1, 1, 1; 0, 2, 5, -1; 2, 5, -1, 1; 2, 2, 2, 1 ]");
  B = MxFromString("[ 7; -5; 28; 13 ]" ); // single vertical vector B

  printf( "Solving A * X = B\n" );
  printf("Matrix A\n");
  printf( MxToString( A ) );
  printf("\nMatrix B\n");
  printf( MxToString( B ) );
  
  X = MxSolve( A, B );

  printf("\nSolution X\n");

  Example 2:

  A = MxFromString("[ 1, 1, 1, 1; 0, 2, 5, -1; 2, 5, -1, 1; 2, 2, 2, 1 ]");
  B = MxFromString("[ 7, 14 ; -5, -10; 28, 56; 13, 26 ]" ); // 2 right-hand side vertical vectors

  printf( "Solving A * X = B\n" );
  printf("Matrix A\n");
  printf( MxToString( A ) );
  printf("\nMatrix B\n");
  printf( MxToString( B ) );
  
  X = MxSolve( A, B );

  printf("\nSolutions X\n");

  printf( MxToString( X ) ); // two solutions

  (Highly) Technical note about numerical precision:

  Despite the fact that both MxSolve and MxInverse use double precision arithmetic solving/inverting matrices is subject to numerical precision of double IEEE
  and for example zero result may come up as something like 1.4355e-16 (0.0000000000000001) due to the fact that double precision is still limited in accuracy (16 digits).

  The result of
  X = MxInverse( A ) @ B;
  although mathematically the same as solving the system of equations, would yield slightly different result because if you do the inverse the returned matrix is converted back
  to single precision and matrix product is performed with single precision. When you use MxSolve you are performing all calcs using 64-bit (double) precision and
  only end result is converted back to single precision. So for example polynomial fit code works better with MxSolve than MxInverse

  
  // Least Squares Polynomial Fit test

order = Param( "n-th Order", 15, 1, 25, 1 );
length = 60;

lvb = BarCount - 1;
fvb = lvb - length;

yy = Matrix( length + 1, 1, 0 );
xx = Matrix( length + 1, order + 1, 1 );

yy = MxSetBlock( yy, 0, length, 0, 0, Ref( C, fvb ) );

x = BarIndex() - length/2;

for( j = 1; j <= order; j++ )
{
     xx = MxSetBlock( xx, 0, length, j, j, x ^ j );
}

xxt = MxTranspose( xx );
aa = MxSolve( xxt @ xx, xxt ) @ yy;
//aa = MxInverse( xxt @ xx ) @ xxt @ yy; // alternative way

rr = Null; // store the fit in rr
for( i = fvb; i <= lvb; i++ )
{
     rr[i] = aa[0][0];

     for( j = 1; j <= order; j++ )
     {
         rr[i] += aa[j][0] * x[ i - fvb ] ^ j;
     }
}

SetChartOptions( 0, chartShowDates );
SetBarFillColor( IIf( C > O, ColorRGB( 0, 75, 0 ), IIf( C <= O, ColorRGB( 75, 0, 0 ), colorLightGrey ) ) );
Plot( rr, "rr", colorWhite, styleLine | styleThick);
Plot( C, "", IIf( C > O, ColorRGB( 0, 255, 0 ), IIf( C <= O, ColorRGB( 255, 0, 0 ), colorLightGrey ) ), styleDots | styleNoLine );

• AFL: MxSort( mx, dim = -1, ascening = True ) - sorts the matrix
  
  Sorts all items in a matrix
  When dim == -1 (the default) it would sort:
  a) a row if there is only one row (vector is horizontal)
  b) a column if there is only one column (vector is vertical)
  c) each column separately if there are more rows and columns than one (so we have actual 2D matrix).
  
  When dim == 0 the function sorts the items in each row separately
  When dim == 1 the function sorts the items in each column separately

  
  // example
  m = MxFromString("[ 9, 5, 6; 8, 7, 3 ]");

  printf( MxToString( m ) + "\n\n" );

  printf("%g, %g\n\n", MxGetSize( m, 0 ), MxGetSize( m, 1 ) );

  m2 = MxSort( m, 0 ) ;

  printf( MxToString( m2 ) + "\n\n" );

  m3 = MxSort( m, 1 ) ;

  printf( MxToString( m3 ) + "\n\n" );
  

• AFL: MxSortRows( mx, ascending = True, col1 = 0, col2 = -1, col3 = -1 )
  
  Sorts the rows of the matrix in ascending/descending order of the col1 column. When the col1 column has equal values, SortRows sorts according to the col2 and col3 columns in succession (if col2 and col3 are specified and >= 0 ).
  Column numbers are zero based.

  Hint: if you want to sort columns instead you can Transpose/Sort rows/Transpose back.

  m = MxFromString("[ 9, 1, 6; 40, 30, 20; 8, 7, 3; 3, 5, 1 ]");

  printf("Input matrix\n");

  printf( MxToString( m ) + "\n\n" );

  printf("Rows %g, Cols %g\n\n", MxGetSize( m, 0 ), MxGetSize( m, 1 ) );

  printf("Sorting every row separately\n");
  m2 = MxSort( m, 0 ) ;

  printf( MxToString( m2 ) + "\n\n" );

  printf("Sorting every column separately\n");
  m3 = MxSort( m, 1 ) ;

  printf( MxToString( m3 )+ "\n\n");

  printf("Sorting rows by contents of first column\n");
  m4 = MxSortRows( m, True, 0 ) ;

  printf(MxToString( m4 )+ "\n\n");

  printf("Sorting rows by contents of second column\n");
  m5 = MxSortRows( m, True, 1 ) ;

  printf(MxToString( m5 )+ "\n\n");

• AFL: MxToString - creates string out of matrix variable in the Wolfram list style like this (for 3x3 matrix): { { x00, x01, x02 }, { x10, x11, x12 }, { x20, x21, x22 } }
  

If you experience any problem with this beta version please send detailed description of the problem (especially the steps needed to reproduce it) to support at amibroker.com