MQLmagazine.com » MQL programming

Guidelines for writing portfolio expert advisors in MQL5

Bogdan Caramalac, MQLmagazine sr.editor — Mon, 24 May 2010 21:29:43 +0000

[Versiunea romaneasca] [MQLmagazine.com in romana] [English edition]

Portfolio expert advisors, as well as the more complex multiasset expert advisors, were awaited by the retail trading community ever since limitations of the MT4 backtester became obvious, that is, about 4 – 5 years ago. The difference between multiasset and portfolio EAs is that multiasset EAs have an integrated strategy where each asset is a component, whether portfolio EAs replicate same strategy, with just some parameter differences, over more instruments. Of course, in the MT4 context, normal EAs could have been run on separate instruments and obtain portfolio effects, however backtesting could never have covered these portfolio effects because it was not possible to backtest on more than one instrument.

The main advantage of portfolying is the yield averaging while spreading the risk. The degree up to which this happens, is dependant on the overall portfolio correlation. The lesser overall correlated, the lesser nonsysthematic risk left uncovered. However, if you don’t pick the portfolio right, instead to balance your overall trading system, it will add extra instability.

A portfolio trading system must be able to:
- apply similar trading logics to a large number of assets;
- differentiate trading logics from one instrument to another;
- control exposure.

It requires a pretty large data structure to take make things manageable.

For instance, this one:

input double MarginUsagePerPosition=0.5;
 
int SymbolsCount;
 
struct IndicatorHandlersStruct
  {
   //...
  };
 
struct SystemParametersStruct
  {
   //...
  };
 
struct SymbolData
  {
    string Symbol;
    IndicatorHandlersStruct IndicatorHandlers;
    SystemParametersStruct SystemParameters;
  };
 
SymbolData SymbolsTable[300];

The first variable, MarginUsagePerPosition is a sort of asset allocation parameter, by controlling the maximal volume of each position. SymbolsCount tells how many symbols are used and it can be used for enumeration, while the big SymbolsTable[] will contain all the needed data, from symbols to handlers. For instance, the following structure will be set up to calculate 4 moving averages (two on M5 and two on H1) per each symbol:

struct IndicatorHandlersStruct
  {
   int MovingAveragesHandlers14[2];
   int MovingAveragesHandlers9[2];
  };
 
struct SymbolData
  {
    string Symbol;
    IndicatorHandlersStruct IndicatorHandlers;
    datetime LastBarTime;
  };
 
SymbolData SymbolsTable[300];

Before OnInit() we have to write a procedure to fill in the SymbolsTable[] with the indicator handlers. MakeIndicatorHandlers() is being called from OnInit() to separate the symbols setup from the indicators setup. While you can easily alter OnInit() to change the list, by adding, deleting or making an automated instrument selector, the MakeIndicatorHandlers() will remain static, with the purpose to fill the structure with the needed indicator handlers.

void MakeIndicatorHandlers()
  {
   datetime lastbar[1];
   for (int i=0;i<SymbolsCount;i++)
      {
       SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers14[0]=iTEMA(SymbolsTable[i].Symbol,PERIOD_M5,14,0,PRICE_OPEN);
       SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers9[0]=iTEMA(SymbolsTable[i].Symbol,PERIOD_M5,9,0,PRICE_OPEN);
       SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers14[1]=iTEMA(SymbolsTable[i].Symbol,PERIOD_H1,14,0,PRICE_OPEN);
       SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers9[1]=iTEMA(SymbolsTable[i].Symbol,PERIOD_H1,9,0,PRICE_OPEN);  
      }
   return;
  }

Within OnInit(), we set up the symbols table. For instance, we are making it to work on 3 forex pairs: EURUSD, USDCHF, GBPJPY:

int OnInit()
  {
   SymbolsTable[0].Symbol="EURUSD";
   SymbolsTable[1].Symbol="USDCHF";
   SymbolsTable[2].Symbol="GBPJPY";
   SymbolsCount=3;
   MakeIndicatorHandlers();
   return(0);
  }

And now OnTick():

void OnTick()
  {
   datetime datetime_array[1];
   for (int i=0;i<SymbolsCount;i++)
      {
       CopyTime(SymbolsTable[i].Symbol,PERIOD_M5,0,1,datetime_array);
       if (datetime_array[0]!=SymbolsTable[i].LastBarTime)
         {               
          TradeLogics(i);
          LastBarTime[i]=datetime_array[0];    
         }//if (datetime_array[0]!=SymbolsTable[i].LastBarTime)
      }//for (int i=0;i
   return;
  }

Now all the pieces are complete. OnTick() will check for each instrument if a new bar appeared and will call TradeLogics() for every instrument in the array, which will implement decision and trading.

We will not stop on TradeLogics() in this article, because TradeLogics() is EA specific. But a few principles should be guides while writing the TradeLogics(). The first division of the TradeLogics() is the asset class that it deals with. It can’t deal with equities the same way it deals with forex. Equities don’t run 24 hours a day 5 days a week. They have specific trading sessions and gaps from a day to another, not mentioning from a week to another. So this is the first division, the asset class. The second, is that it is better to separate the trading functions from the TradeLogics(). For instance, the TradeLogics() could just command how to adjust the position on a given instrument, and leave the trading function to a ManagePosition() procedure.
Of course, TradeLogics() can be in practice triggered by OnTrade() or OnChartEvent(), or even a CEP engine’s EventsCallback().

double UnitsToLots(double units,string symbol)
  {
  double dlotsize=SymbolInfoDouble(symbol,SYMBOL_TRADE_CONTRACT_SIZE);
  double mag=SymbolInfoDouble(symbol,SYMBOL_VOLUME_STEP);
  double mini=SymbolInfoDouble(symbol,SYMBOL_VOLUME_MIN);
  double big=1/mag;    
  double size=dlotsize/big;
  double dlots0,dlots1,back0,back1;
  dlots0=NormalizeDouble(units/size,0)*mag;
  dlots1=dlots0+mag;
  back0=dlots0*big*size;
  back1=dlots1*big*size;
  if (back1-units<units-back0)
    {
     if (dlots1<mini)
       return(mini);  
     else  
       return(dlots1);
    }
  else
    {
     if (dlots0==0)
        return(mini);
     else
       {
        if (dlots0<mini)
          return(mini);
        else
          return(dlots0);
       }//else if (dlots0==0)
    }//else if if (back1-units
  }

UnitsToLots() is a port of a function that I wrote in MQL4 times, and it was working at that time with MarketInfo(). UnitsToLots() will return the appropriate number of lots correspondant to a volume given in units. For instance it may answer 1.0 for 100000 units of EURUSD.

#define FLAT        0
#define LONG        1
#define SHORT       2
 
int GetPositionType(int asset_index)
  {
   string symbol=SymbolsTable[asset_index].Symbol;
   bool sel=PositionSelect(symbol);
   if (sel==false)
     return(FLAT);
   else
    {
     long p=PositionGetInteger(POSITION_TYPE);
     if (p==POSITION_TYPE_BUY)
       return(LONG);
     else
       return(SHORT);
    }
  }

This function is a simple proxy to get a straight position type value. The value for POSITION_TYPE_BUY is 0 in MQL5, and PositionInfoInteger() will answer 0 even if the position was not previously selected.

void ManagePosition(int asset_index,int operation,double forcevolume=0.00)
  {   
   MqlTradeRequest request;
   MqlTradeResult result;
   int p=GetPositionType(asset_index);
   double now_volume;
   double current_volume,v0;
   if (PositionSelect(SymbolsTable[asset_index].Symbol)==true)
     current_volume=PositionGetDouble(POSITION_VOLUME);
   else
     current_volume=0;
   v0=current_volume;   
   request.action=TRADE_ACTION_DEAL;
   request.symbol=SymbolsTable[asset_index].Symbol; 
   request.deviation=Slippage;
   request.type_filling=ORDER_FILLING_AON;
   request.type_time=ORDER_TIME_GTC;
   if (p==FLAT)
     {
      if (DoubleToString(forcevolume,2)=="0.00")
        request.volume=UnitsToLots( (MarginUsagePerPosition/100)*AccountInfoDouble(ACCOUNT_EQUITY)*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
      else
        request.volume=forcevolume;
      if (operation==LONG)
        {
         request.type=ORDER_TYPE_BUY;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
      if (operation==SHORT)
        {
         request.type=ORDER_TYPE_SELL;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
      if (operation==FLAT)
        return;
     }
   else//if (p==FLAT)
     {     
      if (p==LONG&&operation==SHORT)
        {
         if (DoubleToString(forcevolume,2)=="0.00")
           request.volume=current_volume+UnitsToLots( (MarginUsagePerPosition/100)*AccountInfoDouble(ACCOUNT_EQUITY)*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
         else
           request.volume=current_volume+forcevolume;
         request.type=ORDER_TYPE_SELL;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        }
      if (p==SHORT&&operation==LONG)
        {
         if (DoubleToString(forcevolume,2)=="0.00")         
           request.volume=current_volume+UnitsToLots( (MarginUsagePerPosition/100)*AccountInfoDouble(ACCOUNT_EQUITY)*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
         else
           request.volume=current_volume+forcevolume;
         request.type=ORDER_TYPE_BUY;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
      if (p==LONG&&operation==FLAT)
        {       
         if (DoubleToString(forcevolume,2)=="0.00")
           request.volume=current_volume;
         else
           request.volume=forcevolume;
         request.type=ORDER_TYPE_SELL;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        }
      if (p==SHORT&&operation==FLAT)
        {
         if (DoubleToString(forcevolume,2)=="0.00")           
           request.volume=current_volume;
         else
           request.volume=forcevolume;
         request.type=ORDER_TYPE_BUY;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        } 
      if (p==LONG&&operation==LONG)//adjustment of the present LONG position
        {
         if (DoubleToString(forcevolume,2)=="0.00")
           request.volume=UnitsToLots( (MarginUsagePerPosition/100)*AccountInfoDouble(ACCOUNT_EQUITY)*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol )-current_volume;
         else
           request.volume=forcevolume-current_volume;
         if (NormalizeDouble(request.volume,2)>0.00)
           {         
            request.type=ORDER_TYPE_BUY;
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
           }
         else //we close the difference
           {
            request.volume=MathAbs(request.volume);
            request.type=ORDER_TYPE_SELL;         
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
           }
        }        
      if (p==SHORT&&operation==SHORT)//adjustment of the present SHORT position
        {
         if (DoubleToString(forcevolume,2)=="0.00")
           request.volume=UnitsToLots( (MarginUsagePerPosition/100)*AccountInfoDouble(ACCOUNT_EQUITY)*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol )-current_volume;
         else
           request.volume=forcevolume-current_volume;
         if (NormalizeDouble(request.volume,2)>0.00)
           {
            request.type=ORDER_TYPE_SELL;         
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));                  
           }
         else
           {
            request.volume=MathAbs(request.volume);
            request.type=ORDER_TYPE_BUY;
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));            
           }
        }        
     }//if (p==FLAT)]  
   if (DoubleToString(request.volume,2)!="0.00")
     {
      OrderSend(request,result);     
      PositionSetSLTP(StopLoss,TakeProfit);
     }     
  }

What does ManagePosition() do ? It opens a position on the given asset by index, upon request, if it is flattened before ; or it reverts the current position, by recalculating the new lot size (calculates new lot size with the formula, adds to current volume and reverses operation). It can also add or cut from the current position. The forcevolume parameter can be used to force a given volume into a trade, bypassing calculus.
How about setting a stop loss ? Nothing easier:

void PositionSetSLTP(int asset_index,int sl,int tp)
  {
   bool todosl, todotp;
   MqlTradeRequest request;
   MqlTradeResult result;   
   double price=0.00;
   int p=GetPositionType(asset_index);
   if (p==LONG||p==SHORT)
     price=PositionGetDouble(POSITION_PRICE_OPEN);
   request.action=TRADE_ACTION_SLTP;
   request.symbol=SymbolsTable[asset_index].Symbol;   
   todosl=false;
   todotp=false;
   if (DoubleToString(sl,4)!="0.0000")
     {
      todosl=true;      
      if (p==LONG)
        request.sl=NormalizeDouble(price-sl*SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_POINT),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
      if (p==SHORT)
        request.sl=NormalizeDouble(price+sl*SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_POINT),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
     }
   if (DoubleToString(tp,4)!="0.0000")
     {      
      todotp=true;
      if (p==LONG)
        request.tp=NormalizeDouble(price+tp*SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_POINT),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
      if (p==SHORT)
        request.tp=NormalizeDouble(price-tp*SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_POINT),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
     }
   request.deviation=Slippage;   
   if (todosl==true||todotp==true)
     OrderSend(request,result);
  }

Note that the OrderSend() calls are not followed by an analysis of the result. We didn’t dwelve into retcodes because some of them are not completely clear to us.

And finally OnDeinit(), which destroys the indicator handlers.

void OnDeinit(const int reason)
  {
   for (int i=0;i<SymbolsCount;i++)
      {   
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers14[0]);
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers9[0]);
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers14[1]);
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.MovingAverageHandlers9[1]);
      }
   return;
  }

MQL5 : A portfolio moving average sample expert

Bogdan Caramalac, MQLmagazine sr.editor — Mon, 24 May 2010 21:29:26 +0000

[Versiunea romaneasca] [MQLmagazine.com in romana] [English edition]

I wrote this expert advisor to as a full demonstration on writing portfolio expert advisors, following the principles presented in Guidelines for writing portfolio expert advisors in MQL5.

The expert will work on the chosen timeframe, only when a new bar forms up on every participating instrument. Upon checking moving average intersections, it will manage positions. There is also the option to filter by volatility (I repaired the BBoverSTD indicator from the Volatility analysis : bridging the gap from volatility forecasting to price forecasting article).

//+------------------------------------------------------------------+
//|                                                  PortfolioMA.mq5 |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
#property version   "1.00"
 
#define FLAT        0
#define LONG        1
#define SHORT       2
 
input int Slippage=30;
input double MarginUsagePerPosition=0.5;
input bool VolatilityFilter=false;
input int StopLoss=300;
input int TakeProfit=1000;
 
input int SlowMALength=14;
input int FastMALength=9;

This section defines the constants for working with positions (FLAT,LONG,SHORT) and the parameters of the system.

int SymbolsCount;
 
struct IndicatorHandlersStruct
  {
   int SlowMovingAverageHandler;
   int FastMovingAverageHandler;
   int BBoverSTDHandler;
  };
 
struct SymbolData
  {
    string Symbol;
    IndicatorHandlersStruct IndicatorHandlers;
    datetime LastBarTime;    
  };
 
SymbolData SymbolsTable[30];

This section defines the data structure holding the symbols and indicator handlers.

void MakeIndicatorHandlers()
  {
   datetime datetime_array[1];
   int error1,error2,error3;
   for (int i=0;i<SymbolsCount;i++)
      {
       error1=0;
       error2=0;
       error3=0;
       SymbolsTable[i].IndicatorHandlers.SlowMovingAverageHandler=iMA(SymbolsTable[i].Symbol,Period(),SlowMALength,0,MODE_SMA,PRICE_OPEN);
       Print("SymbolsTable[",i,"].IndicatorHandlers.SlowMovingAverageHandler=",SymbolsTable[i].IndicatorHandlers.SlowMovingAverageHandler);
       if (SymbolsTable[i].IndicatorHandlers.SlowMovingAverageHandler<0)
         { 
          Print("Invalid handle.");
          error1=GetLastError();
          Print("Error ",error1," while initializing iMA(",SlowMALength,") for ",SymbolsTable[i].Symbol);
         }
       SymbolsTable[i].IndicatorHandlers.FastMovingAverageHandler=iMA(SymbolsTable[i].Symbol,Period(),FastMALength,0,MODE_SMA,PRICE_OPEN);
       Print("SymbolsTable[",i,"].IndicatorHandlers.FastMovingAverageHandler=",SymbolsTable[i].IndicatorHandlers.FastMovingAverageHandler);
       if (SymbolsTable[i].IndicatorHandlers.FastMovingAverageHandler<0)
         {
          Print("Invalid handle.");
          error2=GetLastError();
          Print("Error ",error2," while initializing iMA(",FastMALength,") for ",SymbolsTable[i].Symbol);
         }
       SymbolsTable[i].IndicatorHandlers.BBoverSTDHandler=iCustom(SymbolsTable[i].Symbol,Period(),"BBoverSTD",20,10,1.5,"SMA",2,0);
       Print("SymbolsTable[",i,"].IndicatorHandlers.BBoverSTDHandler=",SymbolsTable[i].IndicatorHandlers.BBoverSTDHandler);
       if (SymbolsTable[i].IndicatorHandlers.BBoverSTDHandler<0)
         { 
          Print("Invalid handle.");
          error3=GetLastError();
          Print("Error ",error3," while initializing BBoverSTD for ",SymbolsTable[i].Symbol);
         }                   
      }
   return;
  }

This is the MakeIndicatorHandlers() routine tailored for this EA.

Now follow UnitsToLots(), GetPositionType(), PositionSetSLTP() and ManagePosition() that are the ones presented in the Guidelines for writing portfolio expert advisors in MQL5.

string MASignal(int asset_index)
  {
   double maslowarray[1];
   double mafastarray[1];
   double mafast,maslow;
   datetime t;
   int copycount1,copycount2;
   int error1=0;
   int error2=0;
   string signal;
   copycount1=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.SlowMovingAverageHandler,0,0,1,maslowarray);
   if (copycount1==-1)
     {
      Print("Nothing copied (1). Retrieving error.");
      error1=GetLastError();
     }
   copycount2=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.FastMovingAverageHandler,0,0,1,mafastarray);
   if (copycount2==-1)
     {
      Print("Nothing copied (2). Retrieving error.");
      error2=GetLastError();
     }
   maslow=NormalizeDouble(maslowarray[0],SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
   mafast=NormalizeDouble(mafastarray[0],SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));   
   t=TimeCurrent();
   if (maslow<mafast)
     signal="L";
   if (maslow==mafast)
     signal="-";
   if (maslow>mafast)
     signal="S";
   return(signal);
  }

This returns the moving average signal for the requested asset. Signal can be “L”, “S” or “-” .

void TradeLogics(int asset_index)
  {
   string s;
   double stddev[1];
   double bbhi[1];
   double bblo[1];
   int copycount;
   s=MASignal(asset_index);   
   if (VolatilityFilter==true)
     {
      copycount=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.BBoverSTDHandler,0,0,1,stddev);
      copycount=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.BBoverSTDHandler,2,0,1,bbhi);
     }
   if (GetPositionType(asset_index)==FLAT)
     {      
      if (s=="L")
        {
         if (VolatilityFilter==true)
           {
            if (stddev[0]>bbhi[0])
              ManagePosition(asset_index,LONG);
           }
         else
           ManagePosition(asset_index,LONG);
        }
      if (s=="S")
        {
         if (VolatilityFilter==true)
           {
            if (stddev[0]>bbhi[0])
              ManagePosition(asset_index,SHORT);
           }
         else
           ManagePosition(asset_index,SHORT);
        }
     }//else if (GetPositionType(asset_index)==FLAT)
   else//position is not flat
     {
      if (GetPositionType(asset_index)==LONG)
        {
         if (VolatilityFilter==true)
           {
            if (s=="S"||stddev[0]<bbhi[0])
              ManagePosition(asset_index,FLAT);
           }           
         else
           {
            if (s=="S")
              ManagePosition(asset_index,FLAT);           
           }
        }
      if (GetPositionType(asset_index)==SHORT)
        {
         if (VolatilityFilter==true)
           {
            if (s=="L"||stddev[0]<bbhi[0])
              ManagePosition(asset_index,FLAT);
           }
         else
           {
            if (s=="L")
              ManagePosition(asset_index,FLAT);           
           }        
        }
     }           
   return;
  }

This is the TradeLogics(). First it checks if VolatilityFilter is enabled, and only in this case calls the BBoverSTD indicator, to save time for the situation the filter is disabled. Then it tests if position is flat.
If it is, goes long or short according to the MASignal() , eventually filtered by the BBoverSTD values, if VolatilityFilter is enabled. If the position is long or short already, it checks only for contrarian signals (of course, same filtering). If contrarian signal comes, position is flattened, otherwise there is no adaptation of position to the new equity size (although the ManagePosition() is able to do it, but not asked to do so).

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
   string crtsymbol;
   SymbolsTable[0].Symbol="EURUSD";
   SymbolsTable[1].Symbol="USDJPY";
   SymbolsTable[2].Symbol="GBPUSD";
   SymbolsTable[3].Symbol="USDCHF";
   SymbolsCount=1;
   MakeIndicatorHandlers();
   return(0);
  }

This is the OnInit(). It is already preset to work with four currency pairs. If SymbolsCount remains 1 , only EURUSD will be traded. If you set it to 4, all four pairs will be traded. You can alter the SymbolsTable[] however you wish (problems may arise however at the lot calculus while trading CFDs). OnInit() must end with a call to MakeIndicatorHandlers() to get the indicator handlers ready for usage.

//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
   datetime datetime_array[1];
   for (int i=0;i<SymbolsCount;i++)
      {
       CopyTime(SymbolsTable[i].Symbol,Period(),0,1,datetime_array);
       if (datetime_array[0]!=SymbolsTable[i].LastBarTime)
         {
          //Print(SymbolsTable[i].Symbol," :: ",TimeToString(SymbolsTable[i].LastBarTime,TIME_DATE)," ",TimeToString(SymbolsTable[i].LastBarTime,TIME_MINUTES),"  vs. ",TimeToString(datetime_array[0],TIME_DATE)," ",TimeToString(datetime_array[0],TIME_SECONDS) );
          TradeLogics(i);
          SymbolsTable[i].LastBarTime=datetime_array[0];          
         }//if (datetime_array[0]!=LastBarOccurred[i])
      }//for (int i=0;i
   return;
  }

This is the OnTick(). The OnTick() contains the symbol loop, and requires the time of the last bar of the given timeframe on the current symbol given by the cycle. If this time is different than remembered time, it means a new bar appeared on that symbol chart. Then TradeLogics() is called for decision and subsequent call to ManagePosition().

//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
   for (int i=0;i<SymbolsCount;i++)
      {   
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.SlowMovingAverageHandler);
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.FastMovingAverageHandler);
       IndicatorRelease(SymbolsTable[i].IndicatorHandlers.BBoverSTDHandler);
      }
   return;
  }

This is the OnDeinit(). It releases indicator handlers. Should OnInit() be run again, it will make other handlers. It is not completely necessary to do this, however wasting memory with new handlers every time OnInit() runs is not clean programming.

The following backtests were made with no StopLoss and TakeProfit set, also with VolatilityFilter disabled.

As we can see, portfolying was not a good choice. It added extra instability and deeper drawdowns. However, now we are dealing with trading systems, while portfolio management theory supposes a generally long or out strategy. This why, in financial theory, negative correlations would drive risks down, because pluses on some assets would compensate with minuses on other assets. But, what we have now is a portfolio of trading systems. We can think a trading system as an equity function of instruments:

 equity = trading system (instrument)

On two perfectly negatively correlated assets, the trading system function would output two positively correlated equity series. A rigorous approach would ask that a Markowitz or CAPM selection over trading systems to be done with based on trading system outputs rather than market data.

See also the article about trading systems virtualization article, which is analysing the problem from another point of view.

File link:
PortfolioMA.mq5

Trading systems virtualization to achieve equity line fitting – a myth ?

Bogdan Caramalac, MQLmagazine sr.editor — Mon, 24 May 2010 21:29:07 +0000

[Versiunea romaneasca] [MQLmagazine.com in romana] [English edition]

Did you ever think that you’d like to have a moving average over the equity of a trading system, so you can turn it on and off ? This would be called equity line fitting.

The frustrating aspect is that once you start doing this from the trading system that you want to curve it its equity line, you cannot control the process anymore, as the indicators of the equity line will report differently once you hit the switch.

So, what you have to do is to let the system paper trade somehow in parallel to your main trading system, and copy the trades in your main trading system when the switch is on, ignore paper trades when the switch is off.

This can be done by virtualization. Meaningly, you have to write a paper trade position management system, like a virtual account and output its “equity” to the main trading system, which will curve-fit this equity and decide whether to replicate or not the paper system’s trades.

Thus, we begin writing such a system, in file which we’ll call TradeVirtualization.mqh .

//+------------------------------------------------------------------+
//|                                          TradeVirtualization.mqh |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
#property version "1.0"
 
#define FLAT        0
#define LONG        1
#define SHORT       2
 
struct VirtualSymbolData
  {
    string Symbol;    
    double position;
    double price;
    double sl;
    double tp;
    double marked_price;    
  };
 
struct VirtualEquityData
  {
   datetime opentime,closetime;
   double open,high,low,close;  
  };

We begin with the regular defines for position management, followed by two structures. The first is intended to store symbol data in, providing symbol, position (positive – long, negative – short), price (price of initiation for the position), marked price (last price at which the position was marked to market). The second is the intended to store virtual equity data : open and close times of an equity bar ; open, high, low, close for the equity within interval. Now it’s time for the VirtualAccount class.

class VirtualAccount
  {   
   public:
   int VirtualSymbolsCount;    
   VirtualSymbolData VirtualSymbolsTable[30]; 
   VirtualEquityData VirtualEquity[500];
   double VirtualLastAccountEquity;  
   int VirtualLastEquityPos;
   uint VirtualEquityBarSpan;
 
   void VirtualAccountSet(double Equity,uint TimeSpan)   
       {
        datetime tc=TimeCurrent();
        VirtualEquity[0].opentime=tc;
        VirtualEquity[0].open=Equity;
        VirtualEquity[0].high=Equity;
        VirtualEquity[0].low=Equity;
        VirtualEquity[0].close=Equity;
        VirtualEquity[0].closetime=tc;
        VirtualLastAccountEquity=VirtualEquity[0].close;
        VirtualEquityBarSpan=TimeSpan;
        VirtualLastEquityPos=0;
        return;
       }  
 
    int VirtualGetPositionType(int asset_index); 
    double VirtualGetPositionSize(int asset_index);
    void VirtualPositionSetSL(double asset_index,double sl);
    void VirtualPositionSetTP(double asset_index,double tp);
    void VirtualMarkToMarket(int asset_index);
    void VirtualChangePosition(int asset_index,int direction,double price,double lotsize);
    private:
    void VirtualAddNewEquity(double equity);
 
    public:
    VirtualAccount() 
       {
        VirtualAccountSet(AccountInfoDouble(ACCOUNT_EQUITY),0);
       };      
  };

The class contains an array of VirtualSymbolsData and its count, and an array with VirtualEquityData. Other key parameters are VirtualLastEquityPos and VirtualEquityBarSpan (the time interval of a bar, in seconds). VirtualLastAccountEquity is just a proxy for retrieving last recorded equity.

int VirtualAccount::VirtualGetPositionType(int asset_index)     
   {
    if (DoubleToString(NormalizeDouble(VirtualSymbolsTable[asset_index].position,2),2)=="0.00")
      return(FLAT);
    if (VirtualSymbolsTable[asset_index].position>0.00)
      return(LONG);
    if (VirtualSymbolsTable[asset_index].position<0.00)
      return(SHORT); 
    return(0);       
   }
 
double VirtualAccount::VirtualGetPositionSize(int asset_index)     
   {
    if (DoubleToString(NormalizeDouble(VirtualSymbolsTable[asset_index].position,2),2)=="0.00")
      return(0.00);
    return(NormalizeDouble(MathAbs(VirtualSymbolsTable[asset_index].position),2));       
   }       
 
 void VirtualAccount::VirtualPositionSetSL(double asset_index,double sl)
     {
      VirtualSymbolsTable[asset_index].sl=sl;
     }
 
 void VirtualAccount::VirtualPositionSetTP(double asset_index,double tp)
     {
      VirtualSymbolsTable[asset_index].tp=tp;
     }

VirtualGetPositionType() and VirtualGetPositionSize() are pure informational. The position size is always positive, like the POSITION_VOLUME requests to MQL5’s PositionInfoDouble(). The other two functions do what their name says they do.

void VirtualAccount::VirtualAddNewEquity(double equity)
    {
     datetime tc=TimeCurrent();
     uint diff;
     diff=tc-VirtualEquity[VirtualLastEquityPos].opentime;
     if (diff<=VirtualEquityBarSpan)
       {
        if (VirtualEquity[VirtualLastEquityPos].high<equity)
          VirtualEquity[VirtualLastEquityPos].high=equity;
        if (VirtualEquity[VirtualLastEquityPos].low>equity)
          VirtualEquity[VirtualLastEquityPos].low=equity;
        VirtualEquity[VirtualLastEquityPos].close=equity;
        diff=tc-VirtualEquity[VirtualLastEquityPos].closetime;           
        VirtualEquity[VirtualLastEquityPos].closetime=tc;           
        VirtualLastAccountEquity=equity;
        return;
       }
     if (VirtualLastEquityPos!=ArrayRange(VirtualEquity,0)-1)
       {           
        VirtualLastEquityPos++;
        VirtualEquity[VirtualLastEquityPos].opentime=tc;
        VirtualEquity[VirtualLastEquityPos].open=equity;
        VirtualEquity[VirtualLastEquityPos].high=equity;
        VirtualEquity[VirtualLastEquityPos].low=equity;
        VirtualEquity[VirtualLastEquityPos].close=equity;           
        VirtualEquity[VirtualLastEquityPos].closetime=tc;
        VirtualLastAccountEquity=equity;
        return;
       }
     else
       {
        for (int i=0;i<ArrayRange(VirtualEquity,0)-1;i++)
           VirtualEquity[i]=VirtualEquity[i+1];                  
        VirtualEquity[VirtualLastEquityPos].opentime=tc;
        VirtualEquity[VirtualLastEquityPos].open=equity;
        VirtualEquity[VirtualLastEquityPos].high=equity;
        VirtualEquity[VirtualLastEquityPos].low=equity;
        VirtualEquity[VirtualLastEquityPos].close=equity;           
        VirtualEquity[VirtualLastEquityPos].closetime=tc;
        VirtualLastAccountEquity=equity;
        return;
       }        
    }

At the beginning, the class is automatically set up with an equity equal to the account equity it runs on and a timespan of 0. The user may set the account to have a different startup equity and timespan. At every tick, VirtualMarkToMarket(-1) has to be called to ensure that all positions are marked to market with the new tick data. After each mark to market (either individual, with a positive parameter, or overall, with a negative parameter), a new equity value is calculated. Now the recording in the VirtualEquity[] begins from 0. If the timespan is not 0, and the new tick comes within the timespan from last equity bar opening time, the new tick is recorded on that bar, on the close field, stamping also the closetime field. High and low per equity bar is recalculated. If the timespan is 0, or if the new equity comes too late, it enters on the following bar, and so on, up to the maximum subscript (499). When it is to enter on 500, array is slided to the left one position and bar 499 starts to be recorded again. VirtualAddNewEquity() is called only from this class. Thus, it was designated to be a private method.

void VirtualAccount::VirtualMarkToMarket(int asset_index) //marks a new equity value
    {
     double fluctuation=0.00;
     double rprice,new_equity;
     int p;
     if (asset_index<0)//mark all
       {
        for (int i=0;i<VirtualSymbolsCount;i++)
           {
            if (DoubleToString(VirtualSymbolsTable[i].position,2)=="0.00")
              continue;
            if (NormalizeDouble(VirtualSymbolsTable[i].position,2)>0.00)
              rprice=NormalizeDouble(SymbolInfoDouble(VirtualSymbolsTable[i].Symbol,SYMBOL_BID),SymbolInfoInteger(VirtualSymbolsTable[i].Symbol,SYMBOL_DIGITS));
            if (NormalizeDouble(VirtualSymbolsTable[i].position,2)<0.00)
              rprice=NormalizeDouble(SymbolInfoDouble(VirtualSymbolsTable[i].Symbol,SYMBOL_ASK),SymbolInfoInteger(VirtualSymbolsTable[i].Symbol,SYMBOL_DIGITS));                
            fluctuation=fluctuation+MathAbs(VirtualSymbolsTable[i].position)*SymbolInfoDouble(VirtualSymbolsTable[i].Symbol,SYMBOL_TRADE_CONTRACT_SIZE)*(rprice-VirtualSymbolsTable[i].marked_price);               
            VirtualSymbolsTable[i].marked_price=rprice;
            if (DoubleToString(VirtualSymbolsTable[i].sl,5)!="0.00000")
              {
               if (VirtualSymbolsTable[i].position>0.00&&rprice<=VirtualSymbolsTable[i].sl)
                 {
                  VirtualSymbolsTable[i].position=0.00;                     
                  VirtualSymbolsTable[i].tp=0.00;
                  VirtualSymbolsTable[i].sl=0.00;
                 }
               if (VirtualSymbolsTable[i].position<0.00&&rprice>=VirtualSymbolsTable[i].sl)
                 {
                  VirtualSymbolsTable[i].position=0.00;                     
                  VirtualSymbolsTable[i].tp=0.00;
                  VirtualSymbolsTable[i].sl=0.00;
                 }                    
              }
            if (DoubleToString(VirtualSymbolsTable[i].tp,5)!="0.00000")
              {
               if (VirtualSymbolsTable[i].position>0.00&&rprice>=VirtualSymbolsTable[i].tp)
                 {
                  VirtualSymbolsTable[i].position=0.00;                     
                  VirtualSymbolsTable[i].tp=0.00;
                  VirtualSymbolsTable[i].sl=0.00;
                 }
               if (VirtualSymbolsTable[i].position<0.00&&rprice<=VirtualSymbolsTable[i].tp)
                 {
                  VirtualSymbolsTable[i].position=0.00;                     
                  VirtualSymbolsTable[i].tp=0.00;
                  VirtualSymbolsTable[i].sl=0.00;
                 }                    
              }                         
           }
        new_equity=VirtualEquity[VirtualLastEquityPos].close+fluctuation;
        VirtualAddNewEquity(new_equity);  
       }                      
     else
       {
        if (DoubleToString(VirtualSymbolsTable[asset_index].position,2)!="0.00")
          {
           if (NormalizeDouble(VirtualSymbolsTable[asset_index].position,2)>0.00)
             {
              p=LONG;
              rprice=NormalizeDouble(SymbolInfoDouble(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
             }
           if (NormalizeDouble(VirtualSymbolsTable[asset_index].position,2)<0.00)
             {
              p=SHORT;
              rprice=NormalizeDouble(SymbolInfoDouble(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));                
             }
           //if (p==LONG)
           //  Print("Market marking @ ",DoubleToString(rprice,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS))," for ",VirtualSymbolsTable[asset_index].Symbol," ",DoubleToString(VirtualSymbolsTable[asset_index].position,2)," LONG @ ",DoubleToString(VirtualSymbolsTable[asset_index].price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS)),"last mark @ ",DoubleToString(VirtualSymbolsTable[asset_index].marked_price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS))," -> ",DoubleToString(new_equity,2)," by fl. ",DoubleToString(fluctuation,2));
           //if (p==SHORT)
           //  Print("Market marking @ ",DoubleToString(rprice,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS))," for ",VirtualSymbolsTable[asset_index].Symbol," ",DoubleToString(VirtualSymbolsTable[asset_index].position,2)," SHORT @ ",DoubleToString(VirtualSymbolsTable[asset_index].price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS)),"last mark @ ",DoubleToString(VirtualSymbolsTable[asset_index].marked_price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS))," -> ",DoubleToString(new_equity,2)," by fl. ",DoubleToString(fluctuation,2));                
           fluctuation=fluctuation+MathAbs(VirtualSymbolsTable[asset_index].position)*SymbolInfoDouble(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_TRADE_CONTRACT_SIZE)*(rprice-VirtualSymbolsTable[asset_index].marked_price);               
           VirtualSymbolsTable[asset_index].marked_price=rprice;
           if (DoubleToString(VirtualSymbolsTable[asset_index].sl,5)!="0.00000")
             {
              if (VirtualSymbolsTable[asset_index].position>0.00&&rprice<=VirtualSymbolsTable[asset_index].sl)
                {
                 VirtualSymbolsTable[asset_index].position=0.00;                     
                 VirtualSymbolsTable[asset_index].tp=0.00;
                 VirtualSymbolsTable[asset_index].sl=0.00;
                }
              if (VirtualSymbolsTable[asset_index].position<0.00&&rprice>=VirtualSymbolsTable[asset_index].sl)
                {
                 VirtualSymbolsTable[asset_index].position=0.00;                     
                 VirtualSymbolsTable[asset_index].tp=0.00;
                 VirtualSymbolsTable[asset_index].sl=0.00;
                }                    
             }
           if (DoubleToString(VirtualSymbolsTable[asset_index].tp,5)!="0.00000")
             {
              if (VirtualSymbolsTable[asset_index].position>0.00&&rprice>=VirtualSymbolsTable[asset_index].tp)
                {
                 VirtualSymbolsTable[asset_index].position=0.00;                     
                 VirtualSymbolsTable[asset_index].tp=0.00;
                 VirtualSymbolsTable[asset_index].sl=0.00;
                }
              if (VirtualSymbolsTable[asset_index].position<0.00&&rprice<=VirtualSymbolsTable[asset_index].tp)
                {
                 VirtualSymbolsTable[asset_index].position=0.00;                     
                 VirtualSymbolsTable[asset_index].tp=0.00;
                 VirtualSymbolsTable[asset_index].sl=0.00;
                }                    
             }                                       
          }
        new_equity=VirtualEquity[VirtualLastEquityPos].close+fluctuation;
        VirtualAddNewEquity(new_equity);             
       }
     return;
    }

VirtualMarkToMarket() is the heart of the entire class. It can be called for a position or for all positions. It marks positions to the market, calculating the account equity. For each position it calculates the fluctuation in equity from the last marked price to the current market price, which becomes marked price. For the same position, the procedure may clear the stop loss and take profits once these are taken. The user has to call VirtualMarkToMarket(-1) at every tick to mark positions to market.

void VirtualAccount::VirtualChangePosition(int asset_index,int direction,double price,double lotsize) //direction is ORDER_TYPE_BUY or ORDER_TYPE_SELL
    {
     int p=VirtualGetPositionType(asset_index);
     double rprice,fluctuation,new_equity;
     double oldposition;            
     if (p==FLAT)
       {
        VirtualSymbolsTable[asset_index].price=price;
        VirtualSymbolsTable[asset_index].marked_price=price;
        VirtualSymbolsTable[asset_index].sl=0.00000;
        VirtualSymbolsTable[asset_index].tp=0.00000;
        if (direction==ORDER_TYPE_BUY)             
          VirtualSymbolsTable[asset_index].position=NormalizeDouble(lotsize,2);
        if (direction==ORDER_TYPE_SELL)
          VirtualSymbolsTable[asset_index].position=NormalizeDouble(-lotsize,2);
        //Print("Position set to ",VirtualSymbolsTable[asset_index].position," @ price ",DoubleToString(VirtualSymbolsTable[asset_index].price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS) ) );
       }
     else
       {
        oldposition=VirtualSymbolsTable[asset_index].position;
        //local mark to market
        VirtualMarkToMarket(asset_index);           
        //position update
        VirtualSymbolsTable[asset_index].price=price;           
        if (direction==ORDER_TYPE_BUY)
          {
           VirtualSymbolsTable[asset_index].position=NormalizeDouble(VirtualSymbolsTable[asset_index].position+lotsize,2);
           if (oldposition<0.00&&VirtualSymbolsTable[asset_index].position>0.00||DoubleToString(VirtualSymbolsTable[asset_index].position,5)=="0.00000")
             {
              VirtualSymbolsTable[asset_index].sl=0.00000;
              VirtualSymbolsTable[asset_index].tp=0.00000;
             }
          }
        if (direction==ORDER_TYPE_SELL)
          {
           VirtualSymbolsTable[asset_index].position=NormalizeDouble(VirtualSymbolsTable[asset_index].position-lotsize,2);
           if (oldposition>0.00&&VirtualSymbolsTable[asset_index].position<0.00||DoubleToString(VirtualSymbolsTable[asset_index].position,5)=="0.00000")
             {
              VirtualSymbolsTable[asset_index].sl=0.00000;
              VirtualSymbolsTable[asset_index].tp=0.00000;
             }
          }              
        //Print("Position set to ",VirtualSymbolsTable[asset_index].position," @ price ",DoubleToString(VirtualSymbolsTable[asset_index].price,SymbolInfoInteger(VirtualSymbolsTable[asset_index].Symbol,SYMBOL_DIGITS) ) );
       }
    return;
   }

To make a trade, you have to use VirtualChangePosition() which will open a position with the given direction , price and lot size on the specified asset index. This opening is similar to a deal. Thus, only ORDER_TYPE_BUY and ORDER_TYPE_SELL are supported for operation. No pending or StopLimit orders. The function only sets or modifies positions. It also clears the previous stop loss and take profit if a position reverses sign.

Now that the class is done, we can virtualize the portfolio moving average expert from the MQL5 : A portfolio moving average sample expert article in a new file, called VirtualPortfolioMA.mq5 .

//+------------------------------------------------------------------+
//|                                           VirtualPortfolioMA.mq5 |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
#property version   "1.00"
 
#define FLAT        0
#define LONG        1
#define SHORT       2
 
input int Slippage=30;
input double MarginUsagePerPosition=0.5;
input bool VolatilityFilter=false;
input int VirtualStopLoss=300;
input int VirtualTakeProfit=1000;
input int SlowMALength=14;
input int FastMALength=9;
 
input int EquitySlowMAPeriod=14;
input int EquityFastMAPeriod=9;
input int RealStopLoss=300;
input int RealTakeProfit=1000;

The variables are quite the same, with the distinction that now stop loss , take profit and moving average periods are split in two : for the virtual account and for the account the system runs on.

#include 
#include 
 
int SymbolsCount;
 
struct IndicatorHandlersStruct
  {
   int SlowMovingAverageHandler;
   int FastMovingAverageHandler;
   int BBoverSTDHandler;
  };
 
struct SymbolData
  {
    string Symbol;
    IndicatorHandlersStruct IndicatorHandlers;
    datetime LastBarTime;
  };
 
SymbolData SymbolsTable[30];
 
VirtualAccount VirtualSystem;    
 
datetime VirtualLastBarTime;
string RealTrading="FORBIDDEN";

The structures are quite similar. The class is instanced in the VirtualSystem object. A VirtualLastBarTime is added to record the open time of the last equity bar from the virtual account. The RealTrading variable will record the trading state with respect to copying trades from the virtual account, that is, “STRAIGHT”, “REVERSED”, or “FORBIDDEN” (only “STRAIGHT” and “FORBIDDEN” are actually used).

The VirtualMakeIndicatorHanders() takes place of the former MakeIndicatorHandlers(). The code is the same, only the name is changed.

void VirtualPositionSetSLTP(int asset_index,int sl,int tp)
  {
   VirtualSystem.VirtualPositionSetSL(asset_index,sl);
   VirtualSystem.VirtualPositionSetSL(asset_index,tp);
  }  
 
void VirtualManagePosition(int asset_index,int operation)
  {
   MqlTradeRequest request;
   MqlTradeResult result;
   int p;
   p=VirtualSystem.VirtualGetPositionType(asset_index);
   double now_volume;
   double current_volume,v0;
   current_volume=VirtualSystem.VirtualGetPositionSize(asset_index);
   v0=current_volume;   
   request.action=TRADE_ACTION_DEAL;
   request.symbol=SymbolsTable[asset_index].Symbol; 
   request.deviation=Slippage;
   request.type_filling=ORDER_FILLING_AON;
   request.type_time=ORDER_TIME_GTC;
   if (p==FLAT)
     {
      request.volume=UnitsToLots( (MarginUsagePerPosition/100)*VirtualSystem.VirtualLastAccountEquity*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
      if (operation==LONG)
        {
         request.type=ORDER_TYPE_BUY;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
      if (operation==SHORT)
        {
         request.type=ORDER_TYPE_SELL;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
     }
   else//if (p==FLAT)
     {      
      if (p==LONG&&operation==SHORT)
        {
         request.volume=current_volume+UnitsToLots( (MarginUsagePerPosition/100)*VirtualSystem.VirtualLastAccountEquity*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
         request.type=ORDER_TYPE_SELL;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        }
      if (p==SHORT&&operation==LONG)
        {
         request.volume=current_volume+UnitsToLots( (MarginUsagePerPosition/100)*VirtualSystem.VirtualLastAccountEquity*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol );
         request.type=ORDER_TYPE_BUY;
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
        }
      if (p==LONG&&operation==FLAT)
        {
         request.volume=current_volume;
         request.type=ORDER_TYPE_SELL;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        }
      if (p==SHORT&&operation==FLAT)
        {
         request.volume=current_volume;
         request.type=ORDER_TYPE_BUY;         
         request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
        } 
      if (p==LONG&&operation==LONG)//adjustment of the present LONG position
        {
         request.volume=UnitsToLots( (MarginUsagePerPosition/100)*VirtualSystem.VirtualLastAccountEquity*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol )-current_volume;
         if (NormalizeDouble(request.volume,2)>0.00)
           {         
            request.type=ORDER_TYPE_BUY;
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));
           }
         else //we close the difference
           {
            request.volume=MathAbs(request.volume);
            request.type=ORDER_TYPE_SELL;         
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));         
           }
        }        
      if (p==SHORT&&operation==SHORT)//adjustment of the present SHORT position
        {
         request.volume=UnitsToLots( (MarginUsagePerPosition/100)*VirtualSystem.VirtualLastAccountEquity*AccountInfoInteger(ACCOUNT_LEVERAGE),SymbolsTable[asset_index].Symbol )-current_volume;
         if (NormalizeDouble(request.volume,2)>0.00)
           {
            request.type=ORDER_TYPE_SELL;         
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_BID),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));                  
           }
         else
           {
            request.volume=MathAbs(request.volume);
            request.type=ORDER_TYPE_BUY;
            request.price=NormalizeDouble(SymbolInfoDouble(SymbolsTable[asset_index].Symbol,SYMBOL_ASK),SymbolInfoInteger(SymbolsTable[asset_index].Symbol,SYMBOL_DIGITS));            
           }
        }        
     }//if (p==FLAT)
   if (request.volume>0.00)
     {       
      VirtualSystem.VirtualChangePosition(asset_index,request.type,request.price,request.volume);
      VirtualPositionSetSLTP(asset_index,VirtualStopLoss,VirtualTakeProfit);      
     }
   //deciding to replicate , stay out or reverse signals on real portfolio
   if (RealTrading!="FORBIDDEN")
     {
      if (RealTrading=="STRAIGHT")
        ManagePosition(asset_index,operation);
      else
        {
         if (operation==LONG)
           ManagePosition(asset_index,SHORT);
         if (operation==SHORT)
           ManagePosition(asset_index,LONG);
         if (operation==FLAT)
           ManagePosition(asset_index,FLAT);
        }
     }
   return;     
  }

This is the VirtualManagePosition(). It operates like ManagePosition() however its requests are directed towards the virtual account. At the end, after analysing the RealTrading variable, it decides whether to do nothing, to copy or to reverse virtual operations on the actual trading account.

The MASignal(), PositionSetSLTP(), and ManagePosition() remain the same as in the original expert.

void VirtualTradeLogics(int asset_index)
  {
   string s;
   double stddev[1];
   double bbhi[1];
   double bblo[1];
   int copycount;
   s=MASignal(asset_index);    
   if (VolatilityFilter==true)
     {
      copycount=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.BBoverSTDHandler,0,0,1,stddev);
      copycount=CopyBuffer(SymbolsTable[asset_index].IndicatorHandlers.BBoverSTDHandler,2,0,1,bbhi);
     }
   if (VirtualSystem.VirtualGetPositionType(asset_index)==FLAT)
     {      
      if (s=="L")
        {
         if (VolatilityFilter==true)
           {
            if (stddev[0]>bbhi[0])
              VirtualManagePosition(asset_index,LONG);
           }
         else
           VirtualManagePosition(asset_index,LONG);
        }
      if (s=="S")
        {
         if (VolatilityFilter==true)
           {
            if (stddev[0]>bbhi[0])
              VirtualManagePosition(asset_index,SHORT);
           }
         else
           VirtualManagePosition(asset_index,SHORT);
        }
     }//else if (GetPositionType(asset_index)==FLAT)
   else//position is not flat
     {
      if (VirtualSystem.VirtualGetPositionType(asset_index)==LONG)
        {        
         if (VolatilityFilter==true)
           {
            if (s=="S"||stddev[0]<bbhi[0])
              VirtualManagePosition(asset_index,FLAT);
           }           
         else
           {            
            if (s=="S")
              {               
               VirtualManagePosition(asset_index,FLAT);           
              }
           }//if (VolatilityFilter==true)
        }
      if (VirtualSystem.VirtualGetPositionType(asset_index)==SHORT)
        {         
         if (VolatilityFilter==true)
           {
            if (s=="L"||stddev[0]<bbhi[0])
              VirtualManagePosition(asset_index,FLAT);
           }
         else
           {            
            if (s=="L")
              {               
               VirtualManagePosition(asset_index,FLAT);           
              }
           }//if (VolatilityFilter==true)        
        }//if (VirtualSystem.VirtualGetPositionType(asset_index)==SHORT)
     }           
   return;
  }

This is the former TradeLogics() , but modified in order to operate on the virtual account.

int PeriodToMinutes(ENUM_TIMEFRAMES p)
  {
   switch(p)
     {
      case PERIOD_M1:return(1);
      case PERIOD_M2:return(2);
      case PERIOD_M3:return(3);
      case PERIOD_M4:return(4);
      case PERIOD_M5:return(5);
      case PERIOD_M6:return(6);
      case PERIOD_M10:return(10);
      case PERIOD_M12:return(12);
      case PERIOD_M15:return(15);
      case PERIOD_M20:return(20);
      case PERIOD_M30:return(30);
      case PERIOD_H1:return(60);
      case PERIOD_H2:return(60*2);
      case PERIOD_H3:return(60*3);
      case PERIOD_H4:return(60*4);
      case PERIOD_H6:return(60*6);
      case PERIOD_H8:return(60*8);
      case PERIOD_H12:return(60*12);
      case PERIOD_D1:return(60*24);
      case PERIOD_W1:return(60*24*7);
      case PERIOD_MN1:return(60*24*30);
      default: return(0);
     }
  };

This function is used in OnInit() to setup the virtual account’s bar timespan , and it converts periods into minutes.

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
   SymbolsTable[0].Symbol="EURUSD";
   SymbolsTable[1].Symbol="USDJPY";
   SymbolsTable[2].Symbol="GBPUSD";
   SymbolsTable[3].Symbol="USDCHF";
   SymbolsCount=1;
 
   VirtualSystem.VirtualAccountSet(AccountInfoDouble(ACCOUNT_EQUITY),PeriodToMinutes(Period())*60);  
   VirtualSystem.VirtualSymbolsCount=SymbolsCount;      
   for (int i=0;i<SymbolsCount;i++)
      VirtualSystem.VirtualSymbolsTable[i].Symbol=SymbolsTable[i].Symbol;
   VirtualMakeIndicatorHandlers();
   VirtualLastBarTime=VirtualSystem.VirtualEquity[VirtualSystem.VirtualLastEquityPos].opentime;
   RealTrading="FORBIDDEN";
   return(0);
  }

This is OnInit(). It prepares symbols like former OnInit() then copies symbols to the virtual account. Sets up virtual account and makes sure RealTrading is set to “FORBIDDEN”.

//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {   
   double loweq;         
   double fast_ma_now,slow_ma_now,fast_ma_old,slow_ma_old;
   double EquityCloseBuffer[31];
   datetime datetime_array[1];   
   //marking to market virtual positions
   VirtualSystem.VirtualMarkToMarket(-1);
   //looping symbols to apply logics
   for (int i=0;i<SymbolsCount;i++)
      {
       CopyTime(SymbolsTable[i].Symbol,Period(),0,1,datetime_array);
       if (datetime_array[0]!=SymbolsTable[i].LastBarTime)
         {                  
          VirtualTradeLogics(i);
          SymbolsTable[i].LastBarTime=datetime_array[0];
         }//if (datetime_array[0]!=LastBarOccurred[i])
      }//for (int i=0;i
 
   //now we check if we have enough equity bars ; if true, we analyse to see whether we activate or not live trading
   if (VirtualSystem.VirtualEquity[VirtualSystem.VirtualLastEquityPos].opentime!=VirtualLastBarTime)
     {
      if (VirtualSystem.VirtualLastEquityPos>ArrayRange(EquityCloseBuffer,0))  
        {
         for (int i=1;i<=ArrayRange(EquityCloseBuffer,0)-1;i++)
            {         
             EquityCloseBuffer[i]=StringToDouble( DoubleToString(VirtualSystem.VirtualEquity[VirtualSystem.VirtualLastEquityPos-ArrayRange(EquityCloseBuffer,0)+1+i].close,2) );             
             loweq=StringToDouble( DoubleToString(VirtualSystem.VirtualEquity[VirtualSystem.VirtualLastEquityPos-ArrayRange(EquityCloseBuffer,0)+1+i].low,2));                
            }                                    
        }
      fast_ma_now=SimpleMA(ArrayRange(EquityCloseBuffer,0)-1,EquityFastMAPeriod,EquityCloseBuffer);
      slow_ma_now=SimpleMA(ArrayRange(EquityCloseBuffer,0)-1,EquitySlowMAPeriod,EquityCloseBuffer);
      fast_ma_old=SimpleMA(ArrayRange(EquityCloseBuffer,0)-2,EquityFastMAPeriod,EquityCloseBuffer);
      slow_ma_old=SimpleMA(ArrayRange(EquityCloseBuffer,0)-2,EquitySlowMAPeriod,EquityCloseBuffer);
      //bullish ma cross on virtual system's equity line      
      if (fast_ma_now>slow_ma_now&&fast_ma_old<slow_ma_old)
        {          
         RealTrading="STRAIGHT";
         //replicating virtual portfolio into real
         //position sizes are calculated to be proportional with the real account equity
         for (int i=0;i<SymbolsCount;i++)
            ManagePosition(i,VirtualSystem.VirtualGetPositionType(i), 
                    UnitsToLots(
                     (VirtualSystem.VirtualGetPositionSize(i)*
                      AccountInfoDouble(ACCOUNT_EQUITY)/
                      VirtualSystem.VirtualLastAccountEquity)*
                      SymbolInfoDouble(SymbolsTable[i].Symbol,SYMBOL_TRADE_CONTRACT_SIZE)
                      ,SymbolsTable[i].Symbol) );
        }
      //bearish ma cross on virtual system's equity line        
      if (fast_ma_now<slow_ma_now&&fast_ma_old>slow_ma_old)
        {
         RealTrading="FORBIDDEN";
         //closing real positions
         for (int i=0;i<SymbolsCount;i++)
            ManagePosition(i,FLAT);         
        }             
      VirtualLastBarTime=VirtualSystem.VirtualEquity[VirtualSystem.VirtualLastEquityPos].opentime;
     }
   return;
  }

This is the OnTick(). Like former OnTick(), it has a symbol loop and applies the logics for each symbol, on the virtual account this time. It declares the EquityCloseBuffer[] where last values of virtual acount equity bars are copied, after each loop is finished. When this buffer contains enough values, a moving average cross on the virtual account equity line is calculated. When these moving average yield a bullish signal, RealTrading is set to “STRAIGHT”. Portfolio is being replicated from the virtual account on the actual account, proportional to the size of actual account. Of course, the VirtualManagePosition() will have subsequent calls to ManagePosition() and from this time on both accounts will work in the same time. When the moving average cross becomes bearish, RealTrading is set to “FORBIDDEN” and actual account positions are closed.

The OnDeinit() is exactly the same, as the indicator handlers are residing in the same SymbolsTable[].

Surprised? The portfolio run is more unstable than one pair run, as it was for the main trading system. The extra volatility translated into range movements on the equity line, which in turn translated into false positive signals given by moving averages on the equity line, making the virtualization to be a fiasco. There are at least three points to consider:

1. Sampling frequency
What is the optimal sampling frequency for the equity moving average? After all, we don’t know exactly how MetaTrader compresses equity data on bars so we don’t know its own sampling interval. Then, how to sample for the average ?

2. Optimal trading system for the equity line
Nobody said that moving averages over equity are the best way to fit. Other criterias may apply, same as in regular trading.

3. Should you trade all the instruments in the same virtual account, or trade in separate virtual accounts and fit each one’s equity ?

The main conclusion that yields is that trading systems are not to be based on cold, pure application of indicators. Indicators, as their name says, are indicating some of the actions that have to be taken, as counselors that give advices, influencing positively the results of trading systems, but their place is not at the core of trading systems. Rather, sound statistics and arbitrage relationships are at the base of effective trading systems.

File links:
TradeVirtualization.mqh
VirtualPortfolioMA.mq5

MQL5 : Implementation of a simple CEP engine

Bogdan Caramalac, MQLmagazine sr.editor — Mon, 24 May 2010 21:28:51 +0000

[Versiunea romaneasca] [MQLmagazine.com in romana] [English edition]

This edition was harder to make than any of the previous editions. The late release of Strategy Tester and the large number of bugs that surged while making the CEP engine took a lot of time. From this moment, magazine issues will not be monthly anymore, because they will contain a lot of work and updates of past works. This is why, this time we are adding the Forum to the Magazine.

Now back to the CEP engine.

The CEP engine was some of the hardest stuff possible to write in MQL5. It was hard not only for me as programmer, but it had implications over MT5 as well, because the spaghetti data structure that it uses put MetaTrader to the test ; it needed a lot of bug filtering, especially on objects and memory allocations. The code managed to run with minor issues barely on build 270 (yes, already in May), and completely clean on build 271. It’s a long hard code, and I’ll chop it to pieces for commenting. I also left inline comments and Print lines intact to be better understood by readers. The algorithm is quite modified, not the same as the one presented in Anatomy of a Simple CEP Engine .

The following file it’s gonna be called CEPEngine.mqh :

//+------------------------------------------------------------------+
//|                                                    CEPEngine.mqh |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
 
#property version "1.0";
 
//*******************************************
 
string Replicate(string s,int n)
   {
   string res="";
   int k=1;
   if (n<1)
      return(res);                    
   for (k=1;k<=n;k++) 
       {
        StringConcatenate(res,res,s);
       }
    return(res);
   }          
 
string BoolToString(bool b)
  {
   if (b==true)
     return(".T.");
   else
     return(".F.");   
  }
 
//*******************************************

These are very simple string functions, that are needed for the larger functions that translate structures into string messages, descriptions of several CEP structures. The first function will replicate a string, and the second one will return a string for a boolean , in a FoxPro-ish style.

struct SimpleEvent
  {
   public:
   uint EventID;
   uint EventTime;
   uint Reserved;
   ushort BindCount;
   uint Binds[100];
  };
 
   bool BindEvent(uint eventid,SimpleEvent& s)
     {
      bool fd;
      if (s.BindCount==0)
        {
         s.Binds[0]=eventid;
         s.BindCount++;
         return(true);
        }
      else
        {
         fd=false;
         //checking previous bind
         for (int i=0;i<s.BindCount;i++)
            {
             if (s.Binds[i]==eventid)
               {
                fd=true;
                break;
               }
            }//for (int i=0;i
         if (fd==false)
           {
            if (s.BindCount<ArrayRange(s.Binds,0)-1)
              {
               s.Binds[s.BindCount]=eventid;
               s.BindCount++;
               return(true);
              }
            else
              return(false);
           }
         else
           return(false);
        }//else if (s.BindCount!=0)
     }
 
   int FindBind(uint eventid,SimpleEvent& s)
     {
      bool fd;
      if (s.BindCount==0)
        {
         return(-1);
        }
      else
        {
         fd=false;
         //checking previous bind
         for (int i=0;i<s.BindCount;i++)
            {
             if (s.Binds[i]==eventid)
               {
                return(i);
                break;
               }
            }//for (int i=0;i
         return(-1);
        }
     }     
 
   bool UnbindEvent(uint eventid,SimpleEvent& s)
     {
      if (s.BindCount==0)
        return(false);
      int fd;
      fd=-1;
      for (int i=0;i<s.BindCount;i++)
         {
          if (s.Binds[i]==eventid)
            {
             fd=i;
             break;
            }
         }//for (int i=0;i
      if (fd==0)
        return(false);
      else
        {
         for (int j=fd+1;j<s.BindCount;j++)
            {
             s.Binds[j-1]=s.Binds[j];
            }
         s.Binds[s.BindCount-1]=0;
         s.BindCount--;
         return(true);
        }//if (fd==0)
     }

The SimpleEvent structure is at the core of the engine. The events queue that I described in my previous article it is made by tiny SimpleEvent structures. EventID plays the ID, EventTime is the time (in ticks – milliseconds since Windows start – not datetime format), BindCount tells to how many complex events the simple event is binded, and the Binds[] contains these events. The BindEvent(), FindBind() and UnbindEvent() functions are to work with this structure. Normally, I could have opted for a class instead of a structure, but what you gain in class functionality you lose in data maneuvrability : you cannot copy objects, but field by field, otherwise you get Structure have objects and cannot be copied.

struct SimpleEventArrangement
  {
   int EventCount;
   uint EventArray[100];
   int Prebinds[100];
   int NegationsCount[100]; //how many negation events before each event from the array
   uint NegationsArray[10][100]; //store negation events before each event from the array
   bool NegationsChecked[100];//checks of the first negations that match
  };
 
struct ComplexEvent
  {
   uint ComplexID;
   uint TimeConstraint;
   ushort ArrangementsCount;
   SimpleEventArrangement ArrangementsArray[100];
  };

The ComplexEvent structure is the base unit of the complex events table. Remember the pic from the previous article? That is quite what can be found in the ComplexEvent structure. Thus, we have the ID , time constraint, arrangements count (number of rows) , or versions of event queues that make up the same complex event, and finally an arrangements array, made up by SimpleEventArrangement(s).

The SimpleEventArrangement structure describes what can be called to be a row of that matrix. Thus we have a count, and the row (meaningly, the EventArray[]) , which contains the required simple events. The Prebinds[] will note down of course, the prebinds of these events, meaningly the position in the event queue for each simple event that is found. The NegationsCount[] will contain how many negation events are required not to appear before every simple event that is required. Thus, one count per each required simple event. The NegationsArray[] will contain these events required not to appear, on columns, rather than on lines. The NegationsChecked[] array is a sort of shortened prebind array. Instead of storing where the negations appear, a simple negation flag is raised in this array, thus the array is unidimensional.

void MakeSimpleEventArrangement(string s,SimpleEventArrangement& arrg)
  {
   int len=StringLen(s+" ");
   string rdnow="";
   string c;
   uint number,swap;
   int normcount;
   arrg.EventCount=0;
   string first;
   int intg,d,crt;
   double dbl;
   s=s+" ";//to make sure it registers last word
   ArrayInitialize(arrg.EventArray,0);
   ArrayInitialize(arrg.NegationsCount,0);
   ArrayInitialize(arrg.NegationsArray,0);
   ArrayInitialize(arrg.Prebinds,-1);
   ArrayInitialize(arrg.NegationsChecked,false);
   normcount=-1;
   if (len!=0)
     {
      for (int i=0;i<len;i++)
         {
          c=StringSubstr(s,i,1);
          if (c==" ")
            {
             if (rdnow!="")
               {
                first=StringSubstr(rdnow,0,1);
                if (first=="-"||first=="!"||first=="n"||first=="N")
                  {
                   number=StringToInteger(StringSubstr(rdnow,1,StringLen(rdnow)-1));
                   if (normcount!=-1)   
                     {
                      arrg.NegationsArray[normcount][arrg.NegationsCount[normcount]]=number;
                      arrg.NegationsCount[normcount]++;
                     }                  
                  }
                else
                  {
                   normcount++;
                   number=StringToInteger(StringSubstr(rdnow,0,StringLen(rdnow) ));
                   arrg.EventArray[normcount]=number;
                   arrg.EventCount=normcount+1;
                  }
                rdnow="";
               }//if (rdnow!="")                
            }//if (c==" ")
          else
            {
             StringConcatenate(rdnow,rdnow,c);
            }
         }//for (int i=0;i
       if (rdnow!="")
         {
          first=StringSubstr(rdnow,0,1);
          if (first=="-"||first=="!"||first=="n"||first=="N")
            {
             number=StringToInteger(StringSubstr(rdnow,1,StringLen(rdnow)-1));
             if (normcount!=-1)   
               {
                arrg.NegationsArray[normcount][arrg.NegationsCount[normcount]]=number;
                arrg.NegationsCount[normcount]++;
               }                  
            }
          else
            {
             normcount++;
             number=StringToInteger(StringSubstr(rdnow,0,StringLen(rdnow) ));
             arrg.EventArray[normcount]=number;
             arrg.EventCount=normcount+1;
            }
          rdnow="";
         }//if (rdnow!="")
 
      //reversing event order
      if (arrg.EventCount>0)
        {
         dbl=NormalizeDouble(arrg.EventCount/2,0);
         intg=NormalizeDouble(MathRound(NormalizeDouble(arrg.EventCount,0)/2),0);
         if (dbl==intg)
           d=0;
         else
           d=1;           
         crt=0;         
         for (int i=0;i<=intg-1-d;i++)
            {             
             swap=arrg.EventArray[i];
             arrg.EventArray[i]=arrg.EventArray[arrg.EventCount-1-crt];
             arrg.EventArray[arrg.EventCount-1-crt]=swap;             
             for (int l=0;l<ArrayRange(arrg.NegationsArray,0);l++)
                {                 
                 swap=arrg.NegationsArray[l][i];
                 arrg.NegationsArray[l][i]=arrg.NegationsArray[l][arrg.EventCount-1-crt];
                 arrg.NegationsArray[l][arrg.EventCount-1-crt]=swap;
                }
             crt++;
            }
        }
 
     }//if (len!=0)
   return;
  }

The MakeSimpleArrangement() function will build a simple arrangement, from a string, as instructed by the user. For instance, “10 !6 11″ means “10, followed by 11, with a 6 that has not to appear before 11″. Spacebar is separator, and negation flag can be “-”,”!”,”n” or “N”. Note that at the end of the function, data is reversed, from last to first, as I said in the previous article, that complex event arrangements will start from the nearby events going to the older events. This function has not to be called by the user, rather it is called by MakeComplexEvent().

void MakeComplexEvent(string s,uint eventid,uint time,ComplexEvent &cplx)
  {
   int len=StringLen(s+" ");
   string rdnow="";
   string c;
   string arrg;
   ushort cnt;
   s=s+" ";
   cplx.ComplexID=eventid;
   cplx.TimeConstraint=time;
   cplx.ArrangementsCount=0;
   if (len!=0)
     {
      for (int i=0;i<len;i++)
         {
          c=StringSubstr(s,i,1);
          if (c==";")
            {
             if (rdnow!="")
               {
                MakeSimpleEventArrangement(rdnow,cplx.ArrangementsArray[cplx.ArrangementsCount]);
                cplx.ArrangementsCount++;
                rdnow="";
               }//if (rdnow!="")
            }//if (c==" ")
          else
            StringConcatenate(rdnow,rdnow,c);
         }//for (int i=0;i
      //for last word
      if (rdnow!="")
        {
         MakeSimpleEventArrangement(rdnow,cplx.ArrangementsArray[cplx.ArrangementsCount]);
         cplx.ArrangementsCount++;
         rdnow="";
        }//if (rdnow!="")
     }//if (len!=0)
   return;
  }

The MakeComplexEvent() function is to be called by user when telling the engine how the complex events look like. The definition is a big string with more simple event arrangements, separated by semicolon. The other parameters are the complex event ID, the time constraint and a complex event variable to store the result in.

string ComplexEventToString(ComplexEvent& c)
  {
   string res="";
   res=res+"ComplexID "+DoubleToString(c.ComplexID,0)+"\n";
   res=res+"Arrangements "+DoubleToString(c.ArrangementsCount,0)+"\n";
   for (int i=0;i<c.ArrangementsCount;i++)
      {
       res=res+"Events: "+DoubleToString(c.ArrangementsArray[i].EventCount,0)+"\n";
       for (int j=0;j<c.ArrangementsArray[i].EventCount;j++)
          {
           res=res+" "+DoubleToString(c.ArrangementsArray[i].EventArray[j],0);
          }
       res=res+"\n";       
       res=res+"Negations:\n";
       for (int j=0;j<10;j++)
          {
           for (int k=0;k<c.ArrangementsArray[i].EventCount;k++)
              {
               res=res+" "+DoubleToString(c.ArrangementsArray[i].NegationsArray[j][k],0);
              }
           res=res+"\n";
          }
       res=res+"\n";
      }
   return(res);
  }

This function was made for commenting reasons only. The function describes a complex event in a string form. You can output the resulting string with Comment().

class CEPEngine
  {
   //setup elements
   private:
   uint QueueTimeConstraint;
   ComplexEvent ComplexEventsTable[30];
   ushort ComplexEventsCount;
   //data
   SimpleEvent EventsQueue[1000];    
   int OldestIndex;
   uint OldestTime;
 
   public:
   void CEPEngine();//constructor
   void ClearEvents();//cleans event table
   void AddComplexEvent(ComplexEvent &c);   
   virtual void EventsCallback(uint EventID);//here you treat raised events from the CEP Engine
   void SetupEngineMilliseconds(uint ms);
   void RaiseEvent(uint EventID);
   void EngineRun();
   string StatusString();
   };

Finally, we got to the CEP engine class.
QueueTimeConstraint is an important variable, which tells which is the length in time for which the engine will look in the queue. Incoming simple events, that are found in the EventsQueue[] , and go past this “event horizon” are overwritten. The OldestIndex is the end of the array ; it is a dynamic end, calculated at every new RaiseEvent() call from the outside, abiding the QueueTimeConstraint, and it has the time , in ticks, in OldestTime.

Now the small stuff:

   void CEPEngine::ClearEvents()  //cleans event table
     {
      for (int i=0;i<ArrayRange(ComplexEventsTable,0);i++)
         {
          ComplexEventsTable[i].ArrangementsCount=0;
         }
      for (int i=0;i<ArrayRange(EventsQueue,0);i++)
          {
           EventsQueue[i].EventID=0;
           EventsQueue[i].EventTime=0;
           EventsQueue[i].BindCount=0;
           ArrayInitialize(EventsQueue[i].Binds,0);
          }
      OldestIndex=-1;
      OldestTime=0;
      ComplexEventsCount=0;
     }  
 
   void CEPEngine::CEPEngine()
     {
      ClearEvents();
     }
 
   void CEPEngine::AddComplexEvent(ComplexEvent &c)
     {
      ComplexEventsTable[ComplexEventsCount]=c;
      ComplexEventsCount=ComplexEventsCount+1;
     }
 
 
   void CEPEngine::SetupEngineMilliseconds(uint ms)
     {
      QueueTimeConstraint=ms;
      return;
     }
 
 
   void CEPEngine::EventsCallback(uint EventID)//here you treat raised events from the CEP Engine
     {
     }   
 
    string CEPEngine::StatusString()
       {
        string res="";
        string eventtext;
        int etl;
        if (ComplexEventsCount!=0)
          {
           for (int c=0;c<ComplexEventsCount;c++)
              {
               eventtext="["+DoubleToString(ComplexEventsTable[c].ComplexID,0)+"]::";
               etl=StringLen(eventtext);
               res=res+eventtext;
               if (ComplexEventsTable[c].ArrangementsCount!=0)
                 {
                  for (int a=0;a<ComplexEventsTable[c].ArrangementsCount;a++)
                     {                     
                      if (ComplexEventsTable[c].ArrangementsArray[a].EventCount!=0)
                        {                         
                         for (int e=0;e<ComplexEventsTable[c].ArrangementsArray[a].EventCount;e++)
                            {
                             res=res+DoubleToString(ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e],0)+"  ";
                            }
                         res=res+"\n"+Replicate(" ",etl);
                         for (int e=0;e<ComplexEventsTable[c].ArrangementsArray[a].EventCount;e++)
                            {
                             res=res+BoolToString(ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[e])+"  ";
                            }
                         res=res+"\n";
                         if (a!=ComplexEventsTable[c].ArrangementsCount-1)
                           {
                            res=res+Replicate(" ",etl);
                           }                      
                        }//if (ComplexEventsTable[c].ArrangementsArray[a].EventCount!=0)                   
                     }//for (int a=0;a
                 }//if (ComplexEventsTable[c].ArrangementsCount!=0)
              }//for (int c=0;c
          }//if (ComplexEventsCount!=0) 
        return(res);
       }

So, the ClearEvents() clears the simple events queue. This is the reason for which it is called from the CEPEngine() constructor. The AddComplexEvent() adds a complex event to the table. It has to be called by user, after making complex events with MakeComplexEvent(). The engine is ready to work after its time constraint is set up by the user with SetupEngineMilliseconds(). And finally, the EventsCallback() is the virtual method that has to be overridden by the user, being called when the CEP engine raises a complex event.
The StatusString() function produces a comment version of the engine status.

Now some average complicated stuff: RaiseEvent() : when the user raises a simple event

    void CEPEngine::RaiseEvent(uint EventID)
      {
       long time=GetTickCount();
       bool isforbidden=true;
       uint opposite=0;
       uint difft;
       if (OldestIndex>0)
         {
          for (int i=0;i<ArrayRange(EventsQueue,0);i++)
             {
              if (EventsQueue[i].EventTime==0)
                {
                 OldestIndex=i-1;
                 OldestTime=EventsQueue[i-1].EventTime;
                 break;
                }
              if (time-EventsQueue[i].EventTime>QueueTimeConstraint)
                {
                 OldestIndex=i-1;
                 OldestTime=EventsQueue[i-1].EventTime;
                 break;
                }//if (time-EventsQueue[i].EventTime>QueueTimeConstraint)  
             }//for (int i=0;i
         }//if (OldestIndex!=0)
       else
         {
          if (OldestIndex<=0)
            {
             OldestIndex=0;
             OldestTime=time;
            }
         }
       for (int j=OldestIndex+1;j>0;j--)
          {
           EventsQueue[j].EventID=EventsQueue[j-1].EventID;
           EventsQueue[j].EventTime=EventsQueue[j-1].EventTime;
           EventsQueue[j].BindCount=EventsQueue[j-1].BindCount;
          }
       ArrayInitialize(EventsQueue[0].Binds,0);
       EventsQueue[0].EventID=EventID;
       EventsQueue[0].EventTime=time;
       EventsQueue[0].BindCount=0;
       OldestIndex=OldestIndex+1;
 
       if (ComplexEventsCount!=0)
         {
          EngineRun();
         }         
       return;
      }//raise event ends

The function has a moderate degree of complexity.
Lines 7 – 32 establish the event horizon within the queue. That is, which is the oldest event within the queue, abiding the time constraint of the whole queue.
Lines 33 – 38 move the queue one step lower , making room for a new simple event.
Lines 39 – 43 insert the new simple event coming from the user program.
Lines 45 – 48 call the EngineRun() method is to process the new data, if at least one complex event is registered in the table.

At the heart of the CEP engine lies the EngineRun() method. It is the most complicated piece of code of the entire class. But, there would be no car without the engine, right?

   void CEPEngine::EngineRun()
     {
      long time=GetTickCount();
      bool any_binded;
      int CrtColumn[100][100];//address CrtColumn as CrtColumn(event,arrangement)
      bool bypass=false;
      int q,p;
      int j,lastj;
      int test;
      if (ComplexEventsCount==0)
        return;
      ArrayInitialize(CrtColumn,0);
      for (int c=0;c<ComplexEventsCount;c++)
         {
          Print("Arrangements in complex event ",c," are ",ComplexEventsTable[c].ArrangementsCount);
          if (ComplexEventsTable[c].ArrangementsCount!=0)
            {                          
             for (int a=0;a<ComplexEventsTable[c].ArrangementsCount;a++)
                {      
                 //Check if the current simple event within arrangement within current complex event is to be binded
                 //by checking with simple events from the queue
                 p=0;                                       
                 while (time-EventsQueue[p].EventTime<=ComplexEventsTable[c].TimeConstraint)
                   {
                    Print("Looking on entry ",p," where is simple event ",EventsQueue[p].EventID," comparing with ",ComplexEventsTable[c].ArrangementsArray[a].EventArray[CrtColumn[c][a]]," on complex event ",c,", arrg. ",a," (column ",CrtColumn[c][a],") vs count ",ComplexEventsTable[c].ArrangementsArray[a].EventCount);                       
                    if (EventsQueue[p].EventID==ComplexEventsTable[c].ArrangementsArray[a].EventArray[CrtColumn[c][a]])
                      {
                       Print("Found simple event ",EventsQueue[p].EventID," for column ",CrtColumn[c][a]," of arrangement ",a," of event ",c," meaningly id=",ComplexEventsTable[c].ComplexID); 
                       ComplexEventsTable[c].ArrangementsArray[a].Prebinds[CrtColumn[c][a]]=p;
                       CrtColumn[c][a]++;                                                                                        
                       bypass=false;
                       if (CrtColumn[c][a]>1)
                         {
                          q=ComplexEventsTable[c].ArrangementsArray[a].Prebinds[CrtColumn[c][a]-2];
                          //-2 instead of -1 because we already incremented the CrtColumn by ++
                          Print("Checking negations from previous bind, ",q," to current element ",p);
                          q=q+1;
                          //Checking negations; if test passes, simple event is prebinded;                                                          
                          //q is an older event than the current one, p, so q>p
                          if (ComplexEventsTable[c].ArrangementsArray[a].NegationsCount[a]!=0)
                            {                                                
                             while (q<p)
                               {
                                for (int j=0;j<ComplexEventsTable[c].ArrangementsArray[a].NegationsCount[a];j++)
                                   {
                                    lastj=j;
                                    Print("Comparing event on ",q," meaningly ",EventsQueue[q].EventID," with negation ",ComplexEventsTable[c].ArrangementsArray[a].NegationsArray[j][CrtColumn[c][a]-2+1]);
                                    if (EventsQueue[q].EventID==ComplexEventsTable[c].ArrangementsArray[a].NegationsArray[j][CrtColumn[c][a]-2+1])
                                      {
                                       ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[j]=true;
                                       break;
                                      }
                                   }
                                 if (ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[lastj]==true)
                                   break;
                                 q++;
                                }
                            }//if (ComplexEventsTable[c].ArrangementsArray[a].NegationsCount!=0)
                          if (ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[lastj]==true)
                            {
                             Print("Erasing last bind on ",CrtColumn[c][a]-1,". Decreasing current column for complex ",c,", arrangement ",a);
                             ComplexEventsTable[c].ArrangementsArray[a].Prebinds[CrtColumn[c][a]-1]=-1;//erase prebind;                                  
                             CrtColumn[c][a]--;
                             bypass=true;
                             break;
                            } 
                         }//if (CrtColumn[c,a]>1)                            
                       if (bypass==false)//meaning,if no negations are checked                          
                         {
                          if (ComplexEventsTable[c].ArrangementsArray[a].EventCount==CrtColumn[c][a])//all prebinds are done
                            {
                             //checking time constraint again
                             if (GetTickCount()-EventsQueue[ComplexEventsTable[c].ArrangementsArray[a].Prebinds[ComplexEventsTable[c].ArrangementsArray[a].EventCount-1]].EventTime<=ComplexEventsTable[c].TimeConstraint)
                               {
                                //before binding, will check if any of these events is previously binded to the same event
                                any_binded=false;
                                for (int e=0;e<ComplexEventsTable[c].ArrangementsArray[a].EventCount;e++)
                                   {
                                    Print("Prebind",ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e]," bind location of complex event id ",ComplexEventsTable[c].ComplexID," : ",FindBind(ComplexEventsTable[c].ComplexID,EventsQueue[ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e]]));
                                    if (FindBind(ComplexEventsTable[c].ComplexID,EventsQueue[ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e]])!=-1)
                                      {
                                       any_binded=true;
                                       Print("One event is already binded to complex event ",ComplexEventsTable[c].ComplexID);
                                       break;
                                      }
                                   }
                                if (any_binded==false)
                                  {
                                   //binding event
                                   for (int e=0;e<ComplexEventsTable[c].ArrangementsArray[a].EventCount;e++)
                                      {
                                       Print("Binding simple event on ",ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e]," to ",ComplexEventsTable[c].ComplexID);
                                       BindEvent(ComplexEventsTable[c].ComplexID,EventsQueue[ComplexEventsTable[c].ArrangementsArray[a].Prebinds[e]]);
                                      }                       
                                   EventsCallback(ComplexEventsTable[c].ComplexID);//raise complex event for user to handle                                   
                                  }//if (any_binded==false)                                
                                break;//search exhausted for current arrangement of current complex event
                                      //it's either complete, so pointless to look forward
                                      //or one of the simple events is binded to the same complex event, again pointless
                               }//if (GetTickCount()-EventsQueue[ComplexEventsTable[c].ArrangementArray[a].Prebinds[...]].EventTime<=ComplexEventsTable[c].TimeConstraint)
                             else//it's a flop: completed too late; deleting binds
                               {                                      
                                for (int f=0;f<ComplexEventsTable[c].ArrangementsArray[a].EventCount;f++)
                                   {
                                    Print("Unbinding simple event from ",ComplexEventsTable[c].ArrangementsArray[a].Prebinds[f]," to ",ComplexEventsTable[c].ComplexID);
                                    UnbindEvent(ComplexEventsTable[c].ComplexID,EventsQueue[ComplexEventsTable[c].ArrangementsArray[a].Prebinds[f]]);
                                    ComplexEventsTable[c].ArrangementsArray[a].Prebinds[f]=-1;
                                    ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[f]=false;
                                   }//for (int f=0;f
                                 break;//no point to look forward , time`s up already
                               }//else if (GetTickCount()-EventsQueue[ComplexEventsTable[c].ArrangementArray[a].Prebinds[...]].EventTime<=ComplexEventsTable[c].TimeConstraint)
                            }//if (ComplexEventsTable[c].ArrangementsArray[a].EventCount==CrtColumn[c][a])
                         }//else if (bypass==true)                                                                                                 
                      }//if (EventsQueue[p].EventID==ComplexEventsTable[c].ArrangementsArray[a].EventArray[CrtColumn[c,a]])
                    p++;
                   }//while (time-EventsQueue[p].EventTime<=ComplexEventsTable[c].TimeConstraint)
                }//for (int a=0;a
            }//if (ComplexEventsTable[c].ArrangementsCount!=0)
         }//for (int c=0;c
 
      for (int c=0;c<ComplexEventsCount;c++)
         {          
          if (ComplexEventsTable[c].ArrangementsCount!=0)
            {
             for (int a=0;a<ComplexEventsTable[c].ArrangementsCount;a++)
                {                     
                 if (ComplexEventsTable[c].ArrangementsArray[a].EventCount!=0)
                   { 
                    if (CrtColumn[c][a]<ComplexEventsTable[c].ArrangementsArray[a].EventCount)
                      {
                       //removing prebinds
                       for (int f=0;f<ComplexEventsTable[c].ArrangementsArray[a].EventCount;f++)
                          {
                           ComplexEventsTable[c].ArrangementsArray[a].Prebinds[f]=-1;
                           ComplexEventsTable[c].ArrangementsArray[a].NegationsChecked[f]=false;
                          }//for (int f=0;f
                      }//if (CrtColumn[c][a]
                   }//if (ComplexEventsTable[c].ArrangementsArray[a].EventCount!=0)                       
                }//for (int a=0;a
            }//if (ComplexEventsTable[c].ArrangementsCount!=0)
         }//for (int c=0;c
      return;
     }

The function contains two loops. One is the main loop, lines 13 – 117 , and the second loop is 120 – 140. The second loop has a cleanup role to be explained below.

The main loop will go thru the ComplexEventsTable, to check which appear. For each complex event within this table, the arrangements will be checked, and for each arrangement, a series of procedures will be performed.
Inside this loop, another while loop will pass thru the EventsQueue[], (lines 23 – 115) from the first to the last, looking for events that match the time constraint of the current complex event. One of the most important variables inside this method is the CrtColumn[] array. This one stores the current pointer (current column in the unidimensional EventArray[]). Thus, the procedure checks the current required simple event from the table with the one found in the queue (line 26) , prebinds it and increases the CrtColumn[] value for the current complex event arrangement (lines 29 – 31). Since CrtColumn[] is initialized with zeros, if it finds first event, it is increased to 1. Now, if the CrtColumn[] is higher than one (meaning that the second event was found, and preparing to find the third) it is necessary to check if between the previous two events (e.g. first and the second, in this case) there are negation events (the ones that are forbidden from appearing), only if the NegationsCount[] for that specific simple event is nonzero (lines 40 – 59). If one negation event appears, it will break the search – pointless for the current arrangement. It also sets the bypass variable to true, for the following code, however that is just a programming technique to make the listing look better – the following code will be executed only if no negations appear in the meanwhile. Since events array is zero based, the maximum event that can be binded is EventCount-1, and, if CrtColumn[] for the current arrangement is equal to EventCount, then all events were prebinded. The complex event is like already done. What follows happens only if the oldest event still matches the time constraint. It will check if at least one of the pre-binded events of this pass was binded for real to the analysed complex event, already. In this case, will raise the any_binded flag to be true, and break. If the flag is not raised, then the complex event is considered completed in this arrangement, simple events are binded to the complex event , EventsCallback() is called to notify the user and current search is breaked (lines 90-96) . Otherwise, pre-binding is failed – happens too late. Events are unbinded, negations and prebinds are erased, search it’s breaked – no point to look forward if it’s too late already.

Then what comes is a secondary loop, lines 121 – 141. Since events are mapped from the nearest to the oldest, they are reported to the user creating incomplete complex events whose “shadowing” that manifests into future prohibiting same complex event from being reported, should it appear again, but made by other simple events. All the prebinds are removed, only definitive binds remain valid. Note, however, that the same complex event, conceptually, may be triggered more times than once, by different arrangements, if these contain completely different simple event streaks (e.g. 15 and 16 or 11 and 10 , if all four appear , meaning the same event, both arrangements will be reported as complex events with the same ID, but distinct) because the engine will not be able know if the event appeared again or these mean the same one.

You may question about negations, as you see negations are event specific, each event having its own negation events before it and the previous event. The current engine does not support global negations. However, if you want some events to be global negations, they have to be set up between every two consecutive simple events. The first event in the queue may have negations assigned, however these will not be checked, because there would be no known timeframe where to check for them. Negations are valid only between consecutive simple events.

This is a script example using the CEP engine in a test. The test consist in passing abstract events to the CEP engine. Let’s call it ceptest.mq5 .

//+------------------------------------------------------------------+
//|                                                      ceptest.mq5 |
//|                                        Copyright Bogdan Caramalac|
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2010, MetaQuotes Software Corp."
#property link      "http://www.mql5.com"
#property version   "1.00"
 
#include 
 
 
//our class: we needed it to override EventsCallback;
class LocalCEPEngine : public CEPEngine
  { 
    void EventsCallback(uint EventID)
     {
      Print("COMPLEX EVENT RECOGNIZED: ",EventID);
     }   
  };
 
//the CEP engine object
LocalCEPEngine myengine;
 
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
   uint tickstart,tick;
   ulong AbstractEvent;
   ComplexEvent c;
   //setting up engine time span
   myengine.SetupEngineMilliseconds(10000);
   //creating complex events
   MakeComplexEvent("10 !6 11 ; 10 12 13 ; 10 13 12",1111,2000,c); //the event for MSFT;  
   myengine.AddComplexEvent(c);   
   //note the events with ! are negations
   MakeComplexEvent("5 !7 15 ; 5 !7 16 17 ; 5 !7 17 !7 !9 16",2222,3000,c); //the event for ^GSPC;
   myengine.AddComplexEvent(c); 
 
   tickstart=GetTickCount();
   tick=tickstart;
 
//alternate tests:
   myengine.RaiseEvent(10);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(6);   
   Print("*****************************************************************************************************************");   
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(10);
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");
   return;
}

This one generates for instance the following output:

I invite you to check the logs, look how the system is working. Try other tests:
Like this:

//alternate tests:
   myengine.RaiseEvent(10);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(11);
   Print("*****************************************************************************************************************");
   return;

Or like this:

//alternate tests:
   myengine.RaiseEvent(10);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(6);   
   Print("*****************************************************************************************************************");   
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");

Or like this:

//alternate tests:
   myengine.RaiseEvent(10);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(10);
   Print("*****************************************************************************************************************");
   return;

Or like this:

//alternate tests:
   myengine.RaiseEvent(10);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(10);
   Print("*****************************************************************************************************************");
   myengine.RaiseEvent(11);   
   Print("*****************************************************************************************************************");
   return;

Now you gonna ask me how to find out which combinations of simple events have to be wired up into complex events. Which complex events stay at the basis of the mid frequency algos? How to scout for them?

Well, I didn’t graduate Paul Wilmott’s CQF. I cannot tell you that. I am a simple romanian finance bachelor, possibly going to be fried very soon in the coming mess to the public sector where I sadly work. I gave a minor suggestion about the mechanics of this search, in the article about self-tuning systems. If you got the brains and the power to go forward, and if you believe in the face value of the things, fairness of education and other blah blah stuff – try getting yourself a Computational Finance masters – or a Quantitative Finance masters – should be opening enough. If you can’t finance one, try Finland. I see Hanken is pretty fair – if you have the nerve to pass thru papers and questions. Otherwise, study alone, implement at home – with all the limits coming from this!

File links:
CEPEngine.mqh
ceptest.mq5

Combinatorics in MQL5

Bogdan Caramalac, MQLmagazine sr.editor — Mon, 24 May 2010 21:28:41 +0000

[Versiunea romaneasca] [MQLmagazine.com in romana] [English edition]

Long time ago I wrote on my former blog about generation of permutations, however I never dwelved too much into explaining the algorithm and the code. Indeed, there are situations when combinatorics are to be applied – when scouting for arbitrages or while deploying statistics requiring asset selections as prerequisite.

Permutations

Permutations are quite simple to both find and generate, although math manuals quite skip these algorithms. Since permutations are specific orders in which elements of a set are arranged, that means while arranging the elements, the possibilities to arrange the remainder elements shrink. For instance if we have 4 elements in a set, after setting the first we have just 3 places left, and so on, until the last element fits automatically to the single place left. Thus, the composition of the number depicting the permutation number is done like when composing a number in another numeration base, except that the base shrinks at every step.

For instance we have to find this permutation’s number:

|1|2|3|4|
|1|4|2|3|

At the beginning, the numeration base is 4, and our array is empty:

|1|2|3|4| : mathematical notation of positions
|1|4|2|3| : permutation to decypher

|0|0|0|0| : occupied places in the array
|0|1|2|3| : places in the array (zero-based)
|0|1|2|3| : count array

We look on the first position, 1, and read the first element, “1″. For computing reasons we use zero-based arrays.
Permutation number = 1-1 = 0

|1|2|3|4| : mathematical notation of positions
|1|4|2|3| : permutation to decypher

|0|0|0|0| : presently occupied places in the permutation
|1|0|0|0| : occupied places in positions notation after insertion
|0|1|2|3| : count array (conceptual, for counting)
|x|0|1|2| : new count array for remainder elements

Now we look at the second element. The second element reads “4″. Then, as we have just 3 seats free, the numeration base is 3. Permutation number = (0 x 3) = 0. To this, we add the position in the count array for the element “4″, which is 2 : Permutation number = 0 + 2 = 2

|1|2|3|4| : mathematical notation of positions
|1|4|2|3| : permutation to decypher

|1|0|0|0| : presently occupied places in the notation of positions
|1|0|0|4| : occupied places in positions notation after insertion
|x|0|1|2| : count array
|x|0|1|x| : new count array for remainder elements

Now we look at the third element. The third element reads “2″. Then, as we have just 2 seats free, the numeration base is 2. Permutation number = (2 x 2) = 4. To this, we add the position in the count array for the element “2″, which is 0 : Permutation number = 4 + 0 = 4

|1|2|3|4| : mathematical notation of positions
|1|4|2|3| : permutation to decypher

|1|0|0|4| : presently occupied places in the notation of positions
|1|2|0|4| : occupied places in positions notation after insertion
|x|0|1|2| : count array
|x|x|0|x| : new count array for remainder elements

Calculation stops. We need only to insert in calculation n-1 elements. The nth element is presumed to be on the leftover place.

Final permutation number = 4

The reverse algorithm, to generate permutations from permutation numbers, is the following:
Starting from base 2, we do n-1 steps of dividing the previous result by an increasing base, keeping the remainder.
So we have:
4 / 2 = [2], remainder 0
[2] / 3 = 0, remainder 2
[0] / 4 = 0, remainder 0

Then we start building the permutation back. We read the last remainder, and we find it to be 0. This means our first element is indeed “1″, as 0 from the count array stands for 1 in the positions array.

|1|2|3|4| : mathematical notation of positions
|0|1|2|3| : physical subscripts in array variable
|0|1|2|3| : count array
|1|0|0|0| : occupied positions after current element insertion
|x|0|1|2| : new count array for remainder elements
|1|0|0|0| : built permutation

Our base shrinked to 3, as we have 3 seats left. We read the following remainder, in a reversed order, and we find it to be 2. Looking into the new count array, we see that 2 corresponds to element “4″ from the mathematical notation.

|1|2|3|4| : mathematical notation of positions
|0|1|2|3| : physical subscripts in array variable
|x|0|1|2| : count array
|1|0|0|4| : occupied positions after current element insertion
|x|0|1|x| : new count array for remainder elements
|1|4|0|0| : built permutation

Our base shrinked to 2, as we have 2 seats left. We read the following remainder, in a reversed order, and we find it to be 0. Looking into the new count array, we see that position 0 corresponds to element “2″ from the mathematical notation.

|1|2|3|4| : mathematical notation of positions
|0|1|2|3| : physical subscripts in array variable
|x|0|1|x| : count array
|1|2|0|4| : occupied positions after current element insertion
|x|x|0|x| : new count array for remainder elements
|1|4|0|2| : built permutation

We have only last element to be determined, and that is “3″, on the single free seat unoccupied.

|1|2|3|4| : mathematical notation of positions
|0|1|2|3| : physical subscripts in array variable
|x|x|0|x| : count array
|1|2|3|4| : occupied positions after current element insertion
|x|x|x|x| : new count array for remainder elements
|1|4|3|2| : built permutation

Because both algorithms are quite straight, they can be implemented straight into functions. Thus, we are starting to write the Combinatorics.mqh.

//+------------------------------------------------------------------+
//|                                                Combinatorics.mqh |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
 
//*************************************************
 
 string Replicate(string s, int count)
   {
    string res="";
    if (count<=0)
      return(res);
    for (int i=1;i<=count;i++)
      StringConcatenate(res,res,s);
    return(res);
   }
 
 int MathDiv(int a,int b)
    {
     int res;
     res=a-MathRound(MathMod(a,b));
     res=MathRound(res/b);
     return(res);
    }
 
 int Round(double a)
    {
     return(MathRound(NormalizeDouble(a,0)));
    }
 
//*************************************************

Usual stuff that we don’t insist on, we go forward to Permutations:

//********************************************************************
//
//
//                             Permutations
//
//
//********************************************************************
 
 long Factorial(ushort n)
   {
    long res=1;
    for (int i=2;i<=n;i++)
       res=res*i;
    return(res);
   }
 
 int FindPermutation(ushort elem, int& perm[])
    {
     int res=0;
     int rr;
     int positions[30];
     for (int i=0;i<ArrayRange(positions,0);i++)
        positions[i]=i+1;
     for (int e=1;e<elem;e++)//we don`t look for the last element , there is one seat left
        {
         rr=0;
         for (int i=0;i<elem;i++)
            {
             if (perm[e-1]==i+1)
               {
                res=res+rr;//has already -1, it`s zero-based
                //Print("Found element ",perm[e-1]," to be on perm[",i,"]. Adding ",rr, " to res, equals ",res);
                if (e!=elem-1)
                  res=res*(elem-e);
                //Print("Multiplicating by elem-e (",elem-e,") = ",res);                
                positions[i]=-1;
               }
             else
               { //if the searched element is not on current element,
                 //increment place counter only current position was not previously used;
                if (positions[i]!=-1)
                  rr++;
               }
            }
        }//for (int e=1;i<=elem;i++)
      return(res);  
    }
 
void GeneratePermutation(int number,int elem,int &perm[])
    {    
    string elemallow; 
    int stoppoint,crtpoint;       
    ushort AscX=StringGetCharacter("X",0);
    int i,j,base;
    int nnow;
    elemallow=Replicate("a",20);
    base=2;    
    nnow=number;    
    for (i=elem-2;i>=0;i--)
       {
       perm[i]=Round(MathMod(nnow,base));       
       nnow=MathDiv(nnow,base);
       base=base+1;        
       }
    perm[0]=perm[0]+1;//because first element is from 1 to elem, not 0 to elem-1    
    for (i=0;i<=elem-2;i++)
       {
       if (i==0)
         {
          StringSetCharacter(elemallow,perm[0]-1,AscX);
         }             
       else
          {
          stoppoint=perm[i]+1;          
          crtpoint=0;
          for (j=0;j<elem;j++)
             {             
             if (StringGetCharacter(elemallow,j)==AscX)
                continue;
             else
                {
                crtpoint=crtpoint+1;
                if (crtpoint==stoppoint)
                   {
                   perm[i]=j+1;//write new perm[i]
                   StringSetCharacter(elemallow,j,AscX);
                   }                   
                }
             }//for (j=0;j
          }//else if (i=0)    
        }//for (i=0;i
    for (i=1;i<=elem;i++)
       {
       if (StringGetCharacter(elemallow,i-1)!=AscX)
          {          
          perm[elem-1]=i;          
          break;
          }
       }        
    return;  
    }

This time we got lucky. Just two simple functions to call. Won’t be the case for Combinations.

Combinations

The combinations algorithm is way harder to understand than the permutations algorithm. The most used algorithm is the one that generates combinations in a recursive manner.

Let’s look at a simple combination queue: C(5,3)

1, 2, 3
1, 2, 4
1, 2, 5
1, 3, 4
1, 3, 5
1, 4, 5

First of all, every element has a different freedom degree. For instance, first can’t be beyond n-k+1.
Another thing, is that after you generate the first element, what you have to do then are C(n-1,k-1).
For instance, C(4,2) are:

1, 2
1, 3
1, 4
2, 3
2, 4
3, 4

For each instance of these , C(3,1) are:

1
2
3

Now you can see how it integrates:

When the procedure is called to generate C(5,3) it starts enumerating the first element, from 1 to 3. Thus, it has:

1,(1+x)

Then it calls itself to generate C(4,2) , starts enumerating from 1 to 2.
1,(1+1),(2+x)

Finally calls itself to generate C(3,1) , and completes:
1,2,(2+1) = 1,2,3
1,2,(2+2) = 1,2,4
1,2,(2+3) = 1,2,5

Returns to the previous cycle and continues with C(4,2) with the following element, 2:
1,(1+2),(1+2+x)

Starts calling C(3,1) and completes
1,3,4
1,3,5

Then turns back to continue C(5,3) with the following element, 2.

The algorithm continues until first function call ends the enumeration loop.

The implementation of the algorithm is quite murky. This time we opted for a class. Because, since direct generation is not possible, it’s not feasible to generate all the combinations every time you want a certain combination, in the case when you may want some from a list. So, a class is needed, to provide an inner loop generating the combinations while calling a callback function to pass the user the found combinations, with the option of early terminating.

//********************************************************************
//
//
//                             Combinations
//
//
//********************************************************************
 
 
long CombinationsTotal(ushort n,ushort k)
  {
   return(Factorial(n)/(Factorial(k)*Factorial(n-k)));
  }
 
class CombinationObject
  {   
   protected: 
   long counter;
   ushort n,k;
   bool terminate;   
   ushort current_combination[];
   private:
   ushort last_combination[];
   bool found;
 
   long CombinationsTotal(ushort n,ushort k);
   string CombImage(ushort &ccomb[],ushort n);
   void GenCombinationsRec(ushort depth,ushort nn,ushort kk);
 
   public:
   void SetupCombinationObject(ushort nn,ushort kk); 
   void LoopCombinations();
   virtual void CombinationsCallback();
   CombinationObject() {SetupCombinationObject(6,3);};   
  };
 
void CombinationObject::CombinationsCallback()
  {
   Print(CombImage(current_combination,k));
  }
 
void CombinationObject::SetupCombinationObject(ushort nn,ushort kk)
  {
   n=nn;
   k=kk;
   terminate=false;
   ArrayInitialize(current_combination,0);
   ArrayInitialize(last_combination,0);
   ArrayResize(current_combination,n);
   ArrayResize(last_combination,n);
   counter=0;
  }
 
string CombinationObject::CombImage(ushort &ccomb[],ushort n)
  {
   string res="";
   for (int i=0;i<n;i++)
      res=res+DoubleToString(ccomb[i],0)+" ";
   return(res);
  }
 
void CombinationObject::GenCombinationsRec(ushort depth,ushort nn, ushort kk)
  {
   bool okay,r;
   if (terminate==true)
     return;
 
   if (depth==k)
     return;  
 
   if (kk==0)
     return;
 
   int allows=nn-kk+1; 
   for (ushort i=1;i<=allows;i++)
      {
       if (terminate==true)
          return;
       if (depth==0)
         current_combination[depth]=i;
       else
         current_combination[depth]=current_combination[depth-1]+i;
 
       GenCombinationsRec(depth+1,nn-i,kk-1);
 
       okay=false;//checking for identical combination
       for (int j=1;j<=k;j++)
          {
           if (current_combination[j-1]!=last_combination[j-1])
             okay=true;//okay, new one
          }
       if (okay==false)
         continue;
       else
         { //copying current to last
          for (int j=1;j<=k;j++)
             last_combination[j-1]=current_combination[j-1];
         }
 
       CombinationsCallback();
       if (terminate==true)
         return;      
       counter=counter+1;
 
      }//for (int i=1;i<=allows;i++)
   return;
  }
 
void CombinationObject::LoopCombinations()
  {
   GenCombinationsRec(0,n,k);
   return;
  }

This is a way tougher nut to crack. The interesting function here, the engine itself, is GenCombinationsRec(). The depth parameter tells the function on which level it starts generating the combinations. When depth is depleted, that is, at every end of subsequent call queue, a new combination is generated in current_combination[] and counter is increased. What the user has to do, is to inherit this class in a custom class and override the CombinationsCallback(). Also, the terminate variable is available, if that is set to true, once the CombinationsCallback() ends, all GenCombinationsRec() calls will be early terminated and program will resume execution.

Arrangements

Since arrangements are permutated combinations, there is no point in overcomplicating the issue. The class is almost a carbon copy of the CombinationObject class, with the difference that the recursive method here, GenArrangementsRec(), which is quite like GenCombinationsRec(), generates combinations, and then, when every one of them is ready, permutates it and serves every arrangement (permutated combination) to the ArrangementsCallback(). Simple and convenient, yet results are not in the sorted manner like with Permutations and Combinations:

//********************************************************************
//
//
//                             Arrangements
//
//
//********************************************************************  
 
long ArrangementsTotal(ushort n,ushort k)
  {
   return(Factorial(n)/Factorial(n-k));
  }
 
class ArrangementObject
  {
   protected: 
   long counter;
   ushort n,k;
   ushort current_arrangement[];
   bool terminate;  
   private:
   ushort current_combination[];  
   ushort last_combination[];     
   bool found;
 
   long ArrangementsTotal(ushort n,ushort k);
   string ArrgImage(ushort &ccomb[],ushort n);
   void GenArrangementsRec(ushort depth,ushort nn,ushort kk);
 
 
   public:
   void SetupArrangementObject(ushort nn,ushort kk); 
   void LoopArrangements();
   virtual void ArrangementsCallback();
   ArrangementObject() {SetupArrangementObject(6,3);};   
  };
 
void ArrangementObject::ArrangementsCallback()
  {
   Print(ArrgImage(current_arrangement,k));
  }
 
void ArrangementObject::SetupArrangementObject(ushort nn,ushort kk)
  {
   n=nn;
   k=kk;
   terminate=false;   
   ArrayInitialize(current_arrangement,0);
   ArrayInitialize(current_combination,0);
   ArrayInitialize(last_combination,0);
   ArrayResize(current_combination,n);
   ArrayResize(current_arrangement,n);
   ArrayResize(last_combination,n);   
   counter=0;
  }
 
 
 
string ArrangementObject::ArrgImage(ushort &ccomb[],ushort n)
  {
   string res="";
   for (int i=0;i<n;i++)
      res=res+DoubleToString(ccomb[i],0)+" ";
   return(res);
  }
 
void ArrangementObject::GenArrangementsRec(ushort depth,ushort nn, ushort kk)
  {
   int perm[30];
   bool okay,r;
   if (terminate==true)
     return;
 
   if (depth==k)
     return;  
 
   if (kk==0)
     return;
 
   int allows=nn-kk+1; 
   for (ushort i=1;i<=allows;i++)
      {
       if (terminate==true)
          return;
       if (depth==0)
         current_combination[depth]=i;
       else
         current_combination[depth]=current_combination[depth-1]+i;
 
       GenArrangementsRec(depth+1,nn-i,kk-1);
 
       okay=false;//checking for identical combination
       for (int j=1;j<=k;j++)
          {
           if (current_combination[j-1]!=last_combination[j-1])
             okay=true;//okay, new one
          }
       if (okay==false)
         continue;
       else
         { //copying current to last
          for (int j=1;j<=k;j++)
             last_combination[j-1]=current_combination[j-1];
         }
 
 
       //here combination is final and arrangements are generated       
       for (int iperm=0;iperm<Factorial(k);iperm++)
          {
           GeneratePermutation(iperm,k,perm);           
           for (int jperm=0;jperm<k;jperm++)
              {
               current_arrangement[jperm]=current_combination[perm[jperm]-1];                              
              }                      
           ArrangementsCallback();                        
           if (terminate==true)
             return;      
           counter=counter+1;   
          }   
 
       }//for (int i=1;i<=allows;i++)
   return;                   
  }
 
void ArrangementObject::LoopArrangements()
  {
   GenArrangementsRec(0,n,k);
   return;
  }

And now a sample script (combtest.mq5). The script will display arrangements of n forex pairs taken as k, with a rising k, from 3 to n. Forex pairs are extracted at the beginning, by interogating each tradeable instrument about its calculation mode and retaining only the ones answering SYMBOL_CALC_MODE_FOREX , with the specification that symbol has to have 6 or more letters (two forex pairs and a suffix). Pairs are extracted and added to an array. Then a customized arrangements engine is called to display arrangements:

//+------------------------------------------------------------------+
//|                                                     combtest.mq5 |
//|                                       Copyright Bogdan Caramalac |
//|                                           http://mqlmagazine.com |
//+------------------------------------------------------------------+
#property copyright "Copyright Bogdan Caramalac"
#property link      "http://mqlmagazine.com"
#property version   "1.00"
 
 
#include 
 
int CurrenciesCount;
int PairsCount;
string ForexPairs[100];
 
//our class: we needed it to override ArrangementsCallback;
class LocalArrgClass : public ArrangementObject
  { 
    void ArrangementsCallback()
     {
      string image="";
      image=DoubleToString(counter,0)+" :";
      for (int i=0;i<k;i++)         
         image=image+" "+ForexPairs[current_arrangement[i]-1];
      Print(image);      
     }   
  };
 
//the arrangment object
LocalArrgClass myarrg;
 
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
 
void CheckAndAddPair(string pair)
  {
   bool found;
   if (PairsCount==0)
     {
      ForexPairs[PairsCount]=pair;
      PairsCount++;
     }
   else
     {
      found=false;
      for (int i=0;i<PairsCount;i++)
         {
          if (ForexPairs[i]==pair)
            {
             found=true;
             break;
            }          
         }
      if (found==false)
        {
         ForexPairs[PairsCount]=pair;
         PairsCount++;               
        } 
     } 
   return;
  }
 
void OnStart()
  {  
   string crtsymbol,pair1,pair2;
   CurrenciesCount=0;
   PairsCount=0;
   for (int i=0;i<SymbolsTotal(false);i++)
      {
       crtsymbol=SymbolName(i,false);
       if (SymbolInfoInteger(crtsymbol,SYMBOL_TRADE_CALC_MODE)==SYMBOL_CALC_MODE_FOREX&&StringLen(crtsymbol)>=6)
         {          
          pair1=StringSubstr(crtsymbol,0,3);          
          CheckAndAddPair(pair1);
          pair2=StringSubstr(crtsymbol,3,3);         
          CheckAndAddPair(pair2);
         }
      }//for (int i=0;i
    for (int iarrg=3;iarrg<=PairsCount;iarrg++)
       {
        myarrg.SetupArrangementObject(PairsCount,iarrg);
        myarrg.LoopArrangements();      
       }
  }
//+------------------------------------------------------------------+

Here’s a peek on how it looks (though you have to stop it using terminate – otherwise it will jump some of the log):

You might ask why did we opted for permutations to use int instead of uint or long. The answer is the MathMod() and MathRound() issue. These functions don’t work with integer types. Even if they expect and return an integers as meaning they still work with double as type. So any convertions have to rely on a force cast of a double, which is a signed real type, as int is a signed integer type. Using unsigned integers or even longer long types that span over 8 bytes instead of just 6 like double would have overcomplicated the issue. I think these functions have to be re-engineered by MetaQuotes, to work only with integer types and to abide the received integer types that play the operands, just as div and mod work in Pascal implementations. And as for MathRound() , this must be a real rounding function that returns in the integer type that is to receive the result, or in a generic integer type, if it is used in expressions.

File links:
Combinatorics.mqh
combtest.mq5