You should view this article as a continuation of the one about Progress Apama, because it brings institutional issues to MetaTrader5. If you look at that Statistical Arbitrage strategy, you’ll see it doesn’t have but an execution at 1-2 seconds at most. So, that doesn’t seem HFT at all. But becomes HFT in the context of running this in 500 instances – because it will have hundreds of executions per second. However, if you keep up to 10-20 instances, is something that MetaTrader5 can handle.

That strategy inspired me to think about an indicator that draws tick charts. Sure, you can view a tick chart, but in the Market Watch, by clicking on “Tick Chart” near “Symbols”. But what if you want to see more than one at a time? So I wrote an indicator to do it. However I did it quite poor, and Rosh (Rashid Umarov) came to the rescue. This is the indicator body, modified after inserting levels:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127

//+------------------------------------------------------------------+
//|                                                        Ticks.mq5 |
//|                        Copyright 2010, MetaQuotes Software Corp. |
//|                                              http://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "2010, MetaQuotes Software Corp."
#property link      "http://www.mql5.com"
#property version   "1.00"
#property indicator_separate_window
#property indicator_buffers 2
#property indicator_plots   2
//--- plot Bid
#property indicator_label1  "Bid"
#property indicator_type1   DRAW_LINE
#property indicator_color1  Blue
#property indicator_style1  STYLE_SOLID
#property indicator_width1  1
//--- plot Ask
#property indicator_label2  "Ask"
#property indicator_type2   DRAW_LINE
#property indicator_color2  Red
#property indicator_style2  STYLE_SOLID
#property indicator_width2  1
#property indicator_level0  0.00
#property indicator_level1  0.00
#property indicator_level2  0.00
#property indicator_level3  0.00
#property indicator_level4  0.00
#property indicator_level5  0.00
#property indicator_level6  0.00
#property indicator_level7  0.00
#property indicator_level8  0.00
#property indicator_level9  0.00
#property indicator_level10  0.00
#property indicator_level11  0.00
#property indicator_level12  0.00
#property indicator_level13  0.00
#property indicator_level14  0.00
#property indicator_level15  0.00
//--- input parameters
input int      number_of_ticks=1000;
input int      points_indent=10;
input int      LevelsCount=7;
//--- indicator buffers
double         BidBuffer[];
double         AskBuffer[];
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- indicator buffers mapping
   SetIndexBuffer(0,BidBuffer,INDICATOR_DATA);
   SetIndexBuffer(1,AskBuffer,INDICATOR_DATA);
   PlotIndexSetDouble(0,PLOT_EMPTY_VALUE,0);
   PlotIndexSetDouble(1,PLOT_EMPTY_VALUE,0);
   IndicatorSetString(INDICATOR_SHORTNAME,Symbol()+" Tick Chart");
//---
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],
                const double &open[],
                const double &high[],
                const double &low[],
                const double &close[],
                const long &tick_volume[],
                const long &volume[],
                const int &spread[])
  {
   static int ticks=0;
//---
   if(ticks==0)
     {
      ArrayInitialize(AskBuffer,0);
      ArrayInitialize(BidBuffer,0);
     }
   setMaxMinPrice(ticks,points_indent);
//--- CopyRates
   MqlTick last_tick;
   if(SymbolInfoTick(Symbol(),last_tick))
     {
      BidBuffer[ticks]=last_tick.bid;
      AskBuffer[ticks]=last_tick.ask;
      int shift=rates_total-1-ticks;
      ticks++;
      BidBuffer[rates_total-1]=last_tick.bid;
      AskBuffer[rates_total-1]=last_tick.ask;
      PlotIndexSetInteger(0,PLOT_SHIFT,shift);
      PlotIndexSetInteger(1,PLOT_SHIFT,shift);
      Comment("Bid =",last_tick.bid,"   Ask =",last_tick.ask);
     }
 
//--- return value of prev_calculated for next call
   return(rates_total);
  }
//+------------------------------------------------------------------+
//| set Maximum and Minimum for an indicator window based on last values
//+------------------------------------------------------------------+
void setMaxMinPrice(int last_values,int indent)
  {
   int dgs;
   double dist;
   int visiblebars=(int)ChartGetInteger(0,CHART_VISIBLE_BARS);
   int depth=MathMin(last_values,visiblebars);
   int startindex=last_values-depth;
   if (startindex<0) startindex=0;
   int max_index=ArrayMaximum(AskBuffer,startindex,depth);
   max_index=max_index>=0?max_index:0;
   int min_index=ArrayMinimum(BidBuffer,startindex,depth);
   min_index=min_index>=0?min_index:0;
   double MaxPrice=AskBuffer[max_index]+indent*_Point;
   double MinPrice=BidBuffer[min_index]-indent*_Point;
   IndicatorSetDouble(INDICATOR_MAXIMUM,MaxPrice);
   IndicatorSetDouble(INDICATOR_MINIMUM,MinPrice);
   dgs=SymbolInfoInteger(Symbol(),SYMBOL_DIGITS);
   dist=NormalizeDouble( (MaxPrice-MinPrice)/(LevelsCount+1), dgs);
   for (int i=0;i<LevelsCount;i++)
      {
       IndicatorSetDouble(INDICATOR_LEVELVALUE,i,MinPrice+dist*(i+1));
      } 
  }
//+------------------------------------------------------------------+

As a tick indicator, it looks okay. But what if you want to put something else on it, like Moving Averages, Bollinger Bands, or even monitoring spreads, like in the Progress Apama strategy window? That means you have to write over and over again, newer and newer indicators, just to show you what you need. Sure, tick calculations are easy from inside an EA – they will happen mostly in the OnTick() event – but you need indicators to help you visualize the strategy, even if their practical application will be most of the time unnecesary, due to separate calculus in the EA. But keep in mind that there is no tick history. Such indicators draw from scratch. So if you want to see something longer, at least now, when the Strategy Tester is not available to run the indicator on, you have to give it time to display, it’s not instant as for other indicators.

So, if tick data would be available, meaning PERIOD_TICK also, then some good things for us might appear, as:
a. No need to write special indicators for tick level.
b. Tick level charts appear rightaway and span for entire history.
c. Tick history will make the backtester more accurate, because the backtester doesn’t have to invent ticks to cover such ares.
d. (drawback): Huge hard drive space occupied by history, larger memory requirements.

What does it mean for MetaQuotes:
a. Change in history and data aggregation, starting from tick to make the rest, not from M1, keeping M1 as a data source for old history, where tick data was not available;
b. Light modification to data access functions (fields such as openbid, openask, closebid, closeask in MqlRates etc).
c. Light modifications to the backtester;
d. MetaTrader branded as a near institutional platform.

Why should we want this? Well, why the big boys don’t use bar charts? After all, there are no surprises on tick charts – and we will investigate this some other time – so you won’t fear that a spider leg bar will touch your stop loss in a middle of good looking trend.

Thing is, if you have Level II available, and if you automate it, you are very close to the HFT realm. In fact, what is being blamed today is not the entire algorithmic trading – because they never think about users of MetaTrader, E-Signal or even NeoTicker – but the institutional , unfair (to be read “colocated”) trading. At the surface, the discussion is about computerized vs manual trading. But these , both sides of the conflict, act on the Level II, with the possibility to quote and be filled (i.e. buy at bid, sell at ask), and pay tiny fees for these trades. Thing is, if we’d benefit Level II plus MetaTrader automation, with lower brokerage fees, some forms of HFT would become applicable. This is a thing that probably remained hidden, the fact that low latency trading seems to be not the same thing as high frequency trading, even if the terms are used interchangeably. I believe that algorithmic Level II trading can be designed to be HFT even if it doesn’t benefit from low latency execution. Problem is, traders were indoctrinated so much with Level I, that is very hard to make Level II trading accepted. Because Level I trading has a “buy or sell now, and see later if you were right” mentality, whether Level II has a “see this, quote it, if get filled, close later” mentality. The same “later” has different meanings, Level I “later” means from minutes to up, usually hours, whether Level II later means “seconds” (for normal traders) , or “milliseconds” for hedge fund systems. Trailing stop is tighter on Level II, trails pip by pip. And all is meant to make real time money. Not in a few hours, and not tomorrow or not next week. But rather, see equity going up a few times per minute. This gives a way higher degree of confort, way higher than in Level I, where you can watch losses accumulating creepy for hours or days. What is most often forgot in trading, is that trading strategies are not for robots that can wait an ethernity to improve their trading styles. Life happens in real time. We are not immortal and beyond it, youth is damn short. This is one of the drivers for cutting down the latency. On one hand, it is the need to be filled before the competition, on other hand, more and more profits per a lesser and lesser time unit means fatter salaries for management… While this, even the most rational and successful traders that on Level I cannot exceed some limits of the yield. Essentially, the Level II automated systems have this job, to compress the time, and get similar yields as the ones from Level I, but in a hundredth or lesser time. But these yields are relative and small, when looked upon from inside the hedge fund industry. Because, the system’s results are relative to the average liquidity used. That is, a lot of liquidity will remain unused, but has to return yield. And while diversification is way to use more liquidity, the time span is the other side of the coin, because one trade a second is a thing, and a hundred trades in the same second, spanning lots of microseconds each, is another thing. On another hand, HFT platforms cost shameful amounts of money, and smaller institutions do not afford them, because a lot of the yields from the market go on platform costs and PhD salaries. Now you understand why a Level II MetaTrader, even if not low latency, is important? As Irene Aldridge admits too, for the buy side, there aren’t many companies yet selling products. In fact, if we exclude Strategy Runner, X-Trader and Ninja Trader, I can’t think at others. X-Trader has the automation pretty institutional in terms of costs, and Strategy Runner is so cryptic that you have to pay lots of cash for development. None of these platforms integrate options, however, and making a market in options, getting filled there, is a great way to make economies, as option spreads can be really high.

After all, if look on the MQL5 forum, you can see that MetaQuotes at least seems to commit to make MetaTrader 5 the best trading terminal in the world. If MetaTrader5 becomes Level II ready (remember the old motherboards with the Prescott ready sticker ?) and by generating an increased awareness about the importance of tick level action in the retail trading community, it might determine changes on brokerage levels.
As brokers will give more access to tight spread ECNs, the spreads will go down, and actually the spread based model might become even extinct, replaced by a commission model – and I mean small commissions, small enough for the trader to feel that he trades in an ECN not just a retail feed that brands commission instead of spread. This way a lot of strategies from the institutional levels would become portable to MetaTrader and retail world – which will probably become “institutional with big latency”, or, more important for retail traders, FREE, easy-to-deploy HFT solution that is not low-latency based. The problem is, that commissions are driven by volume. The larger the volume, the smaller the commissions. However, at least forex traders are heavily leveraged. (Yes, the americans will soon not be able to do this anymore). Being highly leveraged, small losses are still hitting the equity hard. How much could you support commission losses until volume becomes high enough to get rebates and jump to profit? Let’s not forget, the global economy is in recession, and brokers, as any regular business in this word, live off sales… Which makes them more open for innovation, offering from low to tiny commissions…

On another hand, how could MT be used as a Level II platform the way it is? Sure, the OrderSend() can be used in a Level II manner, but without tick data, and with a backtester that just models ticks, instead of using ticks for real? I guess not. Tick data is a must if the station would be rebranded to a Level II station.
However, there are things to do until there, as MetaQuotes still has older homeworks to complete : the Strategy Tester, a better OnTrade() syntax, the Depth Of Market view and of course the OnBookEvent(). When all of these will be completed, MT5 will be so far that you’ll forget about the old MT4 candies, I’m sure about that…

But if the retail community won’t ask for more bang for the buck, nothing will be achieved, cause automated Level II is something they are not likely to slip for free.

One of the upgrades brought by MetaQuotes to the MetaEditor IDE is the introduction of a debugger.

Across this article I’ll use the article ‘error’ as a reference to the conceptual code errors such as wrongly written conditions or wrongly written calculus expressions… Warning: don’t mistake them with the errors signaled by the compiler.

For the ones that don’t know , the purpose of the debugger is to run line by line to code in order to find conceptual programming errors, and to watch the values of the variables realtime.

In MT4, that lacked a debugger, the programmers were using the Print function to find the errors in the code, this being the single method available. This method is difficult to use and the time needed to find the error is way larger compared to using a debugger, because Prin needed a recompilation every time, re-attaching the EA and reading in logs. If we were calling functions giving a wrong result , because they were badly written, we had to go inside that function and attach Print lines to see where were the errors. After repairs, the code had to be cleaned up of redundant Print lines used to find the error.

Right now the debugger works only to test scripts, but we think it will be integrated with the Strategy Tester to work with experts. This is needed also because several buggy EA modules may be run only in rare market conditions, that will not appear while forward testing, but will be revealed while backtesting.

Now the things got simpler, from the same window you can follow the program being run line by line.

In order to use the debugger we have to call the DebugBreak() function where we want the normal execution stopped in order to start execution line by line. If we attach such a script to a chart, this will not stop running, as it will be stopped only if it’s started with the debug mode of the MetaEditor.

To start the script for debugging we have to press Debug->Start or F5 key or it’s dedicated button in the special bar dedicated to the debugger, bar which contains other 3 buttons for passing thru the code.

The group of buttons from the rectangle labeled 1 are the 6 buttons used at debugging. The first 3 buttons are getting in and out from the debug mode, but also to pause. The last 3 buttons are for passing the code in other manners, in the debug mode:

- passing thru the code by Step Into (F11) we go line by line but we enter the functions referred. For instance if we are with the cursor on the line ‘p=prod(a,b)’ and we push Step Into then the cursor will go into the ‘prod’ function and we will be be able to go line by line thru the code of this function;

- passing thru the code by Step Over (F10) we go line by line thru the code but the debugger will jump over called functions (these will be executed, but not line by line in a debug mode).

- Step Out should make the debugger get out of the function where the cursor is, and not close the debug mode, thing which is happening now.

The green arrow labeled with 2 is the current line. If we push F10 (Step Over), the arrow will move to the next line.

Running a code in the Debug mode a new page opens within the Toolbox pageframe, which is divided in two (areas 3 and 4).

In area 3 we see the name of the file that’s being run, the name of the function and the number of the line where we are in the debug mode.

Area 4 is similar to the Watch window from Borland C++ / Borland Pascal or other older IDEs , where can be watched realtime the values and types of variables.

In picture above in area 4 we have 3 variables which we follow : a, s and p. The variable ‘a’ has the value of 5 because we passed the line ‘a=5′ where we are initializing it. The variable ‘p’ has the value 0, because this is the default value taken by the integer variables when they are defined, while ’s’ is undefined, because that’s the stop where the execution is stopped, right before it being declared.

These being said I wish you luck writing codes and debugging them.