Wp0030

744 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 12, NO. 4, JULY 2001

Computational Learning Techniques for Intraday FX
Trading Using Popular Technical Indicators
M. A. H. Dempster, Tom W. Payne, Yazann Romahi, and G. W. P. Thompson

Abstract—There is reliable evidence that technical analysis, V, while Sections VI–VIII describe in more detail how each ap-
as used by traders in the foreign exchange (FX) markets, has proach can be applied to solve this optimization problem ap-
predictive value regarding future movements of foreign exchange proximately. The computational experiments performed are out-
prices. Although the use of artificial intelligence (AI)-based
trading algorithms has been an active research area over the last lined and their results given in Section IX. Section X concludes
decade, there have been relatively few applications to intraday with a discussion of these results and suggests further avenues
foreign exchange—the trading frequency at which technical of research.
analysis is most commonly used. Previous academic studies have Reinforcement learning has to date received only limited at-
concentrated on testing popular trading rules in isolation or have tention in the financial literature and this paper demonstrates
used a genetic algorithm approach to construct new rules in an
attempt to make positive out-of-sample profits after transaction that RL methods show significant promise. The results also indi-
costs. In this paper we consider strategies which use a collection cate that generalization and incorporation of constraints limiting
of popular technical indicators as input and seek a profitable the ability of the algorithms to overfit improves out-of-sample
trading rule defined in terms of them. We consider two popular performance, as is demonstrated here by the genetic algorithm.
computational learning approaches, reinforcement learning and
genetic programming (GP), and compare them to a pair of simpler
methods: the exact solution of an appropriate Markov decision II. TECHNICAL ANALYSIS
problem and a simple heuristic. We find that although all methods Technical analysis has a century-long history amongst invest-
are able to generate significant in-sample and out-of-sample
profits when transaction costs are zero, the genetic algorithm
ment professionals. However, academics have tended to regard
approach is superior for nonzero transaction costs, although none it with a high degree of scepticism over the past few decades
of the methods produce significant profits at realistic transaction largely due to their belief in the efficient markets or random walk
costs. We also find that there is a substantial danger of overfitting hypothesis. Proponents of technical analysis had until very re-
if in-sample learning is not constrained. cently never made serious attempts to test the predictability of
Index Terms—Computational learning, foreign exchange (FX), the various techniques used and as a result the field has remained
genetic algoriths (GA), linear programming, Markov chains, rein- marginalized in the academic literature.
forcement learning, technical trading, trading systems.
However, due to accumulating evidence that markets are less
efficient than was originally believed (see, for example, [1]),
I. INTRODUCTION there has been a recent resurgence of academic interest in the
claims of technical analysis. Lo and MacKinlay [2], [3] have
S INCE the era of floating exchange rates began in the early
1970s, technical trading has become widespread in the for-
eign exchange (FX) markets. Academic investigation of tech-
shown that past prices may be used to forecast future returns
to some degree and thus reject the random walk hypothesis for
United States stock indexes sampled weekly.
nical trading however has largely limited itself to daily data.
LeBaron [1] acknowledges the risk of bias in this research
Although daily data is often used for currency overlay strate-
however. Since various rules are applied and only the successful
gies within an asset-allocation framework, FX traders trading
ones are reported, he notes that it is not clear whether the returns
continuously throughout the day naturally use higher frequency
achieved could have been attained by a trader who had to make
data.
the choice of rules in the first place. LeBaron argues that to
In this investigation, the relative performance of various op-
avoid this bias it is best simply to look at rules that are both
timization techniques in high-frequency (intraday) foreign ex-
widely used and have been in use for a long period of time.
change trading is examined. We compare the performance of
Neely et al. [4] use a genetic programming based approach to
a genetic algorithm (GA) and a reinforcement learning (RL)
avoid this bias and found out-of-sample net returns in the 1–7%
system to a simple linear program (LP) characterising a Markov
per annum range in currency markets against the dollar during
decision process (MDP) and a heuristic.
1981 to 1995.
In Section II, we give a brief literature review of preceding
Although there has been a significant amount of work in tech-
work in technical analysis. Sections III and IV then introduce
nical analysis, most of this has been based on stock market data.
the GA and RL methods. The stochastic optimization problem
However, since the early 1970s this approach to trading has
to be solved by all the compared methods is defined in Section
been widely adopted by foreign currency traders [4]. A survey
by Taylor and Allen [5] found that in intraday trading 90% of
Manuscript received October 16, 2000; revised February 27, 2001. respondents reported the use of technical analysis, with 60%
The authors are with the Centre for Financial Research, Judge institute of
Management, University of Cambridge, Cambridge, U.K. stating that they regarded such information as at least as impor-
Publisher Item Identifier S 1045-9227(01)05018-4. tant as economic fundamentals. Neely et al. [4] argue that this
1045–9227/01$10.00 © 2001 IEEE

DEMPSTER et al.: COMPUTATIONAL LEARNING TECHNIQUES FOR INTRADAY FX TRADING 745

can be partly explained by the unsatisfactory performance of Evolutionary learning encompasses sets of algorithms that are
exchange rate models based on economic fundamentals. They inspired by Darwinian evolution. GAs are population-based op-
cite Frankel and Rose [6] who state that no model based on timization algorithms first proposed by Holland [21]. They have
such standard fundamentals like money supplies, real incomes, since become an active research area within the artificial intelli-
interest rates, and current-account balances will ever succeed in gence community and have been successfully applied to a broad
explaining or predicting a high percentage of the variation in the range of hard problems. Their success is in part due to their sev-
exchange rate, at least at short or medium-term frequencies. eral control parameters that allow them to be highly tuned to the
A number of researchers have examined net returns due to specific problem at hand. GP is an extension proposed by Koza
various trading rules in the foreign exchange markets [7], [8]. [22], whose original goal was to evolve computer programs.
The general conclusion is that trading rules are sometimes able Pictet et al. [23] employ a GA to optimize a class of exponen-
to earn significant returns net of transaction costs and that this tially weighted moving average rules, but run into serious over-
cannot be easily explained as compensation for bearing risk. fitting and poor out-of-sample performance. They report 3.6%
Neely and Weller [9] note however that academic investigation to 9.6% annual excess returns net of transaction costs, but as
of technical trading has not been consistent with the practice the models of Olsen and Associates are not publicly available
of technical analysis. As noted above, technical trading is most their results are difficult to evaluate. Neely and Weller [9] re-
popular in the foreign exchange markets where the majority of port that for their GA approach, although strong evidence of
intraday foreign exchange traders consider themselves technical predictability in the data is measured out-of-sample when trans-
traders. They trade throughout the day using high-frequency action costs are set to zero, no evidence of profitable trading op-
data but aim to end the day with a net open position of zero. portunities arise when transaction costs are applied and trading
This is in contrast to much of the academic literature which has is restricted to times of high market activity.
tended to take much longer horizons into account and only con-
sider daily closing prices. IV. REINFORCEMENT LEARNING
Goodhart and O’Hara [10] provide a thorough survey of past
Reinforcement learning has so far found only a few financial
work investigating the statistical properties of high-frequency
applications. The reinforcement learning technique is strongly
trading data, which has tended to look only at narrow classes of
influenced by the theory of MDPs, which evolved from attempts
rules. Goodhart and Curcio [11] examine the usefulness of resis-
to understand the problem of making sequences of decisions
tance levels published by Reuters and also examine the perfor-
under uncertainty when each decision can depend on the pre-
mance of various filter rules identified by practitioners. Demp-
vious decisions and their outcomes. The last decade has wit-
ster and Jones [12], [13] examine profitability of the systematic
nessed the merging of ideas from the reinforcement learning and
application of the popular channel and head-and-shoulders pat-
control theory communities [24]. This has expanded the scope
terns to intraday FX trading at various frequencies, including
of dynamic programming and allowed the approximate solution
with an overlay of statistically derived filtering rules. In subse-
of problems that were previously considered intractable.
quent work [14], [15] upon which this paper expands, they apply
Although reinforcement learning was developed indepen-
a variety of technical trading rules to trade such data (see also
dently of MDPs, the integration of these ideas with the theory
Tan [16]) and also study a genetic program which trades com-
of MDPs brought a new dimension to RL. Watkins [25]
binations of these rules on the same data [17]. None of these
was instrumental in this advance by devising the method of
studies report any evidence of significant profit opportunities,
-learning for estimating action-value functions. The nature
but by focussing on relatively narrow classes of rules their re-
of reinforcement learning makes it possible to approximate
sults do not necessarily exclude the possibility that a search over
optimal policies in ways that put more effort into learning to
a broader class would reveal profitable strategies. Gencay et al.
make good decisions for frequently encountered situations
[18] in fact assert that simple trading models are able to earn
at the expense of less effort for less frequently encountered
significant returns after transaction costs in various foreign ex-
situations [26]. This is a key property which distinguishes
change markets using high frequency data.
reinforcement learning from other approaches for approximate
solution of MDP’s.
As fundamental research in reinforcement learning advances,
III. GENETIC ALGORITHMS applications to finance have started to emerge. Moody et al. [27]
examine a recurrent reinforcement learning algorithm that seeks
In recent years, the application of artificial intelligence (AI)
to optimize an online estimate of the Sharpe ratio. They also
techniques to technical trading and finance has experienced sig-
compare the recurrent RL approach to that of -learning.
nificant growth. Neural networks have received the most atten-
tion in the past and have shown varying degrees of success.
V. APPLYING OPTIMIZATION METHODS TO TECHNICAL
However recently there has been a shift in favor of user-trans-
TRADING
parent, nonblack box evolutionary methods like GAs and in par-
ticular genetic programming (GP). An increasing amount of at- In this paper, following [15], [17], [14], we consider trading
tention in the last several years has been spent on these genetic rules defined in terms of eight popular technical indicators used
approaches which have found financial applications in option by intraday FX traders. They include both buy and sell signals
pricing [19], [20] and as an optimization tool in technical trading based on simple trend-detecting techniques such as moving av-
applications [17], [14], [4]. erages as well as more complex rules. The indicators we use are


the price channel breakout, adaptive moving average, relative information (their current position) than the heuristic and MDP
strength index, stochastics, moving average convergence/diver- methods and we might thus expect them to perform better. The
gence, moving average crossover, momentum oscillator, and GA method also has an extra constraint restricting the com-
commodity channel index. A complete algorithmic description plexity of the rules it can generate which is intended to stop
of these indicators can be found in [15], [14]. overfitting of the in-sample data.
To define the indicators, we first aggregate the raw tick data
into (here) quarter-hourly intervals, and for each compute the
VI. APPLYING RL TO THE TECHNICAL TRADING PROBLEM
bar data—the open, close, high, and low FX rates. Most of the
indicators use only the closing price of each bar, so we will The ultimate goal of reinforcement learning based trading
introduce the notation to denote the closing GBP:USD FX systems is to optimize some relevant measure of trading system
rate (i.e., the dollar value of one pound) of bar (here we use performance such as profit, economic utility or risk-adjusted re-
boldface to indicate random entities). turn. A standard RL framework has two central components;
We define the market state at time as the binary string an agent and an environment. The agent is the learner and de-
of length 16 giving the buy and sell pounds indications of the cision maker that interacts with the environment. The environ-
eight indicators, and define the state space as the ment consists of a set of states and available actions for the agent
set of all possible market states. Here a 1 represents a trading in each state.
recommendation for an individual indicator whose entry is oth- The agent is bound to the environment through perception
erwise 0. In effect, we have constructed from the available tick and action. At a given time step the agent receives input ,
data a discrete-time data series: at time (the end of the bar which is representative of some state , where is the set
interval) we see , compute and must choose whether or not of all possible states in the environment. As mentioned in the
to switch currencies based on the values of the indicators in- previous section, is defined here as being a combination of
corporated in and which currency is currently held. We con- the technical indicator buy and sell pounds decisions prepended
sider this time series to be a realization of a binary string valued to the current state of the agent (0 for holding dollars and 1 for
stochastic process and make the required trading decisions by pounds). The agent then selects an action where
solving an appropriate stochastic optimization problem. telling it to hold pounds ( ) or dollars ( ) over
Formally, a trading strategy is a function the next timestep. This selection is determined by the agent’s
, , for some current position ( , dollars, policy ( , i.e., defined in our case as the trading strategy)
or , pounds), telling us whether we should hold pounds ( ) which is a mapping from states to probabilities of selecting each
or dollars ( ) over the next timestep. It should be noted that of the possible actions.
although our trading strategies are formally Markovian (feed- For learning to occur while iteratively improving the trading
back rules), some of our technical indicators require a number strategy (policy) over multiple passes of the in-sample data, the
of periods of previous values of to decide the corresponding agent needs a merit function that it seeks to improve. In RL, this
0-1 entries in . The objective of the trading strategies used in is a function of expected return which is the amount of return
this paper is to maximize the expected dollar return (after trans- the agent expects to get in the future as a result of moving for-
action costs) up to some horizon : ward from the current state. At each learning episode for every
time-step the value of the last transition is communicated to
the agent by an immediate reward in the form of a scalar rein-
(1) forcement signal . The expected return from a state is therefore
defined as
where denotes expectation, is the proportional transaction
cost, and is chosen with the understanding that trading strate-
gies start in dollars, observe and then have the opportunity to
switch to pounds. Since we do not have an explicit probabilistic (2)
model for how FX rates evolve, we cannot perform the expecta-
tion calculation in (1), but instead adopt the familiar approach of
dividing our data series into an in-sample region, over which we where is the discount factor and is the final time step.
optimize the performance of a candidate trading strategy, and an Note that the parameter determines the “far-sightedness” of
out-of-sample region where the strategy is ultimately tested. the agent. If then and the agent myopically
The different approaches utilized solve slightly different tries to maximize reward only at the next time-step. Conversely,
versions of the in-sample optimization problem. The simple as the agent must consider rewards over an increasing
heuristic and Markov Chain methods find a rule which takes as number of future time steps to the horizon. The goal of the agent
input a market state and outputs one of three possible actions: is to learn over a large number of episodes a policy mapping of
either “hold pounds,” “hold dollars” (switching currencies if which maximizes for all as the limit
necessary) or “stay in the same currency.” of the approximations obtained from the same states at the pre-
The GA and RL approaches find a rule which takes as input vious episode.
the market state and the currency currently held, and chooses In our implementation, the agent is directly attempting to
between two actions: either to stay in the same currency or maximize (1). The reward signal is therefore equivalent to ac-
switch. Thus the RL and GA method are given slightly more tual returns achieved from each state at the previous episode.


This implies that whenever the agent remains in the base cur- analysis of the algorithm to enable convergence proofs. As a
rency, regardless of what happens to the FX rate, the agent is bootstrapping approach, -learning estimates the -value func-
neither rewarded nor penalized. tion of the problem based on estimates at the previous learning
Often RL problems have a simple goal in the form of a single episode. The -learning update is the backward recursion
state which when attained communicates a fixed reward and
has the effect of delaying rewards from the current time pe-
riod of each learning episode. Maes and Brookes [28] show
(5)
that immediate rewards are most effective—when they are fea-
sible. RL problems can in fact be formulated with separate state
where the current state-action pair from
spaces and reinforcement rewards in order to leave less of a
the previous learning episode. At each iteration (episode) of the
temporal gap between performance and rewards. In particular
learning algorithm, the action-value pairs associated with all the
it has been shown that successive immediate rewards lead to ef-
states are updated and over a large number of iterations their
`
fective learning. Mataric [29] demonstrates the effectiveness of
values converge to optimality for (4). We note that there are
multiple goals and progress estimators, for example, a reward
some parameters in (5): in particular, the learning rate refers
function which provides instantaneously positive and negative
to the extent with which we update the current -factor based
rewards based upon “immediate measurable progress relative to
on future rewards, refers to how “far-sighted” the agent is
specific goals.”
and a final parameter of the algorithm is the policy followed
It is for this reason that we chose to define the immediate
in choosing the potential action at each time step. -learning
reward function (2) rather than to communicate the cumulative
has been proven to converge to the optimal policy regardless of
reward only at the end of each trading episode.
the policy actually used in the training period [25]. We find that
In reinforcement learning the link between the agent and the
following a random policy while training yields the best results.
environment in which learning occurs is the value function .
In order for the algorithm to converge, the learning rate
Its value for a given state is a measure of how “good” it is for
must be set to decrease over the course of learning episodes.
an agent to be in that state as given by the total expected fu-
Thus has been initially set to 0.15 and converges downwards
ture reward from that state under policy . Note that since the
to 0.000 15 at a rate of , where is the
agent’s policy determines the choice of actions subsequent
episode (iteration) number which runs from 0 to 10 000. The
to a state, the value function evaluated at a state must depend
parameter has been set to 0.9999 so that each state has full
on that policy. Moreover, for any two policies and we say
sight of future rewards in order to allow faster convergence to
that is preferred to , written , if and only if ,
the optimal.
. Under suitable technical conditions there will
With this RL approach we might expect to be able to outper-
always be at least one policy that is at least as good as all other
form all the other approaches on the in-sample data set. How-
policies. Such a policy is called an optimal policy and is the
ever on the out-of-sample data set, in particular at higher slip-
target of any learning agent within the RL paradigm. To all op-
page values, we suspect that some form of generalization of the
timal policies is associated the optimal value function , which
input space would lead to more successful performance.
can be defined in terms of a dynamic programming recursion as

(3) VII. APPLYING THE GENETIC ALGORITHM
The approach chosen extends the genetic programming work
Another way to characterize the value of a state is to con-
initiated in [14] and [17]. It is based on the premise that prac-
sider it in terms of the values of all the actions that can be taken
titioners typically base their decisions on a variety of technical
from that state assuming that an optimal policy is followed
signals, which process is formalized by a trading rule. Such a
subsequently. This value is referred to as the -value and is
rule takes as input a number of technical indicators and gener-
given by
ates a recommended position (long £, neutral, or long $). The
agent applies the rule at each timestep and executes a trade if
the rule recommends a different position to the current one.
(4) Potential rules are constructed as binary trees in which the
The optimal value function expresses the obvious fact that the terminal nodes are one of our 16 indicators yielding a Boolean
value of a state under an optimal policy must equal the expected signal at each timestep and the nonterminal nodes are the
return for the best action from that state, i.e., Boolean operators AND, OR, and XOR. The rule is evaluated
recursively. The value of a terminal node is the state of the
associated indicator at the current time; and the value of a
nonterminal node is the associated Boolean function applied to
The functions and provide the basis for learning algo- its two children. The overall value of the rule is the value of
rithms for MDPs. the root node. An overall rule value of one (true) is interpreted
-learning [25] was one of the most important breakthroughs as a recommended long £ position and zero (false) is taken
in the reinforcement learning literature [26]. In this method, the as a recommended neutral position. Rules are limited to a
learned action-value function directly approximates the op- maximum depth of four (i.e., a maximum of 16 terminals) to
timal action-value function and dramatically simplifies the limit complexity. An example rule is shown in Fig. 1. This


definition of a rule generalizes that used in [17], [14] which
allows trees in the comb form of Fig. 1, but to depth 10.
The fitness score of such a rule is defined as the total return
cumulated over the appropriate data period [cf. (1)], i.e.

Fig. 1. An example rule.
(6)

The genetic algorithm is used to search the space of all such
rules and is tuned to favor rules that trade successfully (i.e.,
achieve high fitness scores) in the in-sample training period. An
initial population of 250 rules is randomly generated and each
rule is evaluated according to (6). A new population of rules is
generated from this in which high scoring rules are preferred to
low scoring rules. This bias means that the fitness scores of the
new population should be greater than those of the old popula-
tion. New rules are generated by two processes: crossover and
mutation.
To use crossover two parent rules are selected from the Fig. 2. Genetic algorithm crossover.
current population. The selection process is biased toward fitter
(better performing) rules: all rules in the current population with a Markov chain on a small set of market states and replaces
are ranked in order of fitness score, and are chosen with the problem of maximizing the profit made over the in-sample
a probability linearly proportional to their rank. A random period with that of maximizing a total expected discounted re-
subtree is chosen from each parent rule and these two subtrees turn assuming Markov dynamics. This approach is described in
are swapped between the parent rules to create two new rules, detail in Section VIII-A.
each of which inherits characteristics from both parents. This The second method is a simple heuristic: with each state we
process is shown in Fig. 2. Potential subtrees to swap are associate a number (which will be interpreted as the expected
checked to ensure that the resulting new rules would not exceed rise in the exchange FX rate over the next trading periods
the specified maximum tree depth. Two new rules meeting this for some ) and consider strategies which buy pounds if this
criterion are inserted into the new population. For mutation, a number exceeds one threshold, and sell pounds when it falls
single rule is selected from the current population (again biased below a second threshold. We then optimize over and the two
toward the better performers) and a random node is replaced thresholds to maximize the in-sample profit. More details on this
with a random node of the same type (e.g., an AND might method are given in Section VIII-B.
become an OR). The mutated rule is then inserted into the new These two methods were used to benchmark the success of
population. the true computational learning approaches in maximizing the
New rules are generated using 75% crossover and 25% mu- in-sample profit. Neither are likely to attain the true optimum,
tation until a total of 250 new rules are generated, when the new but they should perform reasonably well. The heuristic was
population is evaluated for fitness scores. The average score of more successful at solving the in-sample problem with nonzero
the new population should be greater than that of the old due transaction costs than any of the other approaches, but it does
to the favoritism shown to the better performing rules in the old not perform particularly well out-of-sample due to over-fitting.
population. This process is repeated 100 times (generations) and
the best rule found during the entire run is selected as the output
A. A Markov-Chain Linear Programming Approximation
of the genetic algorithm.
The rules found by this process exhibit a number of desirable Recall that denotes the market-state at time (the values of
properties. First, with careful tuning of the GA they should per- the indicator recommendations on which the trading decision at
form well in-sample. Second, a rule can be understood by hu- time is based). In this section we will let denote the set of
mans: it is clear what the rule does, even if it is not clear why it all market states which occur in the in-sample dataset.
does it. Thirdly, this structure limits (but does not prevent) how We will construct a controlled Markov chain on the set of
much a rule can learn in detail (i.e., overfit) the training data set. pairs where denotes a market state present in the
It is to be expected that this enforced generalization will lead to in-sample dataset and indicates
better out-of-sample performance with a possible reduction in whether or not pounds are currently held. The controls at each
in-sample performance. timestep are 0 or 1, indicating the currency we wish to hold over
the next timestep as before.
Denoting by the number of times state appears
VIII. MARKOV CHAIN AND SIMPLE HEURISTIC
in the in-sample dataset (excluding the final data-point), and
In addition to the RL and GA methods we will consider two by the number of times state is immediately fol-
alternative approaches. The first replaces the in-sample dataset lowed by state , we define a controlled Markov chain ,


, by fixing and choosing the prob-
ability of a transition from to using control
to be

For , we define an approximation to the ex-
pected dollar return over the next timestep given we are in state
and hold currency as

We are now in a position to replace the problem of maximizing
the in-sample return with the problem Fig. 3. Cumulative in-sample monthly returns at no slippage.

(7)

where is a feedback map from the state-space of the Markov
chain to the set of controls and we treat the term
as zero (since we must start in dollars). The constant is a dis-
count factor, chosen arbitrarily to be equal to . This is an
approximation to the objective (1).
Since the set is quite small, this problem can be solved
exactly using the technique of linear programming. To see this,
observe that the solution to (7) is characterized by the optimal
value function defined for all as
the optimal value (1) when . The function
satisfies the dynamic programming recursion Fig. 4. Cumulative out-of-sample monthly returns at no slippage.

(8)

whose solution is unique under various conditions (it suffices
that the Markov chain has a finite statespace and that
, which is the case here). But a solution to the system above
can be obtained by solving the linear program

subject to for all and Fig. 5. Cumulative in-sample monthly returns at 1 bp slippage.

B. A Simple Heuristic
(9) One objection to the method of the previous section is
that when transaction costs are large the solution obtained to
the problem in (7) above may perform badly over the actual
which must yield the required optimal value function . The in-sample period; worse even than the trivial strategy ‘always
optimal action in state can be extracted from the hold dollars’ with a return of 0. As an alternative, we consider
optimal value function by choosing any maximizing a heuristic defined in terms of three parameters :
the right-hand side of (8). The solution to the LP is found for
each in-sample period using the CPLEX commercial LP solver. when (10)


Fig. 6. Cumulative out-of-sample monthly returns at 1 bp slippage. Fig. 8. Cumulative out-of-sample monthly returns at 4 bp slippage.

Fig. 7. Cumulative in-sample monthly returns at 4 bp slippage.
Fig. 9. Cumulative in-sample monthly returns at 8 bp slippage.

when (11)

The expected value in (10) and (11) is just the expected return
available if we held pounds for the next days given that the cur-
rent market state is . Since we do not have a stochastic process
model for FX rate movements, this expectation must also be es-
timated from the in-sample data (assuming the ergodic theorem
holds) as

where is the set .
For a several classes of stochastic process models for FX dy-
namics, the optimal strategy for both very low and very high
transaction costs has the form of (10) and (11), making it a plau- Fig. 10. Cumulative out-of-sample monthly returns at 8 bp slippage.
sible heuristic in general.
The optimization of the three parameters of the heuristic is a
from January 1994 until January 1998 using a moving window
nonconvex multiextremal problem and for each in-sample pe-
of one year for training (fitting) followed by one month out-of-
riod is solved by a simple genetic algorithm.
sample testing.
The cumulative (without reinvestment) returns over the
IX. NUMERICAL EXPERIMENTS period are shown in odd-numbered Figs. 3–9 for selected
The results reported below were obtained by applying the ap- in-sample (fitting) cases and evenly-numbered Figs. 4–10 for
proaches described above to the GBP:USD exchange rate data corresponding out-of-sample (testing) cases. The annualised


TABLE I
OUT-OF-SAMPLE AVERAGE ANNUAL
RETURNS

average monthly returns are shown in Table I for the various
approaches in the out-of-sample case.
In-sample fitting performance has been shown for com-
pleteness to demonstrate the learning ability of the various
approaches. It is clear that on the in-sample data set, the
simple heuristic approach consistently outperforms all the
Fig. 11. Indicators used by the genetic algorithm by slippage.
other methods except in the no slippage case, when all methods
were able to fit the data to essentially the same degree. The
out-of-sample test results demonstrate, however, that the
heuristic approach was in fact significantly overfitting the data.
For the out-of-sample back-tests, we note that the genetic al-
gorithm and reinforcement learning approaches tended to out-
perform the others at lower slippage values. In order to gain
further insight into the overall best performing GA, a plot of
how often it inferred rules using each indicator for each slip-
page value is shown in Fig. 11. Fig. 12 shows the frequencies
with which the GA employed specific indicators over the en-
tire four year data period, aggregated for all slippage values and
into quarters, with considerable variability in the patterns evi-
dent. The GAs reduction in trading frequency with decreasing
transaction costs is demonstrated dramatically in Fig. 13. Sim-
ilar results apply to the other methods with the exception of the
heuristic, whose high trading frequency at realistic transaction Fig. 12. Relative frequencies over time of indicators used by the genetic
costs leads to its poor performance in out-of-sample back-tests. algorithm.
Data on the dealing frequency of all the different approaches is
given in Table II in order to shed light on the risk profiles of
the different methods. These results are discussed further in the
final Section X.
In order to evaluate the relative risk-adjusted performance of
the trading models further, we now consider several risk mea-
sures found in the financial literature. A risk-adjusted measure
commonly used to evaluate portfolio models is the Sharpe ratio,
defined as

Sharpe Ratio (12)

The Sharpe ratio evaluations of our trading models, as shown
in Table III demonstrate that on the dataset used we are able to
gain significant risk-adjusted returns up to a slippage value of 1
bp. However the Sharpe ratio is numerically unstable for small Fig. 13. Genetic algorithm position duration by slippage.
variances of returns and cannot consider the clustering of profit
and loss trades [13], [18]. Furthermore the Sharpe ratio penal- TABLE II
izes strategies for upside volatility and its definition in terms of OUT-OF-SAMPLE AVERAGE MONTHLY DEALING FREQUENCY
summary statistics means that a strategy can appear to be suc-
cessful but in fact suffers from significant drawdown [6]. Max-
imum drawdown over a certain period of length
is defined as

(13)


TABLE III hundred signal states that occur within a given year. Each of
OUT-OF-SAMPLE SHARPE RATIOS the four trading strategies is then evaluated out-of-sample at
15-min intervals on the next month of indicator signals and this
back-testing process is then rolled forward a month and repeated
for a total of 48 months.
It is evident that in-sample, all approaches were able to
infer successful trading strategies and also notable that the ge-
netic algorithm consistently underperforms the other methods
in-sample. This is undoubtably due to the constraint imposed on
TABLE IV
OUT-OF-SAMPLE DRAWDOWNS
the complexity of the rules which was specifically imposed to
avoid overfitting. By contrast, the non-GA approaches—in par-
ticular the trading threshold heuristic—may end up exploiting
noise in the in-sample data set. At zero-slippage (no costs of
trading), however, we find that all approaches are able to infer
similar strategies and perform similarly out-of-sample. There
is evidence that the non-GA approaches do in fact overfit as the
GA outperforms the other methods with nonzero transaction
TABLE V costs in the out-of-sample cases up to 8 bp slippage.
OUT-OF-SAMPLE STIRLING RATIOS The fact that the techniques investigated here return positive
results both in-sample and in out-of-sample back-tests implies
that there is useful information in technical indicators that can
be exploited. This is consistent with the tenets of technical anal-
ysis and contradictory to the efficient market hypothesis. Fur-
thermore, the GAs relatively good out-of-sample performance
demonstrates that using a combination of technical indicators
leads to better performance than using the individual indica-
where and are the total returns of the periods from tors themselves. In fact, Dempster and Jones [15], [14] demon-
to and as defined by (6) respectively. In our case was strate that with a few exceptions these indicators are largely
defined as on a monthly basis and the mean over the out-of- unprofitable on the same data when considered in isolation.
sample back-test period is reported in Table IV. We therefore Figs. 11 and 12 demonstrate that some indicators also convey
also quote the Stirling ratio, defined as more information than others depending on the slippage value
and the market state. We note that the relative strength index
Stirling Ratio (14) (buy/sell) indicators are not used at zero transaction costs but
as the slippage is increased, the GA tends to favor them. Indi-
cators such as price channel breakout, stochastics, and moving
which is the average monthly return divided by the maximum
average crossover (buy) are very important at zero slippage, but
drawdown within that month. This value averaged over the 48
the GA appears to disregard the information provided by them at
monthly back-test periods is reported in Table V.
higher slippage values. At zero slippage the GA is able to infer
The current RL implementation requires about eight minutes
successful strategies without using the current position. How-
CPU time on a 650 MHz Athlon per single training opti-
ever, at higher transaction costs, knowing the current position
mization (episode). The GA is implemented in the interpreted
becomes very important. This lends credence to the argument
language Scheme, but evaluation is parallelised over multiple
that this extra position information tends to favor the RL and GA
similar CPUs. It also takes about eight minutes CPU time
approaches, since the Markov chain approach and the heuristic
per optimization on a single machine. The Markov chain and
did not have this information available to them.
heuristic approaches execute in four seconds and approximately
The RL approach, the Markov chain and the heuristic all ex-
four minutes, respectively.
ploit the fact that of the market possible states only a few
hundred actually occur in the in-sample period. This number
X. DISCUSSION AND CONCLUSION is small enough that each state may be considered individually
In this paper we have developed three trading strategies based when deciding a strategy. However, there are two problems with
on computational learning techniques and one simple heuristic such a rule when it is back-tested out-of-sample.
based on trading thresholds over a fixed horizon. The strategies Firstly, we may encounter a state in the out-of-sample data
based on the genetic (programming) algorithm (GA) and rein- which was not present in the in-sample data. In that case some
forcement ( -) learning train at 15-min intervals on the buy-sell arbitrary action must be made and both the RL and the Markov
signals from eight popular technical trading indicators—some chain method choose to hold their current position. This may be
of which require a number of previous observations—and cur- a disadvantage if many new states are encountered and it also ig-
rent positions over a one year period of GBP:USD FX data, nores the fact that some new states may be very similar to states
while the Markov chain strategy uses the entire set of training which were present in the in-sample period. The genetic algo-
data to estimate the relative transition frequencies of the few rithm on the other hand generates a trading rule in the in-sample


training period whose structure tends to take the same actions in [5] M. P. Taylor and H. Allen, “The use of technical analysis in the foreign
similar states. exchange market,” J. Int. Money Finance, vol. 11, pp. 304–314, June
1992.
Secondly, there is a severe danger that these approaches [6] J. A. Frankel and A. K. Rose, “A survey of empirical research on nominal
may learn too well (overfit) the specific in-sample data; in exchange rates,” Nat. Bureau Economic Res., Working Paper No. 4865,
other words the in-sample problem they attempt to solve is too Sept. 1994.
[7] R. Levich and L. Thomas, “The significance of technical trading rule
specific. Indeed the simple heuristic method demonstrates this profits in the foreign exchange market: A bootstrap approach,” J. Int.
quite clearly: it achieves excellent in-sample performance but Money Finance, vol. 12, pp. 451–474, Oct. 1993.
is mediocre out-of-sample and terrible at realistic transaction [8] C. R. Osler and K. Chang, “Head and shoulders: Not just a flaky pattern,”
Federal Reserve Bank of New York, Staff Papers no. 4, Aug. 1995.
costs. The limit on the complexity of the GA is an artificial [9] C. Neely and P. Weller, “Intraday technical trading in the foreign ex-
constraint which reduces the opportunity for the GA to overfit change market,” Federal Reserve Bank of St. Louis, Working Paper
while not prohibiting simple trading rules. This limit effectively 99-016A, 1999.
[10] C. Goodhart and M. O’Hara, “High frequency data in financial markets:
forces the GA to work with generalized (classes of specific) Issues and applications,” J. Empirical Finance, vol. 4, pp. 73–114, 1997.
states. Thus we hope to improve the current reinforcement [11] C. Goodhart and R. Curcio, When support/resistance levels are broken,
learning approach by forcing state generalization and also by can profits be made? Evidence from the foreign exchange market,
London School of Economics, July 1992.
improving its convergence properties. [12] M. A. H. Dempster and C. M. Jones, “Can channel pattern trading be
Another current avenue of research is to find constraints for successfully automated?,” European J. Finance, 2001, to be published.
[13] , Can technical pattern trading be successfully automated? 2. Head
the in-sample optimization problem which force state general- and shoulders, Centre Financial Res., Judge Inst. Management, Univ.
ization (such as the rule-complexity constraint in the GA ap- Cambridge, July 1999.
proach), but for which a heuristic similar to that of Section [14] C. M. Jones, “Automated Technical Foreign Exchange Trading with
High Frequency Data,” Ph.D. dissertation, Centre Financial Res., Judge
VIII-B can be applied. Inst. Management, Univ. Cambridge, June 1999.
A further goal is the exploration of generalization methods [15] M. A. H. Dempster and C. M. Jones, The profitability of intra-day FX
in the context of a broader RL approach. Neuro-dynamic trading using technical indicators, , Centre Financial Res., Judge Inst.
Management, Univ. Cambridge, Dec. 2000.
programming (NDP) [24] attempts to combine neural networks [16] G. W. Tan, “Topics in foreign exchange trading systems,” M.S. thesis,
with the central ideas of dynamic programming in order to Centre Financial Res., Judge Inst. Management, Univ. Cambridge, June
address this goal. NDP’s employ parametric representations 1999.
[17] M. A. H. Dempster and C. M. Jones, “A real-time adaptive trading
of the value function (such as artificial neural networks) to system using genetic programming,” Quant. Finance, 2001, to be
overcome the curse of dimensionality. The free parameters are published.
tuned using regression or stochastic approximation methods [18] R. Gencay, G. Ballocchi, M. Dacorogna, R. Olsen, and O. Pictet,
Real-time trading models and the statistical properties of foreign
used in combination with classical dynamic programming exchange rates, in Internal Document RAG.1988-12-01, Olsen &
methods. Further, Wilson’s X-classifier system [30] attempts to Associates, Zurich, Switzerland, 1998.
merge ideas from reinforcement learning with those from the [19] S. Chen and W. Lee, “Option pricing with gas: A second report,” in Proc.
IEEE Int. Conf. Neural Networks, vol. 1, 1997, pp. 21–25.
Classifier System community in order to incorporate general- [20] N. K. Chidambaran, C. H. J. Lee, and J. R. Trigueros, “An adaptive
ization into a -Learning-like framework. We also intend to evolutionary approach to option pricing via genetic programming,”
investigate this approach in the present context in the future. in Genetic Programming 1998: Proc. 3rd Annu. Conf., J. R. Koza,
W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H.
Garzon, D. E. Goldberg, H. Iba, and R. Riolo, Eds. Madison, WI, July
22–25, 1998, pp. 38–41.
[21] J. H. Holland, Adaptation in Natural and Artificial Systems. Ann
ACKNOWLEDGMENT Arbor, MI: Univ. Michigan Press, 1975.
[22] J. R. Koza, Genetic Programming: On the Programming of Computers
The authors would like to thank Dr. C. Jones for his insightful by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
[23] O. V. Pictet, M. M. Dacorogna, B. Chopard, M. Oudsaidene, R. Schirru,
comments, advice, and proofreading of the paper, and J. Scott and M. Tomassini, “Using genetic algorithm for robust optimization
for lending his expertise in Scheme. in financial applications,” Neural Network World, vol. 5, no. 4, pp.
573–587, 1995.
[24] D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Program-
ming. Waltham, MA: Athena Scientific, 1996.
[25] C. Watkins, “Learning from Delayed Reward,” Ph.D. dissertation, Dept.
REFERENCES Eng., Univ. Cambridge, 1989.
[26] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduc-
[1] B. LeBaron, “Technical trading rule profitability and foreign exchange tion. Cambridge, MA: MIT Press, 1998.
intervention,” J. Int. Economics, vol. 49, pp. 124–143, 1999. [27] J. Moody, L. Wu, Y. Liao, and M. Saffell, “Performance functions and re-
[2] A. W. Lo and A. C. MacKinlay, “Stock market prices do not follow inforcement learning for trading systems and portfolios,” J. Forecasting,
random walks: Evidence froma simple specification test,” Rev. Financial vol. 17, pp. 441–470, 1998.
Studies, vol. 1, pp. 41–66, 1988. [28] P. Maes and R. Brooks, “Learning to coordinate behaviors,” in Proc. 8th
[3] , A Nonrandom Walk Down Wall Street. Princeton, NJ: Princeton Nat. Conf. Artificial Intell., July 29–Aug. 3, 1990, pp. 796–802.
Univ. Press, 1999. [29] M. Mataric, “Learning in multi-robot systems,” in IJCAI-95 Workshop
[4] C. J. Neely, P. A. Weller, and R. Dittmar, “Is technical analysis in the on Adaptation and Learning in Multiagent Systems, 1995, pp. 32–37.
foreign exchange market profitable? A genetic programming approach,” [30] S. Wilson, “Classifier fitness based on accuracy,” J. Evolutionary
J. Financial Quantitative Anal., pp. 405–426, Dec. 1997. Comput., vol. 3, no. 2, pp. 149–175, 1995.


M. A. H. Dempster received the B.A. degree from Toronto University, Toronto, Yazann Romahi received the B.Eng. degree in electronics and computer sci-
ON, Canada, the M.A. degree from Oxford University, Oxford, U.K., and the ence and the M.Sc. degree in artificial intelligence from Edinburgh University,
M.S. and Ph.D. degrees from Carnegie Mellon University, Pittsgurgh, PA, all in Edinburgh, U.K. He is pursuing the Ph.D. degree at the Centre for Financial Re-
mathematics. search at the Judge Institute of Management Studies in the University of Cam-
He is currently Professor of Management Studies (Finance and Management bridge.
Science), Director of the Centre for Financial Research and Director of Re- His Ph.D. work is involved in the investigation of FX Market Dynamics and
search at the Judge Institute of Management Studies in the University of Cam- optimal trading in the FX markets in the presence of transaction costs using
bridge. His present research interests include mathematical and computational artificial intelligence-based techniques.
finance and economics, optimization and nonlinear analysis, stochastic systems,
algorithm analysis, decision support, and applications software and telecom-
munications systems modeling. He is author of more than 100 published re-
search articles and reports and is author, editor or translator of eight books
including Mathematics of Derivative Securities (Cambridge, U.K.: Cambridge
Univ. Press, 1997).
Dr. Dempster is joint Editor-in-Chief of Quantitative Finance and Associate
Editor of Stochastics and Stochastic Reports, Journal of Economic Dynamics
and Control, Computational Finance, and Computational Economics.

G. W. P. Thompson received the M.A. and Ph.D. degrese in mathematics from
Tom W. Payne received the M.A. degree in engineering and computer science the Queen’s College, Cambridge University, Cambridge, U.K.
from Clare College, Cambridge University, Cambridge, U.K., and the M.Sc. He is a Research Associate at the Centre for Financial Research at the Judge
degree in artificial intelligence from Edinburgh University, Edinburgh, U.K. He Institute of Management Studies in the University of Cambridge. His Ph.D.
is currently pursuing the Ph.D. degree at the Centre for Financial Research at work investigated Gaussian models for interest-rates, numerical techniques for
the Judge Institute of Management Studies in the University of Cambridge. pricing complex options in Gaussian markets, and the optimal trading of se-
His work focuses on the application on genetic algorithms to building suc- curities in the presence of transaction costs. His current research interests also
cessful automated trading systems. include optimization, particularly stochastic programming.

Wp0030

More Related Content

Similar to Wp0030 (20)

Recently uploaded (20)

Wp0030