SlideShare a Scribd company logo
Developing Highfrequency Trading Systems
Sebastien Donadio Sourav Ghosh download
https://ebookbell.com/product/developing-highfrequency-trading-
systems-sebastien-donadio-sourav-ghosh-56240736
Explore and download more ebooks at ebookbell.com
Here are some recommended products that we believe you will be
interested in. You can click the link to download.
Developing High Performance Leaders A Behavioral Science Guide For The
Knowledge Of Work Culture Philip Robert Harris
https://ebookbell.com/product/developing-high-performance-leaders-a-
behavioral-science-guide-for-the-knowledge-of-work-culture-philip-
robert-harris-51247316
Developing High Quality Data Models 1st Edition Matthew West
https://ebookbell.com/product/developing-high-quality-data-models-1st-
edition-matthew-west-2354358
Developing High Quality Data Models Matthew West
https://ebookbell.com/product/developing-high-quality-data-models-
matthew-west-56641312
Realworld Solutions For Developing Highquality Php Frameworks And
Applications 1st Edition Sebastian Bergmann
https://ebookbell.com/product/realworld-solutions-for-developing-
highquality-php-frameworks-and-applications-1st-edition-sebastian-
bergmann-2213124
Realworld Solutions For Developing Highquality Php Frameworks And
Applications Sebastian Bergmann
https://ebookbell.com/product/realworld-solutions-for-developing-
highquality-php-frameworks-and-applications-sebastian-bergmann-5475630
National Policies For Developing High Technology Industries
International Comparisons Francis W Rushing
https://ebookbell.com/product/national-policies-for-developing-high-
technology-industries-international-comparisons-francis-w-
rushing-43443958
Realworld Solutions For Developing Highquality Php Frameworks And
Application Sebastian Bergmann
https://ebookbell.com/product/realworld-solutions-for-developing-
highquality-php-frameworks-and-application-sebastian-bergmann-5411638
Methodical Development Of Modular Product Families Developing High
Product Diversity In A Manageable Way Dieter Krause
https://ebookbell.com/product/methodical-development-of-modular-
product-families-developing-high-product-diversity-in-a-manageable-
way-dieter-krause-47663636
Beyond Knowledge Extracognitive Aspects Of Developing High Ability
Illustrated Edition Larisa V Shavinina
https://ebookbell.com/product/beyond-knowledge-extracognitive-aspects-
of-developing-high-ability-illustrated-edition-larisa-v-
shavinina-1365908
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
Developing
High-Frequency
Trading Systems
Learn how to implement high-frequency trading
from scratch with C++ or Java basics
Sebastien Donadio
Sourav Ghosh
Romain Rossier
BIRMINGHAM—MUMBAI
Developing High-Frequency Trading Systems
Copyright © 2022 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the publisher,
except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without warranty,
either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors,
will be held liable for any damages caused or alleged to have been caused directly or indirectly by
this book.
Packt Publishing has endeavored to provide trademark information about all of the companies
and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing
cannot guarantee the accuracy of this information.
Publishing Product Manager: Heramb Bhavsar
Content Development Editor: Shreya Moharir
Technical Editor: Rahul Limbachiya
Copy Editor: Safis Editing
Project Coordinator: Farheen Fathima
Proofreader: Safis Editing
Indexer: Tejal Daruwale Soni
Production Designer: Roshan Kawale
Marketing Coordinator: Priyanka Mhatre
First published: June 2022
Production reference: 1160622
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-80324-281-1
www.packt.com
Contributors
About the authors
Sebastien Donadio has two decades of experience in high-performance computing,
software design, and financial computing. Currently an architect in the Bloomberg office
of the CTO, he has a wide variety of professional experience, including serving as CTO
of an FX/Crypto trading shop, head of engineering, quantitative analyst, and partner
at a high-frequency trading hedge. Sebastien has taught various computer science and
financial engineering courses over the past fifteen years at various academic institutions,
including Columbia University, the University of Chicago, and NYU. Sebastien holds
a PhD in high-performance, an MBA in finance, and an MSc in analytics from the
University of Chicago. His main passion is technology, but he is also a scuba diving
instructor and an experienced rock-climber.
Sourav Ghosh has worked in several proprietary high-frequency algorithmic trading
firms over the last decade. He has built and deployed extremely low-latency, high-
throughput automated trading systems for trading exchanges around the world, across
multiple asset classes. He specializes in statistical arbitrage market-making and pairs
trading strategies for the most liquid global futures contracts. He works as a vice president
at a trading firm based in Sao Paulo, Brazil. He holds a master's in computer science from
the University of Southern California. His areas of interest include computer architecture,
FinTech, probability theory and stochastic processes, statistical learning and inference
methods, and natural language processing.
Romain Rossier brings 19+ years of experience, mostly as a software architect in the
financial industry, specializing in low latency, high-performance Java software design
and development. He is currently the chief architect for the HCTech FX Proprietary
Trading engine. He also built and led the software development team at HCTech,
where he oversaw and developed the HFT platform architecture for FX, Futures, and
Fixed Income. Prior to HCTech, Romain was director of the Currenex lab, where he led
the team responsible for the development of the Currenex Intelligent Pricing System.
Romain holds a Master of Science in Communication Systems from the Swiss Federal
Institute of Technology in Lausanne.
About the reviewers
John Rizzo has applied his passion for technology and building complex computing
systems in support of the financial markets throughout his career. Over the past 17
years, he has held a number of technology leadership roles, including CTO, director, and
infrastructure architect at various hedge funds and other firms in the financial services
industry. He started and sold a company that developed systems to help manage various
aspects of the loan life cycle and the syndicated loan market. Today, he is focused on
infrastructure security in Bloomberg's Office of the CTO.
Phil Vachon works on projects related to identity, authentication, and the application
of data science to operational security challenges at Bloomberg. Prior to that, Phil
co-founded a start-up focused on high-speed packet capture and analysis. He's also
developed high-frequency trading systems, designed and implemented firmware for
identity and security infrastructure devices, built synthetic aperture radar data processing
tools, and worked on data plane traffic engineering for carrier routers. His main interests
are developing threat models that are relevant to business problems, designing secure
embedded systems, and working to improve individual privacy protections in our
increasingly connected world.
Table of Contents
Preface
Part 1: Trading Strategies, Trading Systems,
and Exchanges
1
Fundamentals of a High-Frequency Trading System
History of HFT 4
The post-1930s era 5
The modern era 6
Why have HFT? 7
What makes HFT so different
from regular trading? 7
Effect of dark pools 9
Who trades HFT? 10
What do I need to start an HFT? 10
What are HFT strategies? 11
Asset classes 12
Liquidity 12
Tick-by-tick data and data distribution 12
Liquidity rebates 12
Matching engine 13
Market making 13
Scalping 15
Statistical arbitrage 15
Latency arbitrage 16
Impact of news 17
Momentum ignition 17
Rebate strategies 18
Pinging 18
Illegal activities 19
Summary 20
2
The Critical Components of a Trading System
Understanding the trading
system 22
Trading system architecture 24
Gateways connecting to trading
exchanges 25
vi Table of Contents
Making a trading system trade
with exchanges 27
Examining the API for communication 29
Order book management 30
Order book considerations 32
Strategy making decisions on
when to trade 34
The OMS 36
Critical components 36
Non-critical components 37
Command and control 37
Services 38
Summary39
3
Understanding the Trading Exchange Dynamics
Architecting a trading
exchange for handling
orders at a large scale 42
History of trading exchanges 42
Understanding features of an exchange 43
Exchange architecture 44
General order book and
matching engine 46
Best price scenario 47
Partial fill scenario  48
No match scenario 49
Multiple orders with the same price 50
Summary  53
Part 2: How to Architect a High-Frequency
Trading System
4
HFT System Foundations – From Hardware to OS
Understanding HFT computers 58
CPUs, from multi-processor to multi-core59
Main memory or RAM 62
Shared memory 64
I/O devices 65
Using the OS for HFT systems 65
User space and kernel space 66
Process scheduling and CPU
resource management 67
Memory management 68
Paged memory and page tables 69
System calls 70
Threading71
Interruption management 72
The role of compilers 73
Executable file formats 74
Static versus dynamic linking 74
Summary75
Table of Contents vii
5
Networking in Motion
Understanding networking in
HFT systems 78
Learning about network
conceptual models 78
Network communications
between
systems in HFT 80
Comprehending how switches work 82
Important protocol concepts 89
Using Ethernet for HFT
communication 90
Using IPv4 as a network layer 90
UDP and TCP for the transport layer 92
Designing financial protocols
for HFT exchanges 94
FIX protocol 95
Interior networks versus
exterior networks 101
Understanding the
packet life cycle 102
Comprehending the packet life in the
send/receive (TX/RX) path  104
Software layer receiving the packet  105
Monitoring the network 105
Packet capture and analysis 106
Valuing time distribution 108
Time-synchronization services 109
Summary110
6
HFT Optimization – Architecture and Operating System
Performance mental model 112
Understanding context
switches  112
Types of context switches 113
Why are context switches good 114
Steps and operations involved in a
context
switch operation 115
Why are context switches bad for HFT? 116
Techniques to avoid or minimize
context switches 117
Building lock-free data
structures  118
When/why are locks needed
(non-HFT applications) 118
Types of synchronization mechanisms 119
Problems and inefficiencies
with using locks 119
Pre-fetching and
pre-allocating memory 127
Memory hierarchy 128
Pre-fetching based alternatives to
boost performance 131
Dynamic memory allocation 133
Pre-allocation-based alternatives to
dynamic memory allocation 134
Summary135
viii Table of Contents
7
HFT Optimization – Logging, Performance, and Networking
Comparing kernel space and
user space  138
What is kernel and user space? 139
Investigating performance – kernel
versus user space 140
Using kernel bypass 141
Understanding why kernel bypass is
the alternative 142
Presenting kernel bypass latencies 142
Learning about
memory-mapped files 143
Using cable fiber, hollow fiber,
and microwave technologies 146
Evolution from cable fiber to hollow
fiber to microwave 147
How hollow fiber works 147
How microwave works 148
Diving into logging and statistics
150
The need for logging in HFT 150
The need for online/live statistics
computation in HFT 150
Measuring performance 152
Motivation for measuring performance 152
Linux tools for measuring performance 153
Custom techniques for measuring
performance 156
Summary161
Part 3: Implementation of a High-Frequency
Trading System
8
C++ – The Quest for Microsecond Latency
C++ 14/17 memory model 166
What is a memory model? 166
The need for a memory model 167
The C++ 11 memory model and its rules168
C++ memory model principles 172
Removing runtime decisions 176
Motivation for removing runtime
decisions 177
Virtual functions 178
Performance penalties  179
Dynamic memory allocation 184
Runtime performance penalty 184
Using constexpr efficiently  186
Exceptions impeding
performance  186
Templates reducing the
runtime  188
What are templates? 188
Template specialization  189
Why use templates? 189
Table of Contents ix
Disadvantages of templates 191
Performance of templates  192
Standard Template Library (STL) 193
Static analysis 195
What is C++ static analysis? 195
The need for static analysis 195
Types of static analysis 196
Steps in static analysis 197
Benefits and drawbacks
of static analysis 197
Use case - Building an FX
high-frequency trading system 200
Summary 201
9
Java and JVM for Low-Latency Systems
Introducing the basics of Java 204
Reducing the impact
of the GC  207
What to do to keep GC events
low and fast 208
Warming up the JVM  213
Tiered compilation in JVM 214
Optimizing the JVM for better startup
performance 218
Measuring the performance of
a Java software 219
Why are Java microbenchmarks
difficult to create? 219
Real-time performance measures 220
Java threading  221
Using a thread pool/queue
with threads 222
High-performance
task queue  227
Queues 227
Circular buffer 227
LMAX disruptor 228
Logging and DB access 230
External or internal thread? 231
Summary 232
10
Python – Interpreted but Open to High Performance
Introducing Python 234
Making use of Python for analytics 234
Why is Python slow? 236
How do we use libraries in Python? 238
Python and C++ for HFT 239
Using C++ in Python 240
Using Python with C++ 240
Boost.Python library 241
Using ctypes/CFFI to accelerate
Python code 243
SWIG 244
Improving the speed
of Python code in HFT 246
Summary 248
x Table of Contents
11
High-Frequency FPGA and Crypto
Reducing latencies
with FPGA  250
Evolution of the fierce competition
of speed in HFT 250
Introduction to FPGA 251
Diving into FPGA trading systems 254
Advantages of FPGA trading systems 256
Disadvantages of FPGA
trading systems 257
Final words on FPGAs 258
Exploring HFT with
cryptocurrencies 259
What is crypto? 259
How do crypto transactions work? 259
What is a blockchain? 260
What is cryptocurrency mining?  260
Similarities between traditional asset
trading and cryptocurrency trading 260
Main differences between
traditional asset trading and
cryptocurrency trading 265
Trading with cryptocurrency exchange 267
HFT strategies in crypto 271
Building a high-frequency system
for crypto trading 272
How to build a trading system in
the cloud 275
Summary 280
Index
Other Books You May Enjoy
Preface
The world of trading markets is complex, but it can be made easier with technology. Sure,
you know how to code, but where do you start? What programming language do you use?
How do you solve the problem of latency? The Developing High-Frequency Trading Systems
book answers all these questions.
This practical guide will help you navigate the fast-paced world of algorithmic trading
and show you how to build a high-frequency trading system from complex technological
components supported by accurate data.
Starting with an introduction to high-frequency trading (HFT), exchanges, and the
critical components of a trading system, the book quickly moves on to the nitty-gritty
of optimizing hardware and your operating system (OS) for low-latency trading, such
as bypassing the kernel, memory management, and the danger of context switching.
Monitoring your system's performance is vital, so you'll also get up to speed with logging
and statistics.
As you move beyond the traditional high-frequency trading programming languages,
such as C++ and Java, you'll learn how to use Python to achieve high-performance levels.
And what book on trading would be complete without diving into cryptocurrency?
By the end of this book, you'll be ready to take on the markets with high-frequency
trading systems.
Who this book is for
This book is for software engineers, quantitative developers or researchers, and
DevOps engineers who want to understand the technical side of high-frequency
trading systems and the optimizations needed to achieve ultra-low latency systems.
Prior experience working with C++ and Java will help you grasp the topics covered in
this book more easily.
xii Preface
What this book covers
Chapter 1, Fundamentals of a High-Frequency Trading System, gives an overview of the
history of high-frequency trading. You will learn about the market participants, the
fundamental HFT requirements (low latency connectivity and infrastructure), the trading
time horizons in HFT versus non-HFT, and the holding periods/position management
(HFT versus Ultra HFT). We will also detail HFT-specific strategies to make money.
Chapter 2, The Critical Components of a Trading System, explains in-depth how a trading
system works. You will learn how the market data gets into the system, and the different
functionalities needed to handle the data and send an order to the exchange.
Chapter 3, Understanding the Trading Exchange Dynamics, introduces how trading
exchanges are a part of the microstructure of markets. We will start by giving the general
infrastructure of an exchange, and we will talk about how the matching engine works and
how the orders are matched and promoted to all the market participants.
Chapter 4, HFT System Foundations – From Hardware to OS, clarifies how the hardware
and OS work together. You will have a clear understanding of the functions of the software
interaction with the OS and the hardware. This chapter will go from the processor to
the trading system, explaining all the layers between including the OS, networking, OS
scheduler, and memory.
Chapter 5, Networking in Motion, expresses how networking benefits HFT. You will
have a clear understanding of the functions of the network stack and its use when
communicating between a trading system and an exchange.
Chapter 6, HFT Optimization – Architecture and Operating System, expounds on creating
a HFT system from a regular trading system. This section will cover many modern
techniques to achieve optimal low latency performance for HFT applications specifically.
We will talk about the OS features and its scheduler, and we will do a deep dive into the
kernel function of the OS.
Chapter 7, HFT Optimization – Logging, Performance, and Networking, covers a vital
part of trading systems: logging and networking. You will understand how logging
helps to monitor an HFT system, and we will learn how to make it efficient in a context
of HFT. Finally, we will cover how to use networking to optimize communication with
trading exchanges.
Chapter 8, C++ – The Quest for Microsecond Latency, defines the use of C++ in a
context of an ultra-low latency system by optimizing cache, memory, and code
execution. You will learn about modern C++ features and techniques to write ultra-low
latency code efficiently.
Preface xiii
Chapter 9, Java and JVM for Low-Latency Systems, details the use of Java in a context
of an ultra-low latency system by optimizing garbage collection, communication, and
data structure.
Chapter 10, Python – Interpreted but Open to High Performance, illustrates how to
use Python in an HFT system. This chapter explains how to create and use HFT
libraries in Python.
Chapter 11, High-Frequency FPGA and Crypto, depicts how to use field programmable
gate array (FPGA) to create an even faster HFT system. It will introduce building an HFT
system for crypto in the cloud.
To get the most out of this book
This book assumes that you are familiar with programming, hardware architecture,
and OS. Because this book will discuss the optimizations required to reduce the tick-to-
trade latency, it is essential to have the minimal knowledge of computer engineering.
Most of the HFT systems run with a Unix-based OS. We will recommend using a Linux
OS to apply your knowledge of this book.
This book is a reservoir of knowledge from many computer engineering and finance
domains. We recommend reading other Packt books such as the following:
• Linux Kernel Programming (https://www.packtpub.com/product/
networking-and-servers/9781789953435)
• Java Programming for Beginners (https://www.packtpub.com/product/
application-development/9781788296298)
• C++ High Performance (https://www.packtpub.com/product/
programming/9781839216541)
We also recommend reading books such as Compilers: Principles, Techniques, and Tools,
and Computer Architecture: A Quantitative Approach. These books will give you more
in-depth knowledge of the optimization we are using for HFT.
Download the color images
We also provide a PDF file that has color images of the screenshots and diagrams used
in this book. You can download it here: https://static.packt-cdn.com/
downloads/9781803242811_ColorImages.pdf.
xiv Preface
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names,
filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles.
Here is an example: It offers one producer to one consumer (OneToOneRingBuffer)
or many producers to one consumer (ManyToOneRingBuffer) solutions.
A block of code is set as follows:
/* Put header files here or function declarations like below */
extern int add_1(int n);
extern int add(int n, int m);
Any command-line input or output is written as follows:
 import math
 math.add_1(5)
6
Bold: Indicates a new term, an important word, or words that you see onscreen. For
instance, words in menus or dialog boxes appear in bold. Here is an example: The Load
Data component (annotation 1) will help get historical data.
Tips or Important Notes
Appear like this.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us
at customercare@packtpub.com and mention the book title in the subject of
your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you have found a mistake in this book, we would be grateful if you would
report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
Preface xv
Piracy: If you come across any illegal copies of our works in any form on the internet,
we would be grateful if you would provide us with the location address or website name.
Please contact us at copyright@packt.com with a link to the material.
If you are interested in becoming an author: If there is a topic that you have
expertise in and you are interested in either writing or contributing to a book,
please visit authors.packtpub.com.
Share Your Thoughts
Once you've read Developing High-Frequency Trading Systems, we'd love to hear your
thoughts! Please click here to go straight to the Amazon review page for this book and
share your feedback.
Your review is important to us and the tech community and will help us make sure we're
delivering excellent quality content.
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
By the end of this introduction, you will have a quick overview of the history of
high-frequency trading (HFT). You will know about the market participants, the
fundamental HFT requirements (low latency connectivity and infrastructure), the
trading time horizons in HFT versus non-HFT, and the holding periods/position
management (HFT versus ultra HFT). We will also talk about the places for HFT. This
book is not about the business of trading or HFT; it is about how to implement an HFT
system concretely by using Java, C++, and Python. You will know how a trading system
works and which trading strategies you can run.
This part comprises the following chapters:
• Chapter 1, Fundamentals of a High-Frequency Trading System
• Chapter 2, The Critical Components of a Trading System
• Chapter 3, Understanding the Trading Exchange Dynamics
Part 1:
Trading Strategies,
Trading Systems,
and Exchanges
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
1
Fundamentals of
a High-Frequency
Trading System
Welcome to Developing High-Frequency Trading Systems!
High-Frequency Trading (HFT) is a form of automated trading. For the last twenty years,
HFT has gained recognition in the media and in society. A book called Flash Boys: A Wall
Street Revolt, written by Michael Lewis in 2014, topped the sales on the New York Times
Best Seller list for three weeks. It relates to an investigation into the HFT industry and its
impact on the trading world. Scholars, the financial world, and the non-financial world
are fascinated by this form of trading. Meanwhile, this new era of trading has created a lot
of fear while giving more and more control to machines.
The goal of this book is to review what HFT is and how to build such a system from a
technical perspective. HFT is a multi-disciplinary matter involving thorough knowledge
of computer architecture, operating systems, networking, and programming. By the end
of this book, you will understand how to build a trading system from scratch by using the
most advanced technical choices for optimizing speed and scalability. We chose to divide
this book into three main parts.
4 Fundamentals of a High-Frequency Trading System
In the first part, we'll go through how HFT tactics function and what kind of trading we
may expect from HFT. Then we will go over the functions of an HFT system. We will
conclude this part with a description of how a trading exchange works.
In the second part of this book, we will explain the theory of operating systems and
hardware and the required knowledge to optimize a trading system, taking into account
the hardware and operating system features.
The final part will explain in detail how to use C++, Java, Python, and FPGA to create an
HFT system. We will also extend this knowledge to crypto trading, and we will review
how to build a trading system in the cloud.
In this chapter, we will talk about how we got into HFT. We will review what kind of
trading strategies work for HFT. We will explain in detail what makes HFT so different
from regular trading.
Our objective in this chapter is to cover the following topics:
• History of HFT
• What HFT is
• Who the participants are
• What trading strategies work in HFT
History of HFT
Let's discuss the history of exchanges and financial markets prior to 1930.
When we talk about HFT, it is difficult to give a precise date for when it started. We need
to come back to the primitive times when trade arose from human contact. Before the
invention of modern-day cash, ancient people relied heavily on trading to trade products
and services with one another in a gift economy. Long-distance trade extends back to
almost 150,000 years ago, according to Peter Watson. Year after year, with more people,
more goods, and more money, trading became one of the major activities of humankind.
It is obvious that making money implies more business. One of the parameters was speed.
If you make more transactions, you will make more money. Many stories describe the
ambition of traders to get technologies such as better transportation to make deals more
quickly or to get news more quickly to take advantage of other folks who do not have
access to these new technical means.
History of HFT 5
We did not have to wait for too long before seeing cases of unfair trade involving those
who have technical advantages over others. In 1790, a Georgia representative spoke to the
US House of Representatives to expose high-speed traders. Indeed, Congress was debating
the Secretary of the Treasury Alexander Hamilton's proposal that the US government
absorb the previous debts accrued by the states during the Revolution (Funding Act of
1790). Traders who had learned the decision immediately bought or rented rapid boats.
Their goal was to front-run messengers and buy the old debts since the passage of the Act
would increase the market value. During the twentieth century, the idea of speed trading
or HFT appeared.
The post-1930s era
Trading is the exchange of items for other items. It can be financial products, services,
cash, digital assets, and more. One of the goals of trading is to make a profit from these
transactions. The number of transactions will be correlated with the quantity of money
generated by the exchange of assets. When we manage to increase the ratio between
the number of transactions and the time, we can increase the profitability over time.
Therefore, being capable of increasing the number of transactions is critical. Trading
actors understood very quickly that they needed to shorten the trading time and started
gathering in some specific places. They used to place their orders in these locations, which
we call today the trading exchange (or trading floor). Major events participated in the
expansion of fast-speed automated trading:
• 1969: Instinet was one of the first automated system infrastructures. It speeded up
the adhesion of high-speed transactions.
• 1971: The National Association of Securities Dealers Automated
Quotations (NASDAQ) was created in 1971 with electronic transactions.
It was the world's first electronic stock market. Initially, it only used to
send quotations.
• 1996: Island ECN was the pioneering electronic communication network for
US equities trading, while Archipelago facilitated electronic trading on the
US trading exchange by creating Archipelago Exchange (ArcaEx).
• 2000s: 10% of transactions are HFT transactions.
The financial sector gathered more and more technologists in the early 2000s. By getting
this technological intake, the sector started evolving sharply. Automation, throughput,
performance, and latency became words that were well known by trading firms. The
HFT transactions reached more than 10% of the market. By 2009, 2% of trading firms
accounted for 75% of the equity volume. Nowadays, only a few firms remain in HFT, such
as Virtu, Jump, Citadel, IMC, and Tower.
6 Fundamentals of a High-Frequency Trading System
The modern era
The post-1930s era focused on transparency and regulation in the equities markets
(and the pit in commodities markets). The modern era gives prominence to electronic
trading and improves transparency. In 2000, the US Securities and Exchange
Commission (SEC) proposed the Central Limit Order Book (CLOB). The CLOB is a
transparent system matching orders between participants. Many more exchanges (such
as Island and Arca) came to the trading scene. The number of trading firms, hedge funds,
and electronic players kept increasing. They created their own technology stack to trade
more quickly and stay competitive. After 10 years, only a few trading firms managed to
remain competitive, becoming the 2% of the trading firms accountable for 75% of all
equity volume.
The savoir faire for competing in HFT requires heavy investment: money, people,
and time. It is a marriage of low-level system expertise and quants, as well as smart
money (investors are more and more technology savvy). Engineers capable of creating
performant code for designing ultra-low latency systems are very expensive. Only a
few engineers had these skills. The performance for such a system required specialized
hardware. Routers, servers, and network devices are also expensive. Therefore, the
experience and the barrier of entry will prevent a lot of new incomers and will limit the
competition. On top of the five firms we talked about previously, there are boutique shops
that trade HFT strategies using an edge they found either in the market structure or some
technical fact that other firms are not exploiting. The giant HFT firms are the companies
responsible for moving most of the equity volume. Nowadays, HFT is estimated to
account for at least 50% of the US equity (shares) trading volume. The market share of
HFT has declined, as has profitability, since the peak year (2009).
After 2015, the growth of digital currencies cleared the way for new opportunities for
high-frequency traders. Today, we can see an extensive growth of HFT strategies working
with well-known crypto exchanges such as Coinbase, Binance, and hundreds of other
crypto exchanges.
This modern era has anchored technology and automated trading for good. Trading
models are data driven and model driven. The market data business definitely became a
major part of trading. Exchanges and trading firms started making money by generating
or collecting market data, the raw material of any algorithm trader.
Why have HFT? 7
Why have HFT?
HFT aims at getting many transactions per second. In this way, companies can react more
quickly to a changing market. They can take advantage of more opportunities than they
would have without this speed. Additionally, large institutions benefit from HFT by gaining
a tiny but considerable advantage in exchange for delivering massive volumes of liquidity to
markets. They place millions of orders that their systems are capable of placing. They help
the market and, as a result, are able to boost earnings in their profitable trades and receive
better spreads. Since the return is very low, they must complete many trades to benefit. On
top of this revenue, they will gain rebates or discounted transaction fees, which are given by
trading venues to make their markets more attractive to HFT firms.
What makes HFT so different from
regular trading?
HFT trading should have the shortest feasible data latency (time delays) and the highest
level of automation possible. HFT relates to algorithmic trading and automated trading.
As a result, participants choose to trade in markets that have a high level of automation
and integration in their trading platforms. Firms utilize computers programmed with
precise algorithms to find trading opportunities and execute orders in algorithmic
trading. To increase the speed of transactions, high-frequency traders use automated
trading and fast connections (and cancellations or modifications). This is possible because
of the technology that trading firms have in place but also because of the exchange
technologies. The following exchanges have invested hundreds of millions of dollars in
HFT technologies:
• NASDAQ, New York City, is the first electronic stock exchange in the world.
All of its equities are traded over a computerized network. It revolutionized the
financial markets in 1971 by removing the requirement for a physical trading floor
and in-person trading. It is the world's second-biggest stock exchange by market
capitalization. Half of NASDAQ's composite offering was made up of technology
firms. With less than 20% of the overall composite, the consumer sector came in
second, followed by healthcare.
• New York Stock Exchange (NYSE), New York City, is the world's largest exchange
for the equity market. In 2013, Intercontinental Exchange, Inc. (ICE) bought NYSE.
8 Fundamentals of a High-Frequency Trading System
• London Stock Exchange (LSE), London, UK, is the largest stock exchange in
Europe and the principal stock exchange in the United Kingdom mainly with regard
to trading in company stocks and bonds. It was created about 300 years ago.
• The Tokyo Stock Exchange (TSE), is Japan's largest stock exchange, with its
headquarters in Tokyo. It was founded in 1878. The exchange has more than 3,500
listed businesses. The TSE, which is operated by the Japan Exchange Group, is home
to the world's largest and most well-known Japanese corporations, including Toyota,
Honda, and Mitsubishi.
• The Chicago Mercantile Market (CME), sometimes known as the Chicago Merc,
is a regulated futures and options exchange in Chicago, Illinois. Agriculture, energy,
stock indices, foreign exchange, interest rates, metals, real estate, and weather are
among the industries in which the CME trades futures and, in most cases, options.
• Direct Edge, Jersey City. Its market share rapidly rose to tenth in the US stock
market, and it typically transacted more than two billion shares daily. Better
Alternative Trading System (BATS) Global Markets was a US-based exchange that
traded a variety of assets, including stocks, options, and foreign exchange. CBOE
Holdings purchased it in 2017 after it was created in 2005. BATS Global Market was
one of the largest US exchanges prior to being bought, and it was well known for its
services to broker-dealers, retail, and institutional investors.
• The CBOE Options Exchange, which was founded in 1973, is the world's largest
options exchange, with contracts centered on individual stocks, indexes, and
interest rates.
All the preceding exchanges are controlled on several levels:
• Trading limitations
• Trading system transparency (information shared among market participants on
the specificities of the architecture, as well as the way of handling orders)
• The type of accepted financial instruments
• Constraints by security issuers
What makes HFT so different from regular trading? 9
For most regulated exchanges, the order size is an issue. Large trades have an important
effect on the market (they can create market impact). Traders use Alternate Trading
Systems (ATS), which have much less regulation in comparison to traditional exchanges
(they don't have to be transparent). Dark pools are the most common sorts of ATS. The
USA presently has around 30 dark pools, which represent a quarter of the US consolidated
trading volume.
Dark pools are beneficial to HFTs because they can handle the speed and the level of
automation demands while having reduced fees. This is not the case for any other type of
trading, which makes HFT different from regular trading. In the following section, let's
learn more about dark pools.
Effect of dark pools
For financial security, buy and sell orders are not displayed in dark pools (price and
volume). Dark pools, in other words, are both opaque and anonymous since the order
book is not advertised. Because it is not possible to see the size of the orders in this type of
trading exchange, investors who place huge orders do not impact markets. Since the other
participants do not see the size of the orders, the dark pools execute these large orders at a
fixed price. It reduces the negative slippage given by trading exchanges.
Dark pools are obliged to notify deals once they have occurred, notwithstanding the lack
of pre-trade transparency.
HFTs and dark pools have a complicated interaction. Dark pools rose in popularity
partially as a result of investors seeking protection from HFTs' fraudulent activities on
public exchanges, and HFTs finding it impossible to know the large orders in dark pools
through pinging. Dark pools introduced a lack of transparency in the markets that
allowed ill-equipped players (that is, on the sell side) to keep up with business practices
that didn't match the state of the art at the time. And, of course, Haim Bodek wrote two
books (The Problem of HFT and The Market Structure Crisis) about finding unordinary
order types in dark pools.
On the other hand, a few dark pools encourage HFT traders to trade on their exchange.
HFT strategies increase liquidity and the likelihood of having orders filled. Dark pools
help HFTs to meet their speed and automation demands while still having reduced
expenses. HFTs are responsible for the decrease in order sizes in dark pools. The dark
pools have been hit by pinging trading strategies locating hidden large orders.
10 Fundamentals of a High-Frequency Trading System
As a result, if these HFT tactics are present, the benefits of dark pools may be harmed. For
example, in 2014, the Attorney General of New York filed a lawsuit against Barclays for
its dark pool operations, alleging that it misrepresented the volume of Barclays's activity
in dark pools. In 2016, Barclays paid a $35 million fine to the SEC and $70 million to the
State of New York.
Dark pools can apply certain constraints to prevent HFTs from engaging in predatory
behavior. The goal is to reduce pinging trading strategies. In 2017, Petrescu and Wedow
imposed a minimum order size to minimize this type of strategy.
We could spend more time discussing the pros and cons of the impact of HFTs on dark
pools, but we end up saying that the advantages of having more liquidity and faster
execution are beneficial enough to have some dark pools being in favor of HFTs. It is fair
for investors as long as they have a thorough understanding of how trading venues work
so they can make educated judgments.
We have talked about the location of the major trading exchanges. Now we will introduce
the HFT participants in the next section.
Who trades HFT?
The answer could be summarized in one word: everyone. From the buy side to the sell
side, ECNs, and even the inter-dealer and inter-broker-dealer markets, they all use HFT.
HFT is dominated by proprietary trading businesses and covers a wide range of products,
including stocks, derivatives, index funds, Exchange-Traded Funds (ETFs), currencies,
and fixed-income instruments. Proprietary trading businesses accounted for half of the
current HFT players, multi-service broker-dealer proprietary trading desks accounted for
less than half, and hedge funds accounted for the rest. Proprietary trading businesses such
as KCG Holdings (created by the merger of Getco and Knight Capital) and the trading
desks of major banks are among the major players in the field. There are some new types of
venues (such as Dealerweb's OTR Exchange and IEX) that are looking to provide venues
where dealers on the sell side feel safe to execute trades and HFTs are providing liquidity.
It is worth saying that HFTs have become major players in the market. They are also
capturing retail flow. Citadel is controlling a large part of the retail flow.
What do I need to start an HFT?
Participants in HFTs must have the following:
• Fast computers: HFT focuses on single-core throughput in most cases, and
parallelism is not used by the strategies necessarily.
What do I need to start an HFT? 11
• Exchange proximity: While some countries restrict the use of shared places to have
trading systems and exchange, in the US, we use co-location. This is a place where
all the HFTs participants have their production servers. They will pay to have their
computers co-located with an exchange's computer servers in the same data centers
in order to decrease latency and shorten the time it takes to complete a deal – even
by microseconds. The cables linking trading systems from all market participants
with the server are the same length to guarantee that nobody has an advantage over
another market participant. The SEC has issued a wide request for feedback on
co-location fees, as well as other concerns impacting the equity market structure. To
ensure fairness among market participants, it is important that co-location fees are
reasonably priced. The SEC invites the co-locations to report their fees.
• Low latency: In HFT, latency is the time it takes for data to reach a trader's
computer, for the trader to make an order in response to the data, and for the order
to be accepted by an exchange. The order may enter the market alongside many
other orders issued by other traders at the most profitable time. There is a danger
of competing against a large number of other people in this circumstance. The
order may not be as profitable as it may have been in this scenario. High-frequency
traders are able to make orders at unfathomably quick speeds because of technology
advertised as low-latency or ultra-low-latency. It is important to use gear designed to
reduce the latency of shuffling data from one place to another.
• Computer algorithms, which are at the heart of AT and HFT, and real-time data
feeds, which could damage earnings.
In the previous sections, we learned where high-frequency traders make business. We also
talked about the technological prerequisites to trade faster. Let's now focus on what HFT
is in depth.
What are HFT strategies?
HFT strategies are a subset of algorithmic trading strategies. They are executed in
the order of the microseconds (and sometimes nanoseconds). The strategies must be
aware of this time limitation to be efficient HFT strategies. They deploy cutting-edge
technology advancements to obtain information faster than the competition. The main
game of this type of strategy is the tick-to-trade, which is the response time to send an
order responding to incoming market data. As we will explain in the next chapter, it is
important to host trading strategies on cutting-edge machines, and they must also run in
a co-location.
We will be defining the domain of applications and some vocab to have when talking
about HFT strategies.
12 Fundamentals of a High-Frequency Trading System
Asset classes
HFT strategies can be applied to any asset classes, such as stocks, futures, bonds, options,
and FX. We also have cryptocurrencies being traded using HFT strategies, even if the
definition of speed is different (because of the settlement time).
Liquidity
The desire of players to interact with regard to a certain asset is known as liquidity.
We define depth as the number of price levels for a given asset. We will say that a book is
deep when there are many levels (layers) for a given asset. We will define a book as big or
broad if the volume per layer is high. If a book is deep or large, we will define a liquidity
of a given asset liquid. The consequence of this statement is that it will be easier for a
trader to buy or sell this asset whenever they want to. As a result, trading exchanges with
a lot of liquidities are wanted by traders. Crypto trading exchanges have difficulty finding
liquidities at the moment.
Tick-by-tick data and data distribution
HFT generates orders every microsecond. Since there are a lot of participants, it is likely
to have huge amounts of data. Storage of this data will be key when we study HFT data to
create models for trading strategies.
Thousands of ticks (security price changes from one order to another) are generated per
trading day on liquid marketplaces, which make up high-frequency data. This material is
randomly spaced in time by its very nature. HFT data exhibits fat tail distributions. That
means that the trading strategies need to take into account that we can have big losses.
They distribution of the market data can be grouped into two categories:
• Volatility clustering: Large changes follow large changes whether in terms of signs or
numbers, while minor changes follow smaller changes.
• Long-range dependency (long memory) refers to the pace at which statistical
dependence between two sites decays as the time interval or spatial distance
between them increases.
Liquidity rebates
To support the provision of stock liquidity, most exchanges have used a maker-taker
model. In this arrangement, investors and traders who place limit orders often earn a
modest rebate from the exchange when their orders are executed since they are considered
to have contributed to stock liquidity, or makers.
What do I need to start an HFT? 13
Those who place market orders, on the other hand, are considered takers of liquidity and
are charged a small fee by the exchange. While the rebates are normally fractions of a
penny per share, over the millions of shares exchanged daily by high-frequency traders,
they may add up to large amounts. Many HFT businesses use trading techniques that are
geared to take advantage of as many liquidity rebates as feasible.
Matching engine
The software program that forms the heart of an exchange's trading system and matches
buy and sell orders on a continuous basis, a service traditionally done by trading floor
professionals, called the matching engine is critical for guaranteeing the efficient
operation of an exchange since it matches buyers and sellers for all stocks. The matching
engine is housed on the exchange's computers, and it is the main reason why HFT
businesses strive to be as near to the exchange servers as possible. We will learn about it in
Chapter 3, Understanding the Trading Exchange Dynamics.
Market making
Before going into details on what market making is, we need to explain the difference
between market takers and market makers.
Market taker/maker
Figure 1.1 represents the limit order book on an exchange. When a trading strategy places
an order close to the top of the book (the layer representing the best price for bid and
for ask), we say that this order is an aggressive order. It means that this order is likely to
be matched with another order. If the order is executed, it means that liquidity has been
removed from the market; it is a market taker. We will say that a trader crosses the spread
when they place a buy order at the price of the ask on the top of the book. If the order is
less aggressive (or passive), this order will not remove liquidities from the market; it is a
market maker.
Figure 1.1 – Order book – passive/aggressive order
14 Fundamentals of a High-Frequency Trading System
Let's look at the market-making strategies.
Market-making strategies
A trading corporation can provide market-making as a service on an exchange. Over time,
a market maker assists in the matching of buyers and sellers. Rather than purchasing or
selling securities based on their underlying assets, market makers maintain a continual
offer to buy and sell securities and profit from the spread, which is the difference between
the two offers.
To reduce the risk of keeping stocks for extended periods of time, every purchase should
be matched with a sale and every sell should be matched with a buy. If a stock is trading at
$100, a market maker can keep a buy offer at $99.50 and a sell offer at $100.50. If they are
successful in finding both a buyer and a seller, it allows those who want to sell right now
to do so even if no one else wants to purchase, and vice versa.
Market makers, in other words, supply liquidity—they make trading simpler. For the most
traded stocks, this technique is not important; however, for smaller firms (less traded than
the big ones), it can be critical to increase the trading volume to facilitate trading. Market
making is one approach that many HFT businesses use. They out-compete everyone else
by changing their quotations quickly and reducing the spread even further: they're willing
to make less money each time since their market-making business can readily grow to
massive quantities. However, an HFT firm's technology can be used for other purposes,
such as arbitrage (making money on minor discrepancies between linked securities)
or execution (breaking up huge institutions' trades to minimize market effect). I won't
go into much more detail because the point is that HFT is capable of more than simply
market-making. The only thing that matters is speed.
Market making can be done by the analysis of the order flow:
• A large volume of buy and sell can drive the market price of buying and selling on
the basis of momentum.
• The flow of liquids (how big are the buy and sell orders: small, medium, or big).
• Exhaustion of momentum (when the order flow is drying off it may signal
a price reversal).
Market-making is the most widely used trading strategy for high-frequency traders. We
will talk about the other HFT strategies in the next sections.
What do I need to start an HFT? 15
Scalping
Scalping is a trading method that focuses on benefitting from tiny price movements and
reselling for a quick profit. Scalping is a phrase used in day trading to describe a technique
that focuses on generating large volumes from tiny profits. Scalping necessitates a tight
exit plan since a single major loss might wipe out all of the modest wins the trader has
worked so hard to achieve. For this technique to work, you'll need the necessary tools,
such as a live feed, a direct-access broker, and the endurance to conduct a lot of trades.
The concept behind scalping is that most stocks will finish the first stage of a trend. But
it's unclear where things will go from there. Some stocks stop rising after that early stage,
while others continue to rise. The goal is to benefit from as many minor transactions as
possible. The let your gains run mentality, on the other hand, aims to maximize good
trading results by expanding the size of winning deals. By increasing the number of
winners while compromising on the magnitude of the gains, this technique accomplishes
outcomes. It's very uncommon for a trader with a long time period to produce good
profits while winning just 50% of their transactions, or even less – the difference is that the
wins are far larger than the losses.
Statistical arbitrage
The Efficient Market Hypothesis (EMH) claims financial markets are informationally
efficient, which means that the prices of traded assets are accurate, and at any one moment
represent all known information. Based on this hypothesis, the market should not
fluctuate if there is not any fundamental news. However, this is not the case, and we can
explain that with liquidity.
Throughout the day, many huge institutional trades have little to do with information
and everything to do with liquidity. Investors who believe they are overexposed will
aggressively hedge or sell their positions, impacting the price. Liquidity seekers are
frequently ready to pay a premium to exit their positions, resulting in a profit for liquidity
providers. Although this capacity to benefit from knowledge appears to violate efficient
market theory, statistical arbitrage is based on it.
Statistical arbitrage seeks to profit from the correlation of price and liquidity by gaining
from the perceived mispricing of assets based on the assets' anticipated value given by a
statistical model.
16 Fundamentals of a High-Frequency Trading System
Short-term price discrepancies in the same security sold on separate venues, or short-
term price differences in related securities, are used in statistical arbitrage, often known
as stat arb. Statistical arbitrage is based on the assumption that price differences in
securities markets exist but go away quickly. Because the time period during which a price
difference occurs might be as short as a fraction of a second, algorithmic trading is well
suited to statistical arbitrage.
When trading the same security in several venues, for example, an algorithm tracks
all of the locations where the security is exchanged. When a price difference arises, the
algorithm buys in the lower market and sells in the higher market, resulting in a profit.
Because the window of opportunity for such differences is small (less than 1 millisecond),
algorithmic trading is ideally suited to this form of trade.
Statistical arbitrage becomes more challenging when investing in linked securities. An
index and a single stock within that index, or a single stock and other stocks in the
same sector, are examples of related securities. In linked securities, a statistical arbitrage
approach entails gathering a large amount of historical data and estimating the usual
connection between the two markets. The algorithm makes a buy or a sell whenever there
is a variation from the norm.
Latency arbitrage
Modern equities markets are complicated, requiring highly technical systems to manage
vast volumes of data. Because of its intricacy, data is invariably processed at varying
speeds. Latency arbitrage takes advantage of market players' differing speeds. Latency
arbitrage aims to take advantage of high-frequency traders' greater speed by leveraging
high-speed fiber optics, superior bandwidth, co-located servers, and direct-price feeds
from exchanges, among other things, to place trades ahead of other market players.
The hypothesis behind latency arbitrage is that in the US, the aggregated feed that
determines the National Best Bid and Offer (NBBO) of all US stock exchanges is slower
than the direct data feeds from stock exchanges available to high-frequency traders.
An HFT program's algorithm can read transaction data more quickly than many other
market players, seeing prices a fraction of a second ahead of the Securities Information
Processor (SIP) feed, which is the consolidated US stock exchange price feed, thanks
to its superior speed. This essentially provides information to the HFT software before
it reaches the official market (the SIP feed), allowing high-frequency traders to observe
where prices are heading ahead of other market players.
What do I need to start an HFT? 17
Impact of news
Information is at the heart of all trading, and it is used to make financial decisions. The
utilization of news data by algorithmic trading systems to generate trading choices is
referred to as information-driven strategies.
Algorithms have been developed to read and analyze news reports from major news
organizations, as well as social media. Any news that has the potential to alter market
prices causes the algorithm to purchase or sell.
High-frequency traders have gotten so accustomed to using information-driven methods
that certain news agencies now package their press releases in a way that makes it simple
for computers to analyze them. They employ predetermined keywords to characterize a
favorable or bad occurrence, for example, so that an algorithm can act on keywords in a
news release. Prior to their planned publication, news providers also place news reports
on servers in crucial geographical regions (such as major financial centers). This reduces
the amount of time it takes for data to move from one location to another. For this sort of
service, news service providers charge an additional fee.
As seen by the hacking of the Associated Press Twitter feed, the use of social media for
information-driven initiatives is growing. In 2013, a hacker tweeted that a bomb had gone
off in the White House, injuring the president, causing an instantaneous plunge in equities
markets throughout the world as algorithms analyzed the bad news from a trustworthy
source and began selling in the market.
Next, let's learn about the momentum ignition trading technique.
Momentum ignition
You have the chance to trade financially if an order you send into the market may cause
a price change and you know it can. The goal of momentum-ignition trading techniques
is to achieve this. The objective is to get other algorithms and traders to start trading in
a stock, causing a price change. In essence, a momentum ignition approach attempts to
deceive other market players into believing that a large price movement is going to occur,
causing them to trade. As a result, the price movement becomes a self-fulfilling prophecy:
traders believe a price movement will occur, and their activities cause one to occur.
18 Fundamentals of a High-Frequency Trading System
Sending enormous volumes of orders into the order book and then canceling them is a
momentum-ignition approach. This creates the illusion of a huge shift in volume in the
stock, which might prompt other traders to place orders, resulting in the start of a short-
term price trend. Before attempting to ignite the market movement, the momentum
ignition approach includes executing the real targeted trading position. This means
that a deal is completed initially that does not significantly influence the market. This
permits a trader using the momentum-ignition approach to enter the market before the
price movement is initiated. The momentum ignition is set after the deal is completed
by submitting a flurry of orders and canceling them in the hopes that other traders will
follow suit and move the price.
The trader using the momentum-ignition technique then quits their initial position at a
profit as the price begins to move.
Momentum-ignition methods require the use of specific order types, and traders may only
utilize algorithms that can send and cancel huge numbers of orders in a short amount of
time to execute them.
Rebate strategies
Market order traders must pay a fee to the exchange, whereas the limit order is
reimbursed with rebates when they add liquidities. As a result, traders, particularly those
engaged in HFT, submit limit orders to build markets, which in turn generates liquidity
on the exchange. It is undoubtedly appealing to traders who place a large number of limit
orders due to the pricing scheme's lower risk for the limit order.
There is also a charge structure called trader-maker pricing that is the polar opposite of
market-taker pricing. In certain markets, it entails giving rebates to market order traders
and collecting fees from limit order traders.
Pinging
Pinging is a strategy for learning about huge orders in trading exchanges and dark pools
by placing tiny marketable orders (typically for 100 shares).
To lessen the market effect of large orders, buy-side businesses utilize this trading
technique to split large orders into many small orders. This algorithm feeds these orders
slowly into the exchange. In order to detect the presence of such large orders, HFT
companies arrange bids and offers in 100-share lots for each listed stock.
What do I need to start an HFT? 19
These ping trades will alert HFT participants to the existence of a large order placed by
the buy-side. HFTs will use this information to ensure risk-free profit from the buy-side.
Some significant market participants have compared pinging to baiting because its main
objective is to entice institutions with huge orders to expose their hand.
Illegal activities
The SEC, the Federal Bureau of Investigation (FBI), and other regulators have launched
crackdowns on alleged HFT violations in recent years. The following sections are
examples of possible offenses.
Front-running
Placing an order based on information that has not been publicly released is called
front-running. This technique has been outlawed by SEC and the Financial Industry
Regulatory Authority (FINRA). Some have used the term front-running to describe
a technique in which HFT firms utilize algorithmic trading technology to identify a
large number of new orders for a given instrument. Before the large number of orders
comes to the market, we place orders to benefit from this incoming large quantity. HFT
corporations can earn almost instantly after purchasing assets by selling them to the
original investors. Even if this way of trading is legal, regulators are concerned and may
need to control this behavior moving forward.
Spoofing
Spoofing is not a legal trading strategy. It consists of a spoofing strategy sending orders
that are not intended to be executed, just to have the other market participants react to
these orders. They will probably send orders to get to this price level. Meanwhile, the
initial orders are canceled and the spoofer takes advantage of the other orders remaining
in the market.
The Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 specifically
targeted the practice, and even before that, FINRA regulations barred orders whose goal
is to mislead the market. The first criminal spoofing case disclosed by legislators in 2014
related to a Chicago trader accused of faking futures markets.
20 Fundamentals of a High-Frequency Trading System
Layering
Layering is the same as spoofing except that the orders are placed at different price levels
to give the appearance that there is a lot of interest in a certain security. The outcome
of this strategy is the same as with regular spoofing. Because of the rapid advancement
of technology, massive market manipulation may take place in fractions of a second.
Layering, like generic spoofing, is typically illegal and forbidden under FINRA rules.
Even if these strategies are now outlawed, we need to keep in mind that some exchanges
are less or not regulated. We will see in Chapter 11, High Frequency FPGA and Crypto,
about cryptocurrencies that these strategies can still work.
Summary
In this chapter, we reviewed the origins of HFT. We went through what makes HFT so
special in comparison to regular trading. We also layered the different types of strategies
that any HFT trading system will be able to support. We talked about the history of
trading systems. Our goal in this chapter was to give you a good understanding of what
HFT is and what trading strategies we can use.
In the next chapter, we will talk about the main functionalities of a trading system. We will
describe how to build a trading system.
2
The Critical
Components of a
Trading System
In the previous chapter, we learned how to create high-frequency trading (HFT)
strategies. In this chapter, we are going to study how to convert these strategies into
real-time software that will connect to an exchange to actually apply the theory that
you've previously learned. We will describe the functioning of a trading system capable
of trading assets.
In this chapter we will cover the following topics:
• Understanding the trading system
• Making a trading system trade with exchanges
• Order book management
• Strategy making decisions on when to trade
By the end of this chapter, you will be capable of designing a trading system, connecting
a trading system to an exchange, and building a limit order book.
22 The Critical Components of a Trading System
Understanding the trading system
Designing a trading system for HFT trading requires much more than knowledge of
programming and trading. The following chapters of this book will describe these parts
in depth, which will give you an edge in designing an HFT system. In this section, we are
going to talk about the fundamentals of trading system design. One of the most critical
parts of designing a system is having a detailed description of the requirements. The
goal of a trading system is to support your trading ideas. Any trading strategies start by
getting data and end up with making a decision based on this data. A trading system will
oversee collecting market data (that is, price updates) and sending orders to the exchange.
Additionally, it will collect answers from the exchange containing information on the
orders. These market updates could be representing any state of the orders: canceled,
rejected, filled, or partially filled. It will also compute metrics measuring the performance
of your portfolio (such as profit and loss, risk metrics, or information about the different
processes of the trading system).
When deciding whether to create this type of software, we need to keep the following
points in mind:
• Asset class: Knowing which asset class will be employed in the trading system will
change the data structure of this software. Each asset class is unique and has its own
set of characteristics. Building a trading system for US equities won't be the same as
building a system for foreign exchange (FX). Stocks in the United States are mostly
traded on two exchanges, the New York Stock Exchange (NYSE) and NASDAQ.
These two exchanges have roughly 7,000 firms (symbols) listed. FX contains six
main currency pairs, six minor currency pairs, and six exotic currency pairs, as
opposed to stocks. We can add additional currency pairs, but no more than 100 will
be available. Unlike the US equity market, where we can have two main exchanges,
in the FX market, there will be hundreds of exchanges. The number of symbols and
the number of exchanges will change the architecture of a trading system.
Understanding the trading system 23
• Trading strategy type (high frequency, long-term position): The software
architecture will be influenced by the trading strategy type. HFT tactics necessitate
transmitting orders in a very short period of time. In the case of US stocks, a
standard trading system will decide to send an order in microseconds. A Chicago
Mercantile Exchange (CME) trading system has a latency in the order of
nanoseconds. Based on this finding, technology will play a significant role in the
software design process. If we only consider the programming language, Python
is not well suited to speed, and we would rather choose C++ or Java. If we wish to
take a long-term position, such as one that lasts many days, the speed with which
a trader may obtain liquidity faster than others is irrelevant.
• Number of users (or trading techniques): As the number of traders grows, so does
the variety of trading tactics. This indicates that the number of orders will increase.
Before submitting an order to an exchange, we must ensure that the orders we
are about to send are valid; we must ensure that the overall position for a specific
instrument has not been reached.
Trading strategies are being moderated by an increasing number of rules in the trading
sector. We shall test the compliance of the orders that we wish to send in order to ensure
that our trading strategy complies with the regulations. All of these tests will increase the
amount of time it takes to calculate. If we have a lot of orders, we'll have to conduct all of
these verifications in sequence for one instrument. If the program isn't fast enough, the
orders will take longer to process. The more users you have, the more scalable the trading
system must be.
These variables change the way you think about the trading system you're going to create.
Let's discuss the design of a simple trading system in the following section.
24 The Critical Components of a Trading System
Trading system architecture
The following schema represents the trading system architecture. On the left part of
this diagram, we can see the venues. Venues are a more generic term for any platform
matching buy and sell orders for the securities and/or derivatives of multiple parties.
In other words, a venue can be a trading exchange, an ECN, an aggregator, or a bank.
Trading systems communicate with venues to collect price updates from all the
participants and send orders. To do so, a trading system needs a piece of software called
Gateways, which will ensure the communication between the trading system and the
venues. The Book Builder will build the limit order book from the data collected from
Gateways. Finally, the Strategy will send the order to the venues through the Order
Manager. The Order Manager is responsible for collecting all the orders coming from
the strategies of the system and keeps track of the life cycle of the orders. All of these
components are a part of the critical path of sending orders to the market.
Figure 2.1 – Trading system architecture design
Additionally, we observe other less critical services, such as command and control, in
charge of starting the components of the system. The viewers are critical in algorithmic
trading because they will give you a status of all the components of the system, the orders
and trades, and the metrics you consider important to monitor your trading strategies.
Algorithmic trading automates trading. Therefore, it is important to keep track of the
health of your trading system and trading strategies. It is particularly important to
understand here that in HFT trading, a few microseconds can end up in colossal losses.
Having viewers and user interfaces capable of efficiently reporting alerts is critical.
Understanding the trading system 25
In the next section, we will talk more in-depth about the critical components of
a trading system.
Gateways connecting to trading exchanges
Gateways are the components of a trading system communicating with the exchange and
the trading system. They are essential because they are the greediest in terms of execution
time. By design, they must get data from the network and provide this data to the rest
of the system. This operation is demanding in terms of system resources and operating
system. Price updates are collected by a trading system, which then transmits orders on
your behalf. To do so, you must first code all the procedures you would do if you were
trading without a trading system. If you want to make money by purchasing low and
selling high, you must first decide what product you will trade. You should get the order
from the other merchants after you have chosen these products.
The other traders will inform you of their willingness to trade a financial asset by
identifying the size, price, and quantity. You can pick the trader with whom you will
negotiate a transaction once you have received enough orders for the product you wish to
trade. The price of this item will influence your selection. If you plan to resell this item in
the future, you will need to acquire it for a cheap price. When you reach an agreement on
a price, tell the other trader that you want to buy at the listed price. You will then own this
product when the transaction is completed.
Data collection
Gateways collect price updates from the trading venues you have chosen (exchanges,
ECNs, and dark pools). This component (shown as a Gateway in the following figure) is
one of the most important in the trading system. This component's job is to get the book
for the instruments from the exchange into the trading system. This component will be
connected to the network and will be able to communicate with it via exchanges receiving
and sending streams.
26 The Critical Components of a Trading System
The position of the trade system's gates is depicted in the following diagram. They are the
trading system's inputs and outputs:.
Figure 2.2 – Gateways in charge of collecting price updates and sending orders
The following points are depicted in the preceding diagram:
• Traders, exchanges, ECNs, and dark pools are represented by the venues.
• Different protocols might be used to connect the venues (they are represented
using arrows).
• Wires, wireless networks, the internet, microwaves, and fibers are all options to
transfer data. In terms of speed, data loss, and bandwidth, each of these network
media has its own set of characteristics.
• The arrows for price updates and orders are bidirectional because we can have data
sent to/received from the venues.
• To begin receiving pricing updates, the gateway will establish a network connection
with the venue, verify itself, and subscribe to a certain instrument (we will explain
this part in the next section).
Making a trading system trade with exchanges 27
• The order-processing gateway also receives and sends communications. When an
order is placed, it is forwarded to the venue over the network.
• An acknowledgment of this order will be sent if the venue receives it. A message
will be issued to the trading system after this order meets a matching order. If the
venue does not receive an order, the acknowledgment will not be sent. It is up to the
trading system to declare that an order has timed out. In this situation, a trader will
need to intervene and check the problem that occurred in the system.
Making a trading system trade with exchanges
A trading system contains a number of functional components that are responsible for
trading and risk management, as well as monitoring the trading process on one or more
exchanges. A trading strategy becomes a part of the trading system once it is coded. As
input, you'll need price data, and as output, your orders. This will give out trading signals.
We'll need gateways to finish this flow because they're the most important components.
The functional components of a trading system, the gateway's interface, and the trading
system's interaction with the outside world are depicted in the following diagram:
Figure 2.3 – The functional components of a trading system
The gateways gather and send orders based on pricing and market reactions. Their
primary function is to establish a link and translate data received from the outside world
into the data structure needed by the trading system.
28 The Critical Components of a Trading System
The following points are depicted in the preceding diagram:
• This trading plan will be on your machine when you apply your trading strategy.
The trade will take place on a different computer.
• Because these two devices are located at different locations, they must connect over
a network.
• The methods of communication employed by the system might vary depending on
its location.
• A single wire will be utilized if the trading system is collocated (the machines are in
the same facility), which will minimize network latency.
• The internet might be another mode of communication if we adopt a cloud solution.
The communication will be substantially slower than with a direct connection in
this instance.
Examine the following figure, which displays the communication that occurs between
the gateways:
Figure 2.4 – Communication between exchange and trading system
Making a trading system trade with exchanges 29
We understand the following points from the preceding diagram:
• When we examine the communication handled by the gateways more closely, we
can see that the venues may use various protocols.
• To transform protocols into trading system data structures, gateways will need to be
able to process a variety of protocols.
We learned about how a trading system connects to trading exchanges. We will now talk
about how communication can happen and which protocol we are using for receiving
market updates and sending orders.
Examining the API for communication
The rules of communication between machines are defined by network protocols.
They specify how these devices will be identified on the network and how they will
communicate with one another. The User Datagram Protocol (UDP) and Transmission
Control Protocol (TCP) over Internet Protocol (IP) protocols are used in trading
finance. Furthermore, we employ a software protocol that specifies how to send an order
and obtain a price update. At the software level, the communication API will establish
communication rules. The communication API is provided by the entity with which you
wish to transact. This document contains all of the messages you will need to get price
updates and place orders.
Examples of trading API documents can be found at https://en.wikipedia.org/
wiki/List_of_electronic_trading_protocols. Before we go into the trading
API, let's go over some networking fundamentals.
The fundamentals of networking
The network is responsible for allowing computers to connect with one another. To share
data, networks require a physical layer. For a network to achieve a certain level of speed,
dependability, or even security, selecting the appropriate medium (communication layer)
is crucial. We use the following terms in trade finance:
• Wire: Electrical currents with a narrow bandwidth.
• Fiber: More bandwidth.
• Microwave: It's simple to set up and has a lot of bandwidth, but it's susceptible
to storms.
30 The Critical Components of a Trading System
Depending on the sort of trading technique you choose, the media will change. In the
Open Systems Interconnection (OSI) model (developed in Chapter 5, Networking in
Motion), selecting the appropriate medium is part of the first layer of the network. The
physical layer is the name given to this layer. There are six more layers on top of this one
that describe the sort of communication.
Like most of the communication, finance is also using IP. This is part of the ISO model's
network layer. This IP establishes the rules for network packet routing. The transport
layer is the final layer we'll discuss. TCP and UDP are the two most well-known
protocols in banking. These two procedures are diametrically opposed. TCP is a
protocol that allows two machines to communicate with each other. All messages sent
initially will be delivered first. UDP lacks a means for determining whether network
packets were received by the network. All of the exchanges will use either TCP or UDP
as their protocol.
In Chapter 5, Networking in Motion, we will go deeper into the study of these protocols.
Let's learn about order book management in the following section.
Order book management
The primary goal of data handling is to copy the limit order book from the venues into
your trading system. The book builder will be in charge of gathering the pricing and
categorizing them for your tactics in order to integrate all of the many books you obtain.
The pricing changes are transformed by the gateway and then passed to the book builder,
as shown in the following diagram. The book builder will use the books that the gateways
have received from the venues, as well as gather and sort any pricing changes:
Figure 2.5 – Book builder getting price updates from Gateway IN
Order book management 31
In the following diagram, we use an example of an order book for a given financial
product. The order book will contain two parts, one for the bids and one for the offers. For
each part, we will store the orders represented by the venue, the volume, and the price.
Every venue will send its own order books. The goal of the book builder is to create a book
taking into account the three books coming from each venue. The data represented in this
diagram is artificial:
Figure 2.6 – Trading system building the book out of three different venues
The following is depicted in the diagram:
• You can see that there is an order for each row in these books.
• For example, a trader on Venue 1's offer list is ready to purchase 1,000 shares for
$1.21. On the other hand, there's a list of people who are eager to sell.
• The offer (or ask) price will almost always be greater than the bid price. Indeed, it
would be far too simple to profit if you could purchase for less than you could sell.
• The book builder's job is to collect the three books from the three places that the
gates have collected. The book builder organizes and sorts the three books.
We have learned how the trading system gets price updates and how to build a limit order
book. We will now explain in detail the different functionalities of the order book.
32 The Critical Components of a Trading System
Order book considerations
A limit order book collects all price updates (orders) and arranges them in a way that
makes the trading strategy's work easier. Exchanges utilize the order book to keep track of
bids and offers. When trading, we receive the book from the exchange in order to figure
out an indication of the asset prices, which prices are the best, or just to get a sense of the
market. We'll have to utilize the network to convey changes to the exchange book because
the exchange is on another platform/server. We have two options for doing so:
• The first way entails sending the entire book. Sending the full book is consuming
in terms of time. Indeed, if we have large trading exchanges (such as NYSE or
NASDAQ), there are millions of orders sent within one second. If each time the
exchange receives a new order, the full book is sent, the network will be saturated
and it will take too long to send price updates.
• The second way will send a full snapshot of the book as described previously.
Then, the exchange will send incremental updates. The book is a critical part of
the trading system since it will provide the trading strategy with the information
to decide when to send an order or not. An order book contains bids and orders
that are presently on the exchange. When a price update is sent to our trading
system, the other market participants receive the same update concurrently. All the
other market participants can also decide to run after this price update. When the
exchange receives many orders (explained in detail in Chapter 3, Understanding the
Trading Exchange Dynamics), the orders received first will be executed first. That's
why the book plays a large part in the latency and all the operations of the book
must be optimized.
For the life cycle of the orders, we will need to handle the following operations:
• Insertion: An insertion is a book entry that adds a new order to the book. This
should be a quick operation. Since we must sort the bids and the offers for any
received price updates, the method and data structure we choose for this operation
are crucial. To insert a new order, we'll need to use a data structure with an O(1) or
O(log n) complexity.
• An amendment/modification will use the order ID to seek the order in the book.
This operation should have the same level of difficulty as the insertion.
• Cancelation: Using the order ID, a cancelation allows an order to be withdrawn
from the book.
Order book management 33
The data structure chosen and the method connected with it have a significant impact
on performance. If you're creating an HFT system, you'll need to make the appropriate
decisions. The order book that we are implementing in HFT is called an order-based
book. Since this is a critical component in the system, it is very important to consider the
complexity of the execution of this book.
An efficient data structure to model the order book must ensure the following:
• Constant look-up, fast quantity update: An order book stores a lot of orders
for one given instrument. Large exchanges can get millions of orders per second.
Because we will have a growing number of orders in this book, it is important to
keep a constant look-up time for order IDs. We will have to look up order IDs
millions of times per second to update these orders. Additionally, we will need
to retrieve orders with the best price rapidly. Looking up orders by price cannot
be linear in terms of complexity. Therefore, we will use fast indexing (having a
logarithmic time to find an order for a specific price).
• Iteration in order of prices: When buying or selling large quantities, we may need
to find many orders to reach a given volume. In this situation, we will start with the
best price, then we will go to the second-best price, and we will keep on going like
this. In this situation, it is also critical for the execution speed to reach the next best
price with very low complexity.
• Retrieving best bid and ask in constant time: Since we will mainly work with the
best prices, we need to have a data structure capable of returning the best orders for
bids and offers.
We will need to take into account the following considerations:
• Organize order identifiers to order information in a huge associative array (for C++,
it could be a std::unordered map or std::vector).
• The order metadata includes references to the order book and price-level it belongs
to, therefore, after checking up the order, the order book and price-level data
structures are only a single dereference away. When using an Order Execute or
Order Reduce action, having a reference to the price allows for an O(1) decrease.
You may preserve pointers to the next and previous orders in the queue if you wish
to keep track of time priority as well.
Random documents with unrelated
content Scribd suggests to you:
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
The Project Gutenberg eBook of The Story of
Paul Jones: An Historical Romance
This ebook is for the use of anyone anywhere in the United States
and most other parts of the world at no cost and with almost no
restrictions whatsoever. You may copy it, give it away or re-use it
under the terms of the Project Gutenberg License included with this
ebook or online at www.gutenberg.org. If you are not located in the
United States, you will have to check the laws of the country where
you are located before using this eBook.
Title: The Story of Paul Jones: An Historical Romance
Author: Alfred Henry Lewis
Illustrator: Seymour M. Stone
Phillipps Ward
Release date: January 17, 2017 [eBook #53990]
Most recently updated: October 23, 2024
Language: English
Credits: Produced by David Widger from page images generously
provided by the Internet Archive
*** START OF THE PROJECT GUTENBERG EBOOK THE STORY OF
PAUL JONES: AN HISTORICAL ROMANCE ***
THE STORY OF PAUL JONES
An Historical Romance
By Alfred Henry Lewis
Illustrated by Seymour M. Stone and Phillipps
Ward
G. W. Dillingham Company, 1906
London
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
CONTENTS
THE STORY OF PAUL JONES
CHAPTER I—HIS BAPTISM OF THE SEA
CHAPTER II—IN THE BLACK TRADE
CHAPTER III—THE YELLOW JACK
CHAPTER IV—THE KILLING OF MUNGO
CHAPTER V—THE SAILOR TURNS PLANTER
CHAPTER VI—THE FIRST BLOW IN VIRGINIA
CHAPTER VII—THE BLAST OF WAR
CHAPTER VIII—THE PLANTER TURNS LIEUTENANT
CHAPTER IX—THE CRUISE OF THE “PROVIDENCE”
CHAPTER X—THE COUNSEL OF CADWALADER
CHAPTER XI—THE GOOD SHIP RANGER
CHAPTER XII—HOW THE “RANGER” TOOK THE “DRAKE”
CHAPTER XIII—THE DUCHESS OF CHARTRES
CHAPTER XIV—THE SAILING OF THE “RICHARD”
CHAPTER XV—THE “RICHARD” AND THE “SERAPIS”
CHAPTER XVI—HOW THE BATTLE RAGED
CHAPTER XVII—THE SURRENDER OF THE “SERAPIS”
CHAPTER XVIII—DIPLOMACY AND THE DUTCH
CHAPTER XIX—NOW FOR THE TRAITOR LANDAIS
CHAPTER XX—AIMEE ADELE DE TELISON
CHAPTER XXI—ANTONY AND CLEOPATRA
CHAPTER XXII—THE FÊTE OF THE DUCHESS DE
CHARTRES
CHAPTER XXIII—THE WEDDING WITHOUT BELLS
CHAPTER XXIV—THAT HONEYMOON SUB ROSA
CHAPTER XXV—CATHERINE OF RUSSIA
CHAPTER XXVI—AN ADMIRAL OF RUSSIA
CHAPTER XXVII—THE HOUSE IN THE RUE TOURNON
CHAPTER XXVIII—LOVE AND THOSE LAST DAYS
THE STORY OF PAUL JONES
T
CHAPTER I—HIS BAPTISM OF THE
SEA
his is in the long-ago, or, to be exact, in July, 1759. The new
brig Friendship, not a fortnight off the stocks, is lying in her
home harbor of Whitehaven, being fitted to her first suit of
sails. Captain Bennison is restlessly about her decks, overseeing
those sea-tailors, the sail-makers, as they go forward with their task,
when Mr. Younger, the owner, comes aboard. The latter gentleman is
lowland Scotch, stout, middle-aged, and his severe expanse of
smooth-shaven upper-lip tells of prudence, perseverance and
Presbyterianism in even parts, as traits dominant of his character.
“Dick,” says Mr. Younger, addressing Captain Bennison, “ye’ll have
a gude brig; and mon! ye s’uld have a gude crew. There’ll be none
of the last in Whitehaven, for what ones the agents showed me
were the mere riff-raff of the sea. I’ll even go to Arbigland, and pick
ye a crew among the fisher people.”
“Arbigland!” repeats Captain Bennison, with a glow of approval.
“The Arbigland men are the best sailor-folk that ever saw the
Solway. Give me an Arbigland crew, James, and I’ll find ye the
Rappahannock with the Friendship, within the month after she tears
her anchor out o’ Whitehaven mud.”
And so Mr. Younger goes over to Arbigland.
It is a blowing July afternoon. An off-shore breeze, now freshening
to a gale, tosses the Solway into choppy billows. Most of the
inhabitants of Arbigland are down at the mouth of the little tide-
water creek, that forms the harbor of the village, eagerly watching a
small fishing yawl. The latter craft is beating up in the teeth of the
gale, striving for the shelter of the creek.
The crew of the yawl consists of but one, and him a lad of twelve.
His right hand holds the tiller; with the left he slacks or hauls the
sheets, and shifts the sail when he goes about.
The yawl has just heeled over on the starboard tack, as Mr.
Younger pushes in among the villagers that crowd the little quay.
“They’ll no make it!” exclaims a fisherman, alluding to the boy and
yawl; “they’ll be blawn oot t’ sea!”
“Ay! they’ll make it sure enough,” declares another stoutly. “It’s
little Jack Paul who’s conning her, and he’d bring the yawl in against
a horrycane. She’s a gude boat, too—as quick on her feet as a
dancing maister; and, as for beating to wind’ard, she’ll lay a point
closer to the wind than a man has a right to ask of his lawful
wedded wife. Ye’ll see; little Jack’ll bring her in.”
“Who is he?” asks Mr. Younger of the last speaker; “who’s yon
boy?”
“He’s son to John Paul, gardener to the laird Craik.”
“Sitha! son to Gardener Paul, quo’ you!” breaks in an old fish-wife
who, with red arms folded beneath her coarse apron, stands
watching the yawl with the others. “Now to my mind, he looks mair
like the laird than I s’uld want my son to look, if I were wife to
Gardener Paul.”
“Shame for ye, Lucky!” cries the fisherman to whom she speaks.
“Would ye cast doots on the lad’s mither, and only because the lad in
his favoring makes ye think now and again on Maister Craik? Jeanny
Paul, that was Jeanny Macduff, is well kenned to be as carefu’ a wife
as ever cooked her man’s breakfast in Arbigland.”
“Ye think so, Tam Bryce?” retorts the incorrigible Lucky. “Much ye
s’uld know of the wives of Arbigland, and you to sea eleven months
o’ the year! I tell ye, Jeanny came fro’ the Highlands; and it’ll be
lang, I trow, since gude in shape of man or woman came oot o’ the
Highlands.”
“Guide your tongue, Lucky!” remonstrates the other, in a low tone;
“guide your tongue, ye jade! Here comes Gardener Paul himsel’.‘’
“I’ll no stay to meet him,” says Lucky, moving away. “Puir blinded
fule! not to see what all Arbigland, ay! and all Kirkbean Parish, too,
for that matter, has seen the twal years, that his boy Jack is no mair
no less than just the laird’s bairn when all’s said.”
“Ye’ll no mind her, Maister Younger,” says Tom Bryce, pointing
after Lucky; “although, to be preceese, what the carline tells has in
it mair of truth than poetry.”
“I was no thinking on the dame’s clack,” returns Mr. Younger, his
eyes still on the nearing yawl, “or whether yon lad’s a gardener’s
bairn or a gentleman’s by-blaw. What I will say, in the face of the
sun, however, is that he has in him the rudiments of as brisk a
sailorman as ever walked saut water.”
“There’ll be none that’s better,” observes Tom Bryce, “going in and
oot o’ Solway Firth.” Then, eyeing the yawl: “He’ll win to the creek’s
mouth on the next reach to sta’board.”
Gardener Paul joins Mr. Younger and the fisherman, Tom Bryce.
“We were talking of your son,” says Mr. Younger to Gardener Paul.
“What say ye, mon; will ye apprentice him? I’ll send him with Dick
Bennison, in my new brig Friendship, to the Virginias and Jamaica.”
John Paul, gardener to the laird, Robert Craik, is a dull man,
notably thick of wit, and slow.
“The Virginias!” he repeats. “My son William has been there these
sixteen year. He’s head man for my kinsman Jones, on his plantation
by the Rappahannock. If Jack sails with Dick Bennison, he’ll meet
William that he’s never seen.”
“He’ll see his brother for sure,” returns Mr. Younger. “The
Friendship goes from Whitehaven to Urbana, and that’s not a dozen
miles down the Rappahannock from your cousin’s plantation.”
The yawl has come safely into the creek’s mouth, and lies rocking
at her moorings as lightly as a gull. The lad leaps ashore, and is
patted on the back by the fisherman in praise of his seamanship. He
smiles through the salt water that drips from his face; for beating to
windward is not the driest point of sailing, and the lad is spray-
soaked from head to heel.
“And may I go, father?
“This is Mr. Younger, Jack,” says Gardener Paul, as the lad conies
up. “He wants ye to sail ‘prentice with Dick Bennison, in the new
brig.‘’ The difference to show between Gardener Paul and little Jack
Paul, as the pair stand together on the quay, goes far to justify those
innuendoes of the scandalous Lucky. Gardener Paul’s heavy peasant
face possesses nothing to mark, on his part, any blood-nearness to
the boy, whose olive skin, large brown eyes, clean profile and dark
hair like silk, speak only of the patrician.
“And may I go, father?” asks Jack, a flush breaking eagerly
through the tan on his cheek.
“Ye might as weel, I think,” responds Gardener Paul judgmatically.
“Ye’re the born petrel; and for the matter of gardening, being my
own and Adam’s trade, I’ve kenned for lang ye’ll no mair touch
spade or mattock than handle coals of fire. So, as I was saying, ye
might as weel sail ‘prentice with Dick; and when ye meet your
brother William, gi’ him his father’s gude word. Ye’ll never have seen
William, Jack, for he left hame before ye were born; and so it’ll be a
braw fore-gathering between the twa of ye—being brothers that
never met before.”
And after this fashion the fisher-boy, John Panl, afterward Admiral
Paul Jones, is given his baptism of the sea.
Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh
T
CHAPTER II—IN THE BLACK TRADE
he sun is struggling through the dust-coated, cobwebbed
windows, and lighting dimly yet sufficiently the dingy office of
Shipowner Younger of Whitehaven. That substantial man is
sitting at his desk, eyes fixed upon the bristle of upstanding masts
which sprout, thick as forest pines on a hillside, from the harbor
basin below. The face of Shipowner Younger has been given the
seasoning of several years, since he went to Arbigland that squall-
torn afternoon, to pick up a crew for Dick Bennison. Also, Shipowner
Younger shines with a new expression of high yet retiring
complacency. The expression is one awful and fascinating to the
clerk, who sits at the far end of the room. Shipowner Younger has
been elected to Parliament, and his awful complacency is that
elevation’s visible sign. The knowledge of his master’s election offers
the basis of much of the clerk’s awe, and that stipendary almost
charms himself into the delusion that he sees a halo about the bald
pate of Shipowner Younger.
The latter brings the spellbound clerk from his trance of
fascination, by wheeling upon him.
“Did ye send doon, mon,” he cries, “to my wharf, with word for
young Jack Paul to come?”
The clerk says that he did.
“Then ye can go seek your denner.”
The clerk, acting on this permission, scrambles to his fascinated
feet. As he retires through the one door, young Jack Paul enters. The
brown-faced boy of the Arbigland yawl has grown to be a brisk
young sailor, taut and natty. He shakes the hand of Shipowner
Younger, who gives him two fingers in that manner of condescending
reserve, which he conceives to be due his dignity as a member of
the House of Commons. Having done so much for his dignity,
Shipowner Younger relaxes.
“Have a chair, lad!” he says. “Bring her here where we can chat.”
The natty Jack Paul brings the clerk’s chair, as being the only one
in the room other than that occupied by Shipowner Younger. One
sees the thorough-paced sailor in the very motions of him; for his
step is quick, catlike and sure, and there is just the specter of a roll
in his walk, as though the heaving swell of the ocean still abides in
his heels. When he has placed the chair, so as to bring himself and
Shipowner Younger face to face, he says:
“And now, sir, what are your commands’?”
“I’ll have sent for ye, Jack,” begins Shipowner Younger,
portentously lengthening the while his shaven upper-lip—“I’ll have
sent for ye, for three several matters: To pay ye a compliment or
twa; to gi’ ye a gude lecture; an’ lastly to do a trifle of business wi’
ye, by way of rounding off. For I hold,” goes on Shipowner Younger,
in an admonishing tone, “that conversations which don’t carry a trifle
of business are no mair than just the crackle of thorns under a pot.
Ye’ll ken I’m rich, Jack—ye’ll ken I can clink my gold, an’ count my
gold, an’ keep my gold wi’ the warmest mon in Whitehaven?”
Young Jack Paul smiles, and nods his full agreement.
“But ye’ll no ken,” goes on Shipowner Younger, with proud
humility, the pride being real and the humility imitated—“ye’ll no
ken, I believe, that I’m ‘lected to the Parleyment in Lunnon, lad?”
Shipowner Younger pauses to observe the effect of this
announcement of his greatness. Being satisfied, he goes on. “It’s a
sacrifeece, no doot, but I s’all make it. The King has need of my
counsel; an’, God save him! he s’all have it. For I’ve always said, lad,
that a mon’s first debt is to the King. But it’ll mean sore changes,
Jack, sore changes will it mean; for I’m to sell up my ships to the
last ship’s gig of ‘em, the better to leave me hand-free and head-free
to serve the King.”
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.
More than just a book-buying platform, we strive to be a bridge
connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.
Join us on a journey of knowledge exploration, passion nurturing, and
personal growth every day!
ebookbell.com

More Related Content

PDF
Parallel Trading Systems business proposition
PDF
The Present and Future of High Frequency Trading
PDF
Waters may2010 hft_idc
PDF
iRage - Campus Roles and Work Culture
DOCX
High frequency trading world's capital moving to china with the speed traders...
PDF
High-Frequency Trading in Stock Market
DOCX
GLOBALIZATION OF FINANCE. ARSHAD HIPPARGI
DOCX
Top china, hk and singapore securities firms at speed traders workshop 2012 s...
Parallel Trading Systems business proposition
The Present and Future of High Frequency Trading
Waters may2010 hft_idc
iRage - Campus Roles and Work Culture
High frequency trading world's capital moving to china with the speed traders...
High-Frequency Trading in Stock Market
GLOBALIZATION OF FINANCE. ARSHAD HIPPARGI
Top china, hk and singapore securities firms at speed traders workshop 2012 s...

Similar to Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh (20)

PDF
Pwc High Frequency Trading Dark Pools
PDF
max dama on automated trading
PDF
Catalyst - Thoughts On Efficient Low Latency Trading Infrastructures
PPTX
Big Data in Stock Exchange( HFT, Forex, Flash Crashes)
PPTX
Big Data in Stock Exchange( HFT, Forex, Flash Crashes)
PPTX
The Impact of Algorithmic Trading
PDF
Fixed Income Trading System Architecture
PDF
Evolve | XTP Brochure
DOCX
High frequency trading
PDF
Op Risk High Frequency Trading June 14 Final
PDF
DataArt Financial Services and Capital Markets
PPTX
Linux rt in financial markets
PPT
Fixed Income Trading Platform Architecture
PDF
Visible Market FinTech Demo 07182012
PDF
The Impact of Algorithmic Trading
PDF
Trade Surveillance with Big Data
PDF
Huxley Technology Market Update Q4
PDF
(독일거래소+발표자료)+Krx+conference hft 101004
PDF
Fintech Software Development: A Comprehensive Guide in 2024
PDF
High-Frequency Trading Report
Pwc High Frequency Trading Dark Pools
max dama on automated trading
Catalyst - Thoughts On Efficient Low Latency Trading Infrastructures
Big Data in Stock Exchange( HFT, Forex, Flash Crashes)
Big Data in Stock Exchange( HFT, Forex, Flash Crashes)
The Impact of Algorithmic Trading
Fixed Income Trading System Architecture
Evolve | XTP Brochure
High frequency trading
Op Risk High Frequency Trading June 14 Final
DataArt Financial Services and Capital Markets
Linux rt in financial markets
Fixed Income Trading Platform Architecture
Visible Market FinTech Demo 07182012
The Impact of Algorithmic Trading
Trade Surveillance with Big Data
Huxley Technology Market Update Q4
(독일거래소+발표자료)+Krx+conference hft 101004
Fintech Software Development: A Comprehensive Guide in 2024
High-Frequency Trading Report
Ad

Recently uploaded (20)

PPTX
UV-Visible spectroscopy..pptx UV-Visible Spectroscopy – Electronic Transition...
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
PDF
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
PPTX
Cell Types and Its function , kingdom of life
PDF
Yogi Goddess Pres Conference Studio Updates
PPTX
Radiologic_Anatomy_of_the_Brachial_plexus [final].pptx
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
PDF
Anesthesia in Laparoscopic Surgery in India
PDF
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
PPTX
Introduction-to-Literarature-and-Literary-Studies-week-Prelim-coverage.pptx
PDF
Classroom Observation Tools for Teachers
PDF
Trump Administration's workforce development strategy
PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
PPTX
Cell Structure & Organelles in detailed.
PPTX
Lesson notes of climatology university.
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
PPTX
202450812 BayCHI UCSC-SV 20250812 v17.pptx
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
PPTX
Orientation - ARALprogram of Deped to the Parents.pptx
PPTX
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
UV-Visible spectroscopy..pptx UV-Visible Spectroscopy – Electronic Transition...
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
Cell Types and Its function , kingdom of life
Yogi Goddess Pres Conference Studio Updates
Radiologic_Anatomy_of_the_Brachial_plexus [final].pptx
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
Anesthesia in Laparoscopic Surgery in India
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
Introduction-to-Literarature-and-Literary-Studies-week-Prelim-coverage.pptx
Classroom Observation Tools for Teachers
Trump Administration's workforce development strategy
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Cell Structure & Organelles in detailed.
Lesson notes of climatology university.
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
202450812 BayCHI UCSC-SV 20250812 v17.pptx
Module 4: Burden of Disease Tutorial Slides S2 2025
Orientation - ARALprogram of Deped to the Parents.pptx
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
Ad

Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh

  • 1. Developing Highfrequency Trading Systems Sebastien Donadio Sourav Ghosh download https://ebookbell.com/product/developing-highfrequency-trading- systems-sebastien-donadio-sourav-ghosh-56240736 Explore and download more ebooks at ebookbell.com
  • 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Developing High Performance Leaders A Behavioral Science Guide For The Knowledge Of Work Culture Philip Robert Harris https://ebookbell.com/product/developing-high-performance-leaders-a- behavioral-science-guide-for-the-knowledge-of-work-culture-philip- robert-harris-51247316 Developing High Quality Data Models 1st Edition Matthew West https://ebookbell.com/product/developing-high-quality-data-models-1st- edition-matthew-west-2354358 Developing High Quality Data Models Matthew West https://ebookbell.com/product/developing-high-quality-data-models- matthew-west-56641312 Realworld Solutions For Developing Highquality Php Frameworks And Applications 1st Edition Sebastian Bergmann https://ebookbell.com/product/realworld-solutions-for-developing- highquality-php-frameworks-and-applications-1st-edition-sebastian- bergmann-2213124
  • 3. Realworld Solutions For Developing Highquality Php Frameworks And Applications Sebastian Bergmann https://ebookbell.com/product/realworld-solutions-for-developing- highquality-php-frameworks-and-applications-sebastian-bergmann-5475630 National Policies For Developing High Technology Industries International Comparisons Francis W Rushing https://ebookbell.com/product/national-policies-for-developing-high- technology-industries-international-comparisons-francis-w- rushing-43443958 Realworld Solutions For Developing Highquality Php Frameworks And Application Sebastian Bergmann https://ebookbell.com/product/realworld-solutions-for-developing- highquality-php-frameworks-and-application-sebastian-bergmann-5411638 Methodical Development Of Modular Product Families Developing High Product Diversity In A Manageable Way Dieter Krause https://ebookbell.com/product/methodical-development-of-modular- product-families-developing-high-product-diversity-in-a-manageable- way-dieter-krause-47663636 Beyond Knowledge Extracognitive Aspects Of Developing High Ability Illustrated Edition Larisa V Shavinina https://ebookbell.com/product/beyond-knowledge-extracognitive-aspects- of-developing-high-ability-illustrated-edition-larisa-v- shavinina-1365908
  • 6. Developing High-Frequency Trading Systems Learn how to implement high-frequency trading from scratch with C++ or Java basics Sebastien Donadio Sourav Ghosh Romain Rossier BIRMINGHAM—MUMBAI
  • 7. Developing High-Frequency Trading Systems Copyright © 2022 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Publishing Product Manager: Heramb Bhavsar Content Development Editor: Shreya Moharir Technical Editor: Rahul Limbachiya Copy Editor: Safis Editing Project Coordinator: Farheen Fathima Proofreader: Safis Editing Indexer: Tejal Daruwale Soni Production Designer: Roshan Kawale Marketing Coordinator: Priyanka Mhatre First published: June 2022 Production reference: 1160622 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80324-281-1 www.packt.com
  • 8. Contributors About the authors Sebastien Donadio has two decades of experience in high-performance computing, software design, and financial computing. Currently an architect in the Bloomberg office of the CTO, he has a wide variety of professional experience, including serving as CTO of an FX/Crypto trading shop, head of engineering, quantitative analyst, and partner at a high-frequency trading hedge. Sebastien has taught various computer science and financial engineering courses over the past fifteen years at various academic institutions, including Columbia University, the University of Chicago, and NYU. Sebastien holds a PhD in high-performance, an MBA in finance, and an MSc in analytics from the University of Chicago. His main passion is technology, but he is also a scuba diving instructor and an experienced rock-climber. Sourav Ghosh has worked in several proprietary high-frequency algorithmic trading firms over the last decade. He has built and deployed extremely low-latency, high- throughput automated trading systems for trading exchanges around the world, across multiple asset classes. He specializes in statistical arbitrage market-making and pairs trading strategies for the most liquid global futures contracts. He works as a vice president at a trading firm based in Sao Paulo, Brazil. He holds a master's in computer science from the University of Southern California. His areas of interest include computer architecture, FinTech, probability theory and stochastic processes, statistical learning and inference methods, and natural language processing. Romain Rossier brings 19+ years of experience, mostly as a software architect in the financial industry, specializing in low latency, high-performance Java software design and development. He is currently the chief architect for the HCTech FX Proprietary Trading engine. He also built and led the software development team at HCTech, where he oversaw and developed the HFT platform architecture for FX, Futures, and Fixed Income. Prior to HCTech, Romain was director of the Currenex lab, where he led the team responsible for the development of the Currenex Intelligent Pricing System. Romain holds a Master of Science in Communication Systems from the Swiss Federal Institute of Technology in Lausanne.
  • 9. About the reviewers John Rizzo has applied his passion for technology and building complex computing systems in support of the financial markets throughout his career. Over the past 17 years, he has held a number of technology leadership roles, including CTO, director, and infrastructure architect at various hedge funds and other firms in the financial services industry. He started and sold a company that developed systems to help manage various aspects of the loan life cycle and the syndicated loan market. Today, he is focused on infrastructure security in Bloomberg's Office of the CTO. Phil Vachon works on projects related to identity, authentication, and the application of data science to operational security challenges at Bloomberg. Prior to that, Phil co-founded a start-up focused on high-speed packet capture and analysis. He's also developed high-frequency trading systems, designed and implemented firmware for identity and security infrastructure devices, built synthetic aperture radar data processing tools, and worked on data plane traffic engineering for carrier routers. His main interests are developing threat models that are relevant to business problems, designing secure embedded systems, and working to improve individual privacy protections in our increasingly connected world.
  • 10. Table of Contents Preface Part 1: Trading Strategies, Trading Systems, and Exchanges 1 Fundamentals of a High-Frequency Trading System History of HFT 4 The post-1930s era 5 The modern era 6 Why have HFT? 7 What makes HFT so different from regular trading? 7 Effect of dark pools 9 Who trades HFT? 10 What do I need to start an HFT? 10 What are HFT strategies? 11 Asset classes 12 Liquidity 12 Tick-by-tick data and data distribution 12 Liquidity rebates 12 Matching engine 13 Market making 13 Scalping 15 Statistical arbitrage 15 Latency arbitrage 16 Impact of news 17 Momentum ignition 17 Rebate strategies 18 Pinging 18 Illegal activities 19 Summary 20 2 The Critical Components of a Trading System Understanding the trading system 22 Trading system architecture 24 Gateways connecting to trading exchanges 25
  • 11. vi Table of Contents Making a trading system trade with exchanges 27 Examining the API for communication 29 Order book management 30 Order book considerations 32 Strategy making decisions on when to trade 34 The OMS 36 Critical components 36 Non-critical components 37 Command and control 37 Services 38 Summary39 3 Understanding the Trading Exchange Dynamics Architecting a trading exchange for handling orders at a large scale 42 History of trading exchanges 42 Understanding features of an exchange 43 Exchange architecture 44 General order book and matching engine 46 Best price scenario 47 Partial fill scenario 48 No match scenario 49 Multiple orders with the same price 50 Summary 53 Part 2: How to Architect a High-Frequency Trading System 4 HFT System Foundations – From Hardware to OS Understanding HFT computers 58 CPUs, from multi-processor to multi-core59 Main memory or RAM 62 Shared memory 64 I/O devices 65 Using the OS for HFT systems 65 User space and kernel space 66 Process scheduling and CPU resource management 67 Memory management 68 Paged memory and page tables 69 System calls 70 Threading71 Interruption management 72 The role of compilers 73 Executable file formats 74 Static versus dynamic linking 74 Summary75
  • 12. Table of Contents vii 5 Networking in Motion Understanding networking in HFT systems 78 Learning about network conceptual models 78 Network communications between systems in HFT 80 Comprehending how switches work 82 Important protocol concepts 89 Using Ethernet for HFT communication 90 Using IPv4 as a network layer 90 UDP and TCP for the transport layer 92 Designing financial protocols for HFT exchanges 94 FIX protocol 95 Interior networks versus exterior networks 101 Understanding the packet life cycle 102 Comprehending the packet life in the send/receive (TX/RX) path 104 Software layer receiving the packet 105 Monitoring the network 105 Packet capture and analysis 106 Valuing time distribution 108 Time-synchronization services 109 Summary110 6 HFT Optimization – Architecture and Operating System Performance mental model 112 Understanding context switches 112 Types of context switches 113 Why are context switches good 114 Steps and operations involved in a context switch operation 115 Why are context switches bad for HFT? 116 Techniques to avoid or minimize context switches 117 Building lock-free data structures 118 When/why are locks needed (non-HFT applications) 118 Types of synchronization mechanisms 119 Problems and inefficiencies with using locks 119 Pre-fetching and pre-allocating memory 127 Memory hierarchy 128 Pre-fetching based alternatives to boost performance 131 Dynamic memory allocation 133 Pre-allocation-based alternatives to dynamic memory allocation 134 Summary135
  • 13. viii Table of Contents 7 HFT Optimization – Logging, Performance, and Networking Comparing kernel space and user space 138 What is kernel and user space? 139 Investigating performance – kernel versus user space 140 Using kernel bypass 141 Understanding why kernel bypass is the alternative 142 Presenting kernel bypass latencies 142 Learning about memory-mapped files 143 Using cable fiber, hollow fiber, and microwave technologies 146 Evolution from cable fiber to hollow fiber to microwave 147 How hollow fiber works 147 How microwave works 148 Diving into logging and statistics 150 The need for logging in HFT 150 The need for online/live statistics computation in HFT 150 Measuring performance 152 Motivation for measuring performance 152 Linux tools for measuring performance 153 Custom techniques for measuring performance 156 Summary161 Part 3: Implementation of a High-Frequency Trading System 8 C++ – The Quest for Microsecond Latency C++ 14/17 memory model 166 What is a memory model? 166 The need for a memory model 167 The C++ 11 memory model and its rules168 C++ memory model principles 172 Removing runtime decisions 176 Motivation for removing runtime decisions 177 Virtual functions 178 Performance penalties 179 Dynamic memory allocation 184 Runtime performance penalty 184 Using constexpr efficiently 186 Exceptions impeding performance 186 Templates reducing the runtime 188 What are templates? 188 Template specialization 189 Why use templates? 189
  • 14. Table of Contents ix Disadvantages of templates 191 Performance of templates 192 Standard Template Library (STL) 193 Static analysis 195 What is C++ static analysis? 195 The need for static analysis 195 Types of static analysis 196 Steps in static analysis 197 Benefits and drawbacks of static analysis 197 Use case - Building an FX high-frequency trading system 200 Summary 201 9 Java and JVM for Low-Latency Systems Introducing the basics of Java 204 Reducing the impact of the GC 207 What to do to keep GC events low and fast 208 Warming up the JVM 213 Tiered compilation in JVM 214 Optimizing the JVM for better startup performance 218 Measuring the performance of a Java software 219 Why are Java microbenchmarks difficult to create? 219 Real-time performance measures 220 Java threading 221 Using a thread pool/queue with threads 222 High-performance task queue 227 Queues 227 Circular buffer 227 LMAX disruptor 228 Logging and DB access 230 External or internal thread? 231 Summary 232 10 Python – Interpreted but Open to High Performance Introducing Python 234 Making use of Python for analytics 234 Why is Python slow? 236 How do we use libraries in Python? 238 Python and C++ for HFT 239 Using C++ in Python 240 Using Python with C++ 240 Boost.Python library 241 Using ctypes/CFFI to accelerate Python code 243 SWIG 244 Improving the speed of Python code in HFT 246 Summary 248
  • 15. x Table of Contents 11 High-Frequency FPGA and Crypto Reducing latencies with FPGA 250 Evolution of the fierce competition of speed in HFT 250 Introduction to FPGA 251 Diving into FPGA trading systems 254 Advantages of FPGA trading systems 256 Disadvantages of FPGA trading systems 257 Final words on FPGAs 258 Exploring HFT with cryptocurrencies 259 What is crypto? 259 How do crypto transactions work? 259 What is a blockchain? 260 What is cryptocurrency mining? 260 Similarities between traditional asset trading and cryptocurrency trading 260 Main differences between traditional asset trading and cryptocurrency trading 265 Trading with cryptocurrency exchange 267 HFT strategies in crypto 271 Building a high-frequency system for crypto trading 272 How to build a trading system in the cloud 275 Summary 280 Index Other Books You May Enjoy
  • 16. Preface The world of trading markets is complex, but it can be made easier with technology. Sure, you know how to code, but where do you start? What programming language do you use? How do you solve the problem of latency? The Developing High-Frequency Trading Systems book answers all these questions. This practical guide will help you navigate the fast-paced world of algorithmic trading and show you how to build a high-frequency trading system from complex technological components supported by accurate data. Starting with an introduction to high-frequency trading (HFT), exchanges, and the critical components of a trading system, the book quickly moves on to the nitty-gritty of optimizing hardware and your operating system (OS) for low-latency trading, such as bypassing the kernel, memory management, and the danger of context switching. Monitoring your system's performance is vital, so you'll also get up to speed with logging and statistics. As you move beyond the traditional high-frequency trading programming languages, such as C++ and Java, you'll learn how to use Python to achieve high-performance levels. And what book on trading would be complete without diving into cryptocurrency? By the end of this book, you'll be ready to take on the markets with high-frequency trading systems. Who this book is for This book is for software engineers, quantitative developers or researchers, and DevOps engineers who want to understand the technical side of high-frequency trading systems and the optimizations needed to achieve ultra-low latency systems. Prior experience working with C++ and Java will help you grasp the topics covered in this book more easily.
  • 17. xii Preface What this book covers Chapter 1, Fundamentals of a High-Frequency Trading System, gives an overview of the history of high-frequency trading. You will learn about the market participants, the fundamental HFT requirements (low latency connectivity and infrastructure), the trading time horizons in HFT versus non-HFT, and the holding periods/position management (HFT versus Ultra HFT). We will also detail HFT-specific strategies to make money. Chapter 2, The Critical Components of a Trading System, explains in-depth how a trading system works. You will learn how the market data gets into the system, and the different functionalities needed to handle the data and send an order to the exchange. Chapter 3, Understanding the Trading Exchange Dynamics, introduces how trading exchanges are a part of the microstructure of markets. We will start by giving the general infrastructure of an exchange, and we will talk about how the matching engine works and how the orders are matched and promoted to all the market participants. Chapter 4, HFT System Foundations – From Hardware to OS, clarifies how the hardware and OS work together. You will have a clear understanding of the functions of the software interaction with the OS and the hardware. This chapter will go from the processor to the trading system, explaining all the layers between including the OS, networking, OS scheduler, and memory. Chapter 5, Networking in Motion, expresses how networking benefits HFT. You will have a clear understanding of the functions of the network stack and its use when communicating between a trading system and an exchange. Chapter 6, HFT Optimization – Architecture and Operating System, expounds on creating a HFT system from a regular trading system. This section will cover many modern techniques to achieve optimal low latency performance for HFT applications specifically. We will talk about the OS features and its scheduler, and we will do a deep dive into the kernel function of the OS. Chapter 7, HFT Optimization – Logging, Performance, and Networking, covers a vital part of trading systems: logging and networking. You will understand how logging helps to monitor an HFT system, and we will learn how to make it efficient in a context of HFT. Finally, we will cover how to use networking to optimize communication with trading exchanges. Chapter 8, C++ – The Quest for Microsecond Latency, defines the use of C++ in a context of an ultra-low latency system by optimizing cache, memory, and code execution. You will learn about modern C++ features and techniques to write ultra-low latency code efficiently.
  • 18. Preface xiii Chapter 9, Java and JVM for Low-Latency Systems, details the use of Java in a context of an ultra-low latency system by optimizing garbage collection, communication, and data structure. Chapter 10, Python – Interpreted but Open to High Performance, illustrates how to use Python in an HFT system. This chapter explains how to create and use HFT libraries in Python. Chapter 11, High-Frequency FPGA and Crypto, depicts how to use field programmable gate array (FPGA) to create an even faster HFT system. It will introduce building an HFT system for crypto in the cloud. To get the most out of this book This book assumes that you are familiar with programming, hardware architecture, and OS. Because this book will discuss the optimizations required to reduce the tick-to- trade latency, it is essential to have the minimal knowledge of computer engineering. Most of the HFT systems run with a Unix-based OS. We will recommend using a Linux OS to apply your knowledge of this book. This book is a reservoir of knowledge from many computer engineering and finance domains. We recommend reading other Packt books such as the following: • Linux Kernel Programming (https://www.packtpub.com/product/ networking-and-servers/9781789953435) • Java Programming for Beginners (https://www.packtpub.com/product/ application-development/9781788296298) • C++ High Performance (https://www.packtpub.com/product/ programming/9781839216541) We also recommend reading books such as Compilers: Principles, Techniques, and Tools, and Computer Architecture: A Quantitative Approach. These books will give you more in-depth knowledge of the optimization we are using for HFT. Download the color images We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://static.packt-cdn.com/ downloads/9781803242811_ColorImages.pdf.
  • 19. xiv Preface Conventions used There are a number of text conventions used throughout this book. Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: It offers one producer to one consumer (OneToOneRingBuffer) or many producers to one consumer (ManyToOneRingBuffer) solutions. A block of code is set as follows: /* Put header files here or function declarations like below */ extern int add_1(int n); extern int add(int n, int m); Any command-line input or output is written as follows: import math math.add_1(5) 6 Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: The Load Data component (annotation 1) will help get historical data. Tips or Important Notes Appear like this. Get in touch Feedback from our readers is always welcome. General feedback: If you have questions about any aspect of this book, email us at customercare@packtpub.com and mention the book title in the subject of your message. Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
  • 20. Preface xv Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at copyright@packt.com with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com. Share Your Thoughts Once you've read Developing High-Frequency Trading Systems, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback. Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.
  • 22. By the end of this introduction, you will have a quick overview of the history of high-frequency trading (HFT). You will know about the market participants, the fundamental HFT requirements (low latency connectivity and infrastructure), the trading time horizons in HFT versus non-HFT, and the holding periods/position management (HFT versus ultra HFT). We will also talk about the places for HFT. This book is not about the business of trading or HFT; it is about how to implement an HFT system concretely by using Java, C++, and Python. You will know how a trading system works and which trading strategies you can run. This part comprises the following chapters: • Chapter 1, Fundamentals of a High-Frequency Trading System • Chapter 2, The Critical Components of a Trading System • Chapter 3, Understanding the Trading Exchange Dynamics Part 1: Trading Strategies, Trading Systems, and Exchanges
  • 24. 1 Fundamentals of a High-Frequency Trading System Welcome to Developing High-Frequency Trading Systems! High-Frequency Trading (HFT) is a form of automated trading. For the last twenty years, HFT has gained recognition in the media and in society. A book called Flash Boys: A Wall Street Revolt, written by Michael Lewis in 2014, topped the sales on the New York Times Best Seller list for three weeks. It relates to an investigation into the HFT industry and its impact on the trading world. Scholars, the financial world, and the non-financial world are fascinated by this form of trading. Meanwhile, this new era of trading has created a lot of fear while giving more and more control to machines. The goal of this book is to review what HFT is and how to build such a system from a technical perspective. HFT is a multi-disciplinary matter involving thorough knowledge of computer architecture, operating systems, networking, and programming. By the end of this book, you will understand how to build a trading system from scratch by using the most advanced technical choices for optimizing speed and scalability. We chose to divide this book into three main parts.
  • 25. 4 Fundamentals of a High-Frequency Trading System In the first part, we'll go through how HFT tactics function and what kind of trading we may expect from HFT. Then we will go over the functions of an HFT system. We will conclude this part with a description of how a trading exchange works. In the second part of this book, we will explain the theory of operating systems and hardware and the required knowledge to optimize a trading system, taking into account the hardware and operating system features. The final part will explain in detail how to use C++, Java, Python, and FPGA to create an HFT system. We will also extend this knowledge to crypto trading, and we will review how to build a trading system in the cloud. In this chapter, we will talk about how we got into HFT. We will review what kind of trading strategies work for HFT. We will explain in detail what makes HFT so different from regular trading. Our objective in this chapter is to cover the following topics: • History of HFT • What HFT is • Who the participants are • What trading strategies work in HFT History of HFT Let's discuss the history of exchanges and financial markets prior to 1930. When we talk about HFT, it is difficult to give a precise date for when it started. We need to come back to the primitive times when trade arose from human contact. Before the invention of modern-day cash, ancient people relied heavily on trading to trade products and services with one another in a gift economy. Long-distance trade extends back to almost 150,000 years ago, according to Peter Watson. Year after year, with more people, more goods, and more money, trading became one of the major activities of humankind. It is obvious that making money implies more business. One of the parameters was speed. If you make more transactions, you will make more money. Many stories describe the ambition of traders to get technologies such as better transportation to make deals more quickly or to get news more quickly to take advantage of other folks who do not have access to these new technical means.
  • 26. History of HFT 5 We did not have to wait for too long before seeing cases of unfair trade involving those who have technical advantages over others. In 1790, a Georgia representative spoke to the US House of Representatives to expose high-speed traders. Indeed, Congress was debating the Secretary of the Treasury Alexander Hamilton's proposal that the US government absorb the previous debts accrued by the states during the Revolution (Funding Act of 1790). Traders who had learned the decision immediately bought or rented rapid boats. Their goal was to front-run messengers and buy the old debts since the passage of the Act would increase the market value. During the twentieth century, the idea of speed trading or HFT appeared. The post-1930s era Trading is the exchange of items for other items. It can be financial products, services, cash, digital assets, and more. One of the goals of trading is to make a profit from these transactions. The number of transactions will be correlated with the quantity of money generated by the exchange of assets. When we manage to increase the ratio between the number of transactions and the time, we can increase the profitability over time. Therefore, being capable of increasing the number of transactions is critical. Trading actors understood very quickly that they needed to shorten the trading time and started gathering in some specific places. They used to place their orders in these locations, which we call today the trading exchange (or trading floor). Major events participated in the expansion of fast-speed automated trading: • 1969: Instinet was one of the first automated system infrastructures. It speeded up the adhesion of high-speed transactions. • 1971: The National Association of Securities Dealers Automated Quotations (NASDAQ) was created in 1971 with electronic transactions. It was the world's first electronic stock market. Initially, it only used to send quotations. • 1996: Island ECN was the pioneering electronic communication network for US equities trading, while Archipelago facilitated electronic trading on the US trading exchange by creating Archipelago Exchange (ArcaEx). • 2000s: 10% of transactions are HFT transactions. The financial sector gathered more and more technologists in the early 2000s. By getting this technological intake, the sector started evolving sharply. Automation, throughput, performance, and latency became words that were well known by trading firms. The HFT transactions reached more than 10% of the market. By 2009, 2% of trading firms accounted for 75% of the equity volume. Nowadays, only a few firms remain in HFT, such as Virtu, Jump, Citadel, IMC, and Tower.
  • 27. 6 Fundamentals of a High-Frequency Trading System The modern era The post-1930s era focused on transparency and regulation in the equities markets (and the pit in commodities markets). The modern era gives prominence to electronic trading and improves transparency. In 2000, the US Securities and Exchange Commission (SEC) proposed the Central Limit Order Book (CLOB). The CLOB is a transparent system matching orders between participants. Many more exchanges (such as Island and Arca) came to the trading scene. The number of trading firms, hedge funds, and electronic players kept increasing. They created their own technology stack to trade more quickly and stay competitive. After 10 years, only a few trading firms managed to remain competitive, becoming the 2% of the trading firms accountable for 75% of all equity volume. The savoir faire for competing in HFT requires heavy investment: money, people, and time. It is a marriage of low-level system expertise and quants, as well as smart money (investors are more and more technology savvy). Engineers capable of creating performant code for designing ultra-low latency systems are very expensive. Only a few engineers had these skills. The performance for such a system required specialized hardware. Routers, servers, and network devices are also expensive. Therefore, the experience and the barrier of entry will prevent a lot of new incomers and will limit the competition. On top of the five firms we talked about previously, there are boutique shops that trade HFT strategies using an edge they found either in the market structure or some technical fact that other firms are not exploiting. The giant HFT firms are the companies responsible for moving most of the equity volume. Nowadays, HFT is estimated to account for at least 50% of the US equity (shares) trading volume. The market share of HFT has declined, as has profitability, since the peak year (2009). After 2015, the growth of digital currencies cleared the way for new opportunities for high-frequency traders. Today, we can see an extensive growth of HFT strategies working with well-known crypto exchanges such as Coinbase, Binance, and hundreds of other crypto exchanges. This modern era has anchored technology and automated trading for good. Trading models are data driven and model driven. The market data business definitely became a major part of trading. Exchanges and trading firms started making money by generating or collecting market data, the raw material of any algorithm trader.
  • 28. Why have HFT? 7 Why have HFT? HFT aims at getting many transactions per second. In this way, companies can react more quickly to a changing market. They can take advantage of more opportunities than they would have without this speed. Additionally, large institutions benefit from HFT by gaining a tiny but considerable advantage in exchange for delivering massive volumes of liquidity to markets. They place millions of orders that their systems are capable of placing. They help the market and, as a result, are able to boost earnings in their profitable trades and receive better spreads. Since the return is very low, they must complete many trades to benefit. On top of this revenue, they will gain rebates or discounted transaction fees, which are given by trading venues to make their markets more attractive to HFT firms. What makes HFT so different from regular trading? HFT trading should have the shortest feasible data latency (time delays) and the highest level of automation possible. HFT relates to algorithmic trading and automated trading. As a result, participants choose to trade in markets that have a high level of automation and integration in their trading platforms. Firms utilize computers programmed with precise algorithms to find trading opportunities and execute orders in algorithmic trading. To increase the speed of transactions, high-frequency traders use automated trading and fast connections (and cancellations or modifications). This is possible because of the technology that trading firms have in place but also because of the exchange technologies. The following exchanges have invested hundreds of millions of dollars in HFT technologies: • NASDAQ, New York City, is the first electronic stock exchange in the world. All of its equities are traded over a computerized network. It revolutionized the financial markets in 1971 by removing the requirement for a physical trading floor and in-person trading. It is the world's second-biggest stock exchange by market capitalization. Half of NASDAQ's composite offering was made up of technology firms. With less than 20% of the overall composite, the consumer sector came in second, followed by healthcare. • New York Stock Exchange (NYSE), New York City, is the world's largest exchange for the equity market. In 2013, Intercontinental Exchange, Inc. (ICE) bought NYSE.
  • 29. 8 Fundamentals of a High-Frequency Trading System • London Stock Exchange (LSE), London, UK, is the largest stock exchange in Europe and the principal stock exchange in the United Kingdom mainly with regard to trading in company stocks and bonds. It was created about 300 years ago. • The Tokyo Stock Exchange (TSE), is Japan's largest stock exchange, with its headquarters in Tokyo. It was founded in 1878. The exchange has more than 3,500 listed businesses. The TSE, which is operated by the Japan Exchange Group, is home to the world's largest and most well-known Japanese corporations, including Toyota, Honda, and Mitsubishi. • The Chicago Mercantile Market (CME), sometimes known as the Chicago Merc, is a regulated futures and options exchange in Chicago, Illinois. Agriculture, energy, stock indices, foreign exchange, interest rates, metals, real estate, and weather are among the industries in which the CME trades futures and, in most cases, options. • Direct Edge, Jersey City. Its market share rapidly rose to tenth in the US stock market, and it typically transacted more than two billion shares daily. Better Alternative Trading System (BATS) Global Markets was a US-based exchange that traded a variety of assets, including stocks, options, and foreign exchange. CBOE Holdings purchased it in 2017 after it was created in 2005. BATS Global Market was one of the largest US exchanges prior to being bought, and it was well known for its services to broker-dealers, retail, and institutional investors. • The CBOE Options Exchange, which was founded in 1973, is the world's largest options exchange, with contracts centered on individual stocks, indexes, and interest rates. All the preceding exchanges are controlled on several levels: • Trading limitations • Trading system transparency (information shared among market participants on the specificities of the architecture, as well as the way of handling orders) • The type of accepted financial instruments • Constraints by security issuers
  • 30. What makes HFT so different from regular trading? 9 For most regulated exchanges, the order size is an issue. Large trades have an important effect on the market (they can create market impact). Traders use Alternate Trading Systems (ATS), which have much less regulation in comparison to traditional exchanges (they don't have to be transparent). Dark pools are the most common sorts of ATS. The USA presently has around 30 dark pools, which represent a quarter of the US consolidated trading volume. Dark pools are beneficial to HFTs because they can handle the speed and the level of automation demands while having reduced fees. This is not the case for any other type of trading, which makes HFT different from regular trading. In the following section, let's learn more about dark pools. Effect of dark pools For financial security, buy and sell orders are not displayed in dark pools (price and volume). Dark pools, in other words, are both opaque and anonymous since the order book is not advertised. Because it is not possible to see the size of the orders in this type of trading exchange, investors who place huge orders do not impact markets. Since the other participants do not see the size of the orders, the dark pools execute these large orders at a fixed price. It reduces the negative slippage given by trading exchanges. Dark pools are obliged to notify deals once they have occurred, notwithstanding the lack of pre-trade transparency. HFTs and dark pools have a complicated interaction. Dark pools rose in popularity partially as a result of investors seeking protection from HFTs' fraudulent activities on public exchanges, and HFTs finding it impossible to know the large orders in dark pools through pinging. Dark pools introduced a lack of transparency in the markets that allowed ill-equipped players (that is, on the sell side) to keep up with business practices that didn't match the state of the art at the time. And, of course, Haim Bodek wrote two books (The Problem of HFT and The Market Structure Crisis) about finding unordinary order types in dark pools. On the other hand, a few dark pools encourage HFT traders to trade on their exchange. HFT strategies increase liquidity and the likelihood of having orders filled. Dark pools help HFTs to meet their speed and automation demands while still having reduced expenses. HFTs are responsible for the decrease in order sizes in dark pools. The dark pools have been hit by pinging trading strategies locating hidden large orders.
  • 31. 10 Fundamentals of a High-Frequency Trading System As a result, if these HFT tactics are present, the benefits of dark pools may be harmed. For example, in 2014, the Attorney General of New York filed a lawsuit against Barclays for its dark pool operations, alleging that it misrepresented the volume of Barclays's activity in dark pools. In 2016, Barclays paid a $35 million fine to the SEC and $70 million to the State of New York. Dark pools can apply certain constraints to prevent HFTs from engaging in predatory behavior. The goal is to reduce pinging trading strategies. In 2017, Petrescu and Wedow imposed a minimum order size to minimize this type of strategy. We could spend more time discussing the pros and cons of the impact of HFTs on dark pools, but we end up saying that the advantages of having more liquidity and faster execution are beneficial enough to have some dark pools being in favor of HFTs. It is fair for investors as long as they have a thorough understanding of how trading venues work so they can make educated judgments. We have talked about the location of the major trading exchanges. Now we will introduce the HFT participants in the next section. Who trades HFT? The answer could be summarized in one word: everyone. From the buy side to the sell side, ECNs, and even the inter-dealer and inter-broker-dealer markets, they all use HFT. HFT is dominated by proprietary trading businesses and covers a wide range of products, including stocks, derivatives, index funds, Exchange-Traded Funds (ETFs), currencies, and fixed-income instruments. Proprietary trading businesses accounted for half of the current HFT players, multi-service broker-dealer proprietary trading desks accounted for less than half, and hedge funds accounted for the rest. Proprietary trading businesses such as KCG Holdings (created by the merger of Getco and Knight Capital) and the trading desks of major banks are among the major players in the field. There are some new types of venues (such as Dealerweb's OTR Exchange and IEX) that are looking to provide venues where dealers on the sell side feel safe to execute trades and HFTs are providing liquidity. It is worth saying that HFTs have become major players in the market. They are also capturing retail flow. Citadel is controlling a large part of the retail flow. What do I need to start an HFT? Participants in HFTs must have the following: • Fast computers: HFT focuses on single-core throughput in most cases, and parallelism is not used by the strategies necessarily.
  • 32. What do I need to start an HFT? 11 • Exchange proximity: While some countries restrict the use of shared places to have trading systems and exchange, in the US, we use co-location. This is a place where all the HFTs participants have their production servers. They will pay to have their computers co-located with an exchange's computer servers in the same data centers in order to decrease latency and shorten the time it takes to complete a deal – even by microseconds. The cables linking trading systems from all market participants with the server are the same length to guarantee that nobody has an advantage over another market participant. The SEC has issued a wide request for feedback on co-location fees, as well as other concerns impacting the equity market structure. To ensure fairness among market participants, it is important that co-location fees are reasonably priced. The SEC invites the co-locations to report their fees. • Low latency: In HFT, latency is the time it takes for data to reach a trader's computer, for the trader to make an order in response to the data, and for the order to be accepted by an exchange. The order may enter the market alongside many other orders issued by other traders at the most profitable time. There is a danger of competing against a large number of other people in this circumstance. The order may not be as profitable as it may have been in this scenario. High-frequency traders are able to make orders at unfathomably quick speeds because of technology advertised as low-latency or ultra-low-latency. It is important to use gear designed to reduce the latency of shuffling data from one place to another. • Computer algorithms, which are at the heart of AT and HFT, and real-time data feeds, which could damage earnings. In the previous sections, we learned where high-frequency traders make business. We also talked about the technological prerequisites to trade faster. Let's now focus on what HFT is in depth. What are HFT strategies? HFT strategies are a subset of algorithmic trading strategies. They are executed in the order of the microseconds (and sometimes nanoseconds). The strategies must be aware of this time limitation to be efficient HFT strategies. They deploy cutting-edge technology advancements to obtain information faster than the competition. The main game of this type of strategy is the tick-to-trade, which is the response time to send an order responding to incoming market data. As we will explain in the next chapter, it is important to host trading strategies on cutting-edge machines, and they must also run in a co-location. We will be defining the domain of applications and some vocab to have when talking about HFT strategies.
  • 33. 12 Fundamentals of a High-Frequency Trading System Asset classes HFT strategies can be applied to any asset classes, such as stocks, futures, bonds, options, and FX. We also have cryptocurrencies being traded using HFT strategies, even if the definition of speed is different (because of the settlement time). Liquidity The desire of players to interact with regard to a certain asset is known as liquidity. We define depth as the number of price levels for a given asset. We will say that a book is deep when there are many levels (layers) for a given asset. We will define a book as big or broad if the volume per layer is high. If a book is deep or large, we will define a liquidity of a given asset liquid. The consequence of this statement is that it will be easier for a trader to buy or sell this asset whenever they want to. As a result, trading exchanges with a lot of liquidities are wanted by traders. Crypto trading exchanges have difficulty finding liquidities at the moment. Tick-by-tick data and data distribution HFT generates orders every microsecond. Since there are a lot of participants, it is likely to have huge amounts of data. Storage of this data will be key when we study HFT data to create models for trading strategies. Thousands of ticks (security price changes from one order to another) are generated per trading day on liquid marketplaces, which make up high-frequency data. This material is randomly spaced in time by its very nature. HFT data exhibits fat tail distributions. That means that the trading strategies need to take into account that we can have big losses. They distribution of the market data can be grouped into two categories: • Volatility clustering: Large changes follow large changes whether in terms of signs or numbers, while minor changes follow smaller changes. • Long-range dependency (long memory) refers to the pace at which statistical dependence between two sites decays as the time interval or spatial distance between them increases. Liquidity rebates To support the provision of stock liquidity, most exchanges have used a maker-taker model. In this arrangement, investors and traders who place limit orders often earn a modest rebate from the exchange when their orders are executed since they are considered to have contributed to stock liquidity, or makers.
  • 34. What do I need to start an HFT? 13 Those who place market orders, on the other hand, are considered takers of liquidity and are charged a small fee by the exchange. While the rebates are normally fractions of a penny per share, over the millions of shares exchanged daily by high-frequency traders, they may add up to large amounts. Many HFT businesses use trading techniques that are geared to take advantage of as many liquidity rebates as feasible. Matching engine The software program that forms the heart of an exchange's trading system and matches buy and sell orders on a continuous basis, a service traditionally done by trading floor professionals, called the matching engine is critical for guaranteeing the efficient operation of an exchange since it matches buyers and sellers for all stocks. The matching engine is housed on the exchange's computers, and it is the main reason why HFT businesses strive to be as near to the exchange servers as possible. We will learn about it in Chapter 3, Understanding the Trading Exchange Dynamics. Market making Before going into details on what market making is, we need to explain the difference between market takers and market makers. Market taker/maker Figure 1.1 represents the limit order book on an exchange. When a trading strategy places an order close to the top of the book (the layer representing the best price for bid and for ask), we say that this order is an aggressive order. It means that this order is likely to be matched with another order. If the order is executed, it means that liquidity has been removed from the market; it is a market taker. We will say that a trader crosses the spread when they place a buy order at the price of the ask on the top of the book. If the order is less aggressive (or passive), this order will not remove liquidities from the market; it is a market maker. Figure 1.1 – Order book – passive/aggressive order
  • 35. 14 Fundamentals of a High-Frequency Trading System Let's look at the market-making strategies. Market-making strategies A trading corporation can provide market-making as a service on an exchange. Over time, a market maker assists in the matching of buyers and sellers. Rather than purchasing or selling securities based on their underlying assets, market makers maintain a continual offer to buy and sell securities and profit from the spread, which is the difference between the two offers. To reduce the risk of keeping stocks for extended periods of time, every purchase should be matched with a sale and every sell should be matched with a buy. If a stock is trading at $100, a market maker can keep a buy offer at $99.50 and a sell offer at $100.50. If they are successful in finding both a buyer and a seller, it allows those who want to sell right now to do so even if no one else wants to purchase, and vice versa. Market makers, in other words, supply liquidity—they make trading simpler. For the most traded stocks, this technique is not important; however, for smaller firms (less traded than the big ones), it can be critical to increase the trading volume to facilitate trading. Market making is one approach that many HFT businesses use. They out-compete everyone else by changing their quotations quickly and reducing the spread even further: they're willing to make less money each time since their market-making business can readily grow to massive quantities. However, an HFT firm's technology can be used for other purposes, such as arbitrage (making money on minor discrepancies between linked securities) or execution (breaking up huge institutions' trades to minimize market effect). I won't go into much more detail because the point is that HFT is capable of more than simply market-making. The only thing that matters is speed. Market making can be done by the analysis of the order flow: • A large volume of buy and sell can drive the market price of buying and selling on the basis of momentum. • The flow of liquids (how big are the buy and sell orders: small, medium, or big). • Exhaustion of momentum (when the order flow is drying off it may signal a price reversal). Market-making is the most widely used trading strategy for high-frequency traders. We will talk about the other HFT strategies in the next sections.
  • 36. What do I need to start an HFT? 15 Scalping Scalping is a trading method that focuses on benefitting from tiny price movements and reselling for a quick profit. Scalping is a phrase used in day trading to describe a technique that focuses on generating large volumes from tiny profits. Scalping necessitates a tight exit plan since a single major loss might wipe out all of the modest wins the trader has worked so hard to achieve. For this technique to work, you'll need the necessary tools, such as a live feed, a direct-access broker, and the endurance to conduct a lot of trades. The concept behind scalping is that most stocks will finish the first stage of a trend. But it's unclear where things will go from there. Some stocks stop rising after that early stage, while others continue to rise. The goal is to benefit from as many minor transactions as possible. The let your gains run mentality, on the other hand, aims to maximize good trading results by expanding the size of winning deals. By increasing the number of winners while compromising on the magnitude of the gains, this technique accomplishes outcomes. It's very uncommon for a trader with a long time period to produce good profits while winning just 50% of their transactions, or even less – the difference is that the wins are far larger than the losses. Statistical arbitrage The Efficient Market Hypothesis (EMH) claims financial markets are informationally efficient, which means that the prices of traded assets are accurate, and at any one moment represent all known information. Based on this hypothesis, the market should not fluctuate if there is not any fundamental news. However, this is not the case, and we can explain that with liquidity. Throughout the day, many huge institutional trades have little to do with information and everything to do with liquidity. Investors who believe they are overexposed will aggressively hedge or sell their positions, impacting the price. Liquidity seekers are frequently ready to pay a premium to exit their positions, resulting in a profit for liquidity providers. Although this capacity to benefit from knowledge appears to violate efficient market theory, statistical arbitrage is based on it. Statistical arbitrage seeks to profit from the correlation of price and liquidity by gaining from the perceived mispricing of assets based on the assets' anticipated value given by a statistical model.
  • 37. 16 Fundamentals of a High-Frequency Trading System Short-term price discrepancies in the same security sold on separate venues, or short- term price differences in related securities, are used in statistical arbitrage, often known as stat arb. Statistical arbitrage is based on the assumption that price differences in securities markets exist but go away quickly. Because the time period during which a price difference occurs might be as short as a fraction of a second, algorithmic trading is well suited to statistical arbitrage. When trading the same security in several venues, for example, an algorithm tracks all of the locations where the security is exchanged. When a price difference arises, the algorithm buys in the lower market and sells in the higher market, resulting in a profit. Because the window of opportunity for such differences is small (less than 1 millisecond), algorithmic trading is ideally suited to this form of trade. Statistical arbitrage becomes more challenging when investing in linked securities. An index and a single stock within that index, or a single stock and other stocks in the same sector, are examples of related securities. In linked securities, a statistical arbitrage approach entails gathering a large amount of historical data and estimating the usual connection between the two markets. The algorithm makes a buy or a sell whenever there is a variation from the norm. Latency arbitrage Modern equities markets are complicated, requiring highly technical systems to manage vast volumes of data. Because of its intricacy, data is invariably processed at varying speeds. Latency arbitrage takes advantage of market players' differing speeds. Latency arbitrage aims to take advantage of high-frequency traders' greater speed by leveraging high-speed fiber optics, superior bandwidth, co-located servers, and direct-price feeds from exchanges, among other things, to place trades ahead of other market players. The hypothesis behind latency arbitrage is that in the US, the aggregated feed that determines the National Best Bid and Offer (NBBO) of all US stock exchanges is slower than the direct data feeds from stock exchanges available to high-frequency traders. An HFT program's algorithm can read transaction data more quickly than many other market players, seeing prices a fraction of a second ahead of the Securities Information Processor (SIP) feed, which is the consolidated US stock exchange price feed, thanks to its superior speed. This essentially provides information to the HFT software before it reaches the official market (the SIP feed), allowing high-frequency traders to observe where prices are heading ahead of other market players.
  • 38. What do I need to start an HFT? 17 Impact of news Information is at the heart of all trading, and it is used to make financial decisions. The utilization of news data by algorithmic trading systems to generate trading choices is referred to as information-driven strategies. Algorithms have been developed to read and analyze news reports from major news organizations, as well as social media. Any news that has the potential to alter market prices causes the algorithm to purchase or sell. High-frequency traders have gotten so accustomed to using information-driven methods that certain news agencies now package their press releases in a way that makes it simple for computers to analyze them. They employ predetermined keywords to characterize a favorable or bad occurrence, for example, so that an algorithm can act on keywords in a news release. Prior to their planned publication, news providers also place news reports on servers in crucial geographical regions (such as major financial centers). This reduces the amount of time it takes for data to move from one location to another. For this sort of service, news service providers charge an additional fee. As seen by the hacking of the Associated Press Twitter feed, the use of social media for information-driven initiatives is growing. In 2013, a hacker tweeted that a bomb had gone off in the White House, injuring the president, causing an instantaneous plunge in equities markets throughout the world as algorithms analyzed the bad news from a trustworthy source and began selling in the market. Next, let's learn about the momentum ignition trading technique. Momentum ignition You have the chance to trade financially if an order you send into the market may cause a price change and you know it can. The goal of momentum-ignition trading techniques is to achieve this. The objective is to get other algorithms and traders to start trading in a stock, causing a price change. In essence, a momentum ignition approach attempts to deceive other market players into believing that a large price movement is going to occur, causing them to trade. As a result, the price movement becomes a self-fulfilling prophecy: traders believe a price movement will occur, and their activities cause one to occur.
  • 39. 18 Fundamentals of a High-Frequency Trading System Sending enormous volumes of orders into the order book and then canceling them is a momentum-ignition approach. This creates the illusion of a huge shift in volume in the stock, which might prompt other traders to place orders, resulting in the start of a short- term price trend. Before attempting to ignite the market movement, the momentum ignition approach includes executing the real targeted trading position. This means that a deal is completed initially that does not significantly influence the market. This permits a trader using the momentum-ignition approach to enter the market before the price movement is initiated. The momentum ignition is set after the deal is completed by submitting a flurry of orders and canceling them in the hopes that other traders will follow suit and move the price. The trader using the momentum-ignition technique then quits their initial position at a profit as the price begins to move. Momentum-ignition methods require the use of specific order types, and traders may only utilize algorithms that can send and cancel huge numbers of orders in a short amount of time to execute them. Rebate strategies Market order traders must pay a fee to the exchange, whereas the limit order is reimbursed with rebates when they add liquidities. As a result, traders, particularly those engaged in HFT, submit limit orders to build markets, which in turn generates liquidity on the exchange. It is undoubtedly appealing to traders who place a large number of limit orders due to the pricing scheme's lower risk for the limit order. There is also a charge structure called trader-maker pricing that is the polar opposite of market-taker pricing. In certain markets, it entails giving rebates to market order traders and collecting fees from limit order traders. Pinging Pinging is a strategy for learning about huge orders in trading exchanges and dark pools by placing tiny marketable orders (typically for 100 shares). To lessen the market effect of large orders, buy-side businesses utilize this trading technique to split large orders into many small orders. This algorithm feeds these orders slowly into the exchange. In order to detect the presence of such large orders, HFT companies arrange bids and offers in 100-share lots for each listed stock.
  • 40. What do I need to start an HFT? 19 These ping trades will alert HFT participants to the existence of a large order placed by the buy-side. HFTs will use this information to ensure risk-free profit from the buy-side. Some significant market participants have compared pinging to baiting because its main objective is to entice institutions with huge orders to expose their hand. Illegal activities The SEC, the Federal Bureau of Investigation (FBI), and other regulators have launched crackdowns on alleged HFT violations in recent years. The following sections are examples of possible offenses. Front-running Placing an order based on information that has not been publicly released is called front-running. This technique has been outlawed by SEC and the Financial Industry Regulatory Authority (FINRA). Some have used the term front-running to describe a technique in which HFT firms utilize algorithmic trading technology to identify a large number of new orders for a given instrument. Before the large number of orders comes to the market, we place orders to benefit from this incoming large quantity. HFT corporations can earn almost instantly after purchasing assets by selling them to the original investors. Even if this way of trading is legal, regulators are concerned and may need to control this behavior moving forward. Spoofing Spoofing is not a legal trading strategy. It consists of a spoofing strategy sending orders that are not intended to be executed, just to have the other market participants react to these orders. They will probably send orders to get to this price level. Meanwhile, the initial orders are canceled and the spoofer takes advantage of the other orders remaining in the market. The Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 specifically targeted the practice, and even before that, FINRA regulations barred orders whose goal is to mislead the market. The first criminal spoofing case disclosed by legislators in 2014 related to a Chicago trader accused of faking futures markets.
  • 41. 20 Fundamentals of a High-Frequency Trading System Layering Layering is the same as spoofing except that the orders are placed at different price levels to give the appearance that there is a lot of interest in a certain security. The outcome of this strategy is the same as with regular spoofing. Because of the rapid advancement of technology, massive market manipulation may take place in fractions of a second. Layering, like generic spoofing, is typically illegal and forbidden under FINRA rules. Even if these strategies are now outlawed, we need to keep in mind that some exchanges are less or not regulated. We will see in Chapter 11, High Frequency FPGA and Crypto, about cryptocurrencies that these strategies can still work. Summary In this chapter, we reviewed the origins of HFT. We went through what makes HFT so special in comparison to regular trading. We also layered the different types of strategies that any HFT trading system will be able to support. We talked about the history of trading systems. Our goal in this chapter was to give you a good understanding of what HFT is and what trading strategies we can use. In the next chapter, we will talk about the main functionalities of a trading system. We will describe how to build a trading system.
  • 42. 2 The Critical Components of a Trading System In the previous chapter, we learned how to create high-frequency trading (HFT) strategies. In this chapter, we are going to study how to convert these strategies into real-time software that will connect to an exchange to actually apply the theory that you've previously learned. We will describe the functioning of a trading system capable of trading assets. In this chapter we will cover the following topics: • Understanding the trading system • Making a trading system trade with exchanges • Order book management • Strategy making decisions on when to trade By the end of this chapter, you will be capable of designing a trading system, connecting a trading system to an exchange, and building a limit order book.
  • 43. 22 The Critical Components of a Trading System Understanding the trading system Designing a trading system for HFT trading requires much more than knowledge of programming and trading. The following chapters of this book will describe these parts in depth, which will give you an edge in designing an HFT system. In this section, we are going to talk about the fundamentals of trading system design. One of the most critical parts of designing a system is having a detailed description of the requirements. The goal of a trading system is to support your trading ideas. Any trading strategies start by getting data and end up with making a decision based on this data. A trading system will oversee collecting market data (that is, price updates) and sending orders to the exchange. Additionally, it will collect answers from the exchange containing information on the orders. These market updates could be representing any state of the orders: canceled, rejected, filled, or partially filled. It will also compute metrics measuring the performance of your portfolio (such as profit and loss, risk metrics, or information about the different processes of the trading system). When deciding whether to create this type of software, we need to keep the following points in mind: • Asset class: Knowing which asset class will be employed in the trading system will change the data structure of this software. Each asset class is unique and has its own set of characteristics. Building a trading system for US equities won't be the same as building a system for foreign exchange (FX). Stocks in the United States are mostly traded on two exchanges, the New York Stock Exchange (NYSE) and NASDAQ. These two exchanges have roughly 7,000 firms (symbols) listed. FX contains six main currency pairs, six minor currency pairs, and six exotic currency pairs, as opposed to stocks. We can add additional currency pairs, but no more than 100 will be available. Unlike the US equity market, where we can have two main exchanges, in the FX market, there will be hundreds of exchanges. The number of symbols and the number of exchanges will change the architecture of a trading system.
  • 44. Understanding the trading system 23 • Trading strategy type (high frequency, long-term position): The software architecture will be influenced by the trading strategy type. HFT tactics necessitate transmitting orders in a very short period of time. In the case of US stocks, a standard trading system will decide to send an order in microseconds. A Chicago Mercantile Exchange (CME) trading system has a latency in the order of nanoseconds. Based on this finding, technology will play a significant role in the software design process. If we only consider the programming language, Python is not well suited to speed, and we would rather choose C++ or Java. If we wish to take a long-term position, such as one that lasts many days, the speed with which a trader may obtain liquidity faster than others is irrelevant. • Number of users (or trading techniques): As the number of traders grows, so does the variety of trading tactics. This indicates that the number of orders will increase. Before submitting an order to an exchange, we must ensure that the orders we are about to send are valid; we must ensure that the overall position for a specific instrument has not been reached. Trading strategies are being moderated by an increasing number of rules in the trading sector. We shall test the compliance of the orders that we wish to send in order to ensure that our trading strategy complies with the regulations. All of these tests will increase the amount of time it takes to calculate. If we have a lot of orders, we'll have to conduct all of these verifications in sequence for one instrument. If the program isn't fast enough, the orders will take longer to process. The more users you have, the more scalable the trading system must be. These variables change the way you think about the trading system you're going to create. Let's discuss the design of a simple trading system in the following section.
  • 45. 24 The Critical Components of a Trading System Trading system architecture The following schema represents the trading system architecture. On the left part of this diagram, we can see the venues. Venues are a more generic term for any platform matching buy and sell orders for the securities and/or derivatives of multiple parties. In other words, a venue can be a trading exchange, an ECN, an aggregator, or a bank. Trading systems communicate with venues to collect price updates from all the participants and send orders. To do so, a trading system needs a piece of software called Gateways, which will ensure the communication between the trading system and the venues. The Book Builder will build the limit order book from the data collected from Gateways. Finally, the Strategy will send the order to the venues through the Order Manager. The Order Manager is responsible for collecting all the orders coming from the strategies of the system and keeps track of the life cycle of the orders. All of these components are a part of the critical path of sending orders to the market. Figure 2.1 – Trading system architecture design Additionally, we observe other less critical services, such as command and control, in charge of starting the components of the system. The viewers are critical in algorithmic trading because they will give you a status of all the components of the system, the orders and trades, and the metrics you consider important to monitor your trading strategies. Algorithmic trading automates trading. Therefore, it is important to keep track of the health of your trading system and trading strategies. It is particularly important to understand here that in HFT trading, a few microseconds can end up in colossal losses. Having viewers and user interfaces capable of efficiently reporting alerts is critical.
  • 46. Understanding the trading system 25 In the next section, we will talk more in-depth about the critical components of a trading system. Gateways connecting to trading exchanges Gateways are the components of a trading system communicating with the exchange and the trading system. They are essential because they are the greediest in terms of execution time. By design, they must get data from the network and provide this data to the rest of the system. This operation is demanding in terms of system resources and operating system. Price updates are collected by a trading system, which then transmits orders on your behalf. To do so, you must first code all the procedures you would do if you were trading without a trading system. If you want to make money by purchasing low and selling high, you must first decide what product you will trade. You should get the order from the other merchants after you have chosen these products. The other traders will inform you of their willingness to trade a financial asset by identifying the size, price, and quantity. You can pick the trader with whom you will negotiate a transaction once you have received enough orders for the product you wish to trade. The price of this item will influence your selection. If you plan to resell this item in the future, you will need to acquire it for a cheap price. When you reach an agreement on a price, tell the other trader that you want to buy at the listed price. You will then own this product when the transaction is completed. Data collection Gateways collect price updates from the trading venues you have chosen (exchanges, ECNs, and dark pools). This component (shown as a Gateway in the following figure) is one of the most important in the trading system. This component's job is to get the book for the instruments from the exchange into the trading system. This component will be connected to the network and will be able to communicate with it via exchanges receiving and sending streams.
  • 47. 26 The Critical Components of a Trading System The position of the trade system's gates is depicted in the following diagram. They are the trading system's inputs and outputs:. Figure 2.2 – Gateways in charge of collecting price updates and sending orders The following points are depicted in the preceding diagram: • Traders, exchanges, ECNs, and dark pools are represented by the venues. • Different protocols might be used to connect the venues (they are represented using arrows). • Wires, wireless networks, the internet, microwaves, and fibers are all options to transfer data. In terms of speed, data loss, and bandwidth, each of these network media has its own set of characteristics. • The arrows for price updates and orders are bidirectional because we can have data sent to/received from the venues. • To begin receiving pricing updates, the gateway will establish a network connection with the venue, verify itself, and subscribe to a certain instrument (we will explain this part in the next section).
  • 48. Making a trading system trade with exchanges 27 • The order-processing gateway also receives and sends communications. When an order is placed, it is forwarded to the venue over the network. • An acknowledgment of this order will be sent if the venue receives it. A message will be issued to the trading system after this order meets a matching order. If the venue does not receive an order, the acknowledgment will not be sent. It is up to the trading system to declare that an order has timed out. In this situation, a trader will need to intervene and check the problem that occurred in the system. Making a trading system trade with exchanges A trading system contains a number of functional components that are responsible for trading and risk management, as well as monitoring the trading process on one or more exchanges. A trading strategy becomes a part of the trading system once it is coded. As input, you'll need price data, and as output, your orders. This will give out trading signals. We'll need gateways to finish this flow because they're the most important components. The functional components of a trading system, the gateway's interface, and the trading system's interaction with the outside world are depicted in the following diagram: Figure 2.3 – The functional components of a trading system The gateways gather and send orders based on pricing and market reactions. Their primary function is to establish a link and translate data received from the outside world into the data structure needed by the trading system.
  • 49. 28 The Critical Components of a Trading System The following points are depicted in the preceding diagram: • This trading plan will be on your machine when you apply your trading strategy. The trade will take place on a different computer. • Because these two devices are located at different locations, they must connect over a network. • The methods of communication employed by the system might vary depending on its location. • A single wire will be utilized if the trading system is collocated (the machines are in the same facility), which will minimize network latency. • The internet might be another mode of communication if we adopt a cloud solution. The communication will be substantially slower than with a direct connection in this instance. Examine the following figure, which displays the communication that occurs between the gateways: Figure 2.4 – Communication between exchange and trading system
  • 50. Making a trading system trade with exchanges 29 We understand the following points from the preceding diagram: • When we examine the communication handled by the gateways more closely, we can see that the venues may use various protocols. • To transform protocols into trading system data structures, gateways will need to be able to process a variety of protocols. We learned about how a trading system connects to trading exchanges. We will now talk about how communication can happen and which protocol we are using for receiving market updates and sending orders. Examining the API for communication The rules of communication between machines are defined by network protocols. They specify how these devices will be identified on the network and how they will communicate with one another. The User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) over Internet Protocol (IP) protocols are used in trading finance. Furthermore, we employ a software protocol that specifies how to send an order and obtain a price update. At the software level, the communication API will establish communication rules. The communication API is provided by the entity with which you wish to transact. This document contains all of the messages you will need to get price updates and place orders. Examples of trading API documents can be found at https://en.wikipedia.org/ wiki/List_of_electronic_trading_protocols. Before we go into the trading API, let's go over some networking fundamentals. The fundamentals of networking The network is responsible for allowing computers to connect with one another. To share data, networks require a physical layer. For a network to achieve a certain level of speed, dependability, or even security, selecting the appropriate medium (communication layer) is crucial. We use the following terms in trade finance: • Wire: Electrical currents with a narrow bandwidth. • Fiber: More bandwidth. • Microwave: It's simple to set up and has a lot of bandwidth, but it's susceptible to storms.
  • 51. 30 The Critical Components of a Trading System Depending on the sort of trading technique you choose, the media will change. In the Open Systems Interconnection (OSI) model (developed in Chapter 5, Networking in Motion), selecting the appropriate medium is part of the first layer of the network. The physical layer is the name given to this layer. There are six more layers on top of this one that describe the sort of communication. Like most of the communication, finance is also using IP. This is part of the ISO model's network layer. This IP establishes the rules for network packet routing. The transport layer is the final layer we'll discuss. TCP and UDP are the two most well-known protocols in banking. These two procedures are diametrically opposed. TCP is a protocol that allows two machines to communicate with each other. All messages sent initially will be delivered first. UDP lacks a means for determining whether network packets were received by the network. All of the exchanges will use either TCP or UDP as their protocol. In Chapter 5, Networking in Motion, we will go deeper into the study of these protocols. Let's learn about order book management in the following section. Order book management The primary goal of data handling is to copy the limit order book from the venues into your trading system. The book builder will be in charge of gathering the pricing and categorizing them for your tactics in order to integrate all of the many books you obtain. The pricing changes are transformed by the gateway and then passed to the book builder, as shown in the following diagram. The book builder will use the books that the gateways have received from the venues, as well as gather and sort any pricing changes: Figure 2.5 – Book builder getting price updates from Gateway IN
  • 52. Order book management 31 In the following diagram, we use an example of an order book for a given financial product. The order book will contain two parts, one for the bids and one for the offers. For each part, we will store the orders represented by the venue, the volume, and the price. Every venue will send its own order books. The goal of the book builder is to create a book taking into account the three books coming from each venue. The data represented in this diagram is artificial: Figure 2.6 – Trading system building the book out of three different venues The following is depicted in the diagram: • You can see that there is an order for each row in these books. • For example, a trader on Venue 1's offer list is ready to purchase 1,000 shares for $1.21. On the other hand, there's a list of people who are eager to sell. • The offer (or ask) price will almost always be greater than the bid price. Indeed, it would be far too simple to profit if you could purchase for less than you could sell. • The book builder's job is to collect the three books from the three places that the gates have collected. The book builder organizes and sorts the three books. We have learned how the trading system gets price updates and how to build a limit order book. We will now explain in detail the different functionalities of the order book.
  • 53. 32 The Critical Components of a Trading System Order book considerations A limit order book collects all price updates (orders) and arranges them in a way that makes the trading strategy's work easier. Exchanges utilize the order book to keep track of bids and offers. When trading, we receive the book from the exchange in order to figure out an indication of the asset prices, which prices are the best, or just to get a sense of the market. We'll have to utilize the network to convey changes to the exchange book because the exchange is on another platform/server. We have two options for doing so: • The first way entails sending the entire book. Sending the full book is consuming in terms of time. Indeed, if we have large trading exchanges (such as NYSE or NASDAQ), there are millions of orders sent within one second. If each time the exchange receives a new order, the full book is sent, the network will be saturated and it will take too long to send price updates. • The second way will send a full snapshot of the book as described previously. Then, the exchange will send incremental updates. The book is a critical part of the trading system since it will provide the trading strategy with the information to decide when to send an order or not. An order book contains bids and orders that are presently on the exchange. When a price update is sent to our trading system, the other market participants receive the same update concurrently. All the other market participants can also decide to run after this price update. When the exchange receives many orders (explained in detail in Chapter 3, Understanding the Trading Exchange Dynamics), the orders received first will be executed first. That's why the book plays a large part in the latency and all the operations of the book must be optimized. For the life cycle of the orders, we will need to handle the following operations: • Insertion: An insertion is a book entry that adds a new order to the book. This should be a quick operation. Since we must sort the bids and the offers for any received price updates, the method and data structure we choose for this operation are crucial. To insert a new order, we'll need to use a data structure with an O(1) or O(log n) complexity. • An amendment/modification will use the order ID to seek the order in the book. This operation should have the same level of difficulty as the insertion. • Cancelation: Using the order ID, a cancelation allows an order to be withdrawn from the book.
  • 54. Order book management 33 The data structure chosen and the method connected with it have a significant impact on performance. If you're creating an HFT system, you'll need to make the appropriate decisions. The order book that we are implementing in HFT is called an order-based book. Since this is a critical component in the system, it is very important to consider the complexity of the execution of this book. An efficient data structure to model the order book must ensure the following: • Constant look-up, fast quantity update: An order book stores a lot of orders for one given instrument. Large exchanges can get millions of orders per second. Because we will have a growing number of orders in this book, it is important to keep a constant look-up time for order IDs. We will have to look up order IDs millions of times per second to update these orders. Additionally, we will need to retrieve orders with the best price rapidly. Looking up orders by price cannot be linear in terms of complexity. Therefore, we will use fast indexing (having a logarithmic time to find an order for a specific price). • Iteration in order of prices: When buying or selling large quantities, we may need to find many orders to reach a given volume. In this situation, we will start with the best price, then we will go to the second-best price, and we will keep on going like this. In this situation, it is also critical for the execution speed to reach the next best price with very low complexity. • Retrieving best bid and ask in constant time: Since we will mainly work with the best prices, we need to have a data structure capable of returning the best orders for bids and offers. We will need to take into account the following considerations: • Organize order identifiers to order information in a huge associative array (for C++, it could be a std::unordered map or std::vector). • The order metadata includes references to the order book and price-level it belongs to, therefore, after checking up the order, the order book and price-level data structures are only a single dereference away. When using an Order Execute or Order Reduce action, having a reference to the price allows for an O(1) decrease. You may preserve pointers to the next and previous orders in the queue if you wish to keep track of time priority as well.
  • 55. Random documents with unrelated content Scribd suggests to you:
  • 59. The Project Gutenberg eBook of The Story of Paul Jones: An Historical Romance
  • 60. This ebook is for the use of anyone anywhere in the United States and most other parts of the world at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this ebook or online at www.gutenberg.org. If you are not located in the United States, you will have to check the laws of the country where you are located before using this eBook. Title: The Story of Paul Jones: An Historical Romance Author: Alfred Henry Lewis Illustrator: Seymour M. Stone Phillipps Ward Release date: January 17, 2017 [eBook #53990] Most recently updated: October 23, 2024 Language: English Credits: Produced by David Widger from page images generously provided by the Internet Archive *** START OF THE PROJECT GUTENBERG EBOOK THE STORY OF PAUL JONES: AN HISTORICAL ROMANCE ***
  • 61. THE STORY OF PAUL JONES An Historical Romance
  • 62. By Alfred Henry Lewis Illustrated by Seymour M. Stone and Phillipps Ward G. W. Dillingham Company, 1906 London
  • 64. CONTENTS THE STORY OF PAUL JONES CHAPTER I—HIS BAPTISM OF THE SEA CHAPTER II—IN THE BLACK TRADE
  • 65. CHAPTER III—THE YELLOW JACK CHAPTER IV—THE KILLING OF MUNGO CHAPTER V—THE SAILOR TURNS PLANTER CHAPTER VI—THE FIRST BLOW IN VIRGINIA CHAPTER VII—THE BLAST OF WAR CHAPTER VIII—THE PLANTER TURNS LIEUTENANT CHAPTER IX—THE CRUISE OF THE “PROVIDENCE” CHAPTER X—THE COUNSEL OF CADWALADER CHAPTER XI—THE GOOD SHIP RANGER CHAPTER XII—HOW THE “RANGER” TOOK THE “DRAKE” CHAPTER XIII—THE DUCHESS OF CHARTRES CHAPTER XIV—THE SAILING OF THE “RICHARD” CHAPTER XV—THE “RICHARD” AND THE “SERAPIS” CHAPTER XVI—HOW THE BATTLE RAGED CHAPTER XVII—THE SURRENDER OF THE “SERAPIS” CHAPTER XVIII—DIPLOMACY AND THE DUTCH CHAPTER XIX—NOW FOR THE TRAITOR LANDAIS CHAPTER XX—AIMEE ADELE DE TELISON CHAPTER XXI—ANTONY AND CLEOPATRA CHAPTER XXII—THE FÊTE OF THE DUCHESS DE CHARTRES CHAPTER XXIII—THE WEDDING WITHOUT BELLS
  • 66. CHAPTER XXIV—THAT HONEYMOON SUB ROSA CHAPTER XXV—CATHERINE OF RUSSIA CHAPTER XXVI—AN ADMIRAL OF RUSSIA CHAPTER XXVII—THE HOUSE IN THE RUE TOURNON CHAPTER XXVIII—LOVE AND THOSE LAST DAYS
  • 67. THE STORY OF PAUL JONES
  • 68. T CHAPTER I—HIS BAPTISM OF THE SEA his is in the long-ago, or, to be exact, in July, 1759. The new brig Friendship, not a fortnight off the stocks, is lying in her home harbor of Whitehaven, being fitted to her first suit of sails. Captain Bennison is restlessly about her decks, overseeing those sea-tailors, the sail-makers, as they go forward with their task, when Mr. Younger, the owner, comes aboard. The latter gentleman is lowland Scotch, stout, middle-aged, and his severe expanse of smooth-shaven upper-lip tells of prudence, perseverance and Presbyterianism in even parts, as traits dominant of his character. “Dick,” says Mr. Younger, addressing Captain Bennison, “ye’ll have a gude brig; and mon! ye s’uld have a gude crew. There’ll be none of the last in Whitehaven, for what ones the agents showed me were the mere riff-raff of the sea. I’ll even go to Arbigland, and pick ye a crew among the fisher people.” “Arbigland!” repeats Captain Bennison, with a glow of approval. “The Arbigland men are the best sailor-folk that ever saw the Solway. Give me an Arbigland crew, James, and I’ll find ye the Rappahannock with the Friendship, within the month after she tears her anchor out o’ Whitehaven mud.” And so Mr. Younger goes over to Arbigland. It is a blowing July afternoon. An off-shore breeze, now freshening to a gale, tosses the Solway into choppy billows. Most of the inhabitants of Arbigland are down at the mouth of the little tide- water creek, that forms the harbor of the village, eagerly watching a small fishing yawl. The latter craft is beating up in the teeth of the gale, striving for the shelter of the creek.
  • 69. The crew of the yawl consists of but one, and him a lad of twelve. His right hand holds the tiller; with the left he slacks or hauls the sheets, and shifts the sail when he goes about. The yawl has just heeled over on the starboard tack, as Mr. Younger pushes in among the villagers that crowd the little quay. “They’ll no make it!” exclaims a fisherman, alluding to the boy and yawl; “they’ll be blawn oot t’ sea!” “Ay! they’ll make it sure enough,” declares another stoutly. “It’s little Jack Paul who’s conning her, and he’d bring the yawl in against a horrycane. She’s a gude boat, too—as quick on her feet as a dancing maister; and, as for beating to wind’ard, she’ll lay a point closer to the wind than a man has a right to ask of his lawful wedded wife. Ye’ll see; little Jack’ll bring her in.” “Who is he?” asks Mr. Younger of the last speaker; “who’s yon boy?” “He’s son to John Paul, gardener to the laird Craik.” “Sitha! son to Gardener Paul, quo’ you!” breaks in an old fish-wife who, with red arms folded beneath her coarse apron, stands watching the yawl with the others. “Now to my mind, he looks mair like the laird than I s’uld want my son to look, if I were wife to Gardener Paul.” “Shame for ye, Lucky!” cries the fisherman to whom she speaks. “Would ye cast doots on the lad’s mither, and only because the lad in his favoring makes ye think now and again on Maister Craik? Jeanny Paul, that was Jeanny Macduff, is well kenned to be as carefu’ a wife as ever cooked her man’s breakfast in Arbigland.” “Ye think so, Tam Bryce?” retorts the incorrigible Lucky. “Much ye s’uld know of the wives of Arbigland, and you to sea eleven months o’ the year! I tell ye, Jeanny came fro’ the Highlands; and it’ll be lang, I trow, since gude in shape of man or woman came oot o’ the Highlands.”
  • 70. “Guide your tongue, Lucky!” remonstrates the other, in a low tone; “guide your tongue, ye jade! Here comes Gardener Paul himsel’.‘’ “I’ll no stay to meet him,” says Lucky, moving away. “Puir blinded fule! not to see what all Arbigland, ay! and all Kirkbean Parish, too, for that matter, has seen the twal years, that his boy Jack is no mair no less than just the laird’s bairn when all’s said.” “Ye’ll no mind her, Maister Younger,” says Tom Bryce, pointing after Lucky; “although, to be preceese, what the carline tells has in it mair of truth than poetry.” “I was no thinking on the dame’s clack,” returns Mr. Younger, his eyes still on the nearing yawl, “or whether yon lad’s a gardener’s bairn or a gentleman’s by-blaw. What I will say, in the face of the sun, however, is that he has in him the rudiments of as brisk a sailorman as ever walked saut water.” “There’ll be none that’s better,” observes Tom Bryce, “going in and oot o’ Solway Firth.” Then, eyeing the yawl: “He’ll win to the creek’s mouth on the next reach to sta’board.” Gardener Paul joins Mr. Younger and the fisherman, Tom Bryce. “We were talking of your son,” says Mr. Younger to Gardener Paul. “What say ye, mon; will ye apprentice him? I’ll send him with Dick Bennison, in my new brig Friendship, to the Virginias and Jamaica.” John Paul, gardener to the laird, Robert Craik, is a dull man, notably thick of wit, and slow. “The Virginias!” he repeats. “My son William has been there these sixteen year. He’s head man for my kinsman Jones, on his plantation by the Rappahannock. If Jack sails with Dick Bennison, he’ll meet William that he’s never seen.” “He’ll see his brother for sure,” returns Mr. Younger. “The Friendship goes from Whitehaven to Urbana, and that’s not a dozen miles down the Rappahannock from your cousin’s plantation.”
  • 71. The yawl has come safely into the creek’s mouth, and lies rocking at her moorings as lightly as a gull. The lad leaps ashore, and is patted on the back by the fisherman in praise of his seamanship. He smiles through the salt water that drips from his face; for beating to windward is not the driest point of sailing, and the lad is spray- soaked from head to heel. “And may I go, father? “This is Mr. Younger, Jack,” says Gardener Paul, as the lad conies up. “He wants ye to sail ‘prentice with Dick Bennison, in the new brig.‘’ The difference to show between Gardener Paul and little Jack Paul, as the pair stand together on the quay, goes far to justify those innuendoes of the scandalous Lucky. Gardener Paul’s heavy peasant face possesses nothing to mark, on his part, any blood-nearness to the boy, whose olive skin, large brown eyes, clean profile and dark hair like silk, speak only of the patrician. “And may I go, father?” asks Jack, a flush breaking eagerly through the tan on his cheek.
  • 72. “Ye might as weel, I think,” responds Gardener Paul judgmatically. “Ye’re the born petrel; and for the matter of gardening, being my own and Adam’s trade, I’ve kenned for lang ye’ll no mair touch spade or mattock than handle coals of fire. So, as I was saying, ye might as weel sail ‘prentice with Dick; and when ye meet your brother William, gi’ him his father’s gude word. Ye’ll never have seen William, Jack, for he left hame before ye were born; and so it’ll be a braw fore-gathering between the twa of ye—being brothers that never met before.” And after this fashion the fisher-boy, John Panl, afterward Admiral Paul Jones, is given his baptism of the sea.
  • 74. T CHAPTER II—IN THE BLACK TRADE he sun is struggling through the dust-coated, cobwebbed windows, and lighting dimly yet sufficiently the dingy office of Shipowner Younger of Whitehaven. That substantial man is sitting at his desk, eyes fixed upon the bristle of upstanding masts which sprout, thick as forest pines on a hillside, from the harbor basin below. The face of Shipowner Younger has been given the seasoning of several years, since he went to Arbigland that squall- torn afternoon, to pick up a crew for Dick Bennison. Also, Shipowner Younger shines with a new expression of high yet retiring complacency. The expression is one awful and fascinating to the clerk, who sits at the far end of the room. Shipowner Younger has been elected to Parliament, and his awful complacency is that elevation’s visible sign. The knowledge of his master’s election offers the basis of much of the clerk’s awe, and that stipendary almost charms himself into the delusion that he sees a halo about the bald pate of Shipowner Younger. The latter brings the spellbound clerk from his trance of fascination, by wheeling upon him. “Did ye send doon, mon,” he cries, “to my wharf, with word for young Jack Paul to come?” The clerk says that he did. “Then ye can go seek your denner.” The clerk, acting on this permission, scrambles to his fascinated feet. As he retires through the one door, young Jack Paul enters. The brown-faced boy of the Arbigland yawl has grown to be a brisk young sailor, taut and natty. He shakes the hand of Shipowner Younger, who gives him two fingers in that manner of condescending reserve, which he conceives to be due his dignity as a member of
  • 75. the House of Commons. Having done so much for his dignity, Shipowner Younger relaxes. “Have a chair, lad!” he says. “Bring her here where we can chat.” The natty Jack Paul brings the clerk’s chair, as being the only one in the room other than that occupied by Shipowner Younger. One sees the thorough-paced sailor in the very motions of him; for his step is quick, catlike and sure, and there is just the specter of a roll in his walk, as though the heaving swell of the ocean still abides in his heels. When he has placed the chair, so as to bring himself and Shipowner Younger face to face, he says: “And now, sir, what are your commands’?” “I’ll have sent for ye, Jack,” begins Shipowner Younger, portentously lengthening the while his shaven upper-lip—“I’ll have sent for ye, for three several matters: To pay ye a compliment or twa; to gi’ ye a gude lecture; an’ lastly to do a trifle of business wi’ ye, by way of rounding off. For I hold,” goes on Shipowner Younger, in an admonishing tone, “that conversations which don’t carry a trifle of business are no mair than just the crackle of thorns under a pot. Ye’ll ken I’m rich, Jack—ye’ll ken I can clink my gold, an’ count my gold, an’ keep my gold wi’ the warmest mon in Whitehaven?” Young Jack Paul smiles, and nods his full agreement. “But ye’ll no ken,” goes on Shipowner Younger, with proud humility, the pride being real and the humility imitated—“ye’ll no ken, I believe, that I’m ‘lected to the Parleyment in Lunnon, lad?” Shipowner Younger pauses to observe the effect of this announcement of his greatness. Being satisfied, he goes on. “It’s a sacrifeece, no doot, but I s’all make it. The King has need of my counsel; an’, God save him! he s’all have it. For I’ve always said, lad, that a mon’s first debt is to the King. But it’ll mean sore changes, Jack, sore changes will it mean; for I’m to sell up my ships to the last ship’s gig of ‘em, the better to leave me hand-free and head-free to serve the King.”
  • 76. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com