Machine learning for trading
- 1. Machine Learning for Trading Bharath Rao, Alphamatters Advisors
- 2. Is Trading Profitable? • Is Gambling a Good Habit? I’m afraid that Depends on who you are • Easy to be fooled by the randomness of markets but hard to keep getting lucky • It’s a zero sum game • Transaction Costs make the expected payoff negative
- 3. So why are we here then? • Haar Kar Jeetne Wale Ko Baazigar Kehte hain! • I believe its possible to use math to do a one up on markets • Or am I fooled by randomness?
- 4. Is Machine Learning Relevant to Trading? • It’s relevant wherever there is lots of data – – Trading – Investing – Marketing – Drug Discovery – NLP – Image Processing • The challenge is financial markets are different from every other domain above
- 5. How are ‘markets’ different from ‘marketing’? • Markets have time series data that’s not stationary • Marketing data (consumer behavior) is largely stationary • Geometric Brownian Motion makes markets extremely difficult to model • You may find patterns but they may not repeat
- 6. What is ML to start with? • Optimization – Classification – Regression
- 7. How to use ML in trading? • Compute things like trading rage • Classify Market Condition as to calm or turbulent • Predict Returns • Model Volatility • Predict Reversals
- 8. What Techniques Work • SVM • Neural Net • Boosting • Random Forest • Linear Regression • Deep Learning?
- 9. What features? • What are features? • What can be useful features • Prices • Volumes • OIs • Volatility Measures • Premium / Discount • Fundamentals
- 10. Steps Involved • Understand the Driving Process: See if the data can be modeled using known models • Feature Selection: Identify potential predictors (technical / fundamental) • Feature Rationalization: Prune the feature set by performing PCA on features • Data Transformation: Normalize the data and Transform it if necessary • Data Split: Divide data into in sample of out of sample • Modeling: Build the ML model using in sample data; Test the same on OOS data • Repeat the previous 2 steps about 30 times and tabulate the performance metrics
- 11. How do we Transform Data? • Technical Indicators? • Signal Processing? • Time Series Models?
- 12. Real Life Example • Finding potential Reversals in a day and Using it for Intraday Trading – Volume spikes turn out to be predictors – Estimating the Trading Range is useful – Technical Oscillators are useful as a qualifier
- 13. How to estimate the trading range? • It’s correlated to volatility • It’s correlated to volume • It’s correlated to the average true range of the last few days
- 15. What else can ML be used for? • Fundamentals based investing • Ranking Stocks / other assets by factors • Option Arbitrage by Learning Greeks
- 16. Things to keep in mind while using ML • Have an idea of the underlying stochastic process • Have a large sample size relative to feature set • Perform as many out of sample tests as possible • Avoid snooping (especially while splitting) • Keep it simple and stupid (Occam’s Razor)
- 17. Things to keep in mind contd. • Enquire as to why the hypotheses arrived at by the ML algorithm should work at all • Avoid using it for HFT (unless you have super high performance hardware) • Avoid using it in high kurtosis strategies (like classical trend following) • Diversify, Diversify, Diversify……
- 18. Problems in using ML • Overfitting • Black Box Decision Making • Irrepeatable data patterns • Complex hypotheses • Hard to diagnose failure • Change in market microstructure can make historical data invalid • Impossible to model Black Swans using ML
- 19. Ho do I use ML • Make sure I’m not dealing with geometric brownian motion using statistical tests • Empirically Come up with skeletons of hypotheses and statistically validate the same • Use ML to sharpen the hypotheses and improve performance metrics
- 21. ML Metrics • Classification: Confusion Matrix – Precision – Recall • Regression: Root Mean Squared Error
- 22. Trading Metrics • Annualized Returns • Max Drawdown • Sharpe • Volatility of Returns • Beta • Win Ratio • Profit per Trade • Rolling returns on weekly, monthly, quarterly basis • Win Ratio on weekly, monthly, quarterly basis • Skew • Kurtosis • VaR
- 23. Some Gambling Concepts • Gambler’s Fallacy • Martingale • Kelley’s Criterion • Gambler’s Ruin • Recommendations Business • The 8th Wonder (Compounding)
- 24. Alternatives
- 25. Moral of the Story • Ezekiel 25 17 – The path of the righteous man is beset on all sides by the inequities of the selfish (NSE / CME, Bharat Mata / Uncle Sam….), and the tyranny of evil men (HFT Firms). Blessed is he (Algoji) who in the name of charity and good will shepherds the weak through the valley of darkness. For truly he is his brothers’ keeper and….
- 26. Thank You!