PREDICTING STOCK PRICE MOVEMENTS BASED ON NEWSPAPER ARTICLES USING A NOVEL DEEP LEARNING APPROACH - Big Data Expo 2019
1 like157 views
This document summarizes a novel deep learning approach to predict stock price movements based on newspaper articles. It uses a combination of convolutional and recurrent neural networks, including word embedding, convolutional layers, and long short-term memory (LSTM). The approach achieves an accuracy of 54.82% on a dataset containing news articles and corresponding stock prices. Future work is proposed to improve the model by addressing noise in the dataset and comparing results to other approaches in literature.
PREDICTING STOCK PRICE MOVEMENTS BASED ON NEWSPAPER ARTICLES USING A NOVEL DEEP LEARNING APPROACH - Big Data Expo 2019
- 1. Predicting stock price movements based on newspaper articles using a novel deep learning approach Today Tomorrow 100110101 010101010 100110100
- 2. © 2018 Deloitte The Netherlands • Intuitive explanation of important concepts • Accessible for non-techies • Discussion of results • No code • No mathematics • Not comprehensive You will get to know what deep learning algorithms are and understand why they work Today
- 3. ‘Language is probably the hardest problem in science; nobody really knows how it works, nobody really knows where it came from and yet we can all do it. Michael Corballis Emeritus Professor at the University of Auckland TedX talk – The Origins and Evolution of Language: https://www.youtube.com/watch?v=nd5cklw6d6Q&t=95s
- 5. © 2018 Deloitte The Netherlands Machine Learning Field of study that gives the ability to the computer to self-learn without being explicitly programmed (Input to machine learning models needs to be numeric!) Deep learning Computational models that are composed of multiple processing layers that allow to learn representations with multiple levels of abstraction. Natural Language Processing (NLP) A branch of artificial intelligence that helps computers understand, interpret and manipulate human language. Natural Language for Financial Forecasting (NLFF) The domain that uses NLP approaches for financial forecasting. Definitions First steps in the world of artificial intelligence are taken by knowing its key concepts
- 6. Background domain The ‘Natural Language for Financial Forecasting’ domain has been growing rapidly in the last decade Number of ‘NLFF’ publications in scientific journals
- 7. Problem While stock price movements are known to be mostly influenced by news updates, most financial companies only include stock price information in their predictive models. By not including news updates, a huge opportunity is missed
- 8. © 2018 Deloitte The Netherlands ▪ Data preprocessing ▪ Explanation approach ➢ Word2Vec word embedding ➢ Convolutional neural network ➢ Recurrent neural network ➢ Sigmoid classification • Results • Future research Agenda
- 9. Type of problem Predicting stock price movements based on newspaper articles amounts to classifying each article as being a positive/negative article
- 10. Approach The approach, that combines convolutional- and recurrent layers, originates from the NLP domain and has not been applied to the NLFF domain yet Start: T articles Transforming articles into numerical representations Deep learning approaches to learn the algorithm Final: Classification
- 11. Data preprocessing Data from two separate sources were used to create a dataframe that contains both textual- as well as stock information Stock prices Source: Yahoo Finance News articles Source: Lexis Nexis Academic
- 12. Embedding layer Through a word embedding, textual data is represented numerically
- 13. Word2vec embedding A statistical method for efficiently learning a standalone word embedding from a collection of written texts Intuitively: ‘King’ – ‘Man’ + ‘Woman’, yields a vector similar to ‘Queen’ Large -0.5117 0.9561 0.3594 0.5076 -0.7155 -0.6644 -0.6312 0.4146 0.1242 … US -0.4742 -0.2551 0.6133 0.7608 -0.9923 -0.5035 0.894 0.815 -0.4761 … Technology 0.6217 -0.9585 0.735 0.3087 0.0678 0.6126 0.9418 0.2878 0.4421 … Companies 0.2931 0.6469 -0.0702 -0.7557 -0.3274 -0.7929 0.4023 0.9049 -0.6831 … are -0.1175 -0.7925 0.1921 0.9312 -0.5084 0.4869 0.5165 0.1922 0.0133 … investing -0.8552 -0.2882 0.3138 -0.8883 0.9069 -0.7656 0.1857 -0.6377 -0.7069 … Example 300 columns Concept: “show me your friends, and I’ll tell who you are.”
- 14. Convolutional layer Convolutional neural networks perform well at recognizing objects/patterns
- 15. Intuition: Convolutional Neural Network (CNN) CNNs learn to recognize features at different abstraction levels Face Hand Eyes Nose Mouth Ear Human Face 1 layer 1 layer
- 16. Convolutional Neural Network applied on word embeddings A filter (A) recognizes a pattern in the text (B). The convolutional output (C) represent the text in terms of a feature 0.46 0.51 0.18 0.43 0.53
- 17. Recurrent layer RNNs perform well at capturing long distant information (memory)
- 18. Intuition: Recurrent layer – Long Short Term Memory (LSTM) To make decisions, a LSTM uses information from both the near as well as distant past I start off with a broad knowledge base Not of all of my knowledge is relevant so I forget some Knowledge Me preparing for the Big Data Expo Knowledge New, relevant information is added to my knowledge My presentation combines both prior- as well as new knowledgeExpo Knowledge p=(y|X) p=(y|X)
- 19. Long Short Term Memory – Recurrent Neural Network The cell state contains the current knowledge, to and from which information is added and removed Article input (from convolutional layer) Cell state Forget gate Input gate Candidate values Output gate Current knowledge (Cell state) 1. What can be forgotten from cell state? (Forget gate) New information 2. What new information can be added to the cell state? (Candidate values) 3. How much of the new information should be updated? (Input gate) Output article 4. What information needs to be outputted per article? (Output gate)
- 20. Sigmoid classification The output of a Sigmoid classifier is always a value between 0 and 1
- 21. Results Combining a CNN and RNN, yields the best performance Full model CNN RNN Sentiment analysis Accuracy 56.25% 55.54% 54.82% 52.54% Accuracy is calculated by dividing the total number of true predictions, by the total number of predictions
- 22. Example dataset The current dataset contains a lot of noise Company: Alphabet Article: ‘Eric Schmidt is executive chairman of Alphabet. not chief executive as incorrectly stated in a column on November 23.’ Company: Tesla Article: ‘Mars doesn't have an extradition treaty with the US. - Jim Chanos discussing Tesla with CNBC’ Company: Verizon Article: ‘China surpassed the US to become the top recipient of foreign direct investment in 2014. The inflow to the US fell by 60 per cent. primarily because of the Verizon pullout by Vodafone. Five of the top 10 FDI recipients are developing markets.’ Company: Intel & Tesla Article: ‘Microsoft and Intel's evolution from the PC to the data centre is proving painful. Uber has settled a key class-action lawsuit. Tesla's chief has an idea for public transport. #techFT is a daily newsletter on technology. media and telecoms. You can sign up here.’ Company: Netflix Article: ‘Whether reaching millennial consumers who want to escape marketing messages. or 'cordcutting’ television viewers. who ditch cable and satellite subscriptions in favour of ad-free Netflix. advertisers are having to work harder than ever to find their audience. Read the report’
- 23. Results literature The results are similar to the results obtained in the literature Accuracy Dataset Approach Pang et al, 2018 53.2% Stock data LSTM Matsubara et al, 2018 59.0% News articles Deep neural generative model Huynh at al, 2017 59.98% News articles Combination of LSTM and GRU Rumelhart et al, 2017 64.74% News articles RNN and self-trained word embedding Selvin et al, 2017 55.9% Stock data LSTM and CNN
- 24. Future research The current approach can be enhanced in multiple ways
- 25. Questions
- 26. Contact information Please contact me for further inquiries Emil Rijcken Email: emil@cwi.nl Linkedin: https://www.linkedin.com/in/emilrijcken/ Mobile: 06 53137886
- 27. © 2018 Deloitte The Netherlands Appendix Appendix
- 28. Problem complexity Checking all solutions is unfeasible Assuming: - 2000 dimensions (e.g. there are 16 convolutional layers with 128 filters each) - 10 options per dimension - 31 860 000 000 000 000 calculations per second (fastest computer in the world) Then: it takes approximately 1.99 x 101979 years (!) to calculate all possible solutions
- 29. Finding solution Through trial and error, different solutions are proposed. The slope of the solution determines how parameters are set Example of solution space in 3D-space 1 0