Parallel Processing of Big Data in Finance for Alpha Generation and Risk Management
- 1. An Approach to Parallel Processing of Big Data in Finance for Alpha Generation and Risk Management Yigal Jhirad March 26, 2014 NVIDIA GTC 2014: Deep Learning and AI Conference Silicon Valley
- 2. 2 GTC 2014: Table of Contents I. The Need For Speed — Portfolio and Risk Management: Big Data/Real Time Sensitivity — Applications - Cluster Analysis, Optimization II. Parallel Implementation — Coherency & Correlation: Cluster Kernel — Evolutionary Algorithm: Optimization Kernel III. Summary IV. Author Biographies DISCLAIMER: This presentation is for information purposes only. The presenter accepts no liability for the content of this presentation, or for the consequences of any actions taken on the basis of the information provided. Although the information in this presentation is considered to be accurate, this is not a representation that it is complete or should be relied upon as a sole resource, as the information contained herein is subject to change.
- 3. 3 GTC 2014: The Need For Speed Portfolio/Risk Management analytics demand increasing amounts of computing speed — Big Data: Time Series Data, Interday and Intraday — Capture Trends, Patterns, and Signals — Coherency and Group membership — Alpha Generation, Risk Management, Market Impact Parallel Processing: APL/CPU and CUDA/GPU physics based modeling exploit hardware efficiently — Array Based Processing paradigm Matrix/Vector thought process is key — APL is a programming language whose quantum data object is an array, which is fundamental to parallel processing and can leverage parallel processing across CPU’s — CUDA leverages GPU Hardware Application in Econometrics and Applied Mathematics — Monte Carlo Simulations — Fourier Analysis — Principal Components — Optimization — Cluster Analysis — Cointegration Neural Networks — Rapid application development and testing of idea thesis and innovation
- 4. 4 Cluster Analysis Cluster Analysis: A multivariate technique designed to identify relationships and cohesion — Factor Analysis, Risk Model Development Correlation Analysis: Pairwise analysis of data across assets. Each pairwise comparison can be run in parallel. — Use Correlation as primary input to cluster analysis — Apply proprietary signal filter to remove selected data and reduce spurious correlations
- 5. 5 Cluster Analysis: Correlation Application example: Correlation function removing N/A values Correlation measures the direction and strength of a linear relationship between variables. The Pearson product moment correlation between two variables X and Y is calculated as: For N assets there are unique correlation pairs Given an N x M matrix A in which each row is a list of returns for a particular equity, return an N x N matrix R in which each element is the scalar result of correlating each row of A to every other — Each element of A may be an N/A value — When processing a pair of rows, the calculation must include neither each N/A value nor the corresponding element in the other row. This requires evaluating each pair separately. — As a result the increased computational demand is more effectively implemented through a parallel processing solution. As the matrix size increases the benefits of parallel processing become more significant.
- 6. 6 Optimization: Mixed Integer Optimization Multi-Phase Optimization Analysis — Identify a target portfolio of stocks that is trending consistently over consecutive periods using specialized, possibly time-sensitive, optimization algorithms — Establish a portfolio of stocks that is performing in a cohesive way Identifying and Assessing factors driving outperformance — Optimize a basket of factors to track target portfolio — Look at factors such as Value vs. Growth, Large Cap vs. Small Cap Optimized Factor Attribution of Targeted Portfolio Relative Ranking Cash/Assets 1 Capex/Assets 2 Dividend Yield 3 Dividend Growth (1 and 5 Year) 4 Market Cap (High - Low) 5 Dividend Payout 6 ROIC 7 E/P 8 Indicative factor attribution of target portfolio Period: 2nd Half of 2013
- 7. 7 Mixed Integer Optimization: Coherency and Pattern Identification using Monte Carlo Application example: Maximize the number of Runs of Outperformance max 𝑤 𝑖 𝑘 𝑠. 𝑡. (ς 𝑡=𝑞 𝑞−𝑘 1 (𝑅 𝑝 𝑡 > 𝑅𝐼 𝑡 ) )> 0 where 𝑅 𝑝 𝑡 =σ 𝑤𝑖 × 𝑟𝑖 𝑡 Given a set of assets and periodic prices for each asset, identify sets of long and short weights for those assets that result in portfolios that maximize the number of consecutive positive incremental returns through the most recent period Load matrix of price indexes (assets x periods) into global memory Run independent simulations — Generate random long and short weights for a subset of assets — Evaluate the portfolio and count the outperforming final periods — Store the final count and corresponding random seed in global memory Sort the final counts to identify the best performers Reconstitute the weights for the best performers
- 8. 8 Mixed Integer Optimization: Configuration Hardware Constraints — Compute capability: 1.3 — Max threads per multiprocessor: 1024 — Max blocks per multiprocessor: 8 — Number of shared memory banks: 16 — Coalescence capacity (4-byte words): 64 bytes = 16 contiguous words — Number of streaming multiprocessors: 30 Software Response – Portfolio Evaluation — Run 16 independent simulations per block for output coalescence — Read 16 price indexes per simulation for input coalescence — Thread block configuration: 16 x 16 = 256 threads — Max blocks per multiprocessor: 1 - 4, depending on occupancy — Max contemporaneous blocks: 30 - 120, depending on occupancy — For sort, maximize number of blocks: 8 => 128 threads per block Tesla C1060
- 9. Add width of padding to starting address Process right to left Mixed Integer Optimization: Price Indexes Input Matrix 9
- 10. Width = max # of longs/shorts (area not necessarily rectangular) ● Number of staging areas = # of multiprocessors on card (multiProcessorCount) * max # of blocks per multiprocessor (4 or less, depending on occupancy) -- Creates enough staging areas to accommodate all blocks that may execute simultaneously ● Each staging area holds data for 16 simulations ● Each cell represents 16 * 4-byte floats or integers (one per simulation) ● Each area populated in three loops: long weights one at a time, short weights one at a time, both asset indexes one asset at a time ● Accompanied by an array of 16 * 4-byte integers = 512 bits, each indicating whether the corresponding staging area is currently available (atomicExch with 0 to capture array and mark all areas “not free” while searching, identify and flip lowest “true” bit, atomicOr array back in) Global Memory Temporary storage for weights generated, used and discarded during the life of a block Mixed Integer Optimization: Weights Staging Area 10
- 11. 16 simulations16 simulations 16weights Mixed Integer Optimization: Weights Staging Area Population 11
- 12. Asset Indexes How not to do it Mixed Integer Optimization: Weights Staging Area Population 12
- 13. Mixed Integer Optimization: Weights Staging Area Populated Long Values, same for shorts 13
- 14. Loop by row of staging area: one asset at a time 1assetpersimulation periods periods Mixed Integer Optimization: Portfolio Evaluation periods simulations 14
- 15. Process right to left 16 periods Mixed Integer Optimization: Overall Portfolio Evaluation 16 simulations 16periods 15
- 16. Mixed Integer Optimization: Sort and Store Results 16
- 17. Pairwise merge two 128 element blocks at a time Mixed Integer Optimization: Merge-Sort Results 17
- 18. Once it has written its final row, each block will atomicSwap the value 1 to the first Merge cell The first block to finish finds a 0 in that cell, meaning its counterpart block has not yet written its final row, so it quits The second block to finish finds a 1 that cell, meaning its counterpart block has already written its final row, so it is the one that can continue on to perform the next level of sort Mixed Integer Optimization: Final Merge-Sort 18
- 19. 19 Summary Parallel capabilities in array processing provide a powerful framework for analyzing large amounts of financial time series data — The need for speed calls for broad Multi-CPU and GPU solutions Our approach is based on time-sensitive implementation of Cluster Analysis and Optimization Analysis — Capture Trends, Patterns and Signals — Coherency and Group membership Application in Portfolio Management/Alpha Generation/Trading and Execution — Need to have not nice to have
- 20. 20 Author Biographies Yigal D. Jhirad, Senior Vice President, is Director of Quantitative Strategies and a Portfolio Manager for Cohen & Steers’ options and real assets strategies. Mr. Jhirad heads the firm’s Investment Risk Committee. He has 26 years of experience. Prior to joining the firm in 2007, Mr. Jhirad was an executive director in the institutional equities division of Morgan Stanley, where he headed the company’s portfolio and derivatives strategies effort. He was responsible for developing, implementing and marketing quantitative and derivatives products to a broad array of institutional clients, including hedge funds, active and passive funds, pension funds and endowments. Mr. Jhirad holds a BS from the Wharton School. He is a Financial Risk Manager (FRM), as Certified by the Global Association of Risk Professionals.