Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas
Download as PPT, PDF3 likes2,258 views
The document discusses the evolution of algorithmic trading and the significance of financial data mining, particularly emphasizing the role of XBRL in enhancing transparency and efficiency in trading strategies. It outlines the transition to low-touch trading methods and highlights various data mining approaches, including mathematical-based and logic-based methods for trading signal generation. Additionally, the document presents insights on market regulations and the international readiness for algorithmic trading integration.
Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas
- 1. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 2. Copyright ® 2009, SAS Institute Inc. All right s reserved. Financial Data Mining with Algorithmic Trading Robert Golan DBmind Technologies, Inc. Please Note: This is the view of DBmind only which may not pertain to DBmind’s Client Views
- 3. Copyright ® 2009, SAS Institute Inc. All right s reserved. Financial Data Mining with Algorithmic Trading Algorithmic Trading has changed the world the way the Traders trade and Trade Support supports. There is a Brave New World happening with the "hands on" Trading evolving into "hands off" Algo Trading. Not all trades need to be made in ultra low latency timing. Future trading will rely on a broader set of data which will be mined for relevance. An important series of XBRL Financial Reporting events are happening throughout the world and especially in the USA. A critical mass of financial data will be ready for mining which will be a boon for transparent "low touch" fundamental style algorithmic trading. Also, "low touch" trading such as program trading & direct market access (DMA) will evolve into advanced Algo Trading strategies. Stock and economic indicators combined with XBRL will add value for Algo Trading. This is about a well thought out strategic high latency trading strategy with data mining discovering the governing rules while adding the expert rules with validation. Yes, the trader is still the key to making this all happen. Both fundamental and technical trading rules need to be combined with the expert rules, the data mined rules, and most importantly the regulatory environment rules. RegNMS in the USA and MiFID in Europe have indirectly helped the adoption of electronic trading and it is important to integrate the GRC related rules in an agile way. Agility is the key and thus the rules need to be placed into a rules engine and managed by the experts for proper compliance, risk management, and governance. Japan, China, and the Netherlands with regards to XBRL are ready to be data mined with Algo Trading now. A XBRL US survey is indicating at least 340 of the estimated 500 public companies that the SEC requires to begin filing in XBRL format in June 2009, have already converted their financial statements into XBRL. XBRL US, is the non-profit XML standard setter that developed and maintains the US GAAP taxonomy used by filers to comply with the SEC mandate. Almost $7 trillion in market capitalization will be represented by this XBRL financial data which is over 50% of the total market cap for all publicly traded companies reporting to the SEC. As this XBRL Financial Data ripens, a wonderful harvest awaits us data miners which will enhance the current Algo Trading strategies which use this data.
- 4. Copyright ® 2009, SAS Institute Inc. All right s reserved. Algorithmic trading http://en.wikipedia.org/wiki/Algorithmic_trading • In electronic financial markets, algorithmic trading, also known as algo, automated, black box, or robo trading, is the use of computer programs for entering trading orders with the computer algorithm deciding on certain aspects of the order such as the timing, price, or type (market vs. limit, or buy vs. sell) of the order. It is widely used by pension funds, mutual funds, and other institutional traders to divide up a large trade into several smaller trades in order to avoid market impact costs or otherwise reduce transaction costs. It is also used by hedge funds and similar traders to make the decision to initiate orders based on information that is received electronically, before human traders are even aware of the information. • Algorithmic trading may be used in any market strategy, including market making, intermarket spreading, arbitrage, or pure speculation (including trend following) to make the complete decision on entering trades and electronically executing the trade with no human intervention, other than in writing the computer program. • In 2006 at the London Stock Exchange, over 40% of all orders were entered by algo traders, with 60% predicted for 2007. American markets and equity markets generally have a higher proportion of algo trades than other markets, and estimates for 2008 range as high as an 80% proportion in some markets. Foreign exchange markets also have active algo trading (about 25% of orders in 2006). Futures and options markets are considered to be fairly easily integrated into algorithmic trading, and bond markets are moving toward more access to algorithmic traders.
- 5. Copyright ® 2009, SAS Institute Inc. All right s reserved. Citadel Investment Group Interactive Brokers Credit Suisse Deutsche Bank Goldman Sachs Lehman Bros. Morgan Stanley Susquehanna Investment Group UBS Some of the Algo Players
- 6. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 7. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 8. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 9. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 10. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 11. Copyright ® 2009, SAS Institute Inc. All right s reserved. Statistical Arbitrage in the U.S. Equities Market Marco Avellaneda† and Jeong-Hyun Lee First draft: July 11, 2008 This version: June 15, 2009 Abstract We study model-driven statistical arbitrage in U.S. equities. The trading signals are generated in two ways: using Principal Component Analysis and using sector ETFs. In both cases, we consider the residuals, or idiosyncratic components of stock returns, and model them as mean-reverting processes. This leads naturally to “contrarian” trading signals.The main contribution of the paper is the construction, back-testing and comparison of market-neutral PCA- and ETF- based strategies applied to the broad universe of U.S. stocks. Back-testing shows that, after accounting for transaction costs, PCA-based strategies have an average annual Sharpe ratio of 1.44 over the period 1997 to 2007, with much stronger performances prior to 2003. During 2003-2007, the averageSharpe ratio of PCA-based strategies was only 0.9. Strategies based on ETFs achieved a Sharpe ratio of 1.1 from 1997 to 2007, experiencing a similar degradation after 2002. We also introduce a method to account for daily trading volume information in the signals (which is akin to using “trading time” as opposed to calendar time), and observe significant improvement in performance in the case of ETF-based signals. ETF strategies which use volume information achieve a Sharpe ratio of 1.51 from 2003 to 2007. The paper also relates the performance of mean-reversion statistical arbitrage strategies with the stock market cycle. In particular, we study in detail the performance of the strategies during the liquidity crisis of the summer of 2007. We obtain results which are consistent with Khandani and Lo (2007) and validate their “unwinding” theory for the quant fund drawdown of August 2007. Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, N.Y. 10012 USA †Finance Concepts, 49-51 Avenue Victor-Hugo, 75116 Paris, France.
- 12. Copyright ® 2009, SAS Institute Inc. All right s reserved. XBRL Basics XBRL is XML It is Extensible There is an XBRL specification – tells you how to use XBRL Hinges on taxonomies – the dictionary of terms for business reporting – which includes financial statements
- 13. Copyright ® 2009, SAS Institute Inc. All right s reserved. XBRL Taxonomy Created by XBRL Consortium Consumed Rendered XBRL Creation XBRL Document Created by Preparer TAGGING
- 14. Copyright ® 2009, SAS Institute Inc. All right s reserved. Validation Standardization Calculation Cash = Currency + Deposits Calculation Cash = Currency + Deposits Formulas Cash ≥ 0 Formulas Cash ≥ 0 Contexts US $ FY2004 Budgeted Contexts US $ FY2004 Budgeted Label cashCashEquivalentsAn dShortTermInvestments Label cashCashEquivalentsAn dShortTermInvestments References GAAP I.2.(a) Instructions Ad Hoc disclosures References GAAP I.2.(a) Instructions Ad Hoc disclosures Presentation Cash & Cash Equivalents Presentation Cash & Cash Equivalents XBRL Item XBRL Item XML Item XML Item XBRL Item XBRL Item Presentation Comptant et Comptant Equivalents Presentation Comptant et Comptant Equivalents Presentation Geld & Geld nahe Mittel Presentation Geld & Geld nahe Mittel Presentation Kas en Geldmiddelen Presentation Kas en Geldmiddelen Presentation 现金与现金等价物 Presentation 现金与现金等价物 Presentation 現金及び現金等価物 Presentation 現金及び現金等価物 Presentation Деньги и их эквиваленты Presentation Деньги и их эквиваленты Presentation Гроші та їх еквіваленти Presentation Гроші та їх еквіваленти
- 15. Copyright ® 2009, SAS Institute Inc. All right s reserved. The Business Reporting Supply Chain External Financial Reporting Business Operations Internal Financial Reporting Investment, Lending, and Regulation Processes Participants Auditors Trading Partners Investors Financial Publishers and Data Aggregators Regulators Software VendorsSoftware Vendors Management Accountants Companies XBRL XBRLXBRL XBRL Financial StatementsXBRL-GL The Journal Standard Transaction Standards Collaboration is KEY!!!
- 16. Copyright ® 2009, SAS Institute Inc. All right s reserved. USUS UKUK JPJPESES SESE CNCN ZAZA AUAU DEDEDKDK Financial Banking Regulators Pilot Pilot Committe d Committe d KRKR SGSG FRFR NZNZ NLNL LuXLuX Portuga l Portuga l BEBE EU CEBSEU CEBS
- 17. Copyright ® 2009, SAS Institute Inc. All right s reserved. XBRL Jurisdictions UK CA SPUS AU NZ IR JP KR BE VZ CO BR AR PT RU SG HK NO SE PL FI IT CN IN LB CZ UA LU IAS B AE NL TR GR MT CH FR SI HU AT Established Jurisdictions Provisional Jurisdictions in Construction & in Project DE DK ZA
- 18. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 19. Copyright ® 2009, SAS Institute Inc. All right s reserved.
- 20. Copyright ® 2009, SAS Institute Inc. All right s reserved. CACA UKUK IEIE AUAU NONO JPJP NZNZ NLNL DEDE CNCN Tax Authorities Pilot Pilot Committe d Committe d Tax XML Technical Committee recommends use of XBRL (Oasis-OECD) 29 Tax authorities
- 21. Copyright ® 2009, SAS Institute Inc. All right s reserved. Exchanges & Equity Regulators Sao Paulo Sao Paulo NZSE NZSE ASX ASXJohannesburg Johannesburg Shenzen Shenzen EuroNext EuroNext KOSDAQ KOSDAQ Tokyo Tokyo Singapore Singapore SWX SWXLux Lux Pilot LiveEval TSX TSX OBX OBX LSE LSE CSE CSE Deutsche Börse Deutsche Börse Taipei SEC SEC Kor ea Kor ea Shanghai Shanghai
- 22. Copyright ® 2009, SAS Institute Inc. All right s reserved. Grant Boyd, CA gboyd@aicpa.org Technical Manager – XBRL, AICPA •http://www.icgfm.org/XBRLPresentations.htm
- 23. Copyright ® 2009, SAS Institute Inc. All right s reserved. Data Mining Approaches Three Basic Approaches to Data Mining • Mathematical-based methods, • Distance-based methods, and • Logic-based methods Methods may use supervised or unsupervised variable • Supervised – induction rules for predefined classifications • Unsupervised – rules and classifications determined by data mining method
- 24. Copyright ® 2009, SAS Institute Inc. All right s reserved. Mathematical-based Methods Neural Network • Network of nodes modeled after a neuron or neural circuit • Supervised learning • Weighted values at different nodes • Mimics the processing of the human brain • Form of Artificial Intelligence
- 25. Copyright ® 2009, SAS Institute Inc. All right s reserved. Mathematical-based Methods Discriminant Analysis • Similar to multiple regression analysis uses a non- continuous dependent variable • Approach identifies the variables (features or cases) that best explain the classification • Supervisory learning approach • Loses effectiveness with large complex data sets
- 26. Copyright ® 2009, SAS Institute Inc. All right s reserved. Logic-Based Approach Tree and Rule Induction • Supervised Learning − Uses an algorithm to induce a decision tree from a file of individual cases − Case has set of attributes and the class to which it belongs • Decision tree can be converted to a rule-based view. • Major advantage is ability to communicate and understand information derived from this approach. • Prior research addressed audit areas of: − bankruptcy, bank failure, and credit risk
- 27. Copyright ® 2009, SAS Institute Inc. All right s reserved. Distance-Based Method Clustering • Data mining approach that partitions large sets of data objects into homogeneous groups • Uses unsupervised classification where little manual pre-screening of data is necessary – − useful in situations where there is no predefined knowledge of categories • Classifications based on an object’s attributes • Most commonly used in field of marketing but could be used in auditing
- 28. Copyright ® 2009, SAS Institute Inc. All right s reserved. Selecting Data Mining Approach Criteria: • Scalability - how well data mining method works regardless of data set size • Accuracy - how well information extracted remains stable and constant beyond the boundaries of the data from which it was extracted, or trained • Robustness - how well the data mining method works in a wide variety of domains • Interpretability - how well data mining method provides understandable information and valuable insight to user
- 29. Copyright ® 2009, SAS Institute Inc. All right s reserved. Continuous Auditing, XBRL and Data Mining Presenters: Jennifer Moore, Lumsden & McCormick, LLP Karina Barton, Canisius College Dr. Joseph O’Donnell, Canisius College New York State Society of Certified Public Accountants Technology Assurance Committee June 15, 2004
- 30. Copyright ® 2009, SAS Institute Inc. All right s reserved. Prime Services http://en.wikipedia.org/wiki/Prime_brokeragehttp://en.wikipedia.org/wiki/Prime_brokerage • Prime Brokerage is the generic name for a bundled package of services offered by investment banks to hedge funds. The business advantage to a hedge fund of using a Prime Broker is that the Prime Broker provides a centralized securities clearing facility for the hedge fund, and the hedge fund's collateral requirements are netted across all deals handled by the Prime Broker. The Prime Broker benefits by earning fees ("spreads") on financing the client's long and short cash and security positions, and by charging, in some cases, fees for clearing and/or other services. • The following "core services" are typically bundled into the Prime Brokerage package: − Global custody (including clearing, custody, and asset servicing) − Securities lending − Financing (to facilitate leverage of client assets) − Customized Technology (provide hedge fund managers with portfolio reporting needed to effectively manage money) − Operational Support (prime brokers act as a hedge fund's primary operations contact with all other broker dealers)
- 31. Copyright ® 2009, SAS Institute Inc. All right s reserved. Hedge Fund Hedge Fund Hedge Fund Exchange Clearing Settlement Trading Position keeping Clearing And Settlement Prime Services
- 32. Copyright ® 2009, SAS Institute Inc. All right s reserved. Hedge Fund Mutual Fund Hedge Fund Exchange Exchange Exchange Algorithmic Trading System Prime Service Provider Ice Berg VWAP TWAP
- 33. Copyright ® 2009, SAS Institute Inc. All right s reserved. Client Client Client Exchanges Clearing Settlement Equities Derivatives Fixed Income Reference Data Validation and Enrichment Risk Management Financial Control Clearing and Settlement
- 34. Copyright ® 2009, SAS Institute Inc. All right s reserved. Date: 15 May 2007 Produced by: Chris Swan
- 35. Copyright ® 2009, SAS Institute Inc. All right s reserved. Overview of stock exchanges The main stock exchanges in the world include: America • American Stock Exchange • NASDAQ • New York Stock Exchange • São Paulo Stock Exchange Europe • Euronext • Frankfurt Stock Exchange • London Stock Exchange • Madrid Stock Exchange • Milan Stock Exchange • Zurich Stock Exchange • Stockholm Stock Exchange
- 36. Copyright ® 2009, SAS Institute Inc. All right s reserved. Listing requirements LSE — main market has requirements for a minimum market capitalization of £700,000, three years of audited financial statements, minimum public float of 25 % and sufficient working capital for at least 12 months from the date of listing NASDAQ — to be listed a company must have issued at least 1.25 million shares of stock worth at least $70 million and must have earned more than $11 million over the last three years NYSE — a company must have issued at least a million shares of stock worth $100 million and must have earned more than $10 million over the last three years
- 37. Copyright ® 2009, SAS Institute Inc. All right s reserved. Overview of stock exchanges Australia/Asia/Africa • Australian Stock Exchange • Bombay Stock Exchange • Hong Kong Stock Exchange • Johannesburg Securities Exchange • Korea Stock Exchange • Shanghai Stock Exchange • Taiwan Stock Exchange • Tokyo Stock Exchange • Toronto Stock Exchange
- 38. Copyright ® 2009, SAS Institute Inc. All right s reserved. Participants Broker — an individual or firm which operates between a buyer and a seller and usually charge a commission. For most products a licence is required. Dealer — an individual or firm which buys and sells for its own account. Broker/dealer — an individual or firm buying and selling for itself and others. A registration is required. Principal — a role of broker/dealer when buying or selling securities for its own account. Market maker — a brokerage or bank that maintains a firm bid and ask price in a given security by standing ready, willing, and able to buy or sell at publicly quoted prices (called making a market). These firms display bid and offer prices for specific numbers of specific securities, and if these prices are met, they will immediately buy for or sell from their own accounts. Specialist — a stock exchange member who makes a market for certain exchange-traded securities, maintaining an inventory of those securities and standing ready to buy and sell shares as necessary to maintain an orderly market for those shares. Can be an individual, partnership, corporation or group of firms.
- 39. Copyright ® 2009, SAS Institute Inc. All right s reserved. Prototypical trading systems Call (periodic) auction — selling stocks by bid at intervals throughout the day. The orders are stored for execution at a single market clearing price. Continuous auction — buyers enter competitive bids and sellers place competitive offers simultaneously. Continuous, since orders are executed upon arrival. Dealership market — trading occur between principals buying and selling to their own accounts. Firm price quotations are available prior to order submission. Auction markets are concentrated and order-driven Dealership markets are fragmented and quote-driven
- 40. Copyright ® 2009, SAS Institute Inc. All right s reserved. Examples NYSE — opens with a periodic auction market and then switches to a continuous auction. Same for Tokyo Stock Exchange. NASDAQ and International Stock Exchange (London) are quote-driven systems (continuous dealership market).
- 41. Copyright ® 2009, SAS Institute Inc. All right s reserved. Examples Euronext Paris — the market is segmented into a number of different groups of stocks based on size and liquidity. The trading mechanisms vary depending on the segment. Euronext 100, Next 150 ,CAC40 indices and stocks which have more than 2,500 order book transactions per year — continuous auction. Other stocks — call auction twice a day.
- 42. Copyright ® 2009, SAS Institute Inc. All right s reserved. Order types Market order — immediate execution at the best price available when the order reaches the marketplace Limit order — to execute a transaction only at a specified price (the limit) or better Stop order Good till cancelled Fill-or-kill All or None Day order
- 43. Copyright ® 2009, SAS Institute Inc. All right s reserved. Algorithmic Trading: An Overview ofAlgorithmic Trading: An Overview of Applications And Models.Applications And Models. Ekaterina Kochieva Gautam Mitra Cormac Lucas
- 44. Copyright ® 2009, SAS Institute Inc. All right s reserved. Summary Definitions and examples have been given about XBRL & Financial Data Mining. Key components to make this work are the global adoption of a Business Reporting language. This includes consistent standards being set by both XBRL US & XBRL International. All companies need to be held responsible for their reporting with the current Reg rules adjusted. A global certification process is needed with the proper GRC engagement model followed. The good news is that there will be plenty of data to be mined. How we mine and integrate this into our Trading Strategies is up to us. This is the "special sauce" which will make or break how our Trader trades & Trade Support supports.
Editor's Notes
- #15: XBRL provides more than just XML Schema for addressing these problems; it provides XML Schema in connection with XML Link (or XLink) to provide multidimensionality. The XBRL Item is connected to its label or machine definition via a schema concept; then XBRL goes beyond the use of Schema to link the element to a range of additional concepts: Presentation – providing a human readable label…..in this case with English, French, German, Dutch, Chinese, Japan, Russian, and Ukrainian presentation alternatives for this item label. Reference – can be any other concept such as reference materials, business process guidance, a Web service or even another business rule. Calculation – provides an internal mathematical check or validation on the item. Context – provides the item with transferable context around currency, period, nature (actual, budgeted, forecasted, etc.). The combination of XML Schema and XML Link provide the data with significant standardization and validation context that can be shared across applications, platforms and entities via domain level taxonomies.
- #39: Market maker — a brokerage or bank that maintains a firm bid and ask price in a given security by standing ready, willing, and able to buy or sell at publicly quoted prices (called making a market). These firms display bid and offer prices for specific numbers of specific securities, and if these prices are met, they will immediately buy for or sell from their own accounts. Market makers are very important for maintaining liquidity and efficiency for the particular securities that they make markets in. At most firms, there is a strict separation of the market-making side and the brokerage side, since otherwise there might be an incentive for brokers to recommend securities simply because the firm makes a market in that security.
- #43: A stop order (sometimes known as a stop loss order) is an order to buy or sell a security once the price of the security reaches a specified price, known as the stop price. When the specified price is reached, the stop order is entered as a market order. Stop orders are used to try to limit an investor's exposure in the market. With a stop order, the customer does not have to actively monitor how a stock is performing. However because the order is triggered automatically when the stop price is reached, the stop price could be activated by a short-term fluctuation in a security's price. Once the stop price is reached, the stop order becomes a market order. In a fast-moving market, the price at which the trade is executed may be much different from the stop price. The use of stop orders is much more frequent for stocks, and futures, that trade on an exchange than in the over-the-counter (OTC) market.