Regulatory Reporting using Big Data Tools
- 3. The Last Mile Of Finance - LMOF 3Adhish Pendharkar “The Last Mile of Finance” is a term popularized by Gartner and others to describe the finance processes and activities that occur between the financial close and financial reporting and disclosure. Increasing regulatory reporting and compliance requirements are taking away major chunks of budgets out of much needed product development budgets. Account Reconciliation Financial Reporting Financial Controls Regulatory Reporting
- 4. Regulatory Reports 4Adhish Pendharkar • Online Banking Business Statistics • E-Banking oClient Statistics oClient Exception • Enterprise Credit Info Basic DB • FX Important Issue Report • FCY International Rate oRegistration oSmall Amt • Payment Tools Qtly Report • FI interbank deposit int rate • Credit Note Report • Trade oLiability Summary oStandby LC & Guarantee oCNY discount int rate • Acceptance by Beijing Banks • New Customer transaction report • Bill of Exchange • FX Activity • AML* • DB Views for Auditors • FX Conversion (1-3) China • LLD • LHBU • LKPBU • AML • BILL OC • ILC OC • Nostro Activity • SBLC & GTEE OS • IOCCA Indonesia • FIRC • TDS Cert / Challan • CTR Report • OSA Report • TDS Dedicatee • AML (Int rate) • AML (Residual Maturity) • Form A3 • Balance of O/S DD & TD • BCTT • Basic Statistical Return • DSB TD Balance AMTs • DSB TD Balance Int rate • DSB Top 20 customer • Non Performing Account • Outstanding TD • Service Tax • Vostro Balance • R-Return (Nostro / Vastro / FETERs) • Consignment Note Follow-up • CS Follow-up • BOE Statistics • Import Register • Export Register • TC Register • ATDME 18-21 India • FTU • LTX • AMLO • FCY Deposit (FTX) • Movement for Non- Resident Account • Trade o/s oConfirmed LC oIssued LC oSBLC/GTEE ofinancial booking • BOT Monitoring report • THOCCA Thailand • Inward Remittance • Outward Remittance • Operating Fund report • Non-Res KRW Acct bal • Non-Res KRW Acct txn • Foreign Currency oCheck purchase & Sell oDeposit/Trust txn oloan & repayment odeposit/trust o/s bal • X-border remit <1000USD • Foreign Exchange • Dormant Account Report • X-Border In/Out Remittance • Credit o/s by customer Korea • PIDM • Cash Report • ITIS * • AML • Transaction Amt by ccy • Repair Statistics * • MOCCA Statistics * • CIF2 Customer • DB Views for Auditors • ATIP MOCCA * • CQE MOCCA * • In Remittance Settlement • Out Remittance Settlement • FX Remittance memo • Corp customers' FX deposit • Remittance to/fro Mainland China • Overseas Customer funds flow • Cheque Acc Opening/Change • Outflow for Overseas investment • CDIC Taiwan Malaysia
- 5. MIS Reports – Business Intelligence 5Adhish Pendharkar • Debit client acct is outgoing; Dr DDA / Cr CNY Nostro • Credit client acct is incoming; Cr DDA / Dr CNY Nostro • RMB Outgoing Payment • RMB Incoming Payment • Transaction Activity Report • Transaction Outstanding Report • Loan Activity Report • Loan Outstanding Report • GTS Activity Report • ACBS Activity Report • Return Reject Report • GIRO Outward Transaction Report • FATF Report • Debit Credit Advice Reports • Online Deposit Maturity Projection • Deposit Outstanding Report • Deposit Assignment Report • Interest Accrual Report of Entrustment Loan • Sweep Execution Report • Reversal Sweep Status Report Liquidity • CNY Nostro • Account Balance • Account Details • Account Interest Rate Details • Account Odd Details • Account Customer Details DDA – Demand Deposit Account Other BI Reports Core Systems
- 6. 6Adhish Pendharkar Regulatory&MISReporting Data Elements on one report often comes from different source systems. Challenges
- 7. Is Big Data only for Analysis? 7Adhish Pendharkar BusinessIntelligence PredictiveAnalytics data mining clickstream advanced analytics unstructureddata text analytics statistical analysis linear regression logistic regression multinomialregression ordinalregression elastic net regularization clusteredvariance k-Means clustering 3V’s 5VC
- 8. Loader Data Warehouse Current State 8Adhish Pendharkar CIS Margin Core Systems Market Data Asia Data EMEA Data AMEA Data FI Positions Ref Data/Prices Transactions / Settlements Position Client / Account ODS Position Summary Ref. Data Trade Summary ETL EOD Staging Liquidity Risk Extracts Disclosure of Interest (DOI) Extracts FI Extracts FI Extracts FI Extracts FI Extracts FI Extracts Equity Extracts FI Extracts FI Extracts Client Open Trades FI Extracts FI Extracts Position Region / Country Specific Region / Country Specific Region / Country Specific Reporting Systems Up Stream / Source Type Data Warehouse Down / Reporting Systems Volume Variety Velocity?
- 9. RegulatoryPrograms AML / CFT INTRAC Korea FIU CAMLAC India FIU BNMFIU TIBMOJ Data Validation Data Quality Variance Others Data Quality Content Trend Analysis Ref Data / Pricing Services FX Core Cash and Depository Data Trade, Liquidity, Loans, and FX Data Other Data (not copies of pre and post trade data) Payments Processors GL Funds Cheq Recei vable Payable ACBS Purpo se Cd Bus Rules Prod code Other Sweeping Acct Multi Product / Multi-Source Data Composition Services ExternalData Distribution US Datamart BASEL Regulators Other Reporting Local Reg. Reporting Dodd- Frank Multiple Local Regulators country specific Suspicious Transaction Reporting Large Value Transactions PAMLC SCAD TAMLO Others Depository Transaction Systems Teller DDA Counter Data Control Framework Services Exception Management Reconciliation Others Annual Reporting 9 Keyw ord Country Currency AUSTRAC Reserve Bank of NZ Others Loans & Deposits FCY Transaction Others Financials Reporting Strategic Regulatory Reporting Architecture 9Adhish Pendharkar
- 10. Strategic Regulatory Reporting Platform 10Adhish Pendharkar Generate Reports Deploy Views Process Data Stage Data Stateless Data Store: The big data store holds all asset classes data. Views / GreenPlum Tables: Regional Views: holds generic attributes/data required by most of the countries’ reports. Local Views: holds specific attributes/data only required by one country / LOB Reporting Services Logic Layer: holds business logic required for each of the reports. Presentation and Data submission layer : Performs reports/data extracts formatting and transmission. Reporting Services – Presentation Big Data Store (Hadoop) Views / GPTables ReportingServices– SubmissionLayer
- 11. Loader Data Warehouse Current State 11Adhish Pendharkar CIS Margin Core Systems Market Data Asia Data EMEA Data AMEA Data FI Positions Ref Data/Prices Transactions / Settlements Position Client / Account ODS Position Summary Ref. Data Trade Summary ETL EOD Staging Liquidity Risk Extracts Disclosure of Interest (DOI) Extracts FI Extracts FI Extracts FI Extracts FI Extracts FI Extracts Equity Extracts FI Extracts FI Extracts Client Open Trades FI Extracts FI Extracts Position Region / Country Specific Region / Country Specific Region / Country Specific Reporting Systems Up Stream / Source Type Data Warehouse Down / Reporting Systems Volume Variety Velocity?
- 12. Future State 12Adhish Pendharkar CIS Margin Core Systems Market Data Asia Data EMEA Data AMEA Data FI Extracts FI Extracts FI Extracts FI Extracts FI Extracts Equity Extracts FI Extracts FI Extracts Client Open Trades FI Extracts FI Extracts Position Region / Country Specific Reporting Systems Up Stream / Source Type Data Warehouse Down / Reporting Systems Volume Variety Velocity? Mule(ESB)/EndPointMappersMule(ESB)/EndPointMappersMule(ESB)/EndPointMappers Big Data Store (Hadoop) Liquidit y Risk View Disclosure of Interest (DOI) Extracts Mule(ESB)/EndPointMappersMule(ESB)/EndPointMappersMule(ESB)/EndPointMappers
- 13. A bit of Java: Technology Stack 13Adhish Pendharkar Data (PivotalHD) Apache/ Tomcat In Memory Cache Query Object Cache Ref. Data Object Cache Mule Supporting Services Service Registry System Administrator Users / Operators Application Session Golden Data Sources GL Teller Liquidity DDA Payables Receivables Sweeping FX Vault Regional Views Local Views … Integration / End-Points Mule ESB / Spring Integration (Instance) In Process Invocation Standalone JVM (Tomcat) Application InstanceDependency Injection Dependency Injection Dependency Injection Application Instance Data Access Layer (JPA) JPA Access Layer Quartz (Job) JSP / Bootstrap Spring GemFire Mule Quartz JPA PivotalHD GreenPlum Stack API Manager Anypoint
- 14. A bit of Java: Code Sample 14Adhish Pendharkar
- 15. Appendix 15Adhish Pendharkar TAMLO - Thailand Anti-Money Laundering Office (http://www.amlo.go.th) TIBMOJ - Taiwan Investigation Bureau, Ministry of Justice (http://www.moj.gov.tw) AML/CFT – Anti Money Laundering – Combating the Financing of Terrorism (https://www.imf.org) CAMLAC - China AML Monitoring & Analysis Center SCAD - Singapore Commercial Affairs Department (http://www.cad.gov.sg/) INTRAC - Indonesian Financial Transaction Reports and Analysis Center (http://www.ppatk.go.id) India FIU – India Financial Intelligence Unit (http://fiuindia.gov.in/) Korea FIU – Korea Financial Intelligence Unit (http://www.kofiu.go.kr) PAMLC - Philippines Anti- Money Laundering Council (http://www.amlc.gov.ph/) AUSTRAC - Australian Transaction Reports and Analysis Centre (http://www.austrac.gov.au/) BNMFIU - Bank Negara Malaysia Financial Intelligence Unit (http://amlcft.bnm.gov.my/)