Dev Analytics Overview
- 1. Developer Analytics Overview & Demo
- 2. Agenda • Functional overview • Demo • Architecture overview
- 3. Developer Portal Per app report App Tile App summary Adoption Downloads In-app purchases Usage Ratings Quality Finance Per Developer report Financial summary Store report Explore Store trends Developer Analytics – Sitemap This is an aggregation of downloads, ratings, quality, finance, purchases and in-app purchases report to give an overview of all the metrics useful to the developer. This is an summary of downloads, purchases, in-app purchases, usage metrics It also shows Discovery metrics – App discovery/conversion funnel and app’s web listing page’s web referrers distribution
- 4. Developer Analytics – Partner dependency Fundamental Telemetry 5R – RnR DSP Telemetry Downloads In-app purchases Usage Quality Ratings and Reviews Finance Financial Summary Adoption page Explore Store trends Page views + Conversion funnel Primes PFDb – PartnerFeedDb (Aggregation) BI Cube (Copy to FEs) Telemetry Cube (Copy + Stability feed generation) RnR BCP Feed (Aggregation) ComTransactionDb (Aggregation)
- 7. Architecture overview • Component Grouping • Copy, Validate & Publish SSAS Cube • Extract and Load with basic processing • Extract, Transform and Load • Data Mining for Store Trends • Workitem Framework Block Diagram
- 8. Architecture overview – Cube PAS • Copy, Validate & Publish SSAS Cube • Input – Cube backup • Processing • Restore cube backup on BE • Validate Schema • Publish • Restore new cube to Passive instance • PAS - Passive Active Sync • Copy cube to passive instance of all FEs • Flip Passive to Active instance. Cube (Backup) DCS Pipeline DLS Pipeline Temp Storage Cube SSAS PAS Pipeline Telemetry Cube SSAS Cube SS AS FE Cube SSAS Telemetry Cube SSAS Cube SS AS FE Cube SSAS Passive Active FE Portal
- 9. Architecture overview – Basic ETL • Extract and Load with basic processing • Input – Raw data • Processing • Validate Schema of raw data • Filter unwanted data • Remove duplicate entries • Load • Re-process SSAS Cube and Publish to FE using PAS • Copy processed data to FE DB FE Portal Staging DB DWHProcessing Cube SSAS PAS Pipeline Store Cube SSAS Store Cube SSAS FE Cube SSAS Feed Data Feed Pipeline App Store Analytics DB App Store Analytics DB FE DB DPS Pipeline
- 10. Architecture overview – ETL • Extract, Transform and Load • Input – Raw data • Processing • Data Fetch • Validate Schema of raw data • Filter unwanted data • Remove duplicate entries • Data normalization • Data Processing – Data Aggregation • Load • Copy processed data to FE DB FE Portal Staging DB DWH Feed Data Feed Pipeline App Store Analytics DB App Store Analytics DB FE DB Data Move Processing Source DB Data Fetcher
- 11. Architecture overview - Mining • Data Mining • Input – Acquisition data • Processing • Data Fetch • Validate Schema of raw data • Filter unwanted data • Remove duplicate entries • Data normalization • Data Processing • Data Preparation – Data Filtering, Data Sampling, Data Discretization • Data Mining • Output • Mining Rules i.e. Observed data trends FE Portal DWH App Store Analytics DB App Store Analytics DB Rules DB Data Preparator Source DB Data Fetcher Sampled Data Store Data Mining
- 12. Architecture overview • Workitem Framework DB Based Workitem Queue Workitem Generator1 WI Type 1 State 1 WI Type 1 State 2 WI Type 4 State 10 Workitem Generator2 WI Processor Type 1 State 1 WI Processor Type 4 State 10 WI Processor Type 1 State 2 WI Processor Type 4 State 10
- 13. Architecture overview Partner Feed DB RnR Feed ComTxn Feed DevAnalytics Staging DB MiningDWH DB MiningDWH DBMiningDWH DB Sampled Data Store Sampled Data Store Data Fetcher WIFx Data Feed Pipeline Data Fetcher WIFx Mining Data Preparator WIFx Mining Data Miner – WIFx, SSAS Data Aggregator WIFx Move Data to FE WIFx Data Feed Pipeline Payout Cube SSAS Telemetry Cube DCS Pipeline DLS PipelineTemp Storage Telemetry Cube SSAS PAS Pipeline Telemetry Cube SSAS A. Te le m Telemetry Cube SSAS Manual Cert DB Data Feed Pipeline Certification Reporting DB Mining Rules DB Mining Rules DB Mining Rules DB DevAnalyticsFE DB DevAnalyticsFE DBDevAnalyticsFE DB Analytics Staging DB Analytics DWH Load DWH SSIS Store Cube SSAS PAS Pipeline Store Cube SSAS Store Cube SSAS Store Cube SSAS App Catalog Feed Data Feed Pipeline Control DB Data Fetch Workitem Generator Data Mining Workitem Generator Aggregation Workitem Generator Settelment Feed CP Feed ForEx Feed App Store Analytics DB App Store Analytics DB App Store Analytics DB DPS Pipeline Tracking DB Currency Conversion Tracking DB Dev Portal Dev Portal Dev Portal Dev Portal Mining Rules DB Mining Rules DB Onboarding DB Stability Score Pipeline Stability Score Feed Data Feed Pipeline BI Cube DCS Pipeline DLS Pipeline BI Cube SSAS PAS Pipeline Telemet ry Cube SSAS Tele m e BI Cube SSAS
- 14. Architecture overview – Cube PAS Telemetry Cube DCS Pipeline DLS Pipeline Temp Storage Telemetry Cube SSAS PAS Pipeline Telemetry Cube SSAS Telem etr y Cu be SS AS Telemetry Cube SSAS Stability Score Pipeline Stability Score Feed BI Cube DCS Pipeline DLS Pipeline BI Cube SSAS PAS Pipeline Telemetr y Cube SSAS Tele m e t r y C u b e S S A S BI Cube SSAS
- 15. Architecture overview – Cube PAS Fundamental Telemetry 5R – RnR DSP Telemetry Downloads In-app purchases Usage Quality Ratings and Reviews Finance Finance Summary Adoption page Explore Store trends Page views + Conversion funnel Primes + CP + CTP PFDb -- PartnerFeedDb BI Cube Telemetry Cube RnR BCP Feed ComTransactionDb
- 16. Architecture overview ComTxn Feed Data Feed Pipeline Payout Cube SSAS Analytics Staging DB Analytics DWH Load DWH SSIS Store Cube SSAS PAS Pipeline Store Cube SSAS Store Cube SSAS Store Cube SSASSettelment Feed CP Feed ForEx Feed App Store Analytics DB App Store Analytics DB App Store Analytics DB DPS Pipeline Currency Conversion Data Feed Pipeline
- 17. Architecture overview Fundamental Telemetry 5R – RnR DSP Telemetry Downloads In-app purchases Usage Quality Ratings and Reviews Finance Finance Summary Adoption page Explore Store trends Page views + Conversion funnel Primes PFDb – PartnerFeedDb (Aggregation) BI Cube (Copy to FEs) Telemetry Cube (Copy + Stability feed generation) RnR BCP Feed (Aggregation) ComTransactionDb (Aggregation)
- 18. Architecture overview – Aggregate DB and Mining Partner Feed DB RnR Feed DevAnalytics Staging DB MiningDWH DB MiningDWH DBMiningDWH DB Sampled Data Store Sampled Data Store Data Fetcher WIFx Data Feed Pipeline Data Fetcher WIFx Mining Data Preparator WIFx Mining Data Miner – WIFx, SSAS Data Aggregator WIFx Move Data to FE WIFx Manual Cert DB Data Feed Pipeline Mining Rules DB Mining Rules DB Mining Rules DB DevAnalyticsFE DB DevAnalyticsFE DB DevAnalyticsFE DB App Catalog Feed Data Feed Pipeline Control DB Data Fetch Workitem Generator Data Mining Workitem Generator Aggregation Workitem Generator Tracking DBTracking DB
- 19. Architecture overview – Aggregate DB and Mining Fundamental Telemetry 5R – RnR DSP Telemetry Downloads In-app purchases Usage Quality Ratings and Reviews Finance Finance Summary Adoption page Explore Store trends Page views + Conversion funnel Primes PFDb – PartnerFeedDb (Aggregation) BI Cube (Copy to FEs) Telemetry Cube (Copy + Stability feed generation) RnR BCP Feed (Aggregation) ComTransactionDb (Aggregation)
- 20. Architecture overview – Front End Block Diagram ReportDefinitions.xml CubeQueries.xml DbQueries.xml Report Manager Cube DAL DB DAL DataAccessService DAL Interface DeveloperAnalytics Controller (MVC) Highcharts Agg DB BI Cube DevAnalyticsFE DB TransactionDB DAL Mining Rules API Agg DB App Store Analytics DB Telemetry Cube Store Cube
- 21. Architecture overview Heading Heading Component + Physical Architecture for Dev Analytics for Windows BlueComponent + Physical Architecture for Dev Analytics for Windows Blue Sources Store(Ratings/ Review as bcp) 5/8 MB, 1/12 hrs 50 MB/Day 7MB/Day10 GB, 1/Day Partner Feed(Acquisitions, Page Referrals, Conversions) 2GB/Day Manual Cert DB Telemetry(Quality, Usage as a cube) DSP Telemetry aka BET (Acquisitions, Conversion, Referrers as a cube) Upgrade Model: FE is meant to be active even when BE is down. WebStore Layer is updated first. BE Layer components next. FE components last. V (N) FE components must work with V(N+1) pipelines. Availability : 3 way replication for FE stores . Replication Load prevents addition of a new DC. BE is Active/Passive. Procedure to do failover is not tested or captured as a runbook. Scalability : FE scale model is based on data replication. BE scale model is adhoc. Portal WebStore/SQL Local Caching(for metadata) Internal Partners Only Certification Reporting DB FE Layer(Active/Active))FE Layer(Active/Active)) DS2DS1 Backend Layer(Active/Passive)Backend Layer(Active/Passive) Mining DWH DB(partition 0) Data Fetcher WIFx(Partner Feed DB) Dev Analytics Staging DB Control DB Mining DWH DB (partition 1) Mining Data Preparator WIFx Data Feed Pipeline(RNR, Manual Cert, App Catalogue) Data Aggregator WIFx PS1 Data Fetcher WIFx(Partner Feed DB) Data Fetcher WIFx(Dev Analytics Staging DB) Mining Data Preparator WIFx Sampled Data Store(partiti on 0) Sampled Data Store Data Miner WIFx PS2 Sampled Data Store(partiti on 1) Data Miner WIFx Tx Catalog (Finance, as bcp) 300MB/Day 7MB/Day SSAS Cube Analytics BE CubeAnalytics Pipeline BE Backend LayerBackend Layer Analytics Staging DB Analytics DWH DB Store Cube Telemetry Cube Store BI Cube BET DCS DLS PAS DCS DLS PAS DCS DLS PAS Mining WI Monitoring WIFx Mining Data Preparator WIFx DC1 Payout Cube Frontend Cube ServerFrontend Cube Server Telemetry Cube Store Cube Store BI Cube Frontend Cube ServerFrontend Cube Server Telemetry Cube Store Cube Store BI Cube Frontend Cube ServerFrontend Cube Server Telemetry Cube Store Cube Store BI Cube Frontend DB Server App Store Analytics DB Dev Analytics FE DB Mining Rules DB Onboarding DB Frontend DB Server App Store Analytics DB Dev Analytics FE DB Mining Rules DB Onboarding DB Stability Score Feed
- 22. Architecture overview – Front End Cube Per DC
- 23. Architecture overview – Front End DB Per DC
- 24. Architecture overview – Back End Cube
- 25. Architecture overview – BE DWH & Processing (Pri DC)
- 26. Architecture overview – BE DWH & Processing (Sec DC)
- 28. Developer Analytics – Quality/Usage details • Usage metric & Reliability rates • Sampled data. • CEIP applies. Minimum 15 minute usage threshold to be sampled. • Max 1000 machines per sample. Random sampling. • Latency: 5 days • Metric computed for 3 day periods • Quality page – Top limitations • Triage dumps with only call stacks. • Un-sampled data. (CEIP applies) • Latency: 6 days
- 29. Developer Analytics – Finance page details
Editor's Notes
- #21: All Dev. Analytics pages are instances of Reports and each report is rendered as per its report definition. A report definition includes all information to define the layout and look and feel of the report (ex. filters applicable, types of charts, chart layout, color scheme etc.). All report definitions are maintained in ReportDefinition.xml. ReportDefinition.xml contains other definitions too (ex. localizable dimensions, mining price tiers etc.) all of which get used during report generation or rendering. The Report Manager module loads and provides access to the report definitions. Within each report, there could be multiple filters and charts present. Each chart and data driven filters (non-static filters) are rendered by firing a query against a data source (Cube / DB). Each such query is associated with a unique id (per data source) and is referred to in the ReportDefinition.xml to define the data source and query id for the chart / filter. For Aggregated DB, the queries are stored procedures and the query id to stored procedure mapping is maintained in DBQueries.xml. For cubes, both query definitions and the query id to query definitions mapping are maintained in CubeQueries.xml. Since Dev. Analytics pages render information from multiple sources (Cube / DB), a uniform access mechanism is provided through the DataAccessService layer. DataAccessService (DAS) layer creates an appropriate DAL class instance which in turn queries the data source and returns the data. The respective DAL interfaces create the query (for cube only) and query the underlying data source to return data. Analytics FE follows an MVC pattern. Typical request response flow is as follows: Client makes a web request (which contains Product Guid and other information) for a full page load Controller verifies product to developer association It get requested report’s definition and asks DAS to return the data for rendering filters DAS accesses DAL instance which queries the data source and returns data The returned data is used to populate the view models and a view is returned to the client The view upon loading makes AJAX calls to get and render chart data Again, controller gets requested report’s definition and asks DAS to return the data for rendering for rendering charts The returned data is used to populate the view models and a view is returned which is rendered on the client. When user applies some filters, client makes an AJAX call to get filtered data Controller receives filter request, gets requested report’s definition and asks DAS to return the data for rendering for rendering charts