Integrating Salesforce With Business Intelligence and Data Warehouses
1 like5,274 views
The document discusses the integration of Salesforce with business intelligence and data warehouses at Cort Business Services, emphasizing the extensive use of Salesforce data for enterprise data integration, real-time access, and reporting. It details the tools and methods used, such as SQL Server and DBAmp, for managing Salesforce data, including CRUD operations, data replication, and complex associations. The document illustrates practical applications and system architecture followed at Cort to optimize the use of Salesforce data within their operations.
Integrating Salesforce With Business Intelligence and Data Warehouses
- 1. Integrating Salesforce with Business Intelligence & Data Warehouses Tim Deagan, CORT Business Services @TimDeagan
- 2. All about CORT Business Services. A Berkshire Hathaway Company CORT, a Berkshire Hathaway Company, is the world’s largest provider of rental furniture and much, much more. When life creates transition, CORT swings into action. Whether you are relocating permanently or temporarily and in need of a place to live, furnishings and local knowledge, a business needing swing space furnishings or a new identity, or just looking for total flexibility and lifestyle options, CORT finds innovative solutions that make practical sense. When you have a special event, as a company or individual, like a corporate gala, wedding, trade show, product launch or company meeting, CORT is there to make your event special and memorable. . ▪ ~1000 Salesforce licenses ▪ Customer for 5 years ▪ Force.com active developers ▪ 59 Custom Objects, ~650,000 API requests/day
- 3. Salesforce data is great in the Cloud, until you need it down to earth for… ▪ Enterprise Data Integration • Cross System Reporting • Moving Data within the Enterprise ▪ Reporting across more objects that Salesforce supports • More than four object relationships (~= JOINS) ▪ Advanced Historical Data Queries
- 4. Bring your Salesforce data into an RDBMS ▪ Query and Report with your internal tools • SQL, not SOQL • ODBC to MS Excel, MS Access, etc. ▪ Populate your Data Warehouse with Salesforce fact data • Objects become raw tables • ETL into ROLAP, OLAP, HOLAP, MOLAP, DOLAP
- 6. DBAmp Functionality ▪ A SQL Server Linked Server to Salesforce production and sandbox instances via HTTPS ▪ Real-time CRUD access ▪ Replication procedures to maintain local copies of Salesforce data ▪ Uses Salesforce API security ▪ Salesforce org based license (usable on unlimited servers) ▪ Great on-line support by the author
- 7. Two methods for real-time data access SQL & SOQL ▪ SQL (technically Transact SQL or T-SQL) • Uses ‘4-part’ naming convention – ServerAlias.Catalog.Schema.LinkedTable – Salesforce…Contact (Linked_Server_Name…Table) • PRO – Allows full range of SQL manipulation • CON – Pulls entire table into memory prior to executing WHERE clause or SQL functions
- 8. Two methods for real-time data access SQL & SOQL ▪ SOQL (technically SOQL via OPENQUERY) • OpenQuery executes the passed query on the linked server • PRO – Higher performance • CON – To pass parameters, you must use a stored procedure that builds the query string and executes it
- 9. CRUD access CREATE & READ ▪ Create == Insert (Tables cannot be created) • INSERT INTO Salesforce...Note (Body, IsPrivate, ParentId, Title) VALUES ('Body of Note 2','false', '00130000005ZsG8AAK','ToDelete') • Stored procedure sf_bulkops provides batched bulk inserts ▪ Read == Select • Select * from Salesforce...Contact • Select * from openquery(Salesforce, ‘Select Id, CreatedDate, CreatedById from Account’)
- 10. CRUD access UPDATE & DELETE ▪ Update • Update Salesforce…Account Set AnnualRevenue = 4000 Where Id='00130000005ZsG8AAK' • Update OPENQUERY(Salesforce, 'Select Id, AnnualRevenue from Account where Id=''00130000005ZsG8AAK'' ') set AnnualRevenue = 4000 ▪ Delete (Rows, not tables) • Delete from Salesforce…Account Where Id='00130000005ZsG8AAK' • Delete openquery(Salesforce,''Select Id from Account where Id=''00130000005ZsG8AAK'' ‘)
- 11. Replicating Salesforce data locally Incremental scheduled updates (hourly, daily, weekly, etc.) ▪ Stored Procedures for REFRESH and REPLICATE • SF_Refresh brings over changes • SF_Replicate brings over complete tables ▪ Schema changes are automatically detected and brought over ▪ Archived and deleted rows can be pulled over ▪ DRAMATICALLY faster queries than real time SQL or SOQL ▪ Local backup of Salesforce
- 12. How we use this system at CORT
- 14. CORT Architecture ▪ Tech Stack • SQL Server 2008 R2, SQL Server 2012 • SQL Server Reporting Services (SSRS) • SQL Server Integration Services (SSIS) ▪ ROLAP (Relational Online Analytical Processing) Environment • No cubes • Nightly jobs crunching data into user friendly tables and views (some denormalization) • SSRS reports providing CSV for Excel usage as well as straight reporting • Combines data from ERP, CRM, eCommerce, Call Center and Financials
- 15. Examples at CORT Find and delete unnecessary custom fields ▪ Created “Field Usage” reports ▪ Developed a ‘hit list’ and eliminated hundreds of unnecessary fields, merging the data into existing fields as appropriate Emergency restores ▪ Over-eager Salesforce bulk loader user crufted 10,000+ records ▪ We restored the relevant data from a SQL Server backup
- 16. Examples at CORT Upload data from other systems ▪ Revenue data from our National Accounts portal needed to be linked to Salesforce accounts • SQL was written to get the National Account data, cross link it to the Salesforce Account ID and upload the revenue nightly to Salesforce so salespeople can see it ▪ Data from our HR system is used to mod or delete User records in Salesforce. • Systems are kept in sync with nightly job
- 17. Examples at CORT Salesforce as a CMS ▪ Our District Policies are maintained by the Districts in Salesforce • SSRS Report publishes the Policies to Truck Drivers and Warehouse workers without Salesforce licenses
- 18. Examples at CORT Creation of complicated associations ▪ Our call center gets sales credit for the leases they ‘touch’ ▪ This is determined by any one of a number of activities ▪ Requires determining if they were the Opp owner at any point in the past ▪ We created a View in SQL Server that identifies all the Opportunties that they get credit for touching…
- 19. Were they ever the opp owner? Were they the initial opp owner? Did they ever own the lead that became the opp? Did they create the lead that became the opp?
- 20. Examples at CORT We wanted to segregate a line of business into a new Salesforce instance/org. ▪ Extract all relevant Salesforce data (opps, leads, contacts, accounts, attachments, etc.) for the LOB into SQL Server ▪ Create tables of all ID based relationships • 415 fields containing keys ▪ Upload records to the new org ▪ Find new object IDs ▪ Change all references to the old IDs to the new IDs in SQL Server ▪ Update records with correct relationship IDs
- 21. How much data are we talking about?
- 22. dbo.Account dbo.CallCenter dbo.AccountContactRole dbo.Campaign dbo.AccountFeed dbo.CAMPAIGNCAL__Campaign_Color__c dbo.AccountHistory dbo.CampaignFeed dbo.AccountPartner dbo.CampaignMember dbo.AccountShare dbo.CampaignMemberStatus dbo.AccountTeamMember dbo.CampaignShare dbo.AdditionalNumber dbo.Case dbo.ApexClass dbo.CaseComment dbo.ApexComponent dbo.CaseContactRole dbo.ApexLog dbo.CaseFeed dbo.ApexPage dbo.CaseHistory dbo.ApexTestQueueItem dbo.CaseShare dbo.ApexTestResult dbo.CaseSolution dbo.ApexTrigger dbo.CaseStatus dbo.ApplicationNameValue__c dbo.CaseTeamMember dbo.Approval dbo.CaseTeamRole dbo.APXT_CMTR__Conga_Merge_Template__c dbo.CaseTeamTemplate dbo.APXT_CMTR__Conga_Merge_Template__Share dbo.CaseTeamTemplateMember dbo.Asset dbo.CaseTeamTemplateRecord dbo.AssetFeed dbo.CategoryData dbo.AssignmentRule dbo.CategoryNode dbo.AsyncApexJob dbo.ChatterActivity dbo.Attachment dbo.ClientBrowser dbo.AuthProvider dbo.CMC_Services__c dbo.Blacklist_Audit__c dbo.CMC_Services__History dbo.Blacklisted_Word__c dbo.CollaborationGroup dbo.BLND_DFAC_Shared_View__c dbo.CollaborationGroupFeed dbo.BLND_DFCS_Carousel__c dbo.CollaborationGroupMember dbo.BLND_DFCS_Slide__c dbo.CollaborationGroupMemberRequest dbo.BLND_DFDT_Action_Item__c dbo.CollaborationInvitation dbo.BLND_DFDT_Discussion__c dbo.College__c dbo.BLND_DFDT_Highlight__c dbo.College__History dbo.BLND_DFDT_Issue__c dbo.Community dbo.BLND_DFDT_Linker__c dbo.Contact dbo.BLND_DFDT_Project__c dbo.ContactFeed dbo.BLND_DFDT_Resource__c dbo.ContactHistory dbo.BLND_DFDT_Task__c dbo.ContactShare dbo.BLND_DFFF_Form__c dbo.ContentDocument dbo.BLND_DFFF_Page__c dbo.ContentDocumentFeed dbo.BLND_DFTE_Expense__c dbo.ContentDocumentHistory dbo.BLND_DFTE_Time__c dbo.ContentVersion dbo.BLND_DFTT_Document__c dbo.ContentVersionHistory dbo.BLND_DFTT_Entity_Info__c dbo.ContentWorkspace dbo.BLND_DFTT_Folder__c dbo.ContentWorkspaceDoc dbo.BrandTemplate dbo.Contract dbo.BusinessHours dbo.ContractContactRole dbo.BusinessProcess dbo.ContractFeed dbo.ContractHistory dbo.ContractStatus dbo.Corporate_Housing__c dbo.Corporate_Housing__Share dbo.CORT_Location__c dbo.CORT_Location__History dbo.CORT_Showroom__c dbo.Coupon_Code__c dbo.Coupon_Code__History dbo.Credit__c dbo.Credit__History dbo.CronTrigger dbo.CurrencyType dbo.Customer_Profile__c dbo.Customer_Profile__History dbo.Dashboard dbo.DashboardComponent dbo.DashboardComponentFeed dbo.DashboardFeed dbo.DatedConversionRate dbo.DFSS_SnapShot_Item__c dbo.DFSS_SnapShot_Push__c dbo.District_Policy__c dbo.District_Policy__History dbo.Document dbo.DocumentAttachmentMap dbo.Domain dbo.DomainSite dbo.dsfs__DocuSign_Envelope__c dbo.dsfs__DocuSign_Envelope_Document__c dbo.dsfs__DocuSign_Envelope_Recipient__c dbo.dsfs__DocuSign_Recipient_Status__c dbo.dsfs__DocuSign_Recipient_Status__History dbo.dsfs__DocuSign_Status__c dbo.dsfs__DocuSign_Status__History dbo.dsfs__DocuSignAccountConfiguration__c dbo.EmailMessage dbo.EmailServicesAddress dbo.EmailServicesFunction dbo.EmailTemplate dbo.EntitySubscription dbo.Evaluation_Phone__c dbo.Evaluation_Phone__History dbo.Event dbo.EventAttendee dbo.EventFeed dbo.EventRelation dbo.Events__c dbo.Events__History dbo.Exhibit_House_Client_del_del__History dbo.FeedComment dbo.FieldPermissions dbo.FiscalYearSettings dbo.Folder dbo.ForecastShare dbo.General_Contractor__c dbo.General_Contractor__History dbo.Group dbo.GroupMember dbo.HashtagDefinition dbo.Holiday dbo.Housing_Market__c dbo.Housing_Market__Share dbo.Idea dbo.IdeaComment dbo.Lead dbo.LeadFeed dbo.LeadHistory dbo.LeadShare dbo.LeadStatus dbo.Lease_Types__c dbo.Lease_Types__Share dbo.LineitemOverride dbo.LoginHistory dbo.LoginIp dbo.MailmergeTemplate dbo.Market__c dbo.Market__Share dbo.National_Account__c dbo.National_Account__History dbo.NewsFeed dbo.Note dbo.ObjectPermissions dbo.Opportunity dbo.OpportunityCompetitor dbo.OpportunityContactRole dbo.OpportunityFeed dbo.OpportunityFieldHistory dbo.OpportunityHistory dbo.OpportunityLineItem dbo.OpportunityOverride dbo.OpportunityPartner dbo.OpportunityShare dbo.OpportunityStage dbo.OpportunityTeamMember dbo.Organization dbo.OrganizationProperty dbo.Exhibit_House_Client_del_del__c dbo.OrgWideEmailAddress dbo.Partner dbo.PartnerRole dbo.Period dbo.PermissionSet dbo.PermissionSetAssignment dbo.Pricebook2 dbo.Pricebook2History dbo.PricebookEntry dbo.ProcessInstance dbo.ProcessInstanceStep dbo.ProcessInstanceWorkitem dbo.product_issue_tracker__c dbo.product_issue_tracker__History dbo.product_sku__c dbo.product_sku__History dbo.Product2 dbo.Product2Feed dbo.Profile dbo.Profile_Contact_Role__c dbo.Profile_Contact_Role__History dbo.Project__c dbo.Project__History dbo.purchase_order__c dbo.purchase_order__History dbo.PushTopic dbo.QtlContactRole__c dbo.QuantityForecast dbo.QuantityForecastHistory dbo.QueueSobject dbo.Quote dbo.Quote_To_Lease__c dbo.Quote_To_Lease__History dbo.QuoteDocument dbo.QuoteLineItem dbo.RecordType dbo.Referral_Payment_Account__c dbo.Referral_Payment_Account__History dbo.Referral_Payment_Transaction__c dbo.Referral_Payment_Transaction__History dbo.Referral_Payment_Vendor__c dbo.Referral_Payment_Vendor__History dbo.Report dbo.ReportFeed dbo.RevenueForecast dbo.RevenueForecastHistory dbo.Scontrol dbo.screenstepslive__ScreenStepsLiveSettings__c dbo.screenstepslive__ScreenStepsLiveTags__c 25GB of data in lots of Tables! dbo.SelfServiceUser dbo.SetupEntityAccess dbo.SFDC_520_Approval_Request__c dbo.SFDC_520_Quote__c dbo.SFDC_520_QuoteLine__c dbo.SFGA__Ad_Group__c dbo.SFGA__Ad_Group__Share dbo.SFGA__Google_Campaign__c dbo.SFGA__Google_Campaign__Share dbo.SFGA__Keyword__c dbo.SFGA__Keyword__Share dbo.SFGA__Search_Phrase__c dbo.SFGA__Search_Phrase__Share dbo.SFGA__Text_Ad__c dbo.SFGA__Text_Ad__Share dbo.SFGA__Version_33_0_3__c dbo.SFGA__Version_33_0_3__Share dbo.Site dbo.SiteDomain dbo.SiteFeed dbo.SiteHistory dbo.Solution dbo.SolutionFeed dbo.SolutionHistory dbo.SolutionStatus dbo.StaticResource dbo.TableRefreshTime dbo.Task dbo.TaskFeed dbo.TaskPriority dbo.TaskStatus dbo.Topic dbo.User dbo.User_Load dbo.UserAccountTeamMember dbo.UserFeed dbo.UserLicense dbo.UserPreference dbo.UserRole dbo.UserTeamMember dbo.VASSnaps dbo.vendor__c dbo.vendor__History dbo.Venue__c dbo.WebLink dbo.xtma_Email_Result__c dbo.xtma_Individual_Email_Result__c dbo.xtma_Link_Detail__c
- 23. DBAmp creates all the Foreign Key constraints that allow diagramming. However, pulling all the tables into an ERD makes for a VERY LARGE diagram!
- 24. Tim Deagan CORT Business Services Executive Director, Product Architecture @TimDeagan