BITCAR- Agile, Data Warehouse
Download as DOCX, PDF2 likes410 views
The document introduces the BITCAR mnemonic and collaboration framework designed to enhance story and acceptance writing for data warehouse projects. It provides guidelines on various aspects, including business value, integration, tables, columns, associations, and representation, aimed at improving team collaboration during backlog refinement sessions. By addressing essential components, BITCAR helps professionals like business analysts and architects create better stories that align with business needs.
BITCAR- Agile, Data Warehouse
- 1. Attili, Phani Mnemonic/Collaboration Technique/Framework (BITCAR). Aims to provide a high level guidance enable better Story & Acceptance writing for Data Warehouse projects BITCAR- Agile, Data Warehouse
- 2. PhainAttili Product Owner BITCAR Abstract My first attempt to introduce a Mnemonic/Collaboration Technique/Framework (BITCAR). Aims to provide ahighlevel guidance enable betterStory&Acceptance writingforDataWarehouse projects. Mnemonic that helps Business Analysts/Business System Analysts/Product Owners stay focused on the essentials for the DW Story/Acceptance. This also helps Data Architects and other development communityrolesinthe SCRUMteamto easilyconsume the information/acceptance withgreaterclarity. Provides a basic learning technique that can enable more collaboration in Backlog refinement session, uncover some common questions we often miss, improve participation among the team and Business thru multiple questions which could lead to better stories/acceptance, and identify the dependencies upfront. Everyletterinthe FrameworkBITCARhas significance inbuildingabetterstoryandacceptance. B: Business I: Integration T: Table/s C: Columns/s A: Association R: Representation Post is only intended to focus on better story/acceptance building for Data Warehouse programs, I intentionally did not cover the basics of Agile/SCRUM frame work with an assumption that users/readersof the frameworkare alreadyaware of Agile/SCRUMprocessesandartifacts.
- 3. PhainAttili Product Owner BITCAR Contents Target Audience.................................................................................................................................4 Challenge & Goal................................................................................................................................4 Framework –BITCAR (Business)...........................................................................................................5 Framework –BITCAR (Integration).......................................................................................................6 Framework –BITCAR (Table)................................................................................................................7 Framework –BITCAR (Columns)...........................................................................................................8 Framework –BITCAR (Association).......................................................................................................9 Framework –BITCAR (Representation)...............................................................................................10 Conclusion .......................................................................................................................................10 Example -BITCARframework.............................................................................................................11
- 4. PhainAttili Product Owner BITCAR Target Audience BusinessSystemAnalysts BusinessAnalysts IT/A Product Owners Data Engineers Data Modelers Data Architects SolutionArchitects SCRUM Masters/TPMs BusinessStakeholders CrossFunctional Teams Challenge & Goal The Challenge in Data Warehouse projects lies in detailing the essential components that contribute to a story or acceptance which can demonstrate to the Business value/Impact. The value/Impact can be operational reporting,Analytics,Self Service,DashBoardsand/or Metrics/KPIs. The Goal of the paper is to provide a basic learning technique that can enable more collaboration in Backlog refinement session, uncover some common questions we often miss, improve participation among the team and Business thru multiple questions which could lead to better stories/acceptance, and identifythe dependenciesupfront.
- 5. PhainAttili Product Owner BITCAR Framework –BITCAR (Business) B is the main driving factor for any of the story and acceptance. B – Reminds the BSA/PO/DA/APO about what business value a particular story is trying to achieve, Business Impact if any, Business Attributes, Business Continuity, Business Need. Following are the areas that would cover B. It is not necessary thatevery component described below be utilized in Storywriting and acceptance, few can be utilized at an epic level or a feature level as well. We need to ensure that relevant components are coveredbasedonthe goal of a particularstory andhow itworksfor the team. B BusinessValue Is thisgoinghelp Businessanalytics? Businessinperformance measurement? BusinessinCompliance reporting? Businessin buildingmetrics? BusinessAttributes What BusinessAttributes/BusinessdataSetsaparticularstory istryingto capture? What are the BusinessDefinitionsforthose attributes? How the BusinessAttributes are derived/calculated? How longthe Businessattributesneedtobe storedinthe Data Warehouse? BusinessImpact What BusinessImpacta particulardataset would create, if notavailable What Businesscando betterwith the datawe are planningtoSource? How Long BusinesscanwaitwithoutthisparticulardataSetsinDW What isthe BusinessPriorityof the targetdata setoverrestof the stories BusinessPointof Contacts BusinessContinuity Is newdata Acquisition fromanew Source? Is newdata Acquisitionfromanexisting Source? Identifythe datasources? Will the new data froma New/Existingsource impactthe downstreamprocesses? Will anyexistingdatafeedsbe affectedbynew dataacquisition? How frequently will the existingdatabeingconsumedbyBusiness? How longdowe have to maintainthe new andexistingfeeds? BusinessNeed What isthe Businessneedforthisdata – isthisinterimneed? DoesBusinessneedhistorical data? How muchhistorygoingforward doesthe businessneed? Doesthe Businessneedthe dataforspecifictime period? BusinessProcess Are there any BusinessProcessesassociatedinthe upstreamwhichwill resultinto multiple statesof andataset or an attributes DoesBusinessrequires retentionof changestothe dataover time? Doesthe existingBusinessprocessneedtobe modifiedtomeetBusinessdata Needs? Are there any new BusinessProcesses thatwill impactthe dataacquisition? What isthe impactto the data collectiondue tothe upstream/downstream processfailures?
- 6. PhainAttili Product Owner BITCAR Framework –BITCAR (Integration) I standsfor integration,whichisthe heartof anyDataware house story. In general the BSA/BA/POwill have to partnerwithSolutionArchitectstobuildthe requirements/Stories aroundthe Solutiontomeet the businessneeds. It isnot necessarythatall the pointsmentionedbelow are applicableeverytime.Basedonthe Solution and Businessneeds, the followingtopicsshouldbe covered. I Is the Integration plannedthruCustom OrganizationApproachesinPlace Do we have the right Businesseventsinplace? Do we have the right primarydrivingfactorsinplace? Do we knowthe tablesthat wouldbe impactedbythese businesseventsortriggers? DoesthisintegrationworkforGlobal datasets? How doesthisdata relate toany existingdataalreadyinthe data warehouse? Doesthisintegration require multiplestate changesof attributes? Is the approach scalable? Doesthis provide reference pattern? Is the integrationplannedthruHighWaterMark Low Watermark Approach Do we have the DrivingTable inplace? Do we have the right clarityof Change Sequence IDs? Is the IntegrationPlannedthrua Directpull anddailyrefresh Can the dailyvolumesbe supportedbythisapproach? What isthe frequencyof the dataextract? If the data needstobe restated,how can we getthe previousstate of data thru this approach? Do we need BusinessRulesinordertointegrate dataacross differentdatasetsfromdifferent sources? Is the integration plannedthruB2B Do we have right securitymeasuresinplace? How muchdata at whatfrequencycanbe exchangedthruthisapproach? How isthe communicationmanagedthru processwhenfailurehappens? Is the IntegrationPlannedthruthe SOA What do we need tohave the subscriptionprocess? Do we have informationonadaptorstobe utilized? How muchdata can we persistinthe integrationlayer?
- 7. PhainAttili Product Owner BITCAR Framework –BITCAR (Table) Thissegmentof the frameworkdetailsthe essentialsthata BSA/DA/POneedtounderstandand incorporate inthe stories/acceptancewithrespectto Tables. T What are the Source Tablesto be Targeted? What isthe data that a particulartable contains? How doesthatTable addressthe Businessneeds? WhichSource Schema doesa particulartable belong? Is thisa newtable the teamis goingtoacquire ? Is thisan existingtablewhere we needbringinmore data? Is thisa reference table? Is the data a transaction where changes basedonbusinessprocesses? What isthe rightextractiontechniquetobe used? What isthe data extractionfrequency? What isthe historyrequirementof thistable? What isthe currentvolume andexpectedgrowthof the data? Are there any special requirementstoconsiderwhichmaycause performance issues? How doesthistable fitintooverall datamodel? Do we needto transformthe source data? Do we have Data IntegrityChecksandQualitychecksplanned? What are the differentsourcesthatare goingto feeda particulartable? Do we have an approach to differentiate multiple source feedsintoone table? Didwe profile the table tounderstandthe dataina Source Table? What definesthe primarykeyforthe source table? Isthe PKa systemgeneratedidentifier? Is there a businessdefineduniqueidentifier?
- 8. PhainAttili Product Owner BITCAR Framework –BITCAR (Columns) Thissegmentof the frameworkdetailsthe essentialsthata BSA/DA/POneedtounderstandand incorporate inthe stories/acceptancewithrespectto Columns C What are the Source Tablesforthe columnsthatneedtobe targeted? Do we have definitionsforthe columns? Do we know the businessusage of thiscolumn? Are there any Flex Columnsthatneedtobe sourcedin? Are there any Fillercolumnsthatneedtobe sourcedin? Do we knowthe Source Data Type and Lengthsforthe columns? What columnsneedtosupportMulti Byte/Unicode? What isthe minimum/maximumdatavaluesthatcanbe expectedinagivencolumn? Do we have the Desiredplumbingcolumnsinplace? Do we have the right foreignkey columnsinplace toassociate othertables? Doesorderof the columnshave anysignificance? How dodifferentdatatypesof multiple sources resultinchanges tothe Target table? Do we knowwhat columnscanbe null andnot null? Do we knowthe expecteddatavaluesthatcouldgetpopulatedinthe target columns? Will new Columns orrecordsalterexistingPKsinBase ordownstreamin Package?
- 9. PhainAttili Product Owner BITCAR Framework –BITCAR (Association) ThisSegmentof the framework detailsthe essentialsthatneedbe coveredaspart of the acceptance/storywithrespecttothe Associationsof differentdatasetsandtables A How will multipletablesassociatedwitheachotheronthe source side? How will multipletables/datasets be associatedatthe Targetside? Doesthe Integrationapproachprovide associationsbetweenmultiple datasets? Do the Joins/Associationsonthe Source Side result inmultiple datasetsorsingle datasetonTarget side? Are there any multi-levelassociationsbetweenthe tables thatneedtobe performedtoachieve businessobjectives? Do the Triggers/Businesseventsonthe primarydrivingtableshelpupdate associatedtables? Do we have all the associationsunderstoodtorepresentdatasetsat multiplelevels? Do any level of associationsbetweenthe tablescreate anyperformance challenges? Can we associate the newdatawithany existingdatasetstomeetbusinessneeds? Do changesinthe integrationlayeraffectanyexistingtablesandassociations? Do we have any downstreamimpactsbecause of anynew associationsformed?
- 10. PhainAttili Product Owner BITCAR Framework –BITCAR (Representation) Thissegmentof the frameworkdetailsthe essentialsthatneedto be covered toensure the right Representationof dataisachievedperthe businessneeds. R What isthe Grainof the data to be representedin? Do we needto bringdata to a base layerwithoutlotof transformations? Do we needto bringthe data intoa packagedform? Do we needto enable the datafor reportrepresentation? Do we needto enable the dataforDashboards? What level of normalizationisneeded? Can the requirementbe metthrougha VIEWor doesthe data needtobe in a physical table? Shouldthe data be summarizedbasedonspecific dimensions? What businessrules ortransformationsare requiredtoachieve particularrepresentation? Do we needa trendbasedrepresentation? Do we needhistoricviewof the selecteddata? Do we needto representthe interimstate changesof anydata sets? Do businessneedself-service reporting? Do we needto publishthe reportsoutsidethe firewall? Do we needto schedule andpublishthe reports? What type of securityisneeded? Do we needa schemalayeror do the usershave database skills? Conclusion Thisframework/Mnemoniccanbe utilizedinmultiple waysinaddressingthe essentialsof DW backlog/stories/acceptance.Notall the attributesof everysegmentof the frameworkwill be applicable for eachstory.But dependingonthe goal of the story,a BSA/PO/DA/Teamcancollaborate with the Businessandotherprojectstakeholders suchas SolutionandETL to jointlyownthe storiesforbetter collaborationandclarity. It’snotnecessarythatTeam adoptsto this,if itworksfor yourteam or not dependsonthe SCRUMteams.
- 11. PhainAttili Product Owner BITCAR Example -BITCAR framework Example 1: Assume thatBusinesshasa needtobringinspecificDatasegmenttoDW to utilize for AnalyticsPurpose. Business • Business Attributes List • Attribute Definition- Business • Business Rules • Business needs this data to be retained for N years Integration • Incremental Approach • Frequency of Extarction • Timing of the extraction Tables • List of tables - Source • Tables Nature • Meta data about the Table • Volume of the Records • PK & FK Columns • Mapping of Business Attributes Vs Columns • Data Types • Data Formats • Field Definitions • Distinct Values • Null /Not Null • Association Association • Any transaformations Involved • Any Joins/Associations betweeen Multiple Table Representation • DW Base Layer • DW Package Layer BITCAR
- 12. PhainAttili Product Owner BITCAR Example 2: Assume thatBusinesshasa needto bringinspecificDatasegmenttoDW to utilize forReporting Purpose Business • Business Attributes List • Attribute Definition- Business • Business needs this data to be retained for N years • Business Roll up details • Business users • Business users concurrent • Business Accepted down time Integration • Incremental Approach • Frequency of Extarction • Timing of the extraction Tables • List of tables - Source • Tables Nature Columns • Mapping of Business Attributes Vs Columns Association • Any transaformations Involved • Any Joins/Associations betweeen Multiple Table • Any Aggegations Representation • DW Base Layer • DW Package • Business Requested Report Template • Report SLA for Referesh • Drill Up/Drill Down • Pull/Push Report BITCAR
- 13. PhainAttili Product Owner BITCAR Glossary BSA BusinesssystemAnalyst PO ProductOwner TPM Technical ProgramManager BI BusinessIntelligence DW Dataware House ETL Extract TransformLoad DA Data Architecture/DataArchitect B2B BusinesstoBusiness SOA Service Oriented Architecture