Why Most Migration Projects Fail – Don’t Be a Statistic Webinar
1 like156 views
The document discusses the high failure rate of migration projects, citing that only 16% are completed on time and on budget, largely due to poor planning and unrealistic scoping. It emphasizes best practices for data migration and profiling, as well as the importance of metadata management in ensuring successful migrations. Case studies illustrate the benefits of effective migration strategies, highlighting the potential for cost reduction and improved data governance.
Why Most Migration Projects Fail – Don’t Be a Statistic Webinar
- 1. © Concept Searching 2017 Why Most Migration Projects Fail – Don’t Be a Statistic Michael Paye Chief Technology Officer Concept Searching mikep@conceptsearching.com www.conceptsearching.com marketing@conceptsearching.com Twitter @conceptsearch Robert Piddocke Vice President of Channel and Business Development Concept Searching robertp@conceptsearching.com
- 2. © Concept Searching 2017 Robert Piddocke – Vice President of Channel and Business Development is passionate about information management and governance. He has worked for several information management companies and assisted with a number of migration projects. In addition, he is an information retrieval geek, who has authored two books on SharePoint Search. Michael Paye – Chief Technology Officer at Concept Searching has been the driving force behind many of the company's recent innovations, including the SharePoint Add-in and hybrid search products. He has a wealth of experience across the Microsoft platform and related technologies, and oversees all product development.
- 3. © Concept Searching 2017 Agenda • Who we are and what we do • Success rate and other fun facts • The business requirements • Best practices – data profiling • How long is this going to take? • Records management • Cloud migration • Metadata and auto-classification • The process • Case study • Takeaways
- 4. © Concept Searching 2017 • Company founded in 2002 • Product launched in 2003 • Focus on management of structured and unstructured information • Profitable, debt free • Technology Platform • Delivered as a web service • Automatic concept identification, content tagging, auto-classification, taxonomy management • Only statistical vendor that can extract conceptual metadata • 8 years KMWorld ‘100 Companies that Matter in Knowledge Management’ 8 years KMWorld ‘Trend Setting Product’ • Authority to Operate enterprise wide US Air Force, NETCON US Army, and Canadian SLSA • Client base: Fortune 500/1000 organizations in Healthcare, Financial Services, Manufacturing, Energy, Professional Services, Pharmaceutical, Public sector and DoD • Microsoft Gold Certification in Application Development • Member of SharePoint PAC and TAP programs • Deployed as a full trust Add-in for all versions of SharePoint on-premises and SharePoint Online, including the latest vNext dedicated platform and the government cloud The Global Leader in Managed Metadata Solutions
- 5. © Concept Searching 2017 Concept Searching’s technology platforms deliver semantic metadata generation, auto-classification and taxonomy/Term Store management, and are fully integrated with all versions of SharePoint on-premises, Microsoft Online/Office 365, and OneDrive for Business What Do We Do? These infrastructure platforms integrate not only with SharePoint but also other content repositories, search engines and file shares, enabling our clients to add structure and manage their enterprise content, regardless of environment The resulting classification metadata is used by clients to deliver ‘intelligent metadata solutions’ in areas such as enhanced search, migration, data privacy, records management, policy enforcement, compliance, text analytics, and business and social collaboration
- 6. © Concept Searching 2017 Unique Approach – Compound Term Processing • Remains unique in the industry • Ability to identify and correctly weight multi-word concepts in unstructured text 6 Concept Searching provides Automatic Concept Term Extraction Triple Baseball Three Heart Organ Center Bypass Highway Avoid
- 7. © Concept Searching 2017 Success Rate and Other Fun Facts • Only 16% of migration projects were on time and on budget • 64% were late • Estimate for overtime overruns was 40% • 37% were over budget • More than half of survey respondents blamed poor, inadequate or unrealistic scoping caused budget overruns (Bloor Research Data Migration White Paper) MIGRATION IS AN OPPORTUNITY
- 8. © Concept Searching 2017 Large Scale Projects Don’t Do Any Better • Only 16% of these did not experience either time or cost overruns • 19% of projects did not have a separate budget or a timescale for data migration, so these could not be measured • Of the remainder • 3% went over budget but were delivered on time • 46% went over time but not over budget • 51% went over both time and budget • How can you run over time but not go over budget? • Appears to be easier to exceed timescales and stay within budget than to do the reverse
- 9. © Concept Searching 2017 The Business Requirements Critical First Steps • Identify what information exists • Determine what may need migrating • Determine how the business needs to use the information • Is migration the right response? • How to maintain business continuity • Who needs access and how quickly? • How is the information used? • How do you need to read, edit, reuse, print, publish, share or make use of the information? • Is it being kept because of regulatory or historical preservation? • Is the content still within the retention period? • How long do you need to keep it? • Can it be archived or deleted?
- 10. © Concept Searching 2017 There is a gross underestimation of just how complex and challenging a data migration can become. It is often perceived as a data ‘shunt and grunt’ exercise, tagged on to the end of the much higher visibility target implementation. Data Migration Pro
- 11. © Concept Searching 2017 Best Practices – Data Profiling • The quality of your project planning will influence any application that uses metadata after the migration • Search, data protection, records management, eDiscovery, text analytics • 90% do not use data profiling* – ? • Include quality analysis • Almost all of you use hand coding • Reactive or proactive? • Data profiling by hand is typically unfeasible documents (Bloor Research)
- 12. © Concept Searching 2017 Best Practices – Data Profiling • What content is moving? • What content can we get rid of? ROT • How can it be grouped? • What content requires special handling – privacy, undeclared records? • What content requires changes? • Most likely the metadata • How volatile is the content? • Data profiling outputs include lists that detail • Content to be migrated in logical groups • Content that requires special handling • Content that will require changes along with the scope • Do it before time and budget estimates
- 13. © Concept Searching 2017
- 14. © Concept Searching 2017 Content Optimization • 1% of corporate information is on litigation hold, 5% is in a records retention category, 25% has current business value • This means 69% of data most organizations keep can, and should, be deleted (CGOC) • 60% of documents are obsolete (eLaw) • 3%-5% of files are lost or misplaced – annual losses to a Fortune 1000 company with one million files is $5M (Survey reported in Information Week) • Companies typically misfile 2% - 7% of their records (New York City Chapter, ARMA International) • Costs $4-$7 to recreate it • US managers spend an average of 4 weeks per year searching for misfiled, mislabeled, untracked, or lost papers (Cuadro Associates) • 90% of content is never accessed after creation
- 15. © Concept Searching 2017 Best Practices – Content Optimization Content Optimization • De-duplication • Versioning • Identification of records that were never declared • Identification of content containing security vulnerabilities • Elimination of redundant, obsolete, and trivial content • Addresses noncompliant content and content that should be archived • Can be used to auto-classify content from diverse internal or external repositories 95% of your data is unstructured Annual growth rate of unstructured content is 120%
- 16. © Concept Searching 2017 Migration of Records ISO standard 15489-1:2001 Defines records management as the field of management responsible for the efficient and systematic control of the creation, receipt, maintenance, use, and disposition of records, including the processes for capturing and maintaining evidence of and information about business activities and transactions in the form of records
- 17. © Concept Searching 2017 Best of Breed? Hardly Solutions can be categorized into five broad categories 1. Preserving the original technology used to create or store the records 2. Emulating the original technology on new platforms 3. Migrating the software necessary to retrieve, deliver, and use the records 4. Migrating the records to up-to-date formats 5. Converting records to standard forms Each has pros and cons
- 18. © Concept Searching 2017 Fundamental Problems in Records Management? • The problem of content migration is not specific to records systems – it is a universal problem • Hampers the ability to manage content over a period of time, impacting data privacy, eDiscovery, litigation • The first generation of electronic records management system specifications – everything from the US DoD 5015.2 that came out in 1998 to MoReq2 that came out in 2008 – did not attempt to tackle the problem • They told vendors what types of metadata to put into their products – but they did not tell vendors how to implement that metadata
- 19. © Concept Searching 2017
- 20. © Concept Searching 2017 Cloud Migration 1. Planning and assessment 2. Duplicate environments 3. Staff training 4. Third party migration tools 5. Leases and licenses 6. Migration consulting
- 21. © Concept Searching 2017 Exchange • Emails are opened by only one third of recipients • About 90% is junk • Still must be managed • Classification of emails and attachments as content is created or ingested • Identifies unprotected security vulnerabilities, as well as confidential information as defined by an organization • Removes breach from search • Sends to a secure repository, or person • Disables download unless authorized • Added value for compliance officers who can identify vulnerabilities within the content • Can migrate content to other content sources • Retains existing security AIIM notes that 87% of organizations are concerned about cloud chaos. The desire to shift strategies to embrace newer content management tools has refocused attention around .migration and disposition.
- 22. © Concept Searching 2017 Reducing the Quality of Metadata The process of migration is lossy • Enormous amount of time to move metatada • Most will not manually map content to new system counterpart metadata • What are the approaches? • Maintain the original application • Cost of licenses fees and support • Users will need to go to the old repository to retrieve information – sooner or later they will forget • Try to connect search policies to new and old system and execute search • Poor search results and ranking • Leave old application, fix new only • Poor ranking • Users need access to old system
- 23. © Concept Searching 2017 Auto-classification in Intelligent Migration Solution • Content migration is an opportunity to bring governance to existing processes • Ability to address high value content, content that should be deleted, duplicate documents, records that were never declared, unprotected privacy exposures • Normalize systems that may have fallen out of compliance • Opportunity to get rid of dark data, ROT, content garbage • File shares to file shares, file shares to SharePoint, SharePoint to SharePoint, custom action from any other repository – .NET code and web services, plug-in architecture to custom develop content sources and destination sources After the migration, content is organized in a hierarchical format, managed internally, and is comprised of multi-term metadata that is unique to each discrete piece of content
- 24. © Concept Searching 2017 How Long Is This Going to Take? Specifically for a migration to the web, but most are applicable to an on-premises environment • Content needs to be rewritten for the web because it is not up to a certain standard or has been copied directly from traditional media • Content needs to be reorganized to fit a new information architecture • Content is old or irrelevant and hasn’t been touched in the last five years • Most of the content is no longer performing and not attracting many visitors • No recent content quality analysis based on usage and conversion exists • Project managers underestimate effort to manually convert and enter content • Content is not fully separated from presentations, making automation difficult • Metadata around content needs to be regenerated because classifications or other systems have changed – retagging content becomes a major pain point • Persistency of URLs can become a problem after automated content migration – URLs can change and follow different patterns, turning defining and implementing redirect systems into a difficult new project
- 25. © Concept Searching 2017
- 26. © Concept Searching 2017 Situation: • Global automotive organization • 40,000 users Challenge: • Migration of over 20 million documents from SharePoint on-premises to the Office 365 dedicated vNext platform • Improved enterprise search and collaboration across 30 content sources • Simplified access to information for a variety of stakeholders Solution: • conceptClassifier for Office 365 platform Benefits: • Cost reduction – decommission of 50 on-premises servers to 5 • Content now auto-classified and searchable in the cloud • Ease of access to information • Improved business production Case Study – Not the Norm
- 27. © Concept Searching 2017 How Did the Process Work? • Create a taxonomy • Taxonomy designed for subject-matter experts • Easy to use • Begin auto-classification • Update taxonomy using the taxonomy prompt ‘Suggest Clues from Class’ • Reiterate for content optimization, security breaches, records • Index and classify the whole corpus in alignment with business requirements
- 28. © Concept Searching 2017 What Was the Result? • Reduced on-premises servers from 50 servers to 5 • Achieved immediate improvements in enterprise search and eDiscovery, enabled concept based searching • Accomplished in two weeks • Successful migration of 20 million documents
- 29. © Concept Searching 2017 Takeaways • See migration as an OPPORTUNITY • Reduce your risk through content optimization • Planning is a key component • Failure due to poor planning and budget overruns
- 30. © Concept Searching 2017 Thank You Michael Paye Chief Technology Officer Concept Searching mikep@conceptsearching.com www.conceptsearching.com marketing@conceptsearching.com Twitter @conceptsearch Robert Piddocke Vice President of Channel and Business Development Concept Searching robertp@conceptsearching.com