DataOps: Nine steps to transform your data science impact Strata London May 18

Editor's Notes

• #11: The average colleague doesn't want to be a data person any more than I want the an accountant. You have to hire like Google. Data people who happen to make good product owners.
• #16: Star wars is not a metaphor for good vs Evil but Waterfall vs Agile
• #17: Too much wastage in the process and hard to impact customers directly
• #20: The key to adding value is to adapt and borrow principles from Agile Software development starting with alignment of data science with the rest of the organisation and it's goals.
• #26: Work only on the organisation’s biggest strategic objectives – those stakeholders have aligned behind. Objectives the business hypothesises will add the most value.
• #27: We don’t know upfront what is going to work.
• #35: DataOps is an automated, process-oriented methodology, used by big data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics.[1] DataOps applies to the entire data lifecycle[2] from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.[3] From a process and methodology perspective, DataOps applies Agile software development, DevOps[3] and the statistical process control used in lean manufacturing, to data analytics.[4] In DataOps, development of new analytics is streamlined using Agile software development, an iterative project management methodology that replaces the traditional Waterfall sequential methodology. Studies show that software development projects complete significantly faster and with far fewer defects when Agile Development is used. The Agile methodology is particularly effective in environments where requirements are quickly evolving — a situation well known to data analytics professionals.[5] DevOps focuses on continuous delivery by leveraging on-demand IT resources and by automating test and deployment of analytics. This merging of software development and IT operations has improved velocity, quality, predictability and scale of software engineering and deployment. Borrowing methods from DevOps, DataOps seeks to bring these same improvements to data analytics.[3] Like lean manufacturing, DataOps utilizes statistical process control (SPC) to monitor and control the data analytics pipeline. With SPC in place, the data flowing through an operational system is constantly monitored and verified to be working. If an anomaly occurs, the data analytics team can be notified through an automated alert.[6] DataOps seeks to provide the tools, processes, and organizational structures to cope with this significant increase in data.[7] Automation streamlines the daily demands of managing large integrated databases, freeing the data team to develop new analytics in a more efficient and effective way.[9] DataOps embraces the need to manage many sources of data, numerous data pipelines and a wide variety of transformations.[3] DataOps seeks to increase velocity, reliability, and quality of data analytics.[10] It emphasizes communication, collaboration, integration, automation, measurement and cooperation between data scientists, analysts, data/ETL(extract, transform, load) engineers, information technology (IT), and quality assurance/governance.[11] It aims to help organizations rapidly produce insight, turn that insight into operational tools, and continuously improve analytic operations and performance.[11]
• #38: This is sometimes really hard for Data Scientists who experiment with data on laptops to accept.
• #40: Add Data and Logic Tests
• #41: Version control is the foundation upon which a lot of delivery is built. At a minimum, reviewers of a publication and future researchers should be able to:1) Download all data and software used to generate the results.2) Run tests and review source code to verify correctness.3) Run a build process to execute the computation. Version control makes it possible to maintain an archived version of the code used to produce a particular result.  Examples include Git and Subversion. 3) Automated build systems document the high-level structure of a computation: which programs process which data, what outputs they produce, etc.  Examples include Make and Ant.
• #42: Configuration management tools document the details of the computational environment where the result was produced, including the programming languages, libraries, and system-level software the results depend on.  Examples include package managers like Conda that document a set of packages, containers like Docker that also document system software, and virtual machines that actually contain the entire environment needed to run a computation.
• #43: In an enterprise setting where multiple data scientists could be working on a single project, the first step to doing data science work that scales is implementing version control, whether that’s GitHub, GitLab, Bitbucket, or another solution. Once your team has the ability to track code changes, the next step is to create a process in which they regularly commit their code to the master branch of your repository.
• #44: 2) During development, automated tests make programs more likely to be correct; they also tend to improve code quality.  During review, they provide evidence of correctness, and for future researchers they provide what is often the most useful form of documentation.  Examples include unittest and nose for Python and JUnit for Java.
• #45: You can move beyond Continuous Integration to make deployment even faster. Traditionally, data science deployment has been a multi-step process that puts the onus on engineering: Engineers would refactor, test, and automate or schedule a data scientist’s model before slowly rolling it out, sometimes months after it was originally built. Developers that embrace continuous delivery are pushing new application features or changes into production quickly, sometimes with the click of a button. Increasingly, cloud and data science platforms are filling this void with features such as the ability to deploy models as APIs or schedule code runs which means that as soon as new development passes your tests it can be deployed into production with no dependencies on other teams.
• #46: For IT teams managing the systems that support models in production and data science environments, the ability to monitor and add resources as data science work expands  — while maintaining system availability — is essential. But that’s just one application. For IT teams managing the resources needed for every deployed model and data science environment across an entire company, a data science platform that offers cluster management features and the ability for IT to dictate the size of the resources made available to data science teams can go a long way toward achieving continuous operations.
• #47: Which brings me on to tools Just as chemistry is not about the tubes but the process of experimentation. DataOps is not tied to a particular technology, architecture, tool, language or framework. However, some tools are better at supporting DataOps collaboration, orchestration, agility, quality, security, access and ease of use.
• #48: Whenever, choosing tools is best to never start with the tools themselves. I like to use the spine model by Tre the wey, Roux and Perold. So to decide on the tool you need to understand the practices you employ, in order to understand what practices to employ you need to define your principles, to define your principles you need to know your values, and to know your values you need start with the needs you’re trying to fulfil. We have a set of clear DataOps Practices we want to employ so we have a clear idea of what tools will be fit for purpose. http://spinemodel.info/explanation/introduction
• #49: But first a bit of advice. You should avoid building your own anything or being on the bleeding edge. Any technology or tool that is really useful will end up being refined or commoditised and turned into a service. Let someone else find the bugs, be the beta tester or end up in cul-de-sac. The other factor to take into account is Cost of Delay. It’s nearly always ignored in business cases. On paper be cheaper to build your own solution. However, the months, or years, you’re taking to do that is months, or years, you’re not benefiting from the solution and handing to your competitors. And it always takes twice as long to build your own solutions, even after you’ve factored in it’s going to take you twice as long as you think.
• #50: Because one of our principles is that we want to make data cycles shorter and shorter it’s important Data Scientists can self-serve not just the data but also the infrastructure, tools and packages
• #51: Modern Cloud architecture makes it very easy to create data products rapidly. Specifically, the move from Infrastructure and Platform as a Service to Software as a Service and Serverless architecture. That means you having not hardware or software to configure, you just bring your data and code and all the scaling and optimisation is done for you. The other advantage is you can use the same tools for dev and production, You can also the same data in dev and production as in the SAAS or Serverless word there’s no need for separation of environments. We’re so convinced of the benefits we’re actually moving our Data Analytics stack out of AWS onto Google Cloud Platform. Here’s an example of GCP reference architecture for Big data which isn’t a million miles from our architecture. There’s absolutely no infrastrucutre to manage within the environment. The other thing you can do is use the cloud as a centralised platform helping to break down organisational barriers and makes it easier to enforce governance rules.
• #53: Modern cloud takes care of the underlying tools but you can add further levels of abstraction and self-service to the compute infrastructure and data pipeline. Data Science Platforms provide tools that enable teams to work faster and deploy DataOps methodology very easily from choosing the computer infrastructure and environments to run their code on, to automated version control, collaboration tools and one-click deployment to APIs and Dashboards.
• #54: The requirements for this type of platform haven’t gone unnoticed, these are just some of the vendors we looked at before settling on Domino Data Labs. Each has their strengths and weaknesses, so which one is best depends on your use-cases.
• #55: There’s another positive side-effect of going down the DataOps route. You require fewer roles due to self-service. There no need for Specialist Dev Ops, Infrastructure Engineers, Sys Admins or DBAs. This reduces friction, hand-offs, bottlenecks You’re left with just four key roles, Data Scientists, Data Engineers, Data Analysts (who are a much under-invested in group as everyone wants to be a Data Scientist) and the Line of Business (these are the stakeholders and also those who will help integrate your Data Product into other applications.
• #56: Worrying About Artificial Intelligence when you can’t even produce a Sales report is not going to get you very far. You need to worry about being able to action data instead in alignment with the organisation’s strategy and goals.
• #59: 80% of the battle is knowing what not to work on. You should not work on projects but products. Products are in constant use by consumers and have direct customer and business benefit. The benefit scales according to the number of customers who use them. A data product may be a machine learning model, a segmentation, a recommendation engine, a dashboard. They may be integrated into other products. They have an owner, you get feedback that helps you improve them through iteration. They are not one-off adhoc pieces of insight that get filed away.
• #66: Velocity is th
• #67: We need to solve all the problems with Data Science today: Hamster Wheel Analytics – Doing busywork for the organisation that makes us feel good because we’re putting in a lot of effort and clients appreciate but is never going to move the needle. The work we do that’s not repeatable because it was never documented The aimless crash and burn – Where we explore data to find the magical insights without a clear objective or worse the rest of the business has no interest in The Roadblock – Work we do that has no route to the customers because it is blocked by corporate silos, IT, Security, lack of infrastructure, tools or willingness to integrate into an end product and remains on a laptop. Work we do that does make a customer impact which we can’t measure because the feedback loop was never closed.
• #68: Instead we can move to the DataOps World – What I like to call the Data Science Factory. It starts with alignment with the rest of the business’ strategy to create options for Agile Delivery and collaboration to deliver them. Rapid delivery of Data Products because there is the governance, trust in data, self-service and automation A path to the end-consumer and feedback to measure value for the next iteration.