Big Data Analytics: The Move Toward Rapid Experimentation

Editor's Notes

• #7: Emily : Please check the overall layout, fonts & formatting Naresh – this is the script for this slide Today if you go to any Big data conference, you see 100’s of vendors showcasing technologies, with success stories of how that technology is the silver bullet for your problems. Lot of data gets thrown at you “48% of companies are adopting hadoop to compliment their data warehouse….companies that leveraged social data to complete the customer 360 picture saw 44% better returns on their marketing campaign…etc etc. All these are shown and told in a very glitzy, buzzing environment where just to be part of it…is exciting…. its like a kid in a toy store or if you are like me, maybe a better metaphor is standing in the tools aisle of a home and garden store. We come back excited, with lot of ideas on which technologies, what tools to use what problem / use cases to solve etc etc. The problem with this is that we end up putting too much emphasis on the tools / technology but we forget that each organization is unique, has its unique culture, is in different organization maturity, has different people, has its own secret sauce that made it successful. The technology example you saw might have worked in very different parameters. I have bought a big tool chest, without realizing that I live in an apartment and don’t even have a garage. This laser focus on tools / technology, without taking a step back to look at larger business problems in your own organization context, often causes such projects to fail. “focus on the business problem, leveraging technology, in context with your corporate strategy & culture”
• #8: Emily : Please check the overall layout, fonts & formatting Naresh – this is the script while we just spoke about the challenges with over eager technology adoption approach, the big data analytics market is moving so rapidly that it is giving rise to another challenge, where organization are not able to shift thru the ever changing landscape and seems to be waiting for the market to mature / dust to settle down, before they make a move. If I take the Apache Big Data top level projects as a barometer for innovation, we see there are 29 top level project as of today, and many more in incubation level. Extend this with the number of independent vendors extending these open source projects, and we find ourselves in rapidly evolving market. The map reduce technology which hit prominence just couple of years back is already termed legacy. We are seeing something new in this field every 6 months, however the Time horizon of enterprise decision making is 3 to 5 years due to cost of infra, tech adoption, people training etc. The above gap is causing the imbalance, and is not easy to solve but ask “what is the cost of NOT doing anything”. Remember “your competitors may not be waiting”. In todays uber competitive world 6 month head start in next big idea could be a game changer. I often say “doing is difficult but NOT doing is fatal”
• #9: Emily : Please check the overall layout, fonts & formatting Naresh – The script Lastly what we see in the market is paradigm shift to Unknown Unknow. Let me explain what I mean by that. Almost all of the data problems that we have solved over last couple of decades were from ‘Known Known’ world – ie where we knew what the problem we are trying to solve and also had good understanding of data that is needed to find answers. This was the good old simple world where you pick any book from Ralph Kimball or Bill Inmon and just do what they say and 8 out of 10 times you will be successful. The business case was clear, benefits was clear, requirements were clear, technology choices were also clear so was the process and methodologies. It was a ‘Known Known’ world.   However the current world problems are much more complex, where most of the time we don’t have a clear cut problem scope and almost never have good understanding of the different types of data that has to come together both from internal systems as well as external sources. Business may not know exactly what they need and what problem they want to solve, they at best may have an idea what outcome they want, that too at a very broad level. As we discussed the technology landscape is continuously evolving, and because the entire concept of big data world is so new that hardly there are any best practices established or methodology developed that can guarantee success. So we suddenly find ourselves in this ‘unknown unknown’ world. This ‘unknown unknown’ world where everything is evolving on us viz. problem statements / data / technology / methodology, everything is unknown – the only thing that remains known is High Expectation. We find ourselves with “vague business case and mismanaged expectations”
• #10: Script – the example I am going to share is from one of our retail customer a 27 Billion giant. The engagement was in their Appliance services division. The experiment we setup was to find out causality between symptoms as mentioned by consumer (in service calls) to the parts needed for repair. The business case was to have the technician carry the right part while going for the diagnosis visit so that the problem can be resolved in one trip itself. In current world it was taking on average 2.5 trips to fix the problems. Since the organization already had a big data platform, it enabled us to quickly setup a analytical sandbox on same platform for this experiment. We first Applied text mining on the problem description from consumers, to classify the problems. Then we Correlated the symptoms with spare parts to identify the most likely part for the given problem. We refined the algorithm iteratively by adding other dimensions like appliance make, vintage, # of people in household, weather etc. to increase the prediction accuracy. In the 4 week experiment the team was able to improve the prediction accuracy from 35% to 80%, making it a viable production deployable model. Just for this particular group, based on the reduction of service call visit (from average 2.5 to 1 for 80% of cases) the business benefits were calculated at 9.5M $ per annum. Not a bad ROI for 4 weeks of experiment. However, it would be incorrect to claim this ROI just for this experiment, this was made possible that organization vision to setup an overall Big data analytics platform, we just demonstrated the power of experimentation to find the hidden nuggets or data insights.
• #11: Naresh : I will start this my reminding you that this is not POC or some technology demonstration, it’s about the mindset to find hidden nuggets in your data. To setup the rapid experiments, you need 2 things the platform and the experiments itself. Platform is your organizational knowledge repository, that as we discussed you should establish or lease (today everything is available on pay per use model) keeping your organization strategy, culture in mind. The experiments should be defined clearly, benefit measurement defined, you should be able to develop & refine it in weeks (ideally not more than 6 weeks). The exit criteria should be clear and there should not be any extra pressure to prove it successful. Avoid that trap to continue to to add features to the model to make it successful. If you are not able to prove it in the given time, Just move on to the next thing, Come back and setup it as a new experiment if you believe you have new information. Once you prove the value, make sure you implement in real life. Let it not remain in lab.. The value is realized only when you are able to scale in production environment. I firmly believe the big data analytics has lot of nuggets hidden in it, the challenge is that the benefits are not apparent to start with, hence we need to cultivate this experimentation mindset to be successful.
• #12: Script – the example I am going to share is from one of our retail customer a 27 Billion giant. The engagement was in their Appliance services division. The experiment we setup was to find out causality between symptoms as mentioned by consumer (in service calls) to the parts needed for repair. The business case was to have the technician carry the right part while going for the diagnosis visit so that the problem can be resolved in one trip itself. In current world it was taking on average 2.5 trips to fix the problems. Since the organization already had a big data platform, it enabled us to quickly setup a analytical sandbox on same platform for this experiment. We first Applied text mining on the problem description from consumers, to classify the problems. Then we Correlated the symptoms with spare parts to identify the most likely part for the given problem. We refined the algorithm iteratively by adding other dimensions like appliance make, vintage, # of people in household, weather etc. to increase the prediction accuracy. In the 4 week experiment the team was able to improve the prediction accuracy from 35% to 80%, making it a viable production deployable model. Just for this particular group, based on the reduction of service call visit (from average 2.5 to 1 for 80% of cases) the business benefits were calculated at 9.5M $ per annum. Not a bad ROI for 4 weeks of experiment. However, it would be incorrect to claim this ROI just for this experiment, this was made possible that organization vision to setup an overall Big data analytics platform, we just demonstrated the power of experimentation to find the hidden nuggets or data insights.
• #13: Follow us on our blog : Brillio.com/insights : tweet at brilllioglobal Next Webinar : Designing the Knowledge Repository Reach us at : Tracey Moon ; Naresh Agarwal