IT Summit - Modernizing Enterprise Analytics: the IT Story

Editor's Notes

• #5: Simple, approachable, accessible and understandable Analytics should be easy and fun – something you want to do
• #6: versus “simplistic tool to do one thing” or “don’t make me think” versus “no human in the loop” algorithms versus “automating drudgery” Leverage skills but don’t require them – best practices should be built-in
• #7: Effective data analysis is based on experimentation Our definition of usability should focus on these more than learnability, intuitiveness, memorability, etc.
• #8: We need to give people the tools to persuasively communicate their insights. It is a tragedy when three weeks of analysis become a single static chart in PowerPoint. True change comes form analytics @ scale.
• #10: Ellie start Self-service dominates market requirements: According to Gartner, “It is very likely that 2014 will be a critical year in which the task of making ‘hard types of analysis easy’ for an expanded set of users, along with ensuring, governance, sales and performance for larger amounts of diverse data, will continue to dominate BI market requirements.”
• #12: History tells us that each time, society has fundamentally transformed, it’s because of innovation in technology and transformative leaders like all of you, with a vision, driving change to the masses. We need to… Re-imagine our IT processes and how we support our business Next
• #13: We need to…. Not only empower users with self-service at scale, but deliver it with governance, security, scalability, availability, monitoring and management, Tableau server enables all of these… For the rest of this session, I am going to start building out the architecture blue print of Tableau starting with the basics and going deeper.
• #14: Story: A large bank You heard a few people downloading and using tableau desktop licenses. Soon you heard more than a few people downloading desktop Someone is now running a server under the desk More and more people are using that server Hang on a second – is this server governed, managed, secure, monitored well? So you download the server to play with it yourself.
• #15: Going from a few people using Tableau to a large enterprise deployment can happen much faster than you may anticipate.
• #16: In IT We have too much on our plate. Infrastructure teams are driving toward private clouds, embracing converged infrastructure and have little time to understand every application they have to deploy, monitor and manage. Every application needs integration to the enterprise technology fabric that takes time and effort. And all of this needs to be monitored and managed end to end. However, if we understand Tableau architecture, we can provide the best guidance both to the infrastructure teams and COE’ service desk teams on how to best deliver a high quality of service with Tableau.
• #17: Lets start at the Top. 
• #18: Today, enterprise have data across any number of data sources across a variety of technologies. For us to be ready for the future enterprise, we cannot get locked into one technology choice. Heterogeneous connectivity empowers users to work with their data where ever it is, and gives IT the choice to host the data in the most appropriate technologies
• #19: Tableau end users connect to data from multiple clients. Tableau Server enables self service analytics at scale with security, governance, shared meta data, scalability, high availability and performance. Lets start drilling into the Tableau Server box top down and understand the various components so we can begin to understand what goes on under the hood.
• #20: Tableau Server – logically – contains various tiers. User Tier Storage Tier Management Tier
• #21: Drilling in further, each of these tiers uses specific services and process that deliver capabilities to end users and to IT. Example: User Tier However, we are still looking at a logical view of the architecture. How does this map to physically what gets installed and what each component does?
• #22: Base install components include server processes like the cluster controller, co-ordination service which are always installed on every node in a cluster. This is why you will notice the “base install” in your server configuration dialog, if you have ever noticed it.
• #23: The gateway is apache (under the hood) and it serves primarily as a request handler for all incoming requests. It also does some level of caching (starting in v9) Uses sticky session for routing between cluster nodes Requires an external load balancer in front of a distributed cluster to avoid disruption with process failure.
• #24: The repository is powered by postgres – a highly scalable and performant relational store that houses data. The schema for repository is documented and available for you to use to build your own integrations/customized workflows.
• #25: Search and browse is powered by SOLR a very powerful search technology that powers many internet sites. Search and Browse services deliver the content browsing and administrative user experiences through an application the “App Server” (aka the vizportal.exe process)
• #26: The Application Server handles all of the authentication. The flow for authentication is completely separate from the heavy workload pipeline, this is a key thing to remember and understand if you plan to load test your servers.
• #27: http://www.tableau.com/support/drivers
• #28: VizQL server is the workhorse doing much of the heavy lifting
• #33: Lets see what happens when a user loads a viz. that is extract based. A user always comes in to a cluster via an external load balancer (if one exists) or the gateway on the primary. The gateway reroutes the request to one of the workers that has vizql servers on it. The vizql server checkin with the data engine to see if the required extract is available on the machine for the workbook. The data engine queries the file store and loads the appropriate extract into memory and services the request back to vizql. Vizql server then completes the rendering and serves up the viz to the end users client. The only thing new in this process compared to previous releases is that the data engine now requires a file store on each box it exists. The file store is responsible for maintaining copies of the extracts synchronized across all nodes in the cluster. Now what happens if one of the critical components fails? <Next>
• #34: We can integrate Tableau in our enterprise standards for user authentication. Easily integrated to Active Directory Services Identity management suits like PingFederate, Siteminder, ADFS etc for SAML Kerberos, Smart Cards, 2 way mutual SSL What this really means, is Tableau allows us to leverage our existing investments in identity management technology to provide a seamless user authentication ============================ We can deliver on enterprise standard user authentication today, with Tableau. We can integrate with AD or leverage industry standard authentication mechanisms such as SAML, Kerberos, OpenID which Tableau supports out of the box. In addition, Tableau has a native implementation of single sign on called trusted tickets which could allow you to navigate tight corners. If our users need to use smart card technologies or client side certificates, we can configure server to leverage those authentication mechanism to provide a seamless experience to users. So, what does this really mean? We can leverage the existing investments we have made in user authentication technologies over the years. You can easily integrate Tableau into your environments to provide a seamless single sign on experience While Tableau server has supported SSL for quite some time now, we have recently strengthened that support to include 2 Way Mutual SSL which is key both government and non-government enterprises. IP whitelisting ensures we only accept safe incoming conversation, you can further harden your deployments to connect to our repository using SSL, should you choose.
• #35: Good to authenticate user, but authorization is critical for governance Kerberos constraints based delegation can deliver row level security Flexible User Filters published to data server to accomplish same Ensuring no data leaks is a critical part of data governance. We also need to adhere to compliance regulations. It’s important to be able to manage permissions across groups like HR and other groups like sales. =============================== Know that the user is who he/she is, is the first critical step. But we need to ensure that authenticated users only get access to the data they are authorized to see for governed projects. Many of you have implemented data authorization in your back end databases, such as SQL Server or SAP HANA. By the way, if you missed the recent announcement, Tableau is now SAP HANA certified. With Kerberos constraints based delegation, you can deliver row level security to your enterprise with ease. This will ensure that only people that are authorized to use the data have access to it. If you environment doesn’t allow the use of Kerberos, you can deploy Tableau Server with User Filters that you can publish to the data server. This becomes part of the meta data. So you are probably thinking, this is great – but I want to make sure my marketing users have distinct permissions to both content and data, than the users in the finance organization.
• #38: Server Core License Cost: $650,000
• #39: Tolerance of Down time correlates to maturity as a business relying on data to make decisions
• #40: 40 Cores = 1.3 million 1000 @ 10% Concurrency = 16 Cores 3000 @ 05% Concurrency = 32 Cores 5000 @ 05% Concurrency = 44 Cores Trade Offs: Separate Backgrounders avoid infrastructure resource conflicts and scale extract refresh Doubling VizQL and Cache Servers on Worker 1 help scale user views Setup needs more RAM Distributed Deployment without HA for repository Repository doesn’t support automatic failover.
• #41: This is an example from the field to show what a “real architecture” can potentially look like.
• #43: We can scale up or scale out Tableau Server Describe how to scale out / scale up We can easily break out and isolate workloads, like background processing and scale them individually =============================== You can work with user bases large and small and can connect to data sources whether they are big data or simple text files storing critical information that matters to you.
• #44: We understand there are many aspects to finding the deployment architecture that is best for you and that will scale well. However, these two affect our capacity and sizing conversations the most in providing you with the best guidance. The horizontal axis shows choices on how frequently you need your data to be refreshed to make effective business decisions. The vertical axis shows choices on how frequently your users use analytics to make effective business decisions. There is a good chance that the next time someone sits down with you about sizing, they will likely have a similar conversation with you. In addition to these variables, we expect that you are authoring workbooks following performance best practices. There are several resources we can provide you (in addition to whitepapers) that will set you up for success on ensuring you and anyone on your team is able to design performant workbooks. So, why don’t we explore a few use cases together, so you get a real hang of it. <<NEXT>>
• #45: Lets consider a ship room meeting where you review the status of a software release cycle. Several people get into the room, perhaps the release co-ordinator or project manager is sharing the latest status on open bugs, assigned owners and making decisions on ship blocking bugs. This is a critical business process, but it happens once weekly at the start of the release and daily as we get closer to the release. Also, its most important that data be up to date when the release co-ordinator has this meeting. So, for these use cases, the release co-ordinator will require data is updated once a day. <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #46: Lets another example. A business process around patient capacity in a hospital ward. In such scenarios, the data needs to be updated more frequently as patients check in and check out. If this is an ER, perhaps the data needs more frequent updates than one hour. Perhaps 15 minutes. However, several people access this workbook a bit more frequently than the engineering ship room process. So you should partner with your business teams to figure out what their business process calls for. <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #47: Last but not least, we all have scenarios where you need to have a live data feed. However the need to do analytics on that live feed may be variable. Lets take Air Traffic Control Systems for an example. You cannot make effective decisions if you cannot see the real time data feeds from the radars on the altitude, velocity and other critical vectors of an aircraft. These analytics scenarios need real time feeds. Tableau, can connect to many live data sources and all you have to do is ‘refresh’ the view to get the latest. However, you have different scalability considerations when you have scenarios like these. <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #48: If we filled out a grid with various examples, perhaps you arrive at the following permutations. Is this a full universe of possibilities, absolutely not. You may decide that your data needs refreshes every 15 minutes. That is OK. Some of you may say we do everything….(that is a popular IT joke, I have heard) What we just did, is together build a framework to ask questions of your business teams or if you are in business, share insights with your IT team so you could have an effective conversation around scalability and capacity planning. <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #49: Here is how it specifically impacts scalability and capacity planning. We added external query caching in 9.0. Caching servers require more system resources. Scenarios in the green are great scenarios where external query caching will be very helpful. So more cache servers are perhaps your decision point. Scenarios on the bottom, aren’t as effective for leveraging external query caching techniques. To be clear thought, external query cache is but only one caching mechanism in the product. All the scenarios will benefit from alternate caching techniques like browser caching, browser rendering, session caching etc. which don’t have deployment significance but still benefit end users. <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #50: Lets get concrete. If you are finding that you are going to onboard more users or that your business scenarios are changing where the same people will use more and more frequent analytics, then the server process that are most effective to scale (up or out) are the vizql server, data server, if you are using published data source, data engine and the cache server. On the other hand, if you are finding that to make effective decisions, you need to support more frequent data updates then you need to scale backgrounders. When you are connective live predominantly, backgrounders don’t help with data refresh. However, you will still need backgrounders to process your subscriptions and other system jobs. When you are planning out your deployments, hopefully you will keep these considerations in mind. While we are here though, why don’t we use the time to work through a couple scenarios. Alan, what use cases do we have in store today? <<NEXT>> INTERNAL NOTE: These capacity requirements two dependencies are additive, but are shown here as a matrix for simplicity.
• #52: Tableau is introducing a new paradigm for working with data. We want to invite and encourage people to ask and answer more questions from their data. We want them to get into a “flow” with their data and explore new territory. To be successful we need to provide really fast response times. If updates take more than a couple seconds then people switch from being engaged with their data to “simply running reports”.
• #53: We have invested heavily in Performance in version 9.0, both on the desktop and server side. We have introduced capabilities that will drive faster queries, make working with extracts faster and also drive significant end user experience when using server for various scenarios. <Next>
• #54: Here is a simple example from a Flights data source which has XXX (100million?) rows of data. In this case, the queries that were executing sequentially in 8.3 took ~25 seconds to render the entire dashboard. In v9, the same dashboard, running queries in parallel, loaded in 12 seconds. Note how many of the queries shown in “green” are running together. This is a huge gain. <Next>
• #56: Delivering mission critical highly available services depends on not just the software but also sound operational practices and infrastructure Tableau builds in multiple layers of availability 3. Cluster Level (ELB) 4. Process Level (Resource Monitoring) 5. Data Level (Next Slide: Repository, failure detection, automatic failover workflow) ======================= Delivering on mission critical services depends on not just the software but also sound operational practices and infrastructure. From a software perspective, Tableau has multiple levels of availability built into it. Resource Monitoring and Service Restoration at a Process Levels ensure that if a process fails, we restore service and let you know. If you are not a large enterprise with lots of investment in infrastructure, monitoring and alerting, you can still benefit from these process level availability capabilities because these will work no matter your deployment – large or small. Process level availability is built in for everyone, you just need to ensure you provide sufficient infrastructure to run at least 2 of each server processes. As we extend that to workers and other worker components like gateways, having an External Load Balancer in front of the cluster, extends high availability to Workers. If a worker goes down because of a hardware failure, an ELB can ensure continuity of service. That’s great, but what about the repository?
• #57: Intro Triggered by legitimate failover{ and the new ‘tabadmin failoverRepository’ command (Click) Couple quick points to setup the next illustration First,{ the crown indicates the Cluster Controller Leader It’s purely coincidence{ that it happens to be the Cluster Controller on the Primary{ in this example. {(Click) And second,{ to simplify the illustration… I’m combining all the Coordination process boxes{ into one{ which represents the entire ensemble. Okay, here we go.
• #58: 1 – Active Repo goes down 2 – Local Cluster Controller notices and updates state in coordination service leader 3 – Coordination Leader updates the other Coordination nodes 4 – Leader Cluster Controller sees the change, and starts the failover timer (1 minute by default) 5 – If the Repo is still down after 1 minute, then the Leader promotes the Passive to Active Notes: If you have specified a Preferred node and the Preferred is not already active, then it will become Active as long as it is healthy.
• #59: You don’t have to be a hard core developer to build these integrations. A little bit of JAVA Script or a little bit of Python can get you a long ways – but clearly the more you know the more advantage you can take from the power of these APIs. Tableau 8 introduces a rich set of APIs that allow IT to extend the richness and beauty of Tableau visualizations to the enterprise web applications. A user can interact with a Tableau Viz. from within an enterprise web application or the enterprise app can subscribe to events that get fired when a user interacts with a visualization allowing the enterprise application to react to the user interact on the viz. The API provides integration both ways. This powerful API provides new ways of driving adoption of Tableau in the enterprise. It also supports asynch operations requiring longer operations. The Data Extract API allows IT to publish data into a Tableau Data Extract format giving you the flexibility to pre-process your data (outside of Tableau) prior to creating extracts. This is supported in a variety of languages. Lets see a live demonstration of the JAVA SCRIPT API. [NEXT]
• #61: We reviewed a couple of deployment architectures, now lets review the over all server architecture so that you are equipped to understand any scenario deployment
• #63: Can have multiple search services.
• #68: This is an example from the field to show what a “real architecture” can potentially look like.
• #70: Welcome and thank you so much joining us for this session: Scaling Analytic Cultue with Drive
• #71: Is this how you feel right now? You’ve invested 1, 2, 5, 10 years in classic BI tools and now you’re being asked to move. It doesn’t feel right, I get it.
• #72: So why do it? Why take on this Tableau thing. Why now, and for what benefit.
• #73: If there is one thing I hope you get out of today, it is that Tableau is different. It can help you create better workplaces by building analytic culture.
• #74: Let’s briefly go bak and explain why doing what we’re doing now doesn’t work. Just think about it. Every time business gets an idea, IT gets a waterfall development project.
• #75: You do all this work and business users are not even appreciative. Why? Because candidly, they rarely get what they want. If you nail it, they have other questions. If you miss, they’re upset. This is a no win scenario for everyone. It’s like a Kamikaze mission. We know how this is going to end but do it anyway for the honor.
• #76: It is seductive to think that becoming “Agile” will solve the problem.   But simply shortening development cycles and making the business user more involved doesn’t solve the fundamental problem that the creative work is happening ‘in the back.’   Are we asking business users to give up their day-jobs to supervise developers?
• #77: Until the nature of the work changes — until the business use can do it themselves or see it being done in real-time – .. there’s no way to satisfy their business or emotional needs.   We do precisely that with the Cycle of Visual Analysis.   We’re taking all of those development steps, moving most responsibilities to the business user themselves, and collapsing timelines totally by making this a real-time activity.
• #78: So in the old model, IT develops reports and is a bottleneck. In the new model, business does -- with data provided by IT. In this model, IT is an enabler.
• #79: We do self-service very well and in the words of Clayton Christiansen, we’ve distrupted the industry. We did something really well that was really, really important.  
• #80: So is the ability for business users to ask and answer their own questions… a BIG deal or a small deal? Think about it. Tableau is a company that helps people draw pictures with data. That’s all the software does.
• #81: So how do we explain this. Here’s a picture from our annual conference this year. I arrived a few mintues late. There were more than 10,000 people filling the MGM Grand Garden where Mayweather fights.
• #82: The enthusiasm and passion for this product is nuts. It begs a serious explanation. I’ve been with Tableau 3 years and am still trying to take it all in.
• #83: And part of my frustration is that this is how we sell the software: try it you’ll like it. The software is addicting and once people try it, it spreads like wildfire.
• #84: If you’re like me, you’re probably saying… “What value is this software bringing to my users and to my organization.” We take our IT responsibilities serioiusly and we can’t just go with what feels good. We need to know why.
• #85: So let’s start here. In its last annual issue on Great Places to Work, Fortune Magazine makes this observation: Intangible assets, mostly derived from human capital, have rocketed from 17% of the S&P 500’s market value in 1975 to 84% in 2015. What an amazing increase. http://fortune.com/2015/03/05/perfect-workplace/
• #86: And here’s the thing. You can’t force knowledge workers to do their best work. Their effort is basically voluntary. And that’s why culture is so important.
• #87: Workplace culture are those pervasive behaviors, beliefs, and values that set the tempo of the workplace. A rule of thumb is that if half of employees are truly committed others will follow. What I would like to propose today is that when you deploy Tableau widely you are dialing up organizational culture in a very useful and productive way. “Analytic Culture” includes four dimensions which we will explore individually:Knowledge Participation Engagement Thinking You will change the way every employee feels about the workplace and you will change the trajectory of organizational performance.
• #88: First, let’s talk about knowledge: A shared, baseline understanding of the business: who, what, when, where, why, how. Sounds like a fairly conventional value proposition. Why is understanding what is going on so important to individuals and to the organization?
• #89: We are hard wired to be curious – just like dogs or other animals. We want to know… or as researchers would say, “acquiring knowledge when our curiosity is aroused is pleasurable and intrinsically rewarding.”
• #90: We also need to know. Anxiety about the unknown has a strong basis in evolutionary biology. Even if 99% of the time the rustling noise behind the rock was a mouse rather than a lion, the humans who ran away every time – the one who was anxious about unknowns – had better survival rates.  
• #91: Ever seen the words “breaking news” on the television screen? Do you also focus more carefully? We seem to “need to know” how and why bad things happened. That plane crash. Those emails. Bacon and our longevity. Litman, Jordan A. “Curiosity and the pleasures of learning: Wanting and liking new information.” Cognition and Emotion (Psychology Press, 2005), p. 793. Litman, Jordan A., ibid, p. 794.
• #92: What do we do when we don’t understand something? We create narratives. At a personal level, knowledge is not just a passing interest. We are hard-wired to seek to understand. We get very uncomfortable when we don’t. This is not something that can be repressed or outlawed by HR. It must be addressed or morale will be destroyed.
• #93: Organizations have a more prosaic concern: they want employees to make better, more well informed decisions in order to ultimately -- make money, save money, or advance the mission. Data can start conversations as well as settle arguments. It can make us feel better, allow us to think broader and deeper about cause and effect. It helps us find needles in hay-stacks – and prevent the eventual embarassment of missing them.
• #94: Understanding the “who,” “what,” “when,” “where,” “why,” and “how” is vital. Some of our customers are using Tableau in IT to make the report factory go faster and frankly, that’s better than nothing.
• #95: Before we go further, let’s make sure we’re all working with the same definitions. “Report factory” is a place where analysis is done: By specialists Using specialized skills Complex tools Sole” source for credible analysis
• #96: So what do we call our champions, AKA, Tableau “analyst”…
• #97: It turns out that as more people know more things, and they share that knowledge with one another, overall knowledge gain increases exponentially. If we think about organizations as networks, Metcalfe’s law states that the value is proportional to the square of the number of users sharing information. A smart organization has lots of people doing lots of thinking, and sharing it with one another.
• #98: When we simply don’t have a broad enough understanding of what is going on, we can’t make sense of new information. Did the odds actually change or are we just on a hot streak? Is that gesture playful or life threatening? Without a baseline understanding, we cannot decide if it is important or relevant, and how it is relevant. This concept applies to societies as well as problem domains in organizations. The genesis of the much maligned “Common Core” initiative in the United States is to give everyone the ability to understand fact from fiction and to learn more quickly througout their lives. More knowledge makes more learning as well as thoughtful decision making possible.
• #99: Using data to inform decision making also increases the sense of fairness in the organization. Has anyone seen the capuchin monkey fairness experiment? Monkeys are given a rock and their task is to give it back in exchange for food. Equal task for equal reward. What happens when one monkey gets a grape and the other, a cucumber?
• #100: Data is fair. It does not take sides. It is blind to power and politics. Whomever can use it can have power. As Peter Scholtes says:   “Without data, opinion prevails. Where opinion prevails, whoever has power is king.”  
• #101: Data’s usage in an organization tends to rise with its estimation of the value of truth. Science is a great example. As Neil de Grasse Tyson says that:   “Any time scientists disagree, it's because we have insufficient data. Then we can agree on what kind of data to get; we get the data; and the data solves the problem. Either I'm right, or you're right, or we're both wrong. And we move on. That kind of conflict resolution does not exist in politics or religion.”     https://en.wikipedia.org/wiki/Scientific_method#/media/File:The_Scientific_Method_as_an_Ongoing_Process.svg   http://www.brainyquote.com/quotes/authors/n/neil_degrasse_tyson.html#C00H72OzMvyMzopr.99
• #102: In How Google Works, Eric Schmidt and Jonathan Rosenberg say:   “One of the most transformative developments of the Internet Century is the ability to quantify almost any aspect of business. Decisions once based on subjective opinion and anecdotal evidence now rely primarily on data… We don’t seek to convince by saying ‘I think.’ We convince by saying ‘Let me show you.’” Data is so integral to decision-making at Google, they have dual projectors in every conference room: one for data analysis and another for video conferencing or demonstrations.   What does this mean? Organizations deciding with data are telling employees that they can all have power. ADD FOR LONG VERSION   Google prides itself on meritocracy and expects those closest to the issues operationally, to analyze and understand the data best. As Schmidt and Rosenberg describe:   “The executives were debating some technical issues, and doing a rather poor job of it. So a young Googler who had been listening from the corner stepped up and presented several data points to clarify Google’s position. In a meeting crowded with impressive titles, this young woman with the least seniority was obviously the best-informed person in the room. She ultimately carried the day simply by having the best grasp of the facts.”   Has that happened to you? Schmidt, Eric; Rosenberg, Jonathan (2014-09-23). How Google Works (Kindle Locations 1978-1979). Grand Central Publishing. Kindle Edition. Schmidt, Eric; Rosenberg, Jonathan (2014-09-23), ibid.
• #103: Data saturation also allows a different approach to management, a systems approach. In 1990, Peter Senge famously argued that if we react to events in isolation without considering how various systems interact, we will likely hurt morale and suffer from unintended consequences. A systems approach emphasizes interconnectedness, shared responsibility, and root-cause understanding. The “Fifth Discipline” is called Systems Thinking.   
• #104: So knowledge necessary, but I would also argue, insufficient because running an organization is not a cold arithmetic exercise. You don’t just put decisions in and get profits out. Have you ever seen a scientific paper with a single metric or equation on it? Why not?    What if useful facts simply showed up in manila envelope one day? How would your team feel carrying out an action plan based on those facts?    There are companies now selling devices that can understand words and spit out a simple metrics.   Sales are down. Sales are up. Support tickets were flat this month. The KPI says you’re under-performing. You’re fired.   How does that make you feel? Is knowledge (or data) alone, enough propel people organizations ahead?
• #105: Especially when stakes are high, we need to know WHY a decision was taken and the logical PATH (or Journey) to that decision.   Decisions must be explained in such a way that others will understand, refine, and help to execute. Scientists understand this. Their papers are long because building consensus requires good explanations.
• #106: So that leads to the importance of participation – or in our world, data democratization. No strategy, no matter how brilliant, will work without the support necessary to execute it.
• #107:   Have you ever worked in a company where you and your peers felt “out of the loop?” Did that job seem tedious and frustrating? It is human nature that when we can express our opinion and when we have power – even just a little – we participate and we persevere.
• #108: Even our founding fathers understood that. As Abraham Lincoln said, “No man, is good enough to govern another man without that other’s consent.” Some of us always vote. Some of us rarely if ever vote. But the fact that we CAN vote gives us skin in the game, buy-in. Data democracy increases the number and quality of conversations and results in higher quality decisions that with more support for execution.
• #109: What does empowerment look like in manufacturing? Have you heard of the Andon system?On the production line at Toyota, employees are authorized and encouraged to pull the Andon cord (in 2014 they changed the cord to a wireless yellow button), which lights up a board that everyone can see. Engineers and managers then rush to the shop floor to assess the situation; and if no immediate solution can be found, the production line stops. Just like that. Millions of dollars of production per hour, stopped. When defects are discovered, the entire team analyzes every step and process in order to understand root cause. Broadening participation in important decisions is a powerful and proven way to build consensus, morale, and commitment to act. Jeffrey Liker and Michael Hoseus, Toyota Culture: The Heart and Soul of the Toyota Way (Default Book Series. December 2007) http://www.autonews.com/article/20140805/OEM01/140809892/toyota-cutting-the-fabled-andon-cord-symbol-of-toyota-way  
• #110:   There’s a great story about when a large Ford delegation visited the Toyota plant in Kentucky in the 1980s. They allowed him to go anywhere and ask any question, but after one and half hours, the Ford team looked disappointed and stopped the tour. They hadn’t seen “anything unusual.” After Ford left the Toyota team leader debriefed:   “We learned a very valuable lesson today.  We have the same equipment and systems as Ford, but what. [they] did not see was our competitive advantage, which is our people.  We are successful because we have intelligence, caring, highly successful team members.” The Toyota Production System (TPS) is best known for pioneering lean manufacturing and its use of statistical quality controls… but respect for employees and the contribution that each and every one can make Is their lasting competitive advantage and a core value. https://parts.olathetoyota.com/toyota-production-system-principles.html
• #111: Now let’s talk about engagement. What if I could give you knowledge and participation – but forced you to use a tool that was everyone hated using. It was cumbersome, time consuming, had no community of users per se, and did not allow people to feel connected with the mission and purpose of their work. Would you be able to transform analytic culture? No, you wouldn’t.
• #112: Classic economic theory suggests that we are one-dimensional, money-driven, extrinsically motivated creatures. Like donkeys, we move toward carrots and away from sticks.   TC speaker Daniel Pink calls this the Human Operating System 2.0 and it has been the bedrock of management practice since industrialization.   Although easy to understand, it turns out that feeding and striking is not all that effective at getting people like us to do our best work. Bribing us is hard.
• #113: So are we motivated by perks? We have lots of expensive perks at Tableau. Our developers can make fresh eggs with bacon and fresh berry smoothies in the corporate kitchen every day. But many companies provide such benefits today and nobody changes jobs for the breakfast.   What keeps our employees engaged are continuous challenges that provide opportunities for creativity and achievement and the ability to work with other smart, engaged employees. LONG FORM: Fortune magazine says: “Here’s the simple secret of every great place to work: It’s personal—not perkonal. It’s relationship-based, not transaction-based.” http://fortune.com/2015/03/05/perfect-workplace/
• #114: You’ve probably heard of Abraham Maslow’s pyramid of “higher needs.” The top of the pyramid includes things like creativity, problem solving, morality, spontaneity, and clarity. Below (the order is contested and not terribly important) are ideas of esteem, belonging, appreciation and other emotional needs. Maslow observed that while we all like to experiment we are ultimately drawn to activities that align with our human nature, which is fundamentally striving. http://brilliantnurse.com/nclex-prioritization-questions-maslows-hierarchy-needs-theory/
• #115: Maslow was a university psychology professor, but do you know what Maslow’s last book was about? It was a journal about leveraging psychological principles to create great workplaces. Sounds obvious doesn’t it? His book called “Eupsychian Management” was expanded upon and republished recently as “Maslow on Management.” 
• #116: The modern field of “positive psychology,” builds upon work by Maslow, Peter Drucker, Warren Benis, and Douglas McGregor, and others.   A common theme in this research is that extrinsic factors including money, power, and image lose their motivational potency quickly. Over the long run, employees can be demotivated in any number of ways, but motivating them requires creating conditions for motivation.   Understanding motivational DNA is vital in a knowledge workplace because work cannot be forced; it is given voluntarily. Motivation to think – to execute, critique, or create – and to work cooperatively as a team, comes from within. It is “intrinsic.”
• #117: There is broad agreement on four dimensions of intrinsic motivation -- autonomy, mastery, purpose, and community. How organizations support these things dramatically impacts engagement in the workplace. Supporting autonomy includes providing encouragement, and free choice. Autonomy is a natural inclination and when employees have real control over various aspects of their work they are much more satisfied. They do not find themselves blocked by others.   Mastery is about the thrill of progress and experiencing “flow” when doing tasks well matched to our abilities. The upward ratchet of “goldilocks” tasks that build skills and lead to mastery is deeply motivating.   Purpose involves connecting to a cause bigger than self. Being able to provide great customer service or effecting positive change in the world are examples of purpose aligned work. Know how the organization is tracking toward its purpose is extremely rewarding.   Community is the final, deep source of personal happiness. An analytic community is one that connects us together for support, encouragement, and identity.  
• #118: Workplaces that are engaged support these intrinsic motivators. Hopefully you’re not in the second column.
• #119: Research by Gallup and others has shown that engagement pays huge dividends in absenteism, productivity, earning, margins, and income. http://www.marketinginnovators.com/resources/blog/the-business-impact-of-engaged-employees/
• #120: And most workplaces are not doing so well at engaging employees. In developed countries, enagement hovers around 20% on average. Are we just going to sit back and prove Marx right?
• #121: Most of BI is focussed on enhancing the Customer Value Proposition (CVP). But in a knowledge workforce where human capital is responsible for customer satisfaction, we really need to be looking upstream at the Employee Value Proposition (EVP). Why should employees want to work for you? We need to be asking ourselves whether our current BI solution is making your workplace more or less engaging. It’s an important question.
• #122: Frederick Herzberg wrote in 1968 that “People are most satisfied with their jobs (and therefore most motivated) when those jobs give them the opportunity to experience achievement.” Herzberg, Frederik. “One More Time: How Do You Motivate Employees.” Written in 1968. Archived in Best of HBR, Motivating People, January 2003.
• #123: You might have heard of the “Progress Principle.” Teresa Amabile and Steven Kramer asked knowledge workers in 2011 to keep diaries describing “good days” and “bad days.” What they found is that “Of all the things that can boost emotions, motivation, and perceptions during a workday, the single most important is making progress in meaningful work… even a small win—can make all the difference in how they feel and perform.” How does it feel to know that the only chance you have to answer an urgent question is to submit a report request? How does it feel to get the call or email back after some delay, to verify what you have in mind and then to be told when the first iteration may be complete?    Does the report factory model give business users a sense of progress? Does it deliver “small wins” every day?   Teresa Amabile and Steven Kramer, The Power of Small Wins, Harvard Business Review, May 2011.
• #124: Helping users get their work done is important. How that work gets done is also important.    In the software industry, quality of execution matters a lot. As our CTO, Chris Stolte has written in “Our Beliefs,” the Tableau development organization pays lots of attention to “flow,” a term coined by Mihaly Csikszentmihalyi. “MIHAY, CHIKSENTMEHIGH”    Like Maslow, Csikszentmihalyi studied psychologically healthy people. What made rock climbers and artists and high-performing professionals happy and how did they reach a high-performance state? His subjects described a subconscious energy like “water carrying them along” so he called the state “flow.” Flow activities were fun because they were hard, but not too hard. Without challenge they were boring but if too difficult, they produced anxiety. The most enjoyable activities were well matched to the end-user’s ability. Goldilocks tasks. Just right.   Tasks that produced flow offered immediate feedback and clear indicators of progress. During these activities, worry about failure and self-consciousness disappeared. Challenge led to growth and eventually mastery. Participants found that growth incredibly satisfying.   That all should sound very familiar to Tableau users.
• #125: When users try Tableau they want to keep Tableau. It’s the kind of choice that we in many aspects of our lives. As Maslow suggested, when we have choice, we choose those things that more closely align with our authentic selves. Does anyone still carry a blackberry? Are you eating processed foods – twinkies and TV dinners – every day? 
• #126: More children today are building toys from blocks than using finished ones. Rather than reading Encyclopedia Britannica, we are writing Wikipedia for free.
• #127: We love the community that Facebook provides. We want a sense of purpose and mission at work.
• #128: Consider the workplace. How many of you wear a suit and tie to work every day? Do you recognize this office? IBM: 1960s Here we have the fake formalism of The Office: 2000s WeWork is a communal work environment. Obviously more casual: 2015 Do you remember casual Fridays? Is every day sort of casual Fridays these days?
• #129: Zappos is all-in on a humanizing work environment where employees can be themselves.
• #130: Does traditional reporting support those things we know to affect motivation? - Making progress every day. 2. Having control over ones work. 3. Keeping users in flow and supporting their learning growth. 4. Building and reinforcing human relationships. 5. Supporting purpose and meaning? This stuff matters. We need to pay attention.
• #131: Finally, the last leg of the analytic culture table, THINKING.
• #132: Where will the next great idea come from?Will it be a prediction of a powerful computer algorithm?Will a HAL type voice come on the intercom and say… “plastics.” Or will it come from your employees thinking and collaborating. Remember, deciding is different from imagining. Should BI tools be opening or closing minds? Should they enhance or retard our natural curiosity.
• #133: Do you remember Bloom’s taxonomy from grade school or college? What if work continually challenged us to think deeply and analytically.    When we use Tableau we are continually working our way through critical thinking skills: Evaluate Judge Compare Contrast Critique Choose Rate Select   Synthesize Compose Originate Design Construct Plan Create Invent Organize Combine Predict Revise   Analyze Compare Classify Point out Distinguish Infer Select Dissect Specify Distinguish Categorize
• #134: Thinking hard at work makes us smarter. There’s a whole science behind brain exercise called “neroplasticity,” which means that our neural pathways can change throughout our lives. We can compensate for injury or disease – or to achieve higher performance. With the right kind of exercise, an old brain can learn new tricks. I’m going to say this with a straight face: analyzing data with Tableau makes us smarter.  
• #135: Former president of the American Statistical Association, David S. Moore said that data analysis has very important characteristics. It is not the same as abstract math or computer science. It is concrete, applied, and collaborative.   The liberal arts are defined as those subjects or skills that in classical antiquity were considered essential for a free person to know in order to take an active role in civic life. Moore argues that data analysis is so broadly applicable that it too should be considered a liberal art.   Moore, David S. “Statistics Among the Liberal Arts.” Journal of the American Statistical Association (1998, Vol 93, p. 1253).
• #136: Finally, you have probably already heard that interactive, applied learning can increase understanding and retention by more than 50% compared lecture-style instruction. Outside of beginner training with our “Superstore” data, customers are using Tableau in an applied, personally meaningful context. The usage experice is concrete, visual, active and experiential. There is no better way to understand and remember aspects of your business than to analyze its data. Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American journal of Physics, 66, 64.  
• #137: In the last 30 minutes we’ve talked about how this new paradigm of self reliant analytics is a gateway and catalyst to transforming your analytic culture. But this transformation is not possible if Tableau is not deployed broadly.
• #138: OK, DEEP BREATH. Now… the next question is… how do we scale it out?
• #139: Well… lets walk through how most customers do it. Usually a curious person stumbles upon our website and downloads the product. They like it. It spreads like wildfire. This is called “organic growth.” And that goes pretty well for a while… And then... Chaos. We get there deliberately by taking a very traditional enterprise software deployment path. Patchwork look and feel IT push-back on operations, data and security Duplicate data Lack of access to data Servers overloaded Limited end-user adoption Unappreciated users Wasted time and opportunity
• #140: You can think of the organic growth path as “analytics for fun.” Individual effort Self-starter, self-guided Private/rogue data Dashboard “oohs” and “ahs” “Fend for yourself” Narrow base of adoption Analytics at Scale Community effort Shared resources/division of labor Sanctioned, enterprise data Systematic skill building Programmatic support & encouragement Broad-based adoption The best practices path is “analytics at scale.” The key idea in building a community around analytics is to provide “support and encouragement” for new users. Why is that important?
• #141: In order to achieve analytics at scale we need deliberate, programmatic support.
• #142: Most of us in this room are technologists. We love the challenge and adventure of new tools. By contrast, many business users are concerned about the idea of wasting precious time learning new software. They want to know where to go if they get stuck and how much time this is going to take out of their busy day.
• #143: We can plot effort vs. time on a curve – what we commonly call a learning curve. With traditional BI software, the curve is so steep and the payoff so small, that only a small number of users will ever reach the “threshold of adoption.”
• #144: With Tableau, we have the opportunity to move large numbers of users over the hump, to the thrill of self-reliant BI. The key idea behind Drive is to build a program of support and encouragement so that can occur at scale, with a relatively small program office.
• #145: Drive is a four phase framework. In the first phase – Discovery – we ask whether the organization truly wants to empower business users. The organizations that proceed with Drive believe that the substantial benefits of self-service outweigh the costs of enablement. We then do a traditional gap-analysis to determine what is required to have a successful journey. The next step – Prototyping – is when we work with a cross-functional group to flesh out visualization, data governance, technical and operational issues. We also build up a gallery of useful and inspiring work.   After that, during – Foundation Building – we operationalize what we’ve learned to provide the answers, support, and encouragement that business users require. In this phase we setup the Center of Operations to manage the technical aspects and the Center of Evangelism to manage the program. Scale-up is the end-game – its when we introduce the value proposition, tools, and program for changing analytic culture, one business unit at a time. We know that we now have answers to the questions that business users will ask and a strong programmatic framework in place to support them. We’re not just dropping software on the business users; we’re planning and preparing for change in a responsible way.
• #146: Business owns the creative and analytical work. IT is empowered to do what they do best, better. Great visualizations are the beginning, not the end, of adoption. Drive provides a concrete plan that expands the vision and reduces risk in deploying enterprise-wide analytics whether implemented in-house, with Tableau consulting, or partner consulting.
• #147: When you look at the breakdown of roles and responsibilities you can see a partnership that plays to the strengths and capabilities of both IT and business. IT Role Operations Infrastructure Systems Security Data Production environment Business Role Creative work Data requirements Community Helpdesk Evangelism Sandbox environment With a real-time analysis, self-service paradigm, there are new expectations around data which will require significant IT support and investment. For Business, this is no free lunch. Self-service requires that they own the creative work and build an online analytic community that IT simply provisions and operates.
• #148: I’m going to wrap up today with some key ideas for successful Driving.
• #149: First up, Getting Started… If you could do only one thing from my list, this would be the one. Let’s walk through this scenario together. A new user is interested in a license. Who do they contact? What do they need to know to start using Tableau? What resources are available? On a larger scale, if you’re the only one helping new users and trying to onboard a whole department, there’s some trouble ahead. As the number of users grows, it becomes more difficult to effectively manage and support them. The “Getting Started” experience connects new users with necessary resources on your intranet. Think of all the common questions you get: How do I get a license? When is training? Who else is using Tableau in my department? What data sources are available? Goal is to think it through and get users over the hump and onto using Tableau, instead of struggling and giving up. This is the first step to empower users and build your organization’s Tableau community so write it up and get it out there.
• #150: Chances are, if you’re in this room and working to grow adoption, you’re a champion. It’s a critical role in your Tableau community. Tableau champions see the transformational value of Tableau and possess the skills necessary to implement visual best practices and maximize the impact of analyzing data. The skills pyramid develops as you transfer knowledge to enable new users and create other champions. This distributes the workload and avoids the risk of your available time becoming a bottleneck. Since self-starters like you are leading the push forward, others might think, “They know Tableau, no additional training needed,” but I strongly encourage you to overinvest in champion development for each department so they can build teams. Things like classroom training and certification accelerate and qualify champions. If your champions are not true experts, you risk spreading bad practices, not best practices.
• #151: Often when I sit down with a new customer, they’re inclined to reproduce existing crosstab reports, instead of creating guided analytics with visual best practices in mind. Crosstabs are familiar and comfortable and hard to part with after all these years, dare I say like a bad relationship? So why not give it a new lease on life as a crosstab in Tableau Server? In a few rare cases it can work; in most, it doesn’t. We can accomplish much, much more “the Tableau way,” which focuses on understanding data. In the field, I’ve seen this transition most challenging for financial use cases. This is your opportunity to innovate, not replicate. Business value comes from a real-time conversation with data and being in the flow of analysis, taking users from visibility to insights to action.
• #152: To fuel creativity and enable users to take action on the results of their analysis, you must be responsive. Consider the obstacles users face in traditional BI – a serial development process with wait time when new questions need to be answered. Customers with traditional BI systems may also be inclined to apply these standards to Tableau, but it will inhibit users’ ability to create and share insights. Here’s the catch: IT has a responsibility to govern the systems they own, but the cycle of visual analytics is agile and iterative. What can we do? By defining policies and procedures for production, which typically has publishing rights locked down and requires content promotion, and a sandbox, to allow for prototyping, administrators can create stable and secure environments that mitigate risk, ensure successful adoption, and support innovation.
• #153: There are those who say this can be done without IT. Well guess what? They’re wrong. We need IT for supporting operations, while we need business to drive the creative analysis. Teamwork is critical to your project’s success; without open communication or trying to run Tableau Server on the sneak, you’ll never achieve analytics at scale. A great opportunity to foster the Business-IT relationship is during prototyping where a responsive real-time exchange between both sides results in a published dashboard and published data source. Think again about the previous example. It turned into a win-win situation. IT saw the benefits of faster extract times, while Business gained instant feedback from their dashboard interactions. The result: improved customer satisfaction because both sides worked together.
• #154: Everything we’ve just covered sum up to a strategic implementation. While Drive provides a plan to scale enterprise-wide analytics, installing Tableau and completing each Drive phase are not enough to achieve your goals. When done right, it will transform your organization. A strategic implementation requires planning and people who are capable of using and supporting Tableau so aim high, get all the support you need to succeed, and then over deliver.
• #155: Measuring success is one of my favorite areas to cover with customers. How do you know you’re achieving your business objectives throughout the implementation? As with any major initiative, set goals and measure progress from your baseline. Identify both quantitative and qualitative metrics. Consider how you will communicate the results by audience, such as executive sponsors, system/site admins, compliance, and publishers. Some examples include: Capabilities – Tableau, data source and business expertise Execution – Time to ramp, # of active users, # of data sources, # of views Culture – Job satisfaction, data-driven decision making, confidence in the analysis
• #156: If you have not done so already, please check out tableau.com/drive to learn more about Drive.
• #157: We have a portfolio of service offerings to accelerate your Drive deployment and prevent missteps: Discovery Prototyping Best Practices development Custom training Helpdesk Scale-out Assessment Events
• #158: Before closing, let’s discuss one last big idea. Tableau’s mission is to help people see and understand their data. All people. All their data. As an IT community, this is an unsolved problem.
• #159: But there is light at the end of this tunnel. With Tableau and Drive, we may for the first time have line-of-site to achieving this goal. It is doable. Our speakers today are doing it and you can do it too.
• #160: In closing, it is said that one experiment is worth a thousand expert opinions. Let me ask you to talk to the people who are using Tableau frequently and participating in the online forums. I think you’ll find that they’re more engaged in their work and more connected to it. The reason is that Tableau is helping to provide those things that researchers know make us happy: autonomy, mastery, purpose, and community.   It’s also making us smarter and increasing our sense of personal power and ability to persuade others.
• #161: You can think of Tableau as an “invisible hand” driving Analytic Culture in the organization.   Using Tableau to make the report factory churn faster is just a glimmer of what can be achieved with Tableau + Drive. For the first time perhaps, IT has the opportunity to truly make analytic culture a reality. I am confident that you in this room will change the way every employee feels about their job and change the trajectory of your organization’s performance. THANK YOU.