SlideShare a Scribd company logo
Google Product Development/Management Process
                Notes taken by Evelyn Rodriguez, http://evelynrodriguez.typepad.com

        From a presentation given on Jan 8, 2003 to Silicon Valley Product Management Association
                                by Google Product Manager Marissa Mayer

Google’s Mission Statement: “Organize the world’s information to make it universally
accessible and useful.”

Formula: Smart people + creative environment + outlet for ideas = innovation

Smart people:
  • High the best
  • Including 50 PhD’s, originated with strong ties to universities and continue strong
       ties to universities (especially Stanford).
  • Also hire range of people with broad experience and non-traditional talents such
       as expert in Italian travel, machine learning, etc.
  • Mostly looking for “enthusiasm and creativity”
  • Very flat organizational structure, not very much hierarchy

Creative Environment:
   • Creative office design. Looks like mix between college campus and playground
       with toys.
   • Called the “GooglePlex”.

Most of this presentation is about “outlet of ideas” and “innovation” part of formula.

Key point stressed several times: User-centered design
User-centered design means building products that people really want and start with
users’ needs and desires for designing products and services.

Cornerstones of Product Development
   1. Build products that people really want
         o Search is #2 after email as most used application on Internet
   2. Generate and capitalize on network effects
   3. Values built into products “don’t be evil”

Process
   1.    Accept ideas from everywhere
   2.    Prioritization – top 100
   3.    Small, agile engineering teams
   4.    Self-organization and visibility
   5.    User-centered design

Accept Ideas From Everywhere

Google Product Development Process                                                                  1
•   Acknowledge that great ideas may come from anyone, anywhere
   •   Have various forums and mediums for idea-collection and participation.
           • Since engineers may be quiet in speaking at meetings, also solicit
              feedback and ideas from
                   i. Collaborative workspaces (on Web – editable pages for ideas and
                      casual conversation) and
                  ii. email.
   •   Employees (called Googlers) are best source for ideas at Google – also some of
       the most intense users of Google with well-formed opinions
   •   Product discussion meeting – like a brainstorming session – every week
           • 6 different ideas per hour average
           • Sometimes they have “themes’ or “categories” for meeting
   •   “We get a lot of our ideas from our users” (including customer support queue)

Prioritization into Top 100 List
   •   Compile all new ideas, currently funded projects and ongoing maintenance efforts
       – anywhere where time and resources are spent or may be spent in future into list
   •   Assign attributes and rank to each item in list
           o How useful will it be to users, will it aid in user retention (decrease churn),
               what is chance for success, does it diversity revenue stream, what is level
               of effort required relative to impact, etc.
           o For each item, rank 1-5 (5=better than 4 and less risk, 4=risky but
               exciting…)
   •   Only 4 and 5’s get any funding and staff to keep the project or start the project
   •   Easier to see priority if you must compare two projects directly - one project
       against another

Small, Agile Engineering Teams
   •   3-person units (like start-ups!)
   •   Unit is a project – they don’t have departments
   •   Unit is co-located (sit next to each other) also with PM
   •   Engineers work on project for 3-4 months, then transition to next project
   •   Very fluid
   •   With 180 engineers, they can work on 60 projects – so they can afford to invest
       on high-risk, high-return projects as well. (They call high-risk projects
       “Googlettes”)
   •   Each project manager works with 9-10 people across units. For example, maybe a
       category such as “Enterprise Infrastructure”
   •   The technical lead in each unit of 3 is responsible for technical excellence of
       project.
   •   Documentation
           o Very sparse, only what is needed in Product Requirements Document
               (PRD)
           o Eric Schmidt: “Late binding decision-making process”


Google Product Development Process                                                        2
o Evolves based on feedback
          o Includes information on general market size, revenue in PRD but believe
            that “if you build something users use, there will be a way to make money”
   • Large Projects
        o Example: Enterprise Product – broken into logical modules, thus 4 units
            (of 3 people) = 12 people
   • Monetization teams
        o Larry Page: “No such thing as a successful failure; if it is useful to people,
            later we can make revenue from it in a logical way.”
        o Focus on providing value to user first.
        o Then create team to execute the “monetization” of most useful
            products/services.
                ! Marissa (speaker) was on team to monetize search
                ! Created AdWords, etc.

Self-organization and Visibility
   •   “Sparrow” Pages
          o Launch calendar
          o Shows each project – when launched, status
   •   Snippets
          o Every Monday sent in by each employee
                 ! Organized by project
                 ! Includes link to employee’s home page
          o Helps with re-use across projects – leveraging what is already being done

User-centered Design
Original minimalist design because founder didn’t really know HTML and did the
simplest design. However, one founder also has Masters degree in human-computer
studies.
        • User studies (1x per week)
        • Focus on quality and ask: What does user really care about?
        • Experimentation
               o Try out on public site and see if users respond well
               o Do statistical analysis on results of use
        • Iteration
               o Make product better over time from feedback
        • Expedient Solutions
               o Better to get it out – “a good (rather than best) solution soon”

User Studies
First User Study:
       • First user study conducted at Stanford University.
       • Offered free pizza to students; marketed with flyers around university – 16
           students came.



Google Product Development Process                                                      3
•   Learned a lot of things about their user’s experience.
       •   Now, recruit through Craig’s List (popular San Fransisco Bay Area, California
           online community site, www.craigslist.com)

Current User Studies:
       • 2 people (they talk to each other more), 1 observer (from Google, does not
          interfere with user’s experience or reactions)
       • “Tell us the parts of the page you see starting from top to bottom and left to
          right.”
              o Found that eyes go to results links first and did not see the rest
                  including advertiser logos, navigation bar, “help”, etc.
       • “Tell us what cached means?”
              o Found too many technical terms that users did not understand.
                  “Cached”, “Boolean search”
              o Better explanations and less assumptions were result.
       • Ask user to find answer to a trivia question using the search engine and
          observe their interaction.
       • They have an in-house user studies lab with a one-way mirror. Software that
          records mouse-clicks.
       • Also, use laptops at bookstores – offer $25 gift certificates for participants in
          study.
       • They test before and after releases.

Quality
       •   Initial spell-checker was low-quality (outsourced); eventually built their own
       •   Learned that “Good quality by itself improves usage”
               o Usage doubled on first day without any changes to user interface
               o Eventually improved user interface “Do you mean…” in distinctive
                    red letters – doubled usage again
               o Larger font doubled usage again
               o Repeated “do you mean” at bottom of screen and doubled usage again

Experimentation
       •   If we have a good idea, try it
       •   Thumbnails of the web page next to results link was failure
               o Trial was Excite@Home users and it was default
               o Did not experiment with internal employees first
               o Both trial and Googlers turned off in preferences
                      • Users said most important to have “information density”
                      • Thumbnails decreased results links “above the fold” (that fit on
                          one screen)
                      • Picture not useful if never visited page before
                      • Learned “any new information must be MORE important than
                          what is lost” – limited screen real estate



Google Product Development Process                                                           4
Iteration
       •    Google News was result of watching engineering email (one employee wrote
            the demo after being frustrated trying to read news after Sept 11 event.
       •    Started as a demo created by one engineer on a weekend
       •    Googlers started to use it to read their news
       •    Then assigned 3 people (one was UI designer) and one PM to work on it
       •    Iteration example
                o Original: all news on one screen
                o Pre-release:
                        ! break into sections with a “google” look and feel
                        ! lots of mock-ups of the layout
                o User interface testing (user studies); Asked “Go to Entertainment
                    Page” but users couldn’t find their way around – poor navigation
                o Next iteration: Didn’t make it to user studies because Googlers hated it
                o Next iteration: Finally got to put something on the public site, but
                    found users not using the navigation bar at all
                o Next iteration: simplified navigation and moved it
       •    They use “test marketing” techniques where they try two styles when they
            can’t decide which is best. Statistical analysis shows which is used more.

Expedient Solutions
   •   Internationalization Example
   •   Originally used a translation company, but not working well; answer came from
       users
   •   Volunteer translators from user community worked on portion of web sites
   •   42,000 volunteer translators signed up (88 languages so far)
   •   How:
           o Created a form for “stings of text” needing translation
           o Quality control
                  ! Confidence ratings for each translator – others vote on how good
                      the translation is
                  ! You can override the current translator by proving your credibility
                      on new strings
                          • New strings are added each week, if your rating is higher,
                              you can start being regular contributor

Conclusion
   •   Use idea gathering and prioritization process that works for your company –
       adjust and adapt to your company
   •   Use a combination and mix techniques – statistical, user studies, etc.
   •   The service has to ultimately provide value to users
   •   Service must be designed FOR users

More Information: Google Labs (engineering demos): http://labs.google.com/



Google Product Development Process                                                       5

More Related Content

PDF
UX Bootcamp - August 2016
PPTX
Putting Personas to Work at IIBA Cleveland
PPTX
Faster Usability Testing in an Agile World - Agile UX Virtual Summit 2017 by ...
PPTX
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
PDF
Exploring UX Techniques and Practices 4 Product Development
PDF
Intro to Agile and Lean UX
PPTX
Putting Personas to Work at UX Pittsburgh
PDF
Rapid Prototyping 2015: Its a Mad Mad World
UX Bootcamp - August 2016
Putting Personas to Work at IIBA Cleveland
Faster Usability Testing in an Agile World - Agile UX Virtual Summit 2017 by ...
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Exploring UX Techniques and Practices 4 Product Development
Intro to Agile and Lean UX
Putting Personas to Work at UX Pittsburgh
Rapid Prototyping 2015: Its a Mad Mad World

What's hot (20)

PDF
Beyond co-creation, be radically transparent
PPTX
Making Great User Experiences, Pittsburgh Scrum MeetUp, Oct 17, 2017
PPTX
The Mobile Frontier
PPT
"So UX Designers Just Make Things Pretty, Right?" and Other Myths Debunked
PDF
User Experience Design: an Overview
PPTX
Lean UX: Building a shared understanding to get out of the deliverables business
PDF
Userchecks - Method for agile usability testing
PDF
General Assembly: Sketch Before You Etch
PPTX
Selling UX in Your Organization at Cleveland World Usability Day (WUD)
PDF
Rapid Product Design in the Wild
PPTX
Product Design in Agile Environments: Making it Work at ProductCamp Pittsburgh
PPTX
Lean UX: Getting out of the deliverables business
PPT
Introduction to Prototyping - Scottish UPA - June 2011
PDF
Design and Creativity Day02
PPTX
The Agile UX Equation: Constructing a Powerful, but Lightweight Process
PDF
Surviving the Hype: An Experimental Framework for Scaling Enterprise Design T...
PPTX
Beyond Staggered Sprints: Integrating User Experience and Agile
PDF
The Agile UX Equation: How to Implement UserZoom Within Your Agile Framework
PPTX
Selfish Accessibility: a11y Camp Toronto 2014
PDF
LeanUX - Presentation slides
Beyond co-creation, be radically transparent
Making Great User Experiences, Pittsburgh Scrum MeetUp, Oct 17, 2017
The Mobile Frontier
"So UX Designers Just Make Things Pretty, Right?" and Other Myths Debunked
User Experience Design: an Overview
Lean UX: Building a shared understanding to get out of the deliverables business
Userchecks - Method for agile usability testing
General Assembly: Sketch Before You Etch
Selling UX in Your Organization at Cleveland World Usability Day (WUD)
Rapid Product Design in the Wild
Product Design in Agile Environments: Making it Work at ProductCamp Pittsburgh
Lean UX: Getting out of the deliverables business
Introduction to Prototyping - Scottish UPA - June 2011
Design and Creativity Day02
The Agile UX Equation: Constructing a Powerful, but Lightweight Process
Surviving the Hype: An Experimental Framework for Scaling Enterprise Design T...
Beyond Staggered Sprints: Integrating User Experience and Agile
The Agile UX Equation: How to Implement UserZoom Within Your Agile Framework
Selfish Accessibility: a11y Camp Toronto 2014
LeanUX - Presentation slides
Ad

Similar to Google Product Development Process (20)

PDF
Google Product Dev Process
PPTX
HCI - 5-6 -Design Process Interaksi Manusi Komputer.pptx
PDF
UX in Action: IBM Watson
PDF
Requirements Engineering for the Humanities
PDF
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
PPTX
Selling UX in Your Organization - Stir Trek 2012
PDF
"Open" includes users - Leverage their input
PDF
Starting your career as UX designer during pandemic
PPTX
Prototype: Its methods, techniques, and key features.
PPTX
What is Prototype,Rapid prototyping and Methods.
PDF
UX Prototyping (UXiD) - Handout by Anton Chandra and Bahni Mahariasha
PDF
Pre-Conference Course: UX and Agile: Making a Great Experience -
PPTX
User Experience Design + Agile: The Good, The Bad, and the Ugly
PPT
KEY
Towards an Agile approach to building application profiles
PPTX
Foundations For A Great User Experience
PDF
Open agile is free and open source community agile-
PPTX
Design Thinking : Prototyping & Testing
PDF
The Design Discovery Jetpack
PPTX
Design Thinking: A Common Sense Process
Google Product Dev Process
HCI - 5-6 -Design Process Interaksi Manusi Komputer.pptx
UX in Action: IBM Watson
Requirements Engineering for the Humanities
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
Selling UX in Your Organization - Stir Trek 2012
"Open" includes users - Leverage their input
Starting your career as UX designer during pandemic
Prototype: Its methods, techniques, and key features.
What is Prototype,Rapid prototyping and Methods.
UX Prototyping (UXiD) - Handout by Anton Chandra and Bahni Mahariasha
Pre-Conference Course: UX and Agile: Making a Great Experience -
User Experience Design + Agile: The Good, The Bad, and the Ugly
Towards an Agile approach to building application profiles
Foundations For A Great User Experience
Open agile is free and open source community agile-
Design Thinking : Prototyping & Testing
The Design Discovery Jetpack
Design Thinking: A Common Sense Process
Ad

Recently uploaded (20)

PDF
August Patch Tuesday
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PDF
NewMind AI Weekly Chronicles - August'25-Week II
PDF
Encapsulation theory and applications.pdf
PDF
Getting Started with Data Integration: FME Form 101
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Accuracy of neural networks in brain wave diagnosis of schizophrenia
PPTX
A Presentation on Touch Screen Technology
PDF
project resource management chapter-09.pdf
PDF
A comparative study of natural language inference in Swahili using monolingua...
PDF
Approach and Philosophy of On baking technology
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
Hindi spoken digit analysis for native and non-native speakers
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
A novel scalable deep ensemble learning framework for big data classification...
PPTX
TLE Review Electricity (Electricity).pptx
PPTX
Programs and apps: productivity, graphics, security and other tools
PPTX
Chapter 5: Probability Theory and Statistics
PDF
Zenith AI: Advanced Artificial Intelligence
August Patch Tuesday
Group 1 Presentation -Planning and Decision Making .pptx
NewMind AI Weekly Chronicles - August'25-Week II
Encapsulation theory and applications.pdf
Getting Started with Data Integration: FME Form 101
Building Integrated photovoltaic BIPV_UPV.pdf
Accuracy of neural networks in brain wave diagnosis of schizophrenia
A Presentation on Touch Screen Technology
project resource management chapter-09.pdf
A comparative study of natural language inference in Swahili using monolingua...
Approach and Philosophy of On baking technology
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
MIND Revenue Release Quarter 2 2025 Press Release
Hindi spoken digit analysis for native and non-native speakers
Digital-Transformation-Roadmap-for-Companies.pptx
A novel scalable deep ensemble learning framework for big data classification...
TLE Review Electricity (Electricity).pptx
Programs and apps: productivity, graphics, security and other tools
Chapter 5: Probability Theory and Statistics
Zenith AI: Advanced Artificial Intelligence

Google Product Development Process

  • 1. Google Product Development/Management Process Notes taken by Evelyn Rodriguez, http://evelynrodriguez.typepad.com From a presentation given on Jan 8, 2003 to Silicon Valley Product Management Association by Google Product Manager Marissa Mayer Google’s Mission Statement: “Organize the world’s information to make it universally accessible and useful.” Formula: Smart people + creative environment + outlet for ideas = innovation Smart people: • High the best • Including 50 PhD’s, originated with strong ties to universities and continue strong ties to universities (especially Stanford). • Also hire range of people with broad experience and non-traditional talents such as expert in Italian travel, machine learning, etc. • Mostly looking for “enthusiasm and creativity” • Very flat organizational structure, not very much hierarchy Creative Environment: • Creative office design. Looks like mix between college campus and playground with toys. • Called the “GooglePlex”. Most of this presentation is about “outlet of ideas” and “innovation” part of formula. Key point stressed several times: User-centered design User-centered design means building products that people really want and start with users’ needs and desires for designing products and services. Cornerstones of Product Development 1. Build products that people really want o Search is #2 after email as most used application on Internet 2. Generate and capitalize on network effects 3. Values built into products “don’t be evil” Process 1. Accept ideas from everywhere 2. Prioritization – top 100 3. Small, agile engineering teams 4. Self-organization and visibility 5. User-centered design Accept Ideas From Everywhere Google Product Development Process 1
  • 2. Acknowledge that great ideas may come from anyone, anywhere • Have various forums and mediums for idea-collection and participation. • Since engineers may be quiet in speaking at meetings, also solicit feedback and ideas from i. Collaborative workspaces (on Web – editable pages for ideas and casual conversation) and ii. email. • Employees (called Googlers) are best source for ideas at Google – also some of the most intense users of Google with well-formed opinions • Product discussion meeting – like a brainstorming session – every week • 6 different ideas per hour average • Sometimes they have “themes’ or “categories” for meeting • “We get a lot of our ideas from our users” (including customer support queue) Prioritization into Top 100 List • Compile all new ideas, currently funded projects and ongoing maintenance efforts – anywhere where time and resources are spent or may be spent in future into list • Assign attributes and rank to each item in list o How useful will it be to users, will it aid in user retention (decrease churn), what is chance for success, does it diversity revenue stream, what is level of effort required relative to impact, etc. o For each item, rank 1-5 (5=better than 4 and less risk, 4=risky but exciting…) • Only 4 and 5’s get any funding and staff to keep the project or start the project • Easier to see priority if you must compare two projects directly - one project against another Small, Agile Engineering Teams • 3-person units (like start-ups!) • Unit is a project – they don’t have departments • Unit is co-located (sit next to each other) also with PM • Engineers work on project for 3-4 months, then transition to next project • Very fluid • With 180 engineers, they can work on 60 projects – so they can afford to invest on high-risk, high-return projects as well. (They call high-risk projects “Googlettes”) • Each project manager works with 9-10 people across units. For example, maybe a category such as “Enterprise Infrastructure” • The technical lead in each unit of 3 is responsible for technical excellence of project. • Documentation o Very sparse, only what is needed in Product Requirements Document (PRD) o Eric Schmidt: “Late binding decision-making process” Google Product Development Process 2
  • 3. o Evolves based on feedback o Includes information on general market size, revenue in PRD but believe that “if you build something users use, there will be a way to make money” • Large Projects o Example: Enterprise Product – broken into logical modules, thus 4 units (of 3 people) = 12 people • Monetization teams o Larry Page: “No such thing as a successful failure; if it is useful to people, later we can make revenue from it in a logical way.” o Focus on providing value to user first. o Then create team to execute the “monetization” of most useful products/services. ! Marissa (speaker) was on team to monetize search ! Created AdWords, etc. Self-organization and Visibility • “Sparrow” Pages o Launch calendar o Shows each project – when launched, status • Snippets o Every Monday sent in by each employee ! Organized by project ! Includes link to employee’s home page o Helps with re-use across projects – leveraging what is already being done User-centered Design Original minimalist design because founder didn’t really know HTML and did the simplest design. However, one founder also has Masters degree in human-computer studies. • User studies (1x per week) • Focus on quality and ask: What does user really care about? • Experimentation o Try out on public site and see if users respond well o Do statistical analysis on results of use • Iteration o Make product better over time from feedback • Expedient Solutions o Better to get it out – “a good (rather than best) solution soon” User Studies First User Study: • First user study conducted at Stanford University. • Offered free pizza to students; marketed with flyers around university – 16 students came. Google Product Development Process 3
  • 4. Learned a lot of things about their user’s experience. • Now, recruit through Craig’s List (popular San Fransisco Bay Area, California online community site, www.craigslist.com) Current User Studies: • 2 people (they talk to each other more), 1 observer (from Google, does not interfere with user’s experience or reactions) • “Tell us the parts of the page you see starting from top to bottom and left to right.” o Found that eyes go to results links first and did not see the rest including advertiser logos, navigation bar, “help”, etc. • “Tell us what cached means?” o Found too many technical terms that users did not understand. “Cached”, “Boolean search” o Better explanations and less assumptions were result. • Ask user to find answer to a trivia question using the search engine and observe their interaction. • They have an in-house user studies lab with a one-way mirror. Software that records mouse-clicks. • Also, use laptops at bookstores – offer $25 gift certificates for participants in study. • They test before and after releases. Quality • Initial spell-checker was low-quality (outsourced); eventually built their own • Learned that “Good quality by itself improves usage” o Usage doubled on first day without any changes to user interface o Eventually improved user interface “Do you mean…” in distinctive red letters – doubled usage again o Larger font doubled usage again o Repeated “do you mean” at bottom of screen and doubled usage again Experimentation • If we have a good idea, try it • Thumbnails of the web page next to results link was failure o Trial was Excite@Home users and it was default o Did not experiment with internal employees first o Both trial and Googlers turned off in preferences • Users said most important to have “information density” • Thumbnails decreased results links “above the fold” (that fit on one screen) • Picture not useful if never visited page before • Learned “any new information must be MORE important than what is lost” – limited screen real estate Google Product Development Process 4
  • 5. Iteration • Google News was result of watching engineering email (one employee wrote the demo after being frustrated trying to read news after Sept 11 event. • Started as a demo created by one engineer on a weekend • Googlers started to use it to read their news • Then assigned 3 people (one was UI designer) and one PM to work on it • Iteration example o Original: all news on one screen o Pre-release: ! break into sections with a “google” look and feel ! lots of mock-ups of the layout o User interface testing (user studies); Asked “Go to Entertainment Page” but users couldn’t find their way around – poor navigation o Next iteration: Didn’t make it to user studies because Googlers hated it o Next iteration: Finally got to put something on the public site, but found users not using the navigation bar at all o Next iteration: simplified navigation and moved it • They use “test marketing” techniques where they try two styles when they can’t decide which is best. Statistical analysis shows which is used more. Expedient Solutions • Internationalization Example • Originally used a translation company, but not working well; answer came from users • Volunteer translators from user community worked on portion of web sites • 42,000 volunteer translators signed up (88 languages so far) • How: o Created a form for “stings of text” needing translation o Quality control ! Confidence ratings for each translator – others vote on how good the translation is ! You can override the current translator by proving your credibility on new strings • New strings are added each week, if your rating is higher, you can start being regular contributor Conclusion • Use idea gathering and prioritization process that works for your company – adjust and adapt to your company • Use a combination and mix techniques – statistical, user studies, etc. • The service has to ultimately provide value to users • Service must be designed FOR users More Information: Google Labs (engineering demos): http://labs.google.com/ Google Product Development Process 5