Hardware Workshop 2017: How to Prototype

Editor's Notes

• #2: Introduce the topic Introduce Synapse Introduce yourself, your background, and what you do for Synapse (why are you qualified to talk about this) 
• #3: Hardware product development is inherently risky, given quickly changing market forces, high expectations by consumers, lead times necessary to build physical devices, the cost of development engineering, the cost of tooling and manufacturing bring-up, and many other factors As entrepreneurs, you’re likely risk tolerant, but when it comes to your product, it’s really about being an intentional risk manager The challenge is to identify risks, including the most fundamental business-related product risks to user experience and human factors risks to technical and manufacturing risks, and determine how to best address each.  You have a range of options from risk acceptance and monitoring to several flavors of mitigation Prototyping and testing is one of your mitigation options, and is a powerful approach that can help you make sound design and development decisions and help ensure a successful product 
• #4: You’ll learn just as much from the process of testing your prototype as you will from making it Tried to to test something and realized you can’t? Requires some understanding of the requirements. Worst case scenario: end up writing a new product requirement. The great news: you’ve just identified a gap in your product definition and added clarity to guide your team Build your schedule, budget, and your design as a whole with testing at the front of your mind
• #6: First, here’s Synapse’s traditional product development process.  Recently, this is getting compressed, meaning prototyping has becoming all the MORE important. Usually by the time you hit POC, you have a rough idea of Who, What they need, and some rough product requirements. Not a lot. Maybe a few. As you move into alpha, you’re making looks like, works like units in quantities of more than 1. You may be working with your production manufacturer, and if you’re not you’ll probably use the alpha design files as an engineering package for a contract manufacturer to quote on
• #7: What am I trying to learn? Thoughtful planning is key to your success in prototyping and testing You’re designing a product development strategy that balances your business constraints and objectives with product definition, product quality and reliability, and successful manufacturing.  The challenge is to use your resources efficiently and effectively Keep in mind that unforeseen issues will arise, and you need to preserve some resources to address them You may draw upon a number of risk mitigation strategies, including reverse engineering, modeling & simulation, and prototyping & testing.  The key is to plan. Be intentional about why you use these strategies and ensure that your resources are deployed effectively
• #8: So, how do we plan for prototyping?  We need to first ask the right questions. The first question is about what we’re trying to learn. As much as possible, we are specific about this, and think ahead to the outputs we need from the prototype What we’re trying to learn most often falls into a few different categories The first is product definition and user experience.  These are the most fundamental areas to explore, because you want to make sure early on that you’re developing a product that users want and like.  There are many options for getting at some of the risks in this category (surveys, interviews, benchmarking, etc.), but prototypes are often necessary to get at human factors and user experience features The second category of learning is about technical and engineering challenges.  In this case, we’re addressing risks around technology feasibility, specific engineering design concepts, ability to meet performance requirements within product constraints, and so forth. The third category of learning is essentially an amalgamation of the first two.  Our goal for early stages of the PDP is to develop a product architecture that we’re confident can meet the business, design, functional, and performance requirements set forth, and that it can do so with appropriate design margin.  However, there are almost always tradeoffs inherent in the requirements.  Often, when we exit the Architecture phase, we have a couple of options because we know there are tradeoffs we need to evaluate.  In this case, we will devise a plan for each architecture to evaluate those tradeoffs, often by prototyping and testing.   An example: the tradeoff between product size and battery life.  We see this one all the time.  However, we can’t build a fully functional form-factor prototype during the Architecture phase in order to measure our power consumption, because it would likely result in failures that have nothing to do with the tradeoffs we want to evaluate. Instead, we build a power budget model, which is a spreadsheet-based tool that’s based on the product use cases. From this model, we are confident in the size of the battery required, and we can evaluate that against the Industrial design targets. If there are tradeoffs that require user input at that point, we’ll build prototypes to evaluate those tradeoffs. Second, we think about who we’re trying to impress, so that we can tailor the prototype design for that audience Who we’re trying to impress has a significant impact on the effort required to develop a prototype. Know your audience. If we’re trying to convince an investor or budget guru that you’ve met your funding milestone, then we might need a prototype that looks good …technical stakeholders within our client organizations that we’re on track, then we might build a less ‘pretty’ prototype that’s more technical in nature …ourselves that a certain design feature is viable, we might build a small, cheap, ugly prototype that we’d never show an investor …a manufacturer, we may prototype parts to be put through a process, such as a weld or manufacturing test, so that the CM can get a jump on stabilizing the process
• #9: What types of prototypes do we make? Looks-like prototypes are designed to explore design and user interaction Works-like prototypes are designed to be functional, and don’t meet industrial design constraints Looks-like/Works-like prototypes represent the production-intent form factor and functionality Your prototypes should also generally be of the minimal complexity possible to test what you need to test.  Again, think ahead to the output you need from that prototype! You can mix and match.  It’s not uncommon for us to build works-like and looks-like prototypes in parallel to tell a story to investors or budget owners about the current engineering state of the product, what the final industrial design intent is, and the product development plan that gets us to manufacturing ramp.
• #10: In parallel with figuring out what type of prototype you should make, you also need to consider how you will test the prototype When we say testing, we’re referring to lots of different types of testing - reliability testing, user interaction studies, regulatory compliance, and performance variation over multiple units are some examples Plan for what you can test on each prototype - some testing will dictate specific materials or construction. If one of your big technical risks is surviving drop testing, using rapid prototype techniques may not be representative and you’ll need real injection molded parts. If you can’t test against your biggest product risks, revisit the prototype plan. You’ve got a limited budget and can only afford to build a small number of units - making a test plan that puts the same unit through multiple tests from least to most destructive can maximize your testing ability. Figuring out how many units to test to get a statistically significant sample isn’t easy (the answer is not, as popularly believed, always 30) - it can be helpful to have someone do some analysis on your test plan One of the biggest issues we see with testing is people launching into a new build before they get test results from the previous build - if you have limited budget, build your schedule so you learn from your testing! Don’t rush to hold a schedule… it will only hurt you in the long run. Don’t ignore bad or ambiguous test results to keep to your schedule, react to them and update your prototype plan
• #11: As a hardware startup, you likely have a small team and people are filling multiple roles and stretching Maybe you can’t afford a team of experts all the time, but you can ask for help at some critical points Critical points that we recommend  reviews include Build plan drafts Effective plan? Are you mitigating the right things? Detailed design reviews prior to releasing data for custom PCBs, tooling, or expensive mechanical prototyping Test data reviews to ensure that you’re reaching the correct conclusions from the data and developing a solid plan for next steps
• #13: One of the projects Synapse has worked on that makes a good case study for prototyping is a stadium entry system for the 49ers candlestick park. They wanted to speed up and improve their fan entry experience and to own and be able to analyze the data coming from the entry system. 
• #14: Because the user experience was key for this product, we wanted to make a working prototype that we could take to a game and collect data and test user interactions with as quickly as possible. This was important to build support for the project within the 49ers organization, so they were our key stakeholder to impress. We started with some VERY rough prototypes made out of a Quaker Oats canister to look at some proposed ID form factors. When those looked good we build a working 3D printed prototype with off the shelf electronics in 5 weeks We then spent 4 weeks testing it. 
• #15: Here’s at POC unit at a game. We learned a lot through this process - for example, we decided that the product didn’t need a display. Real world users will interact with your product in ways you don’t expect, like this woman on the right trying to scan her hand instead of her phone or a ticket.   Now that we have this awesome working prototype, turn on the production line, right? Nope.
• #16: Next comes the hard part - moving to a looks like, works like, manufacturable design For this build, we wanted to learn whether we were ready to turn on the production line and it was really about proving that to ourselves This prototype we built of the stadium entry system used production representative materials and processes and it took 12 weeks to design and build (and 8 weeks to test!) - much longer than the POC units 
• #17: Kezar used a BeagleBone Black and off the self boards for sensors and connectivity in the POC units. The wiring was a mess, but that’s OK, we weren’t making many and speed to field test was the primary goal. For the alpha units, we were getting ready for production and building in bigger quantities so we transitioned from the BeagleBone Black to a system on module that’s designed for small to medium volume production but based on the same processor as the beaglebone black to keep SW/FW development moving smoothly. A BeagleBone Black or a Raspberry Pi or an Arduino in a box is not a production ready product. Those boards have a supply chain that’s set up for educational purposes and for people to buy in low quantities. They can be changed, modified, or end of lifed at any time. They’re not tested for performance over wide temp ranges, and you should be very careful about building a product in any significant volume around them. Have a transition plan and keep your production requirements in mind! Kezar’s a good example of a product that the Beaglebone processor was a good fit for, but do you need that much computing power? Would a smaller, lower cost, lower power processor be better for your application?
• #18: The Airfloss is a device in the Philips Oral Healthcare product family intended to augment flossing It’s fairly compact, and injects a high-pressure mix of air and liquid, either water or an antiseptic like Listerine, into the interstitials between teeth
• #19: The first example of prototype we built for the Airfloss was an early breadboard prototype, or proof of concept We knew our pump system could be risky, even from a high-level conceptual standpoint, and we wanted to learn about some particulars around component interactions, system friction, and so forth We were trying to impress our client’s technical team with this prototype, so we didn’t worry about its appearance. We were trying to determine whether we could power a piston pump for air delivery in the device using a reasonably sized spring and motor.  In this case, we used an oversized motor and monitored its power draw to verify the motor sizing we had predicted, and we tested a range of springs within a small size range to determine characteristics that would yield the output force we needed.  We also verified the use of a rack and pinion interface between the drivetrain and piston The system worked, and we learned about some details of component sizing and important interfaces It’s important to point out here that the design was inspired by that of an Airsof gun.  Reverse engineering is an important risk management tool that can help you identify viable technical concepts early and concentrate your design and prototyping efforts in ways that prevent you from reinventing the wheel
• #20: The next breadboard prototype we built was to help us understand whether our integrated fluid delivery system worked within the volume that we knew was required based on the industrial design In this case, we were trying to impress the client’s technical team and convince ourselves that the integrated system would deliver the output mechanical energy required And, the architecture we were verifying was to simultaneously drive the piston pump to deliver air and a peristaltic pump to deliver water to the mixing chamber The motor and drivetrain were not final, detailed designs, but they were closely representative of the selected architecture We built confidence in our selected architecture from a functional standpoint, and were convinced that we could fit the system within the target form factor provided by the Industrial Design team
• #21: The next step was to develop an integrated prototype to help us understand a few things Does our sealing strategy for the housing work as required?  We had an IP rating we had to meet since this thing is used in the bathroom, often over the sink. Do our miniaturization details work?  We were packing a lot of stuff in a small box, including a complex mechanical drivetrain, a battery, a circuit board, wiring, a fluid reservoir, and a mixing chamber.  Does it still function and perform as intended?  What sort of assembly gotchas will we uncover? How will it perform in drop testing? In this case, we were trying to impress the client’s technical and executive teams, and ourselves again We built a set of Looks-like/Works-like prototypes and performed an extensive suite of tests on them, with special emphasis on drop performance We identified a number of component interfaces that needed to be updated, as well as assembly and reliability concerns that needed to be addressed After fairly extensive cycle testing of units from this build, we discovered some structural failures of the drivetrain components
• #22: We needed to understand the cause of those failures, so we put together some prototypes to hold the integrated fluid delivery system securely in place outside the enclosure so we could film the system with a high-speed camera This is a good example of the educated unforeseen.  In this case, the team suspected that this sort of thing could happen, but the dynamics are nonlinear, and it would be difficult to predict the specific failure mode and effects. Ultimately, we had to prototype it
• #23: For our next case study, we’re going to look at a GPS watch that Synapse built with Nike and TomTom This one’s a good example of the prototyping challenges you run into when you try to fit high performance electronics into a small, wearable form factor
• #24: This product was interesting because it had two key stakeholders with different priorities The TomTom brand is about high performance GPS, so that was their primary concern The Nike brand is all about design, so we needed to fit that high performance GPS in a form factor that fits well and is comfortable to run in. When we looked at the overall schedule to design this product (which you should always do), we identified firmware development as the critical path. It’s very difficult for a FW engineer to debug on a tiny board, and we can’t test out multiple electrical design options in the small form factor of the end product. On almost all of our space constrained, ID driven products, we do a special development board spin for the FW team. We didn’t only use this prototype for firmware development, we also took it out in the world for urban environment GPS testing to give ourselves confidence in GPS performance to satisfy TomTom. We also used it to inform industrial design by trying out multiple lens designs for the watch by fabricating prototypes that sat on top of the NFF board display. With good planning, you may be able to leverage one prototype to learn multiple things.
• #25: For Nike, our key design stakeholder, we made looks like prototypes The product used a directional antenna that needed to point towards the sky when the user was running. The angle between this antenna and the display affected fit and the range of wrist sizes we could accommodate, so we tried out multiple configurations We wanted a looks-like, FEELS-like model, so we chose cast urethane to make the prototypes durable enough for the designers to take out running. They also looked like injection molded plastics, which was the intended path for these parts. You could easily do a lower cost study with 3D printed parts for many products. 
• #26: Prototypes don’t have to be elaborate, they can be very targeted Industrial designers wanted to hide a USB connector in the end of the watch band, and this required running a flexible printed circuit from the end of the band to the main electronics under the display. Watch bands have to survive large tensile loads - our spec was to survive a 15kg tensile load. As a first pass sanity check, we took some flex circuits lying around the office and pulled them to failure to see if we were in the right ballpark and give us confidence to move forward with the design 
• #27: Once we had confidence in our electronics design from the non-form factor board and knew we had an industial design with a good fit and a decent chance of the flex cable surviving, we were ready to build a looks like / works like prototype There were some things that we couldn’t prove until we moved into form factor - for example, the GPS antenna would perform differently in the final form factor with plastic around it. We wanted to prove that the form factor and design were ready to transition to a CM. We wanted to show TomTom that the GPS worked well and Nike needed looks like / works like prototypes to get executive level buy in for the project. Looks like / works like prototypes take some imagination and creativity: We built the boards with a breakaway header to preserve development and debug ability for the firmware team - thinking through debugging, programming, and testing is important with small form factor designs. What good are test pads that you can’t reach?! Since the PCBs were available before the cast urethane parts, we also used some rapid prototype plastic parts to do fit checks. The cast urethane parts were used again for realistic appearance and shorter lead times than soft tools. We made a trade off in confidence in our reliability tests. Cast urethane wasn’t plan of record but was deemed sufficiently close for this prototype. Again, it comes down to what are we trying to learn from this.
• #28: Here are some detailed resources for prototyping. We’re going to move through these quickly, but you’ll get the deck and can reference them later.
• #29: Can mention sheet metal forming and stamping, etching, laser cutting, weldments, etc. in speaking
• #30: Make sure to mention quantities and variation in test between units Update text & notes
• #31: Here are some good resources for off the shelf electronics components that are helpful in prototyping. Hobbyist boards are great for getting up and running quickly and having a support community. If you know what parts you’re using in the end design, look and see whether the semiconductor manufacturer makes and evaluation board - you can solder multiple evaluation boards together and get your firmware team up and running If your volumes are low, consider modules instead of designing custom boards - especially for radios, where modules often come with certifications and can make sense up to 10s of k units Gumstix, a single board computer manufacturer, has an interesting tool to make semi customized boards with standard computing platforms It’s easy to buy low quality batteries, Powerstream is a US distributor that stocks batteries from a decent quality Chinese factory
• #32: Here are some resources for building custom PCBs Lead time and cost can vary quite a bit Expect greater cost and longer lead time for flexible circuits and high density boards with laser drills 
• #33: Once you get custom boards, you’ll need to get the components soldered onto them For very simple boards you can do it yourself, for more complex designs here are some partners we use