Experiments & Pretotyping

Idea of a lean start up develops the confidence in you; that the assumptions and claims you're making are backed up by data. This helps you make a case for your start up. Helps you understand your customer segment and talking to others to validate what you are thinking and proposing is correct. The Art of Prototyping is a MVP to show people. People will believe in what you have to say with this done. Methods for testing as mentioned before include:

1. Interviews ➡️ teach you verbalizable opinions; Mom test
   • Try not to get the opinions I was fishing for
   • Observations ➡️ teach you the un-articulatable, the hidden; IDEO; Thick Data
   • Sit and observe
   • Not everything valuable is measurable / quantifiable
   • Careful with how you use data. Thick vs Big Data
   • When human behaviour enters the equation, data is less reliable.
2. Focus Groups ➡️ diverse views
3. Surveys ➡️ larger $n$ for broader validation
4. Experiments ➡️ Prompt [Non-]Action of earlyvangelists

The general principles of "pretotyping" are:

• Main inquiry: Would people be interested? Would they use it?
• Goal: Building the "Right it" versus Building the "Wrong It"
• Rationale: Data beats opinion - Generate your own data (YODA)
• How: Skin in the Game!! Willingness to purchase.

Pretotyping is the art of pretending. You don't want to spend weeks and months creating a prototype for no-one to be interested. For a pretotype, the entire point is to test certain questions associated with that pretotype.

For example, Jeff Hawkins created a wood model of the Palm Pilot by taking a piece of wood, plastic, and a chopstick to give himself a sense of what the product would be like. A Pinocchio prototype can help test the physical form factor of a product. As it is in fact a dumb prototype, it works best to convince yourself and your team, not others, that your idea is on the right track. The first thing to do when planning any kind of test or experiment, is to figure out what you want to test.

Pretotyping Techniques

The pretotype is the illusion of a product. Here the dumb artifact was the best proxy of the real thing, and allowed him to learn. It validated the product features - "would I use it".

• What? An inanimate (or "dumb") artifact acts as a proxy for the potentially real thing
• Why? Solution doesn't exist and you want to validate product features
• How? Use proxy to validate certain parameters of the product like form factor, features and usability
• Where? Real-life situation where innovation would be used

There are many methods to pretotype (opens in a new tab):

Fake Door

• Description: Test the Initial Level of Interest (ILI) in a yet-to-be-developed product or service by creating artifacts that suggest the product exists and is available to gauge if people would buy it. Advertising new product or feature, tracking click-through & response. Where? Web tech: online ads & landing pages & simple response form
• Example: Placing McSpaghetti on the McDonald's menu to see if anyone orders it, without actually cooking any pasta.

Facade

• Description: Test ILI in an existing but not yet broadly available/scalable product or service by creating artifacts that suggest greater availability or scale. Simulate stable or complex infrastructure for as-yet undeveloped idea by borrowing or renting expensive equipment, space, and assets. Why? Avoid investment in expensive infrastructure while validating & learning about complex process while in motion. How? Deliver experience while communicating current state of product/service
• Example: Bill Gross's experiment with CarsDirect, selling cars online without actual inventory. Also, no tech was involved with Zappos; he delivered the shoes himself. To test the feature, everything else is dropped. And the product is simplified.

Pinocchio

• Description: Create a non-operational version of your product and use your imagination to pretend it works, to see if and how you would use it.
• Example: Jeff Hawkins testing the form factor and usage of a wooden Palm Pilot prototype.

Mechanical Turk

• Description: Before investing in complex mechanisms or back-ends, use human skills to simulate the desired outcome.
• Example: IBM's speech-to-text experiment using a hidden typist to simulate computer processing.

YouTube

• Description: Use videos to make products that don’t yet exist come to life, to see how people react. Have viewers show interest by signing up, tweeting and becoming part of ambassador program.
• Example: Google Glass's introduction via a YouTube video showing the world through the glasses.

Provincial

• Description: Test a new product or service in a smaller, more private and informal context to gauge interest before a large-scale launch.
• Example: BestBuy's NextPlay service tested in a store parking lot for electronic gear swapping.

One-night Stand

• Description: Offer a pretotype version of your product or service on a very limited time basis to gauge interest. Deliver full experience with extremely narrow geo scope and time frame.
• Example: Airbnb founders renting out air-mattresses for one night in San Francisco.

MVP (Minimum Viable Product)

• Description: Create a first iteration of your product with the absolute minimum set of features that would make it valuable.
• Example: The first iPhone version with limited functionality but still high demand.

Infiltrator

• Description: Take advantage of customer traffic in an existing store to place an artifact of your idea to see if people would buy it.
• Example: Upwell Labs' founder putting prototypes in an IKEA store to gauge customer interest.

Impostor

• Description: Use an existing product or service as a starting point for your new idea, modifying it to impersonate your new product.
• Example: Elon Musk's modification of a Lotus roadster to create a prototype for Tesla's electric car.

Skin in the Game

Different types of evidence have different levels of "skin in the game".

Business Experiments

In the book "Testing Business Ideas (opens in a new tab)", they have collated a library of business experiments that can be used to validate your idea:

1. Customer Interview
2. Expert Stakeholder Interviews
3. Partner & Supplier Interviews
4. A Day in the Life
5. Discovery Survey
6. Search Trend Analysis
7. Web Traffic Analysis
8. Discussion Forums
9. Sales Force Feedback
10. Customer Support Analysis
11. Online Ad
12. Link Tracking
13. 404 Test
14. Feature Stub
15. Email Campaign
16. Social Media Campaign
17. Referral Program
18. 3D Print
19. Paper Prototype
20. Storyboard
21. Data Sheet
22. Brochure
23. Explainer Video
24. Boomerang
25. Pretend to Own
26. Product Box
27. Speed Boat
28. Card Sorting
29. Buy a Feature
30. Clickable Prototype
31. Single Feature MVP
32. Mash-Up
33. Concierge
34. Life-Sized Prototype
35. Simple Landing Page
36. Crowdfunding
37. Split Test
38. Presale
39. Validation Survey
40. Wizard of Oz
41. Mock Sale
42. Letter of Intent
43. Pop-Up Store
44. Extreme Programming Spike

Experimentation Evolution

When you have a new idea, you need to run business experiments to reduce risks and uncertainty. Initially, focus on running cheap and quick experiments. Only consider more expensive experiments with greater investment once the risk has been reduced. Why waste your time and resources when you have no evidence?

Ethics of Preto-Typing

• Be honest: Make it known to those that engage that this is a not yet completed product
• Don't be unethical - consider implications and responses thoroughly
• Be up front with and reward those clients that engage with your pretotype
• Lift the veil immediately in dicey situation. Its not worth it.
• Don't make promises you cant keep. If you make promises, than you have to deliver!
• Use case is very limited (don't raise unnecessary expectations, no medical devices, small N only)

Books

• The Startup Owner's Manual, Bob Dorf and Steve Blank
• The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed, Alberto Savoia
• Running Lean: Iterate from Plan A to a Plan That Works, Ash Maurya