OkTrends, the OkCupid blog about data analysis, is a great place to see what is possible with good data analysis. I think if you work in the field of data analysis, you should consider it a work obligation to read OkTrends.

One example of this is how they analyzed profile questions to determine which questions were both casual (easy to bring up) and strong predictors of compatibility. Read their post for the full analysis.
If you just want to cut to the chase, here are the key questions.
If you want to know if you and your date have long term potential, then ask:
  • Do you like horror movies?
  • Have you ever traveled around another country alone?
  • Wouldn’t it be fun to chuck it all and go live on a sailboat?
If you and your date answer the same way to all three questions, congratulations! You are 32% likely to become long term partners, compared to an average chance of 5% otherwise.
On the other hand, if all you want is sex on the first date, you just need to ask:
  • Do you like the taste of beer?
People who like the taste of beer are 60% more likely to have sex on the first date.

This was a great talk by Kathy Sierra this year at SXSW 2009.

She spoke about how to achieve breakthroughs. Basically, when it came to improvement, whether that was personal improvement, product improvement, or company improvement, there is a “big f***ing wall” that stands between you and your goal. At a certain point, incremental  movement will not suffice to get you through the wall.
She spoke about certain kinds of goals. For example, to become an expert, you first have to get through a certain “suck” threshold. But then you are in the land of mediocracy. Experts are the people who just keep on pushing and pushing to improve. She cited a book or study showing that it takes 10,000 hours to become an expert. (Sorry, I didn’t get the citation. If anyone knows, please post a comment.) But not everyone has 10,000 hours – so how can they achieve it in vastly fewer, like say, 1,000 hours.
Here are her roughly 15 ideas for achieving breakthroughs:
  1. Play the Superhero Game. Imagine you had to pick a superpower: either flight or invisibility. Which would you pick, and why? What argument could you make for the other superpower? Now that you are thinking about superpowers, imagine giving a superpower to your users. What superpower would you pick, and what would be a credible case for that super power. She had some good examples: “Photoshop Channels Guy”: this is a credible superpower, because once you have mastered channels, you can do powerful things in Photoshop. “Pivot Table Man” is the equivalent for spreadsheet. But some bad examples are: “Spelling auto-correct man”: it’s just lame. And “Productivity Man” is good for you, but boring as brocolli.
  2. Play the Superset Game. Don’t just take on your competitor. What is the bigger, cooler thing that you can take on? If you blog about your company, that’s not the coolest thing from the perspective of your reader. Imagine you are a cooking appliance company: your readers are passionate about cooking, not the manufacture of cooking appliances.
  3. Deliberate practice. You can use several techniques to really practice, and achieve expertise in <>
  4. Make the right things easy and the wrong things hard. Do you have treadmill equipment gathering cobwebs in the corner? It’s not in the corner because you don’t use it, you don’t use it because it’s in the corner. Take the couch and everthing you sit on out of the media room, and put in exercise balls and exercise equipment.
  5. Get better gear (and/or offer it to your customers). It’s more expensive because it is better. A touchscreen Wacom tablet is just way better than any alternative. A $4,000 horseback riding sadle offers immediate improvement in riding, even for a mediocre rider. Be on the lookout for difficulties justifying the expense: You want more monitors, but your boss things you’ll just be playing games.
  6. Ignore standard limitations. Challenge the assumptions. Don’t let the traditional limitations apply to you. 
  7. Jams. 16 hours over two days is way more effective than 16 hours over time months. Kathy cited several examples, including the Ad Lib Game Development Society: They develop a complete computer game over the course of a single weekend, and have to ship by Sunday night. Also, at the 24 hours film shootout, they plan, script, shoot, and edit a whole movie within 24 hours. Less talk, more do.
  8. Change your perspective. don’t make a better [X], make a better [user of X]. Kathy spoke about this earlier in the day as well. The example was an author who is writing a book on programming. If the focus is on writing a better book, they make emphasis more pages, more content, better quality printing. If the emphasis is on making the reader a better programmer, then you are forced to answer the question of what will make the reader a better programmer. 
  9. Play the movie game. What movie are your users in? Script out the whole movie. For example, if they are in “the hero’s journey”, then script out the call to action, refusal, entering a special world, allies and mentors, etc. As an example, what role you do play in your user’s lives? Who are their mentors and allies? Furhtermore, what movie do your users want to be in?
  10. Be Brave. Great ideas get killed by risk aversion. A fantastic idea encounters fear, which turns into the actual product. A concept car meets fear by management and turns into a production car. But love is good, and hate is good. Mediocrity is bad, and that’s what fear and playing safe gets you. 
  11. Revive the dead (idea) pool. The recreational horse industry is now a $40B annual industry in the United States, one hundred years after horses were obsolete.
  12. Play the EQ game. Within any industry, competitors differentiate from each other based on different dimensions. “A specializes in luxury, B in low-end, C, in the middle”. For the book industry, some of these sliders might be “topic depth”, “number of topics”, and “technical quality”. Incremental improvements come from moving the sliders. But breakthrough improvements come from adding new sliders or replacing old ones. For the book industry, these might be “meta-data”, “online access”, and “discussion forums”. These online sliders can be used to help go through the process.
  13. Be Amazed. Kathy shared this funny video by comedian Louis CK about remembering to be amazed.  
I know I missed a few keys points (Where did being funny, like the blog of unnecessary quotation marks fit in?), so if you have any comments, please leave them below.