So, you’re an expert?

This week, I gave a talk at a company that’s starting with data-driven practices and A/B experimentation specifically. My talk was concerned with the enablers required for this, such as continuous deployment (or DevOps), the specific ways in which organizations can apply data-driven practices and A/B testing and the importance of value modeling so that you know what you’re optimizing for.

The interesting situation was that several participants were either very quiet or had remarks that seemed to question the entire approach. Although I was initially surprised by this and focused on the content, over time it became clear to me that something else was going on: the critics were so-called experts in their specific part of the product portfolio and, apparently, were concerned with the possibility of the data showing that their expert opinions might actually be wrong.

My favorite definition of an expert is someone who tells you why something can’t be done and to some extent, this viewpoint was confirmed as part of this talk. We all appreciate being experts at something and our sense of self-worth stems from the identity confirmation that comes with being an expert. Within groups, the status of individual members is derived from their reputation, for instance, as an expert. So, what I was doing with my talk was presenting people with the possibility that their reputation and status within the group, as well as their sense of self-worth, might be taken away from them because they’ve ‘built their house on quicksand’ and data from the field might not back up their beliefs.