Skewed Samples in the Decision Matrix

A friend is considering changing phones. It’s down to the Android or the iPhone. He has done smart thing, and talked to some people that have had both. He spoke to a few people that had an iPhone and switched to the Android. To a person they all said they preferred the Android phones.

I asked if he had talked to anyone that has switched from Android to an iPhone? He had not. His sample was skewed.

Anyone who decided to leave a iPhone is necessarily unsatisfied with it, and looking for something else. So from the start, they have an issue with the iPhone (or AT&T). If you only talk to people who have left the iPhone, you will naturally hear why they left and what they like better abut their new situation. In order to have a more consistent sample, you should also talk to people who have switched from a Droid to an iPhone. Or at least, who are happy with the iPhone.

But we make this kind of decidion all the time. The church gets ne email complaining, and we change a whole ministry program, when in reality the email representing the views of maybe 1%. Assuming the complaint is legit and something we should be concerned about, we must be careful to weigh issues and complaints, making sure we know the temperature of our people.

When we began using a jib in the service, we placed it to one side of the Worship Center; The same side where most of the pastoral leadership sat. We had just a few complaints, when I saw the end coming. We moved the jib to the other side, and complaints slowed.

For us, it did not matter which side we used. In fact, the new position gave us a better angle on some instruments. It did matter that decision makers were hearing complaints on a regular basis., from people who had access. The complaints represented a very small number of people who were affected by the jib’s position, but they had an audience. Many had been sitting in that location for years and knew the ministers well.

If I had polled the pastors who sat on that side, I’m sure they would have all said that the congregation is distracted by the jib’s movements, and does not want it in the service. Their sample was skewed, and decisions made from it were not representative of the real situation. Most people were fine with it, but a vocal few were not. We forestalled a confrontation by moving the jib elsewhere. Since then, we have had one real complaint.

It is easy to fall into the trap of skewing samples. I can probably find people with any range of views you want. Like this, hate that, want the other thing. But, we must be smart about who we are listening to, and who they are listening to.