Bad Research

After many years in another industry, a corporate CEO left to lead a large publishing company. After a month or so on the job, he grew unsettled at how different publishing was from the consumer product industry he was familiar with, especially the highly “intuitive” approach publishing utilized to make decisions.

He recounted a key moment in his first month when he asked a long-time employee if he was missing something.

“So, as far as I can tell, we publish hundreds and thousands of new products for the consumer every year without doing any real market research, relying entirely on our own intuition and opinions. We make a best guess as to how many we need to print and then we ship as many as we can to retailers. If consumers don’t buy them, the retailers send them back to us.”

The long time publishing employee said, “Yes, you’ve got it.”

I’ve addressed this topic in general in a related post on how publishers make decisions. A link is provided to that post below.  Spoiler alert – publishing decisions are not as logical or scientific a process as one might think.

A few weeks ago, I mentioned that everyone was a theologian. Today, I want to add another item to our job description…researcher. We are all researchers. And most of us are really, really terrible at it.

Because everyone views the world through the lens of their own life experience, we all make many decisions or write things based only on our personal perspective.

Why is that so bad?

Because we all live with blinders on and naturally see the world only through our own subjective lenses. So how cloudy and limiting are those lenses?

Once, a person told me they disagreed with research showing the average American household having 1.7 children because they had three children and they were a typical family. And all their friends had three or more children. So the research was flawed.

Still another told me that they didn’t believe NASCAR racing was a popular sport in the U.S. because they didn’t know one person who had ever gone to a race.

Another person told me that the reason most people fall behind paying their bills is that they don’t keep enough money in their checking accounts.

(Cricket sounds)

We all use personal experience as our primary guide in life and decision-making. But we can dig a great big hole to fall in if that is our only guide.

At some point, everyone must make a decision that is not connected to anything we know or experience from our limited personal lives.

In publishing, a middle-aged male is making a decision about a book for teenage girls. If you think you can use your experience from 20 years ago with your daughter you are a victim of bad research.

If you are a millennial, single female making a decision about a book for men in midlife crisis, it might be a good idea to think beyond yourself or else fall victim to the same curse.

Writers expressing opinions about something from their perspective and limited experience can lead their work down a path where at best it is small and flat or worse, dangerously wrong.

I’ve been in meetings where a book was judged deficient because, “My son would never read this.”

“I don’t like the color orange.”

“I would never read that. We shouldn’t publish it.”

Bad research is all around us, but it is quicker than actually taking the time to understand an issue, so speed is the allure. And it is just less stressful than learning something new to challenge a comfortable position.

So, if you agree that it is probably best to make decisions using a bit more than our personal perspective and bias, here are a few suggestions:

  1. Recognize you have a problem. (“My name is Dan, I make decisions and write things based on my personal biases.”)
  2. Catch yourself every time you are about to express an opinion based only on your own perspective.
  3. Don’t express your opinion until you consider wider information derived from more than what you or your first grader would like.
  4. Make one decision or express one thought based entirely on information outside of your experience or something you recently learned.
  5. Wash, rinse, repeat.

Eventually you will develop a new habit, and find it quite satisfying when you view the world through the eyes of another.

To conclude, something that pulls this together in some weird way.

Wilson Mizner, an American playwright in the early 20th century said, “If you steal from one author it’s plagiarism; if you steal from many it’s research.”

Click here for my 2014 post on how publishers make decisions.

9 Responses to Bad Research

  1. Avatar
    Lisa Evola August 4, 2015 at 6:07 am #

    I guess that’s why they have teams of people and meetings….to get other perspectives. As a leader of many past groups, I was always the first to admit that I had a limited perspective and invited the opinions of others. Often people don’t want to give them because they are afraid that they will get more work dumped on them 🙂 I know this from experience…any who…It make me wonder how anything ever manages to get published…..we all know the saying about opinions!! Thanks for a great perspective 😉

  2. Avatar
    Lisa Evola August 4, 2015 at 6:07 am #

    I guess that’s why they have teams of people and meetings….to get other perspectives. As a leader of many past groups, I was always the first to admit that I had a limited perspective and invited the opinions of others. Often people don’t want to give them because they are afraid that they will get more work dumped on them 🙂 I know this from experience…any who…It make me wonder how anything ever manages to get published…..we all know the saying about opinions!! Thanks for a great perspective 😉

  3. Avatar
    Jan Cline August 4, 2015 at 7:08 am #

    Wow, Dan. This is something that has long been a pet peeve of mine. I write historical fiction and love, love to research. It always irks me when I mention my topic to someone and then hear all the uneducated opinions born from bias, hearsay, and personal preferences. But it has taught me to be more careful about giving an opinion without the knowledge to back it up. As for the industry, I’m not sure if I’m completely comfortable with the way decisions are made, but I don’t know enough about it to think it should change. Then again, it’s usually good to re-evaluate our methods once in a while. What “feels” right isn’t always accurate. Thanks so much for posting this. Something to re-read and share.

  4. Avatar
    Steve Hooley August 4, 2015 at 9:38 am #

    Dan, great post.

    This is certainly thought-provoking. And I agree with everything you said. But I’m left with the question, “How do I find answers to perspective issues, subjective data, etc?” Is the answer found in using a larger (still subjective) group?

    I hope you’ll write another post with more specifics on how we can “consider wider information derived from more than…”

    You’ve got us thinking. Now how do we find that wider perspective?

    • Dan Balow
      Dan Balow August 4, 2015 at 10:41 am #

      I don’t think a wider perspective is as difficult as it sounds.

      Obviously, spiritual matters need to be founded on God’s truth from Scripture. I heard a speaker once say that the scariest moment of any sermon is when the preacher closes the Bible and keeps talking.

      Authors need to look into and quote primary sources, which avoids perpetuating myths and incorrect information. Many people have “radio talk show” sound-bite based opinions. Most of which are not always correct.

      Simply asking, “Is this true?” and then dig in to find out, whether it be a spiritual or worldly issue is something every believer should do, but even more important for writers who can either perpetuate truth or falsehood.

  5. Avatar
    Carol Ashby August 4, 2015 at 10:35 am #

    You can keep even more money in your checking account if you don’t pay your bills at all, but that leads to other problems.

    Many years earning my living as a scientist trained me to always read the professional literature to learn what others were doing in my area. For my historical writing, I now buy relevant books and use, with caution, the web. Whenever possible, I go to the original source of the information to verify whether it is being properly interpreted by someone else before I rely on their interpretation. There is a lot of incomplete and even incorrect information out there on the web to cause the researcher to stumble. Unlike the formal review process in scientific publishing, there are no gatekeepers to ensure quality of information or truthfulness of anything on the web. Caveat emptor to the nth degree!

    Things get squishier for subjective topics that don’t depend completely on facts and logical analysis. It seems to me that trying to understand the opposing position and arguments would improve the writing at the very least because you would have thought through the objections to your position and addressed them in a more convincing manner.

    Whenever possible, I try to discuss what I think I know with someone who is a professional or “life expert” in an area that isn’t my own special expertise. It’s fun, too. Talking with people very different from me is always exhilarating.

  6. Avatar
    Jeanne Takenaka August 4, 2015 at 11:40 am #

    Dan, you bring up such great points. Looking beyond/RESEARCHING beyond my own perspective gives me a much better understanding of the people and the world around me. It takes intentional effort to educate ourselves enough to share opinions that are more well-balanced.

  7. Avatar
    Linda Riggs Mayfield August 4, 2015 at 1:10 pm #

    Dan, I think your post is highly persuasive about not settling for skim-the-top research, but it is also cautionary about the quality of writing. If the writing isn’t good enough to engage the prospective agent or publisher within the first few lines, that individual will never know whether your research is deep and defensible or shallow and questionable. For a writer who likes to kind of tiptoe into the story with a lot of scene-setting, that was a good reminder that just being a fanatic about accuracy and validity in research isn’t enough if the words don’t sing. Thanks!

  8. Avatar
    Sandra Allen Lovelace August 4, 2015 at 4:09 pm #

    Thanks, Dan. Your points are well taken, all the more because of the humor that drives them home. I’ll be looking beyond my personal domain in the days ahead.

Leave a Reply

Get New Posts by Email

Get New Posts by Email

Each article is packed with helpful info and encouragement for writers. You can unsubscribe at any time with one click. 

You have Successfully Subscribed!