An Experimental Psychologist's Take on Beta Reading Part I: Subject Pools

I didn't plan on running my novel critiques like a psych experiment. I just wanted some folks to read the manuscript and recruited/blackmailed some friends accordingly.

A week into the process, I noticed that I was using principles from experimental psychology to ensure better data from my beta readers. Another week after that, I realized that it wasn’t normal to use words like "data" when talking about beta feedback. Then I began emailing my writing group with phrases like "I'm starting to notice clear trends emerging in the responses," and it was all downhill from there.

By now, I’ve embraced the “beta reading as psych experiment” analogy. But nerdy or not, I kinda like the way things turned out. Some writer friends requested that I post about my experience in more detail, and that was the inspiration for this next series: An Experimental Psychologist’s Approach to Beta Reading.


In the next four posts, I will detail what I did last month. My approach is not the best for everybody, but hopefully folks will find some useful tips here and there.

Let's start at the beginning. You (a.k.a. intrepid author/mad scientist), have just finished your manuscript and are ready to start the great experiment. Let's call it "Affective Outcomes of [insert manuscript name] On the Reading Population." In other words, "Does My Novel Suck?  And If So, How?"

First step is to recruit subjects. (Actually, the real first step would be to design the experiment and test it on pilot subjects to ensure things are working. You could draw an analogy to fixing obvious kinks with your writing group before showing it to other readers. But anyways…)

How many subjects do you recruit? I started out shooting for 10 readers, and due to various unforeseen circumstances, ended up with 16. Why did I want so many? For the same reason you want large subject numbers in psych experiments. There’s a lot of individual variation in taste, and unless you have enough readers, it's hard to tell which viewpoints are representative and which viewpoints are outliers.

How do you find 16 people to read your novel? The issue of subject recruitment has always been a tricky one, and where you find your subjects can have a huge effect on experimental results. For my beta readers, I recruited from three main populations, each with its advantages and disadvantages.

1. Friends and family who have been asking to read your novel

Pros: They will finally stop bugging you. Also, you know these people pretty well -- their preferences and their personalities. This is helpful for interpreting their feedback.

Cons: The biggest concern is whether you can hear their feedback without taking it personally and damaging your friendship. Know yourself and act accordingly.



2. Friends and acquaintances who read in your genre.

Pros: They know the genre.

Cons: I can't think of many downsides that apply to the whole group. If you don't know them very well, you might feel awkward asking them for help. Same warning applies about protecting friendships.



3. Writer friends

Pros: They can give you feedback from a writer's perspective and generally give more detailed feedback then non-writers.

Cons: The biggest practical issue is that beta reading is usually reciprical, so be careful about overcommitting yourself to reading too many manuscripts.

In the next installment, we'll get to the mechanics of recruitment and running the "experiment."  



In the meantime, tell me. Where do you find your beta readers?

To get regular updates from this blog, use one of the subscription options in the left sidebar.

18 comments:

  1. You seem to be accepting beta readers without any regard to their knowledge of the genre or of writing. That's a big con for group #1; they might not give any useful feedback.

    The biggest con I can think of for group #2 is the feeling that you may be imposing on them. It's not the act of critique that damages the friendship, it's the act of asking a favor that they may or may not follow through on and for which they may or may not want reciprocation.

    ReplyDelete
  2. I'm curious: from an experimental perspective, would it be better to draw subjects from outside of our circle of family/friends/acquaintances? Does our relationship with this pool bias the results? If so, would it be more ideal (if also more difficult) to get a random sampling of strangers in the pool?

    The pros/cons above are quite practical, but I wondered about the pros/cons of each group in terms of the reliability of the data they provide.

    ReplyDelete
  3. I have found readers for technical papers by going to talks by experts in the field, commenting on their work afterward, and then asking them to read my paper. Their main incentive is reciprocation for my interest and understanding of their work.

    ReplyDelete
  4. For my Beauty in Pain manuscript, I asked family and two friends. FYI, one of the friends does have editing experience.

    Family input was good, but limited. The editing friend gave excellent feedback. The other friend did too, but I had to ask for it. I also had to ask the right questions.

    Alice from Alice Benton's Blog
    http://www.alicebentonsblog.blogspot.com

    ReplyDelete
  5. Sending a novel for beta reading is actually a very relevant psych experiment. It fits so nicely into the experiment nook that I'm quite surprised.

    My beta readers so far consist of three people. My brother and two friends. There might be one or two other people that would read my things, but they have no interest in my genre, so I don't bother asking them.
    Other than that, unless I ask strangers that I meet in supermarkets, there is no other beta readers at this point.

    I think it's important to get views from people who read your genre, and to get a writer opinion as well (I'm still working on that last part) to get better results. Readers might be able to tell you what's wrong (if they pay enough attention) but writers can tell you how to fix it (maybe).

    Anyway, thanks for the post, and don't forget to include a graph for your psych experiment.

    ReplyDelete
  6. Critique groups. I can't praise them enough. I'm in two at present; one meets monthly, the other is online and ongoing. We all write speculative fiction, and we critique each other's work.

    I started out with Critters, an online group. I've left it now, but it was a great start. If my existing groups collapsed, I'd go to a crit group on one of the writing forums for my genre.

    Beyond that, I have three writer friends for who I provide rapid feedback if they need it, and they do the same for me. This is particularly nice if I want to try out a couple of iterations that would strain a group's patience.

    ReplyDelete
  7. I belong to a critique group and they are really picky. However, like you said, they demand a critique back.

    ReplyDelete
  8. Subject Pools. I love it, Livia. ;)

    To find beta readers for Thirty Decibels, I asked writer friends and family members I thought would be interested (always encouraging them to pass it on to young readers, too). But what really helped me set the stage for more readers was my interview on Dorothy Dreyer's blog last fall: http://bit.ly/eYUnLC
    I figured a lot of book-lovers read her blog, so I went ahead and asked for beta readers as part of the interview. From that alone, I heard from roughly 20 people who wanted to read for me. :)

    I'm also a bit nerdy when it comes to my feedback (data). My nerdery shows in my use of google forms. At the end of each chapter in my PDF, a reader can click a link to a quick form asking three questions: What works? What was confusing? What was boring? Not required of course, but a good place for a reader to go if they want to tell me something. And all the feedback goes directly into a spreadsheet for each chapter. :)

    The other nerdy bit is that I'd planned to do a second round of beta readers. I'm always hungry for more feedback... more readers welcome!

    ReplyDelete
  9. So far, I've only called upon friends and family to critique my work, and it's pretty much as I expected: excellent feedback from my two writer-friends, some helpful ideas (but mostly opinionated drivel) from my sister, and little of use from anyone else. Next up: the local writers' group.

    Great stuff so far, Livia. I'm looking forward to finding out what other principles of experimental psychology you used to get all your beta data.

    ReplyDelete
  10. Remus -- You're right about the genre. That could be a downside to both groups one and three. I agree somewhat about the knowledge of writing too, but sometimes nonwriters offer a good perspective. Writers sometimes get so caught up in the art and craft of words that they miss other things.

    James -- Yes, all the factors you mentioned would affect the results. We'll think more about trying to get unbiased replies in later posts.

    David – that's a good idea.

    Alice – great point about asking questions. More on that coming up as well.

    Jake – regarding the graph – I won't disappoint.

    Keyen -- I adore my critique group as well. They were awesome in helping with the first version, although because we do everything in 3000 word increments, I felt like I needed more readers to read the whole thing all the way through.

    Clarissa -- yeah, it's too bad we can't spend all our time writing and critiquing.

    Margo – that's awesome what you did with the Google docs. I'll have to write about you in a follow-up post.

    Nate – I prefer the term "unusual outlier" to "opinionated drivel" :-)

    ReplyDelete
  11. Hi Livia,

    I'm new to this, but am making plans re: beta readers for my WIP, my first novel.

    I'm thinking of going after two kinds of readers. One would be avid readers and/or writers who could comment on the craft and give constructive criticism as well as telling me how they like it.

    The second group would consist of a sampling of all the demographics represented IN the novel as well as a sampling from those I see as target audience. So this would include a span of ages, both male and female. Since my MC is a a college-aged MK (missionary kid) revisiting the place where he grew up, I will have MKs read it (how do they like it? does it represent them well? accurate? etc.) The book is set in the Philippines, so I will have Filipinos read it (accurate? enjoyable? patronizing or negative in any way? do they like it? etc.) I would get older and younger both to read it since I hope to market the book to those ages in the Philippines, and also since I have characters in both age groups. Women could comment on female characters, helping me since I'm a man trying to put them on the page in an accurate way. Etc.

    White mice... never thought those, though. Hmmm...

    ReplyDelete
  12. A week into the process, I noticed that I was using principles from experimental psychology to ensure better data from my beta reader

    This makes perfect sense, both in that it's a pretty good analogy and that it's so easy to model things you do on other things you know how to do. I'm steeped in the culture of software development, so I number my drafts like software versions, think of revision in terms of fixing bugs, etc.

    I pull my beta readers from the same pool you do. I also have a few alpha readers who I let see my first drafts, and they understand that at that point I'm looking for only the most general, gentle feedback and a pat on the back.

    ReplyDelete
  13. I had a lofty plan in place for acquiring beta readers for my first novel. But then I found that although they had agreed to read (some had even asked), very few actually followed through and *did* read, and even fewer offered any sort of feedback. So now I tend to stick with other writers (since they expect a critique from me in return, they're more likely to follow through) and generally have fewer beta readers than I'd ideally want.

    That being said, I think non-writers are very capable of giving valuable input. But I think many of them need guidance as to what to pay attention to. They might not be able to tell me *why* something isn't working or how to fix it, but they can still signal where I have a problem.

    ReplyDelete
  14. Bill – interesting idea, although you might have some trouble getting some of those groups to finish the book. Given that they are so diverse, what appeals to one is not likely to appeal to all of them. You could get around that by pulling out the relevant sections and having those groups only read the sections that pertain to them.

    Lisa – software development is another good analogy. My husband keeps trying to convince me to use version control all my manuscripts (and to write them in LaTeX), but I'm too lazy to implement that.

    Amy – my next blog entry is about how to get people to finish your book, and the one after that is on extracting good feedback.

    ReplyDelete
  15. Hmmmm... Livia, if members of my target audience can't finish the book, I'm in trouble! =)

    ReplyDelete
  16. Bill -- Haha, true. But you only need one target audience, though it's useful to have broader appeal

    ReplyDelete
  17. I help authors with their nonfiction books, but I do see an overlap in soliciting feedback. Like Margot, I suggest asking: Where is it boring? Where was it confusing? Where do I lose you? I also ask questions like, What would you like to see more of? Are there areas I didn't cover that you'd like to see covered? (more a nonfiction question).

    ReplyDelete
  18. Getting 10 beta readers is amazing. Sixteen is insane.

    ReplyDelete