Creating Effective Customer Surveys: Why & How

July 25, 2007

The following article was adapted from the seminar "How to Do Customer Surveys: A Case Study," presented by the American Booksellers Association's Director of Education and of BookSense.com, Len Vlahos, at the 2007 Winter Institute.

Over the past decade, retailers of all shapes and sizes have felt the increasing crunch of competition as new technologies have made it ever easier for start-ups to enter the marketplace, for giants to grow more gigantic, and for customers to be more discerning about where they do their shopping ... and spend their hard-earned money. New technology has also made it much easier for retailers both big and small to garner crucial data about their customers, ranging from where they buy their first cup of coffee in the morning, to what newspapers they read, to their favorite types of books.

Independent bookstores' competitors have entire departments devoted to understanding customer behavior. "It's critical that our members go beyond an anecdotal knowledge of their customers in order to remain competitive," said Len Vlahos, ABA's director of Education and of BookSense.com.

"Among the various methods of conducting market research, customer surveys are among the least expensive and most effective in enabling you to better know your customers. This is something that all booksellers, no matter how big or small, can do."

While it's understandable that some booksellers think they already know who their customers are without the assistance of a survey, rest assured, the results of a good customer survey will result in a critical surprise ... or two. As the saying goes, perception is not always reality.

"We all make judgments about what our customers think and sometimes without information," said Chuck Robinson of Village Books in Bellingham, Washington. As an example he cited a store survey conducted by Village Books with the goal of finding the best local media in which to place store advertisements. The results indicated a lot more people than Robinson thought were reading the Bellingham Herald. "Even though people would all say, 'Oh, no one reads that paper,'" he said, "we found out it was more important than we thought it was."

Casey Coonerty Protti of Bookshop Santa Cruz, who worked as a consultant to ABA and was the primary author of the association's customer survey education materials, echoed Robinson. "I think you want to make sure you know your customers, and what their needs are," she told BTW. "In Santa Cruz, the demographic keeps shifting, and I don't ever want to take for granted that I know what my customers needs are. A survey also tells us how we can improve our store, what do we need to do better, and what are the things we are doing well." Without this customer feedback, she added, it would be easy to get complacent about setting out to improve your bookstore.

With that in mind, it seems only fitting to kick off a nuts-and-bolts article on creating customer surveys with a question: What are the steps to creating an effective customer survey? This question, of course, demands more than a few simple choices and check boxes.

To start, one needs to understand the reason for creating a customer survey. Simply put, retailers conduct customer surveys to better understand the demographics, attitudes, and behaviors of customers -- the ultimate goal being to use customer knowledge to a) better serve their customers and b) to use this knowledge to increase sales.

"Before you can even create your customer survey, there are a couple of things you should have in mind: what is the objective of your survey, what information do you need, and how can you collect it?" said Vlahos. "It's important to fashion a survey with questions and answers that will allow you to easily understand the results so that it will provide you with an actionable strategy. Remember, you will need to measure whether the action dictated by the customer survey results was, in fact, successful. So there's a lot to plan out prior to writing that first question."

Vlahos also stressed that customer surveys are just one way to get to know your customers. "Market research works best when you couple it with database information and other types of customer feedback," he said.

With that in mind, here is a five-step framework for creating and implementing an effective customer survey.

There are two steps in identifying the objective for a customer survey.

First, identify your strategic objective -- this should always be related to a decision or a sales goal. To identify a strategic objective, you should think about what it is you are looking to accomplish with your business. If you want to increase sales, who would you target the survey to and what incentives would you provide to get them to buy more?

Secondly, identify your survey objective. What information do you need to be able to take action on your strategic objective?

For example, a few years ago, Bookshop Santa Cruz, was looking to increase the number of their loyalty club members in order to drive up sales (the strategic objective). As such, store management determined to find out four things (the survey objectives): What did non-members want out of the store? Did they know about the membership program? What did they want out of a membership program? And would they be willing to join the program?

• Exploratory research. There is a tried-and-true science to conducting surveys, so there is no need for booksellers to reinvent the wheel. Following a few simple rules significantly increases the validity and usefulness of your results. Hence, it recommended that you conduct a "survey before the survey," in the form of exploratory research.
  
  "Exploratory research helps you better frame your objective, understand how your customers may react to the topics in the survey, and help you develop questions and answers for your survey," Vlahos noted. "It helps you make sure you get your survey right."
  
  For instance, you can create a focus group consisting of eight to 10 target customers in a one- to two-hour session with an experienced facilitator. Be forewarned: focus groups tend to be more expensive than other options and are only as good as the facilitator. An alternative would be one-on-one interviews -- ask 10 of your customers to talk with you one-on-one for 10 minutes.
  
  In one-on-one interviews, Bookshop Santa Cruz asked non-loyalty club members: "What do you like about Borders?" and found out a number of customers mentioned Border's travel section. As a result, the store added a survey question asking people where they shop in the store by section.

• Survey structure. Key to the success of the survey is, of course, its structure and focus. Most of the questions on the survey should be related to your objective. Then there should be some "longitudinal" queries that are asked in every survey you conduct to enable you to track information over time. These questions may range from how many books the customer has purchased in the last month to the average number of visits to the store per week. And finally, there should always be demographic questions, though these questions should be asked at the end, otherwise the response rate will drop (more on that later).
  
  Importantly, it's crucial that the survey isn't too long. Said Village Books' Robinson, "I think you should keep it fairly focused so that the customer can complete the survey... in under five minutes. And ask people: Would you take three minutes to fill out a survey?" Robinson noted that you will always find a few who will happily fill out a long survey, but it's best to err on the side of brevity.
  
  Vlahos concurred: "Keep it short, sweet, and simple. At most, seven to 10 minutes, but no more than four pages. You must time it and revise it. Remember, one survey cannot answer every question."
  

• Question types. There are numerous ways to frame questions in a survey, including:
  • Scale (On a scale from 1 to 5, please rate the following....)
  • Selection (Why do you shop here? Check all that apply.)
  • Open-ended (What do you like about our store?)
  • Rankings (Please rank these bookstores from 1 to 4 in order of preference.)
  • Range (How many purchases have you made at our store in the last month?

Wherever possible, you should be sure to quantify your results. While open-ended questions can provide you with deeper insight into your customers' satisfaction or dissatisfaction regarding a particular aspect of your store, it is hard to compare this kind of data on an annual basis. That is not to say you should avoid an open-ended question, rather use open-ended questions sparingly.

Conversely, scale queries allow you to compare results by fractions of a number in order to detect even small changes in attitudes over time. Furthermore, forcing respondents to make choices is important because it will give you a better understanding of not only their behavior, but also, what is important to them.

In his store surveys, Robinson said that most of the questions were multiple choice, but that he did mix in different types of questions just to make it "a little more lively." Moreover, he added, "Different kinds of information require different kinds of questions."

• How not to write questions. The way you ask a question is as important as what the question focuses on. Here are some "don'ts" of survey question writing:
  
  * Don't lead customers to a certain response. For example, don't ask a question that begins, "Our bookstore has excellent customer service -- what do you like most about it?" Never assume what your customers think.
  
  * Don't ask people something they can't remember: "How many books did you buy in the past year?"
  
  * Don't ask customers to respond to two attributes at once: "Are our employees welcoming and professional?"
  
  * Don't overwhelm customers with too many options: "Which of these 31 flavors do you like best?"
  
  * Don't give answer choices you are not willing to implement: "Would you like us to add a drive-through window?"
  
  * Don't give answer choices that aren't defined: "Would you use our RSS feed?" (RSS should be defined.)
  
  Added Robinson: "Don't ask what you do badly. They will focus too intently on that information." If you are seeking input on how to improve some aspect of your store, frame the question in a positive light. Instead of asking a respondent what aspect of the store they dislike, offer up a question that asks them to rank from 1 to 5 their level of satisfaction with each of the following attributes, such as customer service, selection of books, and store layout. (A sample survey template, in Word format, is available to ABA bookstore members only.)
  
  
• Sensitivity & Privacy. These days, customer privacy is more crucial than ever, so it's important to state your privacy policy on all documents and inform customers of the policy before they fill out the survey. "By stating your privacy policy up front," Vlahos said, "you are building your brand as a protector of privacy and consumer rights."
  
  With that in mind, asking for demographic information from your customers is a delicate matter and timing is everything. Ask for demographic details at the beginning of a survey and you run the risk of scaring customers off and curtailing response rate. By the end of the survey, however, a customer may be more comfortable with what you are asking and will probably be more willing to give you personal information.
  
  
• Choose a Medium... or two. There are myriad ways to distribute your customer survey -- in-store handouts; mail; e-mail/online; or on the street. And there is no need to box yourself in by choosing only one method. In fact, choosing more than one medium will ensure higher participation rates and diverse viewpoints.
  
  Certainly, Internet is the easiest medium to use, and there are a number of user-friendly online survey tools to choose from, including SurveyMonkey.com and ConstantContact.com.
  
  "We plan to conduct more surveys in the future," Robinson said. "We use Constant Contact, and now that it has survey tools, I'm sure we will use it."
  
  Coonerty Protti said her most recent survey was conducted online using SurveyMonkey. "We received over 1,000 responses," she reported. "It was so much easier [than in-store]."
  
  There are many pros and cons to each medium. (A full list of options, in PDF format, is available to ABA bookstore members only.)
  
  

• Administering the survey. Now that you have prepared your survey, the fun begins: administering the survey. A few tips before you begin -- set your sample size and ensure random sampling; identify and train staff/volunteers to administer the survey; and engage customers.
  
  "Once you have a draft survey written and a medium chosen, you have to plan for implementation to ensure that you end up with a viable sample -- and that your interactions with customers are as positive as possible," Vlahos said.
  
  To garner unbiased samples for in-store surveys, it's recommended that you administer the survey weekdays and weekends. Implement the survey over two consecutive weeks and pay special attention to the season to make sure that the survey is representative (e.g., not during school holidays, Christmas, etc.). Collect at least 50 surveys overall and 20 in each comparison group (e.g., 20 males and 20 females). The larger the sample size, the better.
  
  In terms of staffing the survey project, there are a couple of different options. For one, you might find business classes at local colleges and universities where students can do this as a school project. Also, you could select and train staff, and have them cover different days and times (one benefit to this method is that, by including staff, they will feel more invested in the outcome).
  
  Whichever option you choose, it is wise to provide the survey project staff with a training guide that outlines how to conduct a survey in your store, along with common questions and answers that you might hear from customers. (A sample-training guide, in PDF format, is available to ABA bookstore members only.)
  

• Final Prep. Test the survey with five to 10 customers and revise. Ask the test audience if they found anything confusing or unclear. "While not strictly speaking a survey, the butterfly ballot used by Palm Beach County, Florida, in the 2000 presidential election is a classic example of what can go wrong if your survey is designed badly," said Valhos. "If election supervisors had tested the ballot in advance of the election, the outcome -- for good or ill -- may not have been the same."

• Engaging Customers. An effective way to bump up response is to offer an incentive for participation, such as a discount coupon or entry into a contest to win a gift card.
  
  Other methods for engaging customers are to develop tools to help staff approach customers. Provide staff with a script about how the survey will be used, the time needed, confidentiality and incentives, as well as responses to typical complaints. (A sample talking script and a letter for an e-mail or mail customer intercept, in Word format, is available to ABA bookstore members only.)

It's crucial that, in analyzing your results, you are meticulous and consistent in your methods. Here's how:

1. Number the surveys. Each customer needs a unique identifier.
   

2. Review the surveys for minimum quality standards -- eliminate surveys that are mostly incomplete or have an obvious lack of interest (e.g., rated a "5" on every question). Be consistent with quality standards that you impose to limit the effect of abnormal surveys.
   

3. For in-store or mail surveys, you should create an excel worksheet: each customer should have one row with all of their answers, and the columns should have every possible answer from every question. Also, remember to include a notes column for non-quantifiable answers. (If you are conducting an online survey as well as an in-store survey, you could save time and not have to produce this worksheet. Instead, you could enter all the results from the in-store survey into your online survey. The online survey will then automatically calculate the results -- just make sure you have a way to distinguish what surveys were conducted in-store versus online.)
   

4. Calculate the overall answers.
   

5. Calculate the answers by customer groups and compare.
   

Patterns you find in your survey responses will dictate what strategies you implement in your store, but more than that, a good customer survey will provide you with a great deal of data that can be parsed and used in a number of effective ways.

Here's a look at various ways to take action:

• Targeted marketing. Identify customer groups with similar preferences and target marketing to address their interests.
• Strategic planning. Look at trends over time to understand where you should direct resources.
• Identify and reach new markets. Identify customer groups that have positive attitudes towards the store, but with whom you have not directly communicated in the past.
• Develop a marketing message. Try to discern what aspect of your store resonates with your customers.
  

At Bookshop Santa Cruz, their loyalty club survey revealed that 89 percent of non-members had not heard of the store's loyalty club but would consider joining it, so the store increased the marketing toward the loyalty club, Coonerty Protti noted. Additionally, the surveys revealed that 32 percent of non-members said Bookshop Santa Cruz should offer more discounts, so the store crafted an effective marketing message about the benefits of the loyalty club.

Among the other actions the store took were:

• sending a mailing to customers in a zip code with a high percentage of supportive non-members;
• increasing the number of in-store displays;
• simplifying loyalty club sign-up procedures; and
• highlighting discounts and exclusive sales for members in communications.

And it worked, Coonerty Protti reported, "really well. We gained several thousand new members.... People in the loyalty program buy a lot more, so it increased sales among a group of people and now we have direct communication with these people." Moreover, the store continues to reap the benefits of changes made to its loyalty club.

In a more recent survey, Coonerty Protti found that Amazon.com now poses a bigger competitive threat than the Borders down the street. She noted that, while many of the store's customers refuse to shop at Borders out of loyalty to her store, they don't have the same misgivings about Amazon. The respondents revealed that they feel they get better value at Amazon. "It was a lot about convenience, how if we didn't have a book in stock they would have to come back," she reported. "It was a time issue, and we found out that a lot of people didn't know we had a website."

Coonerty acknowledged that many readers might be thinking, Of course, Amazon is a big competitor! But, she added, "This information helps you to prioritize and focus your energy." As a result, the store is planning a big marketing push centered on its website and will soon be participating in Ingram's Direct-to-Home program, which provides the bookstore with a kind of virtual warehouse. "If someone comes into the store and the book is not in stock, Ingram ships it directly to the customer," she said. "We hope to launch this in a month."

Similarly, Village Books conducted a survey to help decide where it should direct its advertising. "We wanted to find out where our customers were getting information about the store," Robinson said. As noted at the start of the article, Village Books' simple survey brought back surprising results. Aside from the Herald being more important than initially thought, "two weekly independent papers popped up," Robinson said. "It was good for us to figure out what people are paying attention to." The survey results allowed the store to shift its advertising to those newspapers and magazines that Village Books customers read, making the most of their advertising budget. --David Grogan

Topics: Education,