When you have 100s (or 1000s) of responses from your target audience – how do you analyze the answers to open-ended questions?

Basically, we draw on techniques developed for analyzing qualitative data – we are looking for patterns and trends in the responses so we can reach some conclusions as to what we are seeing. Here are some main steps that you could undertake:

1. Read through the responses.Yes, as laborious as it may seem, you must read through each response to get a feeling for the data. As you read through the responses, you will probably see some common themes emerging.

2. Create response categories. The second step is to develop categories for the different themes you are seeing. For example, with a question asking for people’s feedback on a website, you will probably be able to group comments into categories such as “content”, “design”, “features”, “service”, etc.

3. Label each comment with one or several categories. As you read through the comments, assign at least one category to each response. This is what is called “coding” and best done in an excel sheet with responses in one column and your category (s) in the next column.

4. Look at what you have. In the example about feedback for a website, you might label half of your responses as “content”. You can then divide the responses on “content” into smaller categories, e.g. “corporate content”, “product content”, etc. By doing this you will start to see what are the trends in the data and the main issues raised by your respondents.

5. Think what are the responses about? Once you have categorised and coded data, it doesn’t do you much credit just to say “some half of people spoke about content; most of these people spoke about the corporate pages on the website”. You must be able to explain what is being said about the subject or theme. For example in the case of “content” – what were people saying about content? Imagine if a respondent said:

“I consult regularly the corporate pages. This information is well-presented but not up-to-date. I never seem to be able to find information on latest priorities and management profiles”

This example contains different comments on aspects related to design, site updating, navigation and missing content. Notice that the comment on navigation is actually not a “content” issue – but would be considered as a “design” issue and needs to be coded accordingly.

6. Identifying the patterns and trends: once the data has been studied and categories determined, the next step is to see what categories are related and where can trends and patterns be identified: are there common themes emerging? Or are there a series of unrelated points being mentioned?

7. Writing up the analysis: Once you have analyzed the data and identified the major patterns and trends your next step is to write a summary of what you have found. This would normally be a descriptive text incorporating comments directly from the respondents. For example:

“In providing feedback on the website, some half of the respondents spoke about content. The main issues raised included the inability to find content and the lack of up-to-date content on management themes. To a lesser extent, the high quality of the product information and the desire for more information on the management team were mentioned. The following comment from a respondent illustrates these points:

“I find the quality of the product information very good. However, the information is often difficult to find and is hidden on the website”.

As you see, I use terms such as “some half”, “main issues” and “to a lesser extent” to illustrate the magnitude of the trends identified. Some prefer to transfer such an analysis into quantifiable terms – such as “some 50%” or “under 30%”. I prefer not to – but if you are dealing with very few responses, it’s better to mention the precise numbers such as “5 out of 20 responses preferred…”.

There are opensource tools you might want to consider for deeper qualitative data analysis. Check out http://www.softwareadvice.com/resources/easiest-to-use-free-and-open-source-text-analysis-software/ for some ideas.

Good luck with your analysis!