I was an undergraduate research assistant at the BBER since August 2022. Over that time, I had worked on several projects, but I’d have to say that my favorite was the Economic Impacts of the Potential I-35 Conversion in Downtown Duluth. For this project, I completed two tasks that were instrumental to the project—creating a survey using Qualtrics software and coding responses that were gathered from in-person interviews and the survey so that they could be analyzed easier. I learned some best practices for both tasks and share some of them below.
How to create a survey
When creating a survey, it’s important to give potential participants some background information on the project, why the survey is important to the project, and other relevant information such as if the responses are confidential or why participants were selected.
For potential I-35 conversion project, I created two versions of the same survey – one for use in in-person interviews of key stakeholders (survey questions were asked of the stakeholders and answers recorded) and another online for stakeholders who were not available for in-person interviews. While each survey had to collect the same information, each needed to be intuitive for the individual completing it. The in-person interview survey was easy to follow and had room for open-ended (qualitative) responses, where interviewers recorded what stakeholders said about the questions.
The version taken online by key stakeholders needed to avoid being too long or overwhelming. It did not need as much open-ended response options as the in-person interview survey, but rather it needed more questions that were multiple choice or check-box style. When making a survey for people to take online, it is important that the survey is as short and as uncomplicated as possible so participants will complete the entire survey. Incomplete surveys will not provide the comprehensive data needed for a detailed analysis.
How to analyze qualitative responses
Open-ended survey responses are a valuable way to capture a broad range of answers and allow participants to share detailed feedback. But qualitative data is more challenging to analyze than multiple choice survey answers. Fortunately, there are several software options specifically for qualitative data analysis. We used NVivo for this project.
After loading the survey responses from Qualtrics into Excel, the first thing I did to start analyzing the open-ended survey responses was to read through the responses to each question and identify common themes for each question. However, there were so many responses that many important topics were identified—far too many. To help make sense of the common themes, I and the rest of the research team used Nvivo’s mind map tool for visualizing the relationships between themes.
Figure 1 shows one of the mind maps created for the project. As you can see from the figure, the top level, “Duluth Characteristics,” refers broadly to characteristics of Duluth’s downtown waterfront area. Below that, there are two categories, “What would you change?” and “What would you keep or retain?” These categories were based on two related questions from the survey. Below those categories are some of the most common themes in the responses that emerged from the two questions: “improved public safety,” “historic feel,” and “public access to the waterfront.” In total, six mind maps helped us organize the common themes for the eight open-ended survey questions.
The benefit of using a mind map is that, once you have visually organized the themes and categories into a map, the software automatically creates categories (called codes), using the relationships in the mind map. Our team then had the task of assigning every survey response to one or more of the codes in the software.
The coding process involves reading the survey response and assigning it to the code (or codes) that align most closely with the respondents’ comments. The results allow the analyst to see which topics and themes were most common among survey respondents. For example, after coding all the open-ended comments to the question “What private sector development opportunities would you see as being most likely in Duluth’s downtown waterfront area” (as shown in the figure above), we learned that housing was mentioned by our survey participants 53 times, mixed-use commercial 39 times, and leisure and hospitality 37 times.
The tricky part of coding is that is a very time-consuming process, which meant that this task was done by the entire research team. Because multiple people were working on coding, it was important for us to develop definitions for each code for consistency.
Learning how to work with qualitative data was not only fun, but it was also a great learning opportunity.
I will be continuing my education in pursuit of a master’s degree in industrial/organizational psychology. I have always been interested in how people relate to and interact with their workplaces, since we spend nearly a third of our lives at work. A lot of the data collection that happens in industrial/organizational psychology is qualitative, so being able to familiarize myself with the coding process and software will be hugely beneficial.