Girls in Tech 2021 Reflections: Complexities of survey data and understanding experiences

By Ali Roberts, Ariana Bui, and Katie Hearther

CGEST partnered with Hawaiʻi Pacific University (HPU) to host the five-day “Girls in Tech STEM Camp'' on the island of Oahu in June 2021. Thirty-three middle and high school students from public schools across the Hawaiian islands came together to explore how participants use Science, Technology, Engineering, and Math (STEM) to positively impact their communities and inspire positive change. The campers were housed and based out of HPU’s Aloha Tower Marketplace facility but excursions to locations such as the Bishop Museum and HPU’s Oceanic Institute ensured diverse hands-on learning experiences throughout the week. Different STEM fields were the theme of each day’s activities including computer science, astronomy, botany, and marine science. Each field was examined through the lens of technology, and the campers were tasked with constructing an app that would address a persistent issue in their communities.

The camp was funded by a National Science Foundation grant, which ensured that the cost of housing, flights, and programming would not serve as a barrier to students seeking to understand their potential in STEM fields. The program was originally scheduled to take place on the island of Hawaiʻi in 2020 but was ultimately postponed due to the COVID-19 pandemic. As a result, this program was the largest in-person gathering that many participants and staff had attended for over a year and the excitement was palpable. We were eager to examine how this experience might shift the camper’s views on the uses of STEM and their own potential to utilize it in meaningful ways. We attempted to unravel the major themes hidden in these topics through a complex web of survey data.

During the week, campers participated in interviews, pre-surveys, and post-surveys. The aim of the pre-survey and post-survey was to gauge how participating in the camp might influence the way campers define STEM, how they might consider its impact on their lives and communities, and whether they could see themselves pursuing STEM-related studies with confidence. The surveys consisted of a combination of open-ended response questions about overall camp experiences and three rating scale questions about twelve different subject areas within STEM. These subject areas included science, computer science, astronomy and navigation, botany, marine science, technology, robotics, app design, coding, artificial intelligence, virtual reality, and circuitry. These rating scale questions asked campers to rate their levels of interest, levels of confidence, and the likelihood that they would consider pursuing study in each of the twelve subject areas. The open-ended response questions asked campers to describe what STEM means to them, what they learned during the camp, whether they would sign up again, and what their favorite aspects of the camp were.

Data analysis of the rating scale results showed that the mean levels of interest and levels of confidence actually decreased in some subject areas, despite overall positive feedback given in the open-ended response questions. Through conversations with the Girls in Tech leadership team, we considered whether campers may have experienced something akin to “survey fatigue” [url to hyperlink: https://methods.sagepub.com/reference/encyclopedia-of-survey-research-methods/n480.xml] at the end of such a busy and exciting week. We hypothesized that a few months may have provided space for reflection, so we decided to re-distribute the post-survey in an attempt to gain a deeper understanding of these results.

At the follow-up survey’s close in late February 2022, we received responses from 13 out of the 33 campers, which meant that our analysis could not be extrapolated to the entire group. We decided to compare the answers of these particular 13 campers to their original answers using their participant identifiers. Interestingly, comparing the answers from these 13 campers showed a shift in responses from their post-survey answers. Most subject areas saw an increase in mean responses, with marine science showing the highest increase in expressed interest. Although these rating scale responses do not enable a broad understanding of the campers' experiences, this shift can suggest the importance of allowing a reflection period for soliciting rich feedback. Additionally, the qualitative data received from open-ended questions provide valuable narratives that illuminate how these campers experienced and were influenced by the Girls in Tech summer camp.

Overall, the responses to the open feedback questions strengthened the idea that participants had a positive experience at the camp. When asked the questions “Would you sign up for another STEM summer camp like Girls in Tech in the future?” and “Would you recommend this camp to a friend?” one hundred percent of participants said yes in the reissued post-survey. One participant specifically noted “Yes, I would definitely recommend STEM camp. This is not only a great opportunity to learn how important STEM is to our every day life but its [sic] a great opportunity to be yourself, explore and make new friends and memories to last forever.” General sentiment indicated that the participants view their time at the camp in an enjoyable light. What was your favorite part of this camp?

Furthermore, the feedback effectively revealed that the participants largely benefited in three main forms - camp curriculum, cultural knowledge, and general life skills. A noteworthy aspect of the new post-survey was that it allowed insight into how the participants used what they learned over time. When asked about what they learned, one participant responded “I learned that I have the potential to make extremely good speeches. Because of what I did for our final project in the camp, I joined my school's Speech & Debate team.” While it was not initially apparent how the camp would influence the participants long-term in the original survey, the new feedback signifies potential long-term beneficial impact.

As we gear up in preparation for Girls in Tech 2022, these results are guiding us in future survey construction and in how we present information throughout the program. For example, in future survey questions, we will better define and differentiate the 12 subject areas since “technology”, “coding”, and “robotics” all overlap significantly. The initial post-survey results showing decreased confidence levels in different subject areas could be the result of information overload, survey fatigue, and being intimidated by complex subject matter. We will attempt to address this in the 2022 camp by having each speaker and staff member discuss their job, career path, and tangible steps they took to get into their current field. We hope this insight, combined with hearing a wide diversity of job options and career paths, will help this information to be less overwhelming. We also look forward to addressing suggestions mentioned in the qualitative data including exploring more of the intersection between technology and natural sciences, and further highlighting the STEM expertise embedded in Hawaiian culture. Although 94% of respondents said they would recommend the camp to others, we look forward to emphasizing the diverse career paths that exist in STEM and helping campers identify tangible actions that would help them pursue their careers with confidence. We are incredibly grateful to the campers for providing this data, which will ensure that any and all further programming has clear intentions and a positive impact.

References:

Lavrakas, P. J. (2008). Encyclopedia of survey research methods (Vols. 1-0). Thousand Oaks, CA: Sage Publications, Inc. doi: 10.4135/9781412963947. https://methods.sagepub.com/reference/encyclopedia-of-survey-research-methods/n480.xml