Keywords: Challenges of comparative research and International Survey Projects, cross-cultural concerns in data collection and measurement issues

Comparative studies aim to find differences between populations; otherwise, it is merely a replication. However, if variances derive from inadequacy of measurements, rather than actual cultural differences, the results render less meaningful. When valid differences do emerge, a deep understanding of the context is crucial for result interpretation.
As demonstrated in the present study, variation in attitudes toward gun regulation, among samples from the United States and South Africa, are rooted in factors, such as conceptualization of guns ownership as a civic right, perspectives on criminal use of guns, racial social identities, and media coverage. Differences, which we could solidly point to based on analysis of open-ended questions that were included in online surveys, and gave a complementary layer to the quantitative analysis.

Method

Data collection. Data was collected as part of a larger study examining effects of fact checks on partisan beliefs and perceptions of bias in media. An online experimental design focused on the debate over gun regulation, specifically addressing the relation between gun policy and gun-related violence. In the present study, we analyze responses to open-ended questions inserted along the survey, that concerned attitudes and beliefs toward gun-related issues, as well as the media coverage of those issues.

Participants. Respondents were recruited from four prominent national organizations in the U.S. and S.A. – two advocating for gun rights , and two advocating for policies to reduce gun violence . In addition, two national representative samples were recruited through SSI and Panel Services Africa. After cleaning data and filtering incomplete surveys, our final sample consisted of 5561 responses.

Analysis. We integrate two complementary methods; a qualitative and quantitative content analysis of the open-ended questions. In a first step, we examined a random sample of responses from each country, and identified major recurring themes and general tones. Following the qualitative analysis, we conducted a computerized text analysis of all responses, using the Linguistic Inquiry and Word Count (LIWC2015) system, to gauge the sentimental meaning of word choices and along multiple psychological dimensions.

Results & Discussion

It is eminent that the concept of gun regulation differs in the two countries. While many U.S responses referred to constitutional rights and the second amendment, most S.A responses directed the discourse to illegal gun ownership, and criminal use. Furthermore, U.S. responses scored higher on negative emotions, mainly anger, a majority of which derived from the Anti-regulation advocates, and concerned the term "Semi-automatic assault rifles." The S.A. sample did not mentioned this term even once, indicating that it only carries connotation within the U.S., potentially related to terminology used in media coverage, and gun advocates' claims of media bias. A section of the analysis is dedicated to media coverage. We present examples of responses and discuss conclusions in the full paper.

Our results are a testimony to the importance and empirical value of including open-ended questions in quantitative oriented studies, and the benefits of a mixed method approach, especially when examining culture differences.