Keywords: Challenges of comparative research and International Survey Projects, cross-cultural concerns in data collection and measurement issues

The goal of a translation of scales in a cross-cultural comparative survey is to produce an instrument that “measures the same construct(s) in exactly the same way” across all language versions (Byrne, van de Vijver, 2010). The difference in form, structure and concepts between languages may affect perception of the scales and, consequently, challenge the assumption that the instrument ensures measurement and structural equivalence.

Word-by-word translation of all the formal or semantic elements of a scale from one language to another does not necessarily lead to functional equivalence. Different types of linguistic and cultural aspects come into play, e.g. when maintaining semantic distance between points on the scale. Once it is established that the construct can be measured in different cultures, it remains a challenge to strike the best possible balance between accuracy and fluency, because one needs to take into account possible measurement effects such as social desirability or acquiescence, but also culture effects such as differences in conversational norms.

Empirical evidence of the impact of linguistic and cultural parameters on the reliability and validity of translated or adapted scales exists. This paper presents a work in progress that consists in compiling existing translations of widely used response scales and linking these to language/item interactions observed during the analysis of results, whenever such documentation is available. New survey questions can be automatically matched with this repository with a view to flagging known problems and possibly prevent their replication. In this way, a number of challenges and biases related to questionnaire scales that are already known can be identified in the earliest stages of survey design, using a combination of human expertise and automated recognition of previously observed patterns. This identification process can be embedded in questionnaire development.

Multidisciplinary discussions at different stages of questionnaire development, supported by a well-maintained repository of existing issues, can lead to informed decisions. Systematic documentation and robust knowledge management will enable researchers to follow-up on any issue that may show at data collection or analysis of results.