Keywords: Challenges of comparative research and International Survey Projects, cross-cultural concerns in data collection and measurement issues

In the tradition of the social sciences, many high-quality general population surveys consist of long questionnaires, lasting up to one hour or more. Recently, rising costs move surveyors to consider the web mode to replace the common personal interviewing face-to-face or by telephone. As short questionnaires are recommended in self-administered web survey, length may be a problem when the survey is planned with a long questionnaire. It is then necessary to split the survey in several parts in order that each person answers only to a smaller form. But then some issues of comparability might arise between the different groups of respondents : this paper address a methodological experiment measuring to what extent a modified questionnaire in order and length produces the same measure.
Several countries of the EVS project have run in parallel with the traditional face-to-face interviews a separate web version, where the one-hour questionnaire has been split in two parts. Each respondent answer only to half of the whole survey before to be invited to answer the complementary part, so completing the full questionnaire. In order to control the effect of this two-parts surveying, the design of the questionnaire has been redrawn so that not all the respondents answer the questions in the same order. The questionnaire has been split according to thematic rules in order to create six versions. In Switzerland, two control groups are added to receive a full-length questionnaire: one of them has a modified order (second part comes before the first part).
The analysis focus on the influence of the position of questions in the questionnaire according to two parameters: the preceding theme addressed and the burden due to the number of questions already answered. We will address two issues: does the context or the burden influence responses? Is the influence random and if not, what kind of bias should we expect? For the questions analysed, the different groups will be characterised according to the preceding block: in our design, each question can be preceded by three different contexts. The experiment with long questionnaires allows us to disentangle the effect of the context and the effect of the burden: the questions appears in the same context after a much lengthier questionnaire completed.
The paper provides insights on the feasibility of long auto-administered surveys. It will characterise different errors observed according to the burden sensitivity or context sensitivity. These issues will be balanced with two benchmarks. The split of the questionnaire in two parts distant for some weeks will be considered: does the error due to the burden is of the same kind as the error due to the time shift? The results will also be related to the face-to-face: are the differences between different groups of web respondents larger than the differences between web and face-to-face respondents?