To Örebro University

High nonresponse occurs in many sample surveys today, including important surveys carried out by government statistical agencies. An adaptive data collection can be advantageous in those conditions: Lower nonresponse bias in survey estimates can be gained, up to a point, by producing a well-balanced set of respondents. Auxiliary variables serve a twofold purpose: Used in the estimation phase, through calibrated adjustment weighting, they reduce, but do not entirely remove, the bias. In the preceding adaptive data collection phase, auxiliary variables also play a major role: They are instrumental in reducing the imbalance in the ultimate set of respondents. For such combined use of auxiliary variables, the deviation of the calibrated estimate from the unbiased estimate (under full response) is studied in the article. We show that this deviation is a sum of two components. The reducible component can be decreased through adaptive data collection, all the way to zero if perfectly balanced response is realized with respect to a chosen auxiliary vector. By contrast, the resisting component changes little or not at all by a better balanced response; it represents a part of the deviation that adaptive design does not get rid of. The relative size of the former component is an indicator of the potential payoff from an adaptive survey design.

In the design of surveys, a number of input parameters such as contact propensities, participation propensities, and costs per sample unit play a decisive role. In ongoing surveys, these survey design parameters are usually estimated from previous experience and updated gradually with new experience. In new surveys, these parameters are estimated from expert opinion and experience with similar surveys. Although survey institutes have fair expertise and experience, the postulation, estimation, and updating of survey design parameters is rarely done in a systematic way. This article presents a Bayesian framework to include and update prior knowledge and expert opinion about the parameters. This framework is set in the context of adaptive survey designs in which different population units may receive different treatment given quality and cost objectives. For this type of survey, the accuracy of design parameters becomes even more crucial to effective design decisions. The framework allows for a Bayesian analysis of the performance of a survey during data collection and in between waves of a survey. We demonstrate the utility of the Bayesian analysis using a simulation study based on the Dutch Health Survey.

One objective of adaptive data collection is to secure a better balanced survey response. Methods exist for this purpose, including balancing with respect to selected auxiliary variables. Such variables are also used at the estimation stage for (calibrated) nonresponse weighting adjustment.

Earlier research has shown that the use of auxiliary information at the estimation stage can reduce bias, perhaps considerably, but without eliminating it. The question is: would it have contributed further to bias reduction if, prior to estimation, that information had also been used in data collection, to secure a more balanced set of respondents? If the answer is yes, there is clear incentive, from the point of view of better accuracy in the estimates, to practice adaptive survey design, otherwise perhaps not.

A key question is how the regression relationship between the survey variable and the auxiliary vector presents itself in the sample as opposed to the response. Strength in the relationship is helpful but is not the only consideration. The dilemma with nonresponse is one of inconsistent regression: a regression model appropriate for the sample often fails for the responding subset, because nonresponse is selective, non-random.

In this article, we examine how nonresponse bias in survey estimates depends on regression inconsistency, both seen as functions of response imbalance. As a measure of bias we use the deviation of the calibration adjusted estimator from the unbiased estimate under full response. We study how the deviation and the regression inconsistency depend on the imbalance. We observe in empirical work that both can be reduced, to a degree, by efforts to reduce imbalance by an adaptive data collection.

Recently, various indicators have been proposed as indirect measures of non-response error in surveys. They employ auxiliary variables, external to the survey, to detect non-representative or unbalanced response. A class of designs known as adaptive survey designs maximizes these indicators by applying different treatments to different subgroups. The natural question is whether the decrease in non-response bias that is caused by adaptive survey designs could also be achieved by non-response adjustment methods. We discuss this question and provide theoretical and empirical considerations, supported by a range of household and business surveys. We find evidence that more balanced response coincides with less non-response bias, even after adjustment. © 2016 The Royal Statistical Society and John Wiley & Sons Ltd.

Responsive Design is a trend in recent survey literature concerned notably with managing data collection, through planning and appropriate intervention, so as to realize a well-balanced final set of respondents. In this effort, auxiliary variables, including paradata, are central. But regardless of what is done in the data collection, accurate estimation despite nonresponse is the ultimate goal. The auxiliary variables are important at the estimation stage as well, as when calibrated weights are used in the nonresponse adjustment. For accuracy, two factors intervene: (1) in the data collection, the level of imbalance achieved with the auxiliary information; and (2) in the estimation, the degree to which the auxiliaries explain the study variable. In practice, both objectives are less than completely satisfied. Reduced imbalance in data collection does not by itself guarantee low bias in the estimates. We ask: Is balancing worth a perhaps costly and demanding effort in data collection? Could one have done equally well by saving the use of the auxiliary information until the estimation stage? Complete bias elimination is not achieved at either stage. We outline a theory for a two-factor explanation of accuracy, and apply it to two important surveys at Statistics Sweden. The factors-thedegreeof imbalance and the degree of explanation-are systematically varied, and their joint effect on the accuracy of the estimates is evaluated empirically. The results show that reduced imbalance makes the adjustment of the simple estimate lose some of its importance. More importantly, the calibration-adjusted estimate realizes some accuracy improvement by having been preceded in data collection by a reduced imbalance. The explanation of why this happens is not simple, but a theoretical justification is outlined.

In recent literature on survey nonresponse, new indicators of the quality of the data collection have been proposed. These include indicators of balance and representativity (of the set of respondents) and distance (between respondents and nonrespondents), computed on available auxiliary variables. We use such indicators in conjunction with paradata from the Swedish CATI system to examine the inflow of data (as a function of the call attempt number) for the 2009 Swedish Living Conditions Survey (LCS). We then use the LCS 2009 data file to conduct several "experiments in retrospect". They consist in interventions, at suitable chosen points and driven by the prospects of improved balance and reduced distance. The survey estimates computed on the resulting final response set are likely to be less biased. Cost savings realized by fewer calls can be redirected to enhance quality of other aspects of the survey design.