As far as we’re concerned, you can’t argue about panel quality – making sure that our panels are of the highest quality is the basis of our business.
Obviously you’re interested in the methodology and technology we use to ensure this quality long-term and we’re happy to give you the full details. ESOMAR, the worldwide market research organisation, has already drawn up a comprehensive list of questions for you as a buyer of sampling services. And we have answered them in detail.
Click on any of the ESOMAR questions in this diagram and then the exact wording of the question and our answer will appear automatically.
Since the company was established in October 2005, respondi AG has carried out several hundred online surveys per year through our online access panels. Among our over 200 customers are Germany’s biggest market research institutes and a large number of international companies and agencies.The samples in these projects range from very specific target groups with incidence rates of up to 250 participants to large tracking studies with over 25,000 respondents.
Many of our staff have over ten years of experience in dealing with online sampling, i.e. they have been working in this research field virtually since its beginnings. In addition we are continually developing our own methodology, both in collaboration with our clients, and also thanks to our cooperation with the Center for Survey Design and Methodology (CSDM). This level of experience allows us to give our clients competent and up-to-the-minute advice on using online samples.
respondi AG maintains an actively managed online access panel. For the technical side of operations, we use a powerful software solution to manage our panel.
With a unique combination of various online and offline recruitment methods, our panels reflect the Internet population. We recruit members of these panels by topic only via our opinion platforms sozioland and demandi. For this reason, over 200 surveys per year on a wide range of topics are linked to over 4,000 partner websites.
This process allows us to find our panel members in their favourite web hangouts: in topic-specific platforms, in forums and communities. In addition, we also use other sources such as CATI recruitment using native speakers.
Our combined recruitment approach using online and offline sources is unique in this marketplace. Panellists are not attracted by monetary incentives but by their intrinsic motivation.
On the one hand, this approach achieves a quantitatively high response rate of 60%, and on the other, the response quality from the panellists has proved to be particularly high.
Furthermore, in our recruitment process we do not select by socio-demographic or pyschographic characteristics – this is confirmed by our regular dropout analyses. This makes our respondi panel a faithful reflection of the entire Internet population.
The samples from the respondi panel are used exclusively for the purposes of market research.
To widen the scope of the respondi panel as far as possible, respondi uses multiple recruitment sources: for example, its own opinion platforms, on-site questionnaires, search engines and telephone recruitment through our partner institutes in the market research industry. These diverse sources allow us to approach and recruit a wide variety of different target groups.
Our recruitment process is designed to be transparent and fair. Each new participant is given detailed information about the conditions for taking part before they start. Participation in the respondi panel is voluntary and can be ended by both parties at any time. All personal data held on the respective participant is destroyed immediately once the member leaves the panel.
Panel members are invited to take part in each survey by email. They are informed in this email of the expected length of the survey, the period of participation and the incentive, i.e. bonus points or a prize draw.
Each participant may only register once for the panel.
As some surveys are directed at panellists with specific characteristics, i.e. car drivers, school pupils or those in employment, the respondents are selected to take part in each project separately. This means that not all members can take part in all surveys and therefore as a rule there is no right to take part in any given project.
Respondi rewards all members for taking part and they receive bonus points for their answers.
The details given by the panellists are treated in accordance with Germany’s strict laws on data protection. The survey results are evaluated anonymously, i.e. the answers from each of the surveys are never linked to names or addresses of the panellists in the analysis. Data is never passed on to third parties. Details are used solely for the purpose of market and social research. Panel members will not receive any unsolicited commercial emails or other spam.
The high quality of the panel ensures that panel fluctuation remains at a low, but natural level. Per year around 15% of panel members are removed from the database on account of panel mortality or quality controls. Reasons for deleting a member from the database could be a participation rate of zero percent over a period of 12 months, double registrations or the panellist deliberate falsifying answers in several surveys. We check these parameters continually. Constant checks and a moderate invitation frequency prevent negative aspects such as panel bias or professional respondents.
respondi uses the double opt-in process. Once each new member has registered following the above process they receive a confirmation email inviting them to take part in a master data questionnaire. On the start page of this first questionnaire the panellist must actively agree to the regulations and data protection guidelines. Panellists who do not take part in the master data questionnaire are not invited to take part in surveys and are completely deleted from the panel after a period of four weeks.
A system check integrated into the registration process prevents double registration under the same email address. After registration a duplicate check highlights participants with similar addresses, IP addresses and browser configurations. We continually monitor this process and filter out and completely remove fraudulent panellists and double-registered participants. Furthermore, we also ask for bank details, in order to recognise false identities. We also compare profile and survey data regularly and also check constantly both other panellist data and the response quality of each panellist in surveys.
In the master data questionnaire, which must be filled out on registration, we gather around 300 basic details on each member. A further 600 pieces of profile data are then collected on a voluntary basis in the topic-specific specialist panels. In addition, we log each member’s entry method to the respondi panel and we also save all data on current and past projects. The master data is updated every 12 months.
respondi defines active panellists as those who have filled out the master data questionnaire after the double opt-in registration process and have taken part in at least one survey in the past 12 months. Overall our respondi panels total around 220,000 members. The current data changes constantly on account of the dynamics of the panel business. The number of participants and distribution can be found on our website.
We draw a random sample from the target group based on the master data, given quotas and the required screening. We form groups or sub-panels with the help of group filters. Filter criteria can be master data, performance criteria, field data and project data. Groups defined in this way are then ready to be used as samples for the project. In the course of drawing the sample we also use a stratified or quota module. This allows us to set exact quotas for any number of master data criteria according to absolute or relative values. Samples can therefore be used time and again. When issuing invitations we are also able to define time zones, and regions etc. Panel membership statistics allow you to see an overview of the participants by number of sample memberships.
In the invitation email we inform respondents how long the interview will last and how high the incentive will be. The email also includes a link to the survey allowing the respondent to access the questionnaire. We always point out to our panellists that the questionnaire is anonymous. However, the topic of the survey is not revealed at any point in the invitation email.
Panel members receive respondi points (rps) for taking part in the survey. A ten-minute interview is rewarded with 100 points. These 100 points are equivalent to the value of one euro. In addition, we hold regular prize draws. On reaching a redemption level of ten euros, the amount can be transferred to a bank account or converted into a gift voucher. Alternatively the panellist may make a donation of any amount to one of our selected charities.
On average, each member is given the opportunity to complete two surveys per month, i.e. each panellist will belong to the target group of a survey twice.
Filters allow us to form groups of panellists that have already exceeded the limit of a pre-defined invitation frequency within a defined time period. This group is then allowed to have an appropriate period of rest.
In principle we save all data that is relevant to panel participation. Our tracking systems log absolute and relative participation values: invitation frequency, response rate, dropout rate and number of completed interviews.
Panellist information is treated in accordance with the strict laws on data protection. We always carry out surveys anonymously. Under no circumstances do we pass on personal data to third parties. We guarantee members of the respondi panel that their data will be used solely for the purposes of market and social research and that they will not receive any commercial emails or other spam. In accordance with the guidelines of German data protection laws all data held on a participant must be destroyed completely on request or if the member leaves the panel at their own wish. respondi is a member of the German Society for Online Research (DGOF), the Association of German Market and Social Researchers (BVM) and ESOMAR, the world organisation for market researchers. We are therefore subject to strict industry guidelines, which we feel bound to observe rigorously.
Each member is given the assurance that registration and survey data are treated with the utmost confidentiality and will not be combined at any time. It is absolutely out of the question that address data could be passed on to our clients as we store address data and personal data separately.
Every member is assigned a nine-figure pseudonym and the survey results and personal data are linked to this pseudonym. To keep personal data secure, the panel website is protected by an SSL certificate. This allows respondents to exchange points for cash via a secure connection.
Our software offers a comprehensive catalogue of professional solutions. It enables us to automate processes such as data cleaning in surveys and duplicate checks in the panellist database. A quality control system assigns each survey action with a quality variable. This variable enables us to identify the respondents who just click through surveys and who complete the questionnaire in well under the average time.
Double registrations under the same email address are already eliminated during the registration process. Identical or similar postal addresses, IP addresses and browser settings enable us to automatically identify duplicates.
In addition, the panel management uses a benchmarking system to log and process all the relevant panel data. This includes technical and content data such as response rate, full utilisation of the panel capacity and also communication with panellists.
A third element of our quality management system is the respondi satisfaction index. We survey this every month with a representative cross-section of panellists. It measures the panellists’ satisfaction, motivation and loyalty.
The minimum age for taking part in the respondi panel is 14 . Surveys are carried out with younger children solely via their parents registered in the panel.
Like many panel providers, respondi also works with a number of different panel partners. This is particularly the case for those countries in which respondi does not operate its own panels. If we do need to use the services of other panel providers, we will always notify our clients in advance.
We prefer to work with national providers in each country. We select these partners carefully and screen the quality and reliability of their work.
To maintain the response quality at the same high standard for multi-panel surveys, we compare panellist identities anonymously in order to avoid overlapping samples.
Just to clarify: the response quality of an individual panellist is not measured exclusively by the number of his or her panel memberships.
respondi works with the RelevantID technology from RelevantView to exclude certain groups of respondents from surveys and to avoid duplicates. RelevantID logs over 80 components of questionnaire data points (IP address, country, town, region, browser, PC data etc.) and compares this data with characteristics from other participants. Furthermore we marginalize anomalous panel characteristics. With tracking studies in particular this enables us to gather valid and reliable results.
The average participation rate for a survey with five field days is 60%. This participation rate is calculated using the sum of completed interviews, screen outs, quota fulls and non-completed interviews, divided by the number of invitations.
The start rates are dependent on the incidence. The large amount of master data in the respondi panel keeps the screen out rates low. A clever field management system ensures that participants do not constantly encounter full quotas. Dropout rates are less than 10%, which indicates a high level of motivation and loyalty among our participants.
We document the development steps of every panel member throughout their panel life cycle. We log the entry method and entry date for each panellist. Numerous tracking variables record all the relevant respondent data in the survey field phases.
At the same time members are assessed in an internal evaluation system with the help of these tracking variables in terms of their response behaviour and their willingness to take part. This panel compliance index identifies qualitatively weak and unreliable participants.
The internal evaluation system used by respondi continually monitors and measures the response behaviour of panellists. For example, we continually compare profile and current survey data, as well as continually cleaning data and carrying out time measurement. These checks are carried out during and outside of individual projects.
Once the project has ended we carry out a regular quality control. This generates a quality variable. This allows us to ascertain whether a participant has really engaged with the questions or has simply clicked through the questionnaire. Alongside these kinds of automated processes, respondi also uses manual methods such as variance checks and time checks as part of its project management.
Once the survey has ended, we regularly ask how satisfied the panellists were with the content and design of a questionnaire. Another measurement of satisfaction is panellist feedback. Our panel management provides excellent opportunities to respond individually to the needs of the panellists. In addition, every month we survey a satisfaction index of our panellists. This index is gathered from a representative cross-section of our panellists – it measures satisfaction, motivation and loyalty of the panellists.
After the project has finished we give each client an automatically generated field report containing all the relevant information on the course of the project such as number of participants, screen outs, dropouts, field time and access times etc.
During the project each client will be provided with reporting access to the field and results report, which displays the results in the form of graphs and allows the open questions to be read in real time.
Panel quality: we take it seriously
Our answers to ESOMAR’s 26 questions to determine the quality of online samples
and online panels 
Interested?
More information?
