ARF foq2 Router Glossary
1 like2,687 views
The document defines key terms related to assignment algorithms and routing environments: - Assignment algorithms are the logical procedures routers use to assign respondents to studies, determined by fixed and variable instructions. - Priority assignment assigns screeners non-randomly based on predefined rules. - Other terms defined include blocking traffic, click balancing, completion and conversion rates, contact quotas, deduplication, incidence rates, intermediary pages, landing pages, languages, link types, prioritization, publishers, qualification, and qualification rates. - Quotas are used to manage the distribution of sample for characteristics like age, gender, or study subgroups. Reallocation and reassignment redirect disqualified respondents to additional screening opportunities.
ARF foq2 Router Glossary
- 1. Assignment Algorithm Assignment Algorithm The logical procedures a router uses for assigning a study or study screener to respondents in a routing environment. The algorithm or set of instructions determines the functional methods used by the router. Some of these instructions might be “fixed” while others might be variable (i.e., changeable) by the team assigned to mange the router and routing environment. Assignment Algorithm – Priority The act of assigning a screener for attempted qualification in a non-random manner, based on a set of predefined business rules. See the document titled “Router Definitions-Examples-Visuals” for a better understanding of router types and how randomization and prioritization are deployed. Block (or Shut Off) Routed Traffic An ON/OFF toggle that prevents routed traffic from coming into a router, screener, or survey for a specified sample segment. Click Balancing / Balanced Clicks In general, balancing on “clicks” is a specific process in which quotas are used to manage the overall distribution of sample that lands on a survey’s first question or screening question. Depending on router configuration, suppliers may do this differently (e.g., at start of survey screening, or at a specific screener question). Uses include drawing sample that meets certain distributional requirements, such as drawing a “gen pop” sample reflective of the target population in terms of age, gender and region. It is also used to account for different non-response rates by assuring that the sample that “clicks through” reflects the target sample of interest. Also see "Contact Quota.” Completion Rate (aka Study Conversion Rate) The percentage of respondents who complete a survey relative to survey “starts”: "Total survey completes" divided by "Total survey starts" for screened, qualified respondents. Completion rate is different from “response rate” as defined by major survey research associations such as AAPOR. Contact Quota (aka Click Quota, Control Quota or Fielding Quota) Quotas that balance the overall distribution of sample delivered to screening for a survey, thereby ensuring screening is conducted among a sample that mirrors a desired population on specific characteristics (e.g., age, gender, etc.). This is not to be confused with a survey completion quota, as Contact Quotas register at the beginning of the questioning process rather than at the end. Contact Quotas may be applied at the start of screening or a survey (upon entry). It is also used in the sample delivery method known as "click balancing / balanced clicks." Conversion Rate (aka Net Conversion) A percentage describing the number of completes divided by the number of net respondents who attempted to qualify for that survey. This rate is a sample usage or productivity rate encompassing both qualification incidence and survey completion rate. See "Qualification Rate" and "Completion Rate" for comparison.
- 2. Correlation - Among Studies Occurs when two studies in the routing environment have characteristics that are correlated in some way, such as in their qualification criteria. This could arise from studies that have directly conflicting sample need, or a situation in which inappropriate assignment might severely skew the sample of one of the studies (e.g., a study of smokers and a study of tooth whitener users). Deduplication The process or processes used to reduce the likelihood that the same respondent is assigned to take a specific study more than once, complete a specific survey more than once, or is found in the final sample disposition more than once. In the context of routers, it is generally applied in pre-processing rather than in post-processing. This is to prevent the same person from being routed into the same study multiple times, which is important for both one-off ad hoc studies and ongoing trackers. It could occur in situations when 1) the same person enters a router multiple times due to invitations from multiple router sources, or 2) a person engages in the same router at multiple points in time (e.g., visits monthly). This also applies to techniques to prevent the same respondent completing multiple surveys within a router environment – in cases where a router has participation limits (e.g., only one incentivized survey per week). Deduplication is typically also applied within a survey, to prevent the same respondent from completing the same survey more than once. There are two prominent approaches to deduplication in use today, one involves “machine ID’s” and “digital fingerprints” rather than “people” and the other involves verification of specific personal information such as name and address against information found in a known database. Neither method is 100% accurate and depending on settings, could either pass in a “true” duplicate or exclude a “true” unique. Incidence Actual Actual fielded incidence across all suppliers and quotas in a routed sample. This is often calculated differently for real-time sample (vs. traditionally targeted sample), but the concept is the same for both types of sample. Bid / Expected Expected incidence for a given survey in a routing environment. Used in the proposal stage of a project and is generally estimated or gauged from past experience. Quota Actual fielded incidence for a quota cell in a routed sample. Supplier Actual fielded incidence for a given supplier in a routed sample. Intermediary Page A page that is part of the routing experience that is displayed to respondents to provide additional messaging, such as instructions. An intermediary page does not perform qualification. Landing Page A page that presents relevant information about a survey to a respondent, before they start the actual survey. It is generally presented when the respondent clicks on a link in the survey or invitation process. A landing page may or may not record an action, but it generally provides information to a respondent during their experience (usually an intro or exit script). Typically one party will provide another party a landing page to which respondents are directed (or redirected) based on the respondent’s disposition or status in a survey. Statuses used include: complete, termination, overquota, and more recently quality termination. A landing page typically contains logic that allows or prevents further respondent actions, such as taking another survey, general redirecton, or termination from the router process.
- 3. Language Language used in the routing process (e.g., English, Spanish, other). The language presented is typically determined by a respondent profile and country of origin, and will be one of multiple translations of the router questions. Link Types Routers use multiple Link Types to control the path of a particular respondent based on data outcomes and a set of rules. They generally direct (and re-direct) respondents in and out of the routing process, providing information back to a sample provider about the disposition of a respondent by using variables embedded in the URL. Examples of link types are Disqualified, Quota Full, and Completed Interview. Prioritization Prioritization involves presenting a screener or a study to a respondent according to predefined business rules, in a non-random manner. Publisher A third party, contracted by a router owner, who drives online traffic or respondents to a router or surveys. Generally used to describe non-research oriented online sources of respondents. Qualification The act of classifying a respondent as being "qualified" or “eligible” to take a survey. The qualification of a potential respondent may be determined using available databased information (e.g., date of birth) previously collected as well as by information collected in-the-moment as answers to screening questions served up during the routing process. Qualification criteria might include: demographics, stated response data (e.g., purchase decision-maker), quota availability, sample price thresholds, agreed participation, de-duplication, and passing quality tests. Qualification Rate The rate with which respondents who come into a router qualify for a survey, or the rate at which respondents who screen for a particular survey qualify for that survey or are “eligible” to take the survey. The qualification rate considers all factors (including availability of quotas), not just incidence. Quota Total Quota The total number of completed interviews required by a study. Quota Cell / Sub- Quota / Quota Group The number of target subgroup completed interviews required by a survey. For example, a study requiring 50% males and 50% females may have a total quota of n=400 and sub-quotas of 200 for each gender. Quota Matrix / Nested Quotas A set of quota cells that are defined by more than one type of qualification. For example, the sub-quotas of Age by Gender might form a matrix of four cells such as: Males aged 18-39, Males aged 40- 65, Females aged 18-39, and Females aged 40-65. Contact / Click / Control / Fielding Quotas Distributions or “counts” (such as age or gender) that are defined by the clicks or responses that land on some initial part of the survey (e.g., a particular screening question). Not to be confused with a survey completion quota. May be applied at the screener or survey level (on entry). This approach is used in the sample delivery method known as "click balancing."
- 4. Reallocation / Reassignment The act of reassigning a respondent to additional screening questions or screeners (associated with another study in the routing environment), upon disqualification from a prior survey. Used primarily within the serial routing environment. Reassignment allows a respondent who is willing to take a survey, additional opportunities to “screen in” to a survey for which he or she is eligible, after “screening out” of a study for which the respondent was ineligible. Business rules determine how many ineligible screeners a respondent in a routing environment is allowed to complete before the decision is made to not screen that particular respondent any further. Router A technology-based mechanism for allocating online surveys placed on the router to a sample of respondents who enter the routing environment. Routers use software and business rules that take the form of algorithms (see "Algorithms") to assign online surveys to a stream of incoming potential online respondents who enter the environment, usually at a general “landing page”, without any knowledge of which one of the screeners associated with the active surveys in the routing environment they may be assigned to answer. In some environments, as soon as a respondent is found to meet all of the qualifications of a particular study’s screening criteria, the respondent is passed onto that survey. In other routing environments, respondents may complete the criteria of several screeners and a process selects a survey for the respondent from among the ones he or she is found qualified to take. Routing A selection mechanism by which one of the many studies within the routing environment is assigned in real-time to a respondent to complete, if he or she meets all of the eligibility (or screening) requirements. Router Configuration / Rules Customizable rules used within a specific router that control how/when or what order, surveys will be served up or routed to a respondent. This is based on sampling practices and business needs. Router Types Mini-Router / Common Screener A subset of a router process that allows a respondent access to a limited number of surveys that typically have some element in common (e.g., survey type, demo target, same end client). See also “Specialty router.” Parallel In its most basic form, parallel routing is a process where a respondent is exposed to a set of pre-screening questions from all or a subset of the surveys in the routing environment, simultaneously, on the same page. After the respondent answers these pre-screening questions, he or she is assigned to one of the surveys for which the person appears to be qualified. At that point, more specific screening may be conducted before the respondent is deemed eligible to complete the survey. Serial In its most basic form, serial routing is a process whereby a respondent is screened sequentially for the available studies in the routing environment. Upon qualification for a study, the respondent is often immediately routed to that particular survey. Upon disqualification during the screening process, the respondent is reallocated, that is, he or she is re-routed to another screener (associated with another study within the environment). The process repeats until the respondent meets the qualification for an active survey within the routing environment, OR, based on business rules (e.g. length of time in the
- 5. process, number of screening questions respondent has answered by clicking, number of screeners respondent has been administered, the process ceases and the respondent is thanked and terminated. Sample Exchange A router that allows multiple sample sources to direct sample to the exchange so that multiple parties can buy and sell sample with each other, in the form of a marketplace. Specialty A router designed specifically for a unique type of respondent or certain research situations (e.g., B2B, Physicians). See also “Mini-Router.” Sample Reallocation The process of re-routing another study to a respondent upon disqualification from a survey to which they were initially routed. The algorithm that the router uses chooses the next survey to display to the respondent. Segment A subset of sample. A segment is typically defined as a cell, subgroup, or a supplier. Sampling Real-time A type of sampling in which sample assignment to studies is done in real-time (e.g., as respondents enter a router). Traditional / Panel A type of sampling in which an estimated amount of sample is drawn from a panel database at a point in time, according to a study’s specific needs (e.g., balanced according to census) for completion. This sample is then sent inbox invitations with a link to the specific survey for which it was drawn. Once sample for a traditional online study is drawn, it is locked out of being drawn as sample for any other traditional study sample until certain business rules are met. In contrast, sample for a router is generally not drawn to meet the requirements of a specific study (although there are exceptions). Rather, when panel sample is drawn for a routing environment, the general requirement are tp draw sample that will meet the needs of the routing environment studies. Inbox invitations can be sent to sample earmarked for a router, however, the link included usually takes respondents to a general area (not a specific study) and once in that general area, one of the studies in the routing environment (including screening criteria) is assigned to a respondent who “hits” the general landing area. Sample Types Hybrid / Blended When more than one sample source or type is used for a survey. There are a number of different blending processes that one or more organizations employ for combining samples. Insourced Using sample that a research or sample company owns and manages themselves, -- as opposed to buying sample from third parties. One example is sending a research or sample company's own panel sample into a router. Panel (Managed Access) A database of generally double opted-in respondents who have signed up to be members of a panel for the purposes of completing surveys. Incentives (such as cash, points that can be exchanged for cash or gifts, products, sweepstakes, charity donations, etc.) are offered for survey completion. Panels are typically comprised of deeply profiled respondents that are typically invited to surveys via email or a panel website. Non-Panel Sample that comes into a survey from online sources other than a
- 6. managed access panel. This could include “real time” sample, sample obtained by “pop-up” intercepts and banner ads on content provider sites, email lists of clients or that are rented for one time use, etc. Rewards / Affiliate A flow of opt-in respondents who are participating in an online loyalty program. Incentives (e.g., cash, sweepstakes, website-related rewards) can be earned via surveys as well as other activities. These databases typically comprised of lightly profiled respondents who are typically invited to surveys via an email or a reward website. River / River- type The process of inviting or directing respondents into a router or survey via a direct ad placement or other similar real-time invitation mechanism (interstitial ad, presented link, etc.). Social Network A flow of opt-in respondents from social websites who are invited to take surveys for incentives (e.g., cash, website-related rewards). These incentives can be earned via surveys as well as other activities. Social media networks are a source of lightly profiled respondents who are invited to surveys via a socially-oriented website. Stand-alone Sample that is sent to a survey but will not be routed beyond that survey. The “traditional” online approach often uses this sample, although, such surveys could obtain sample that has come from a routing environment. Targeted Sample that is targeted for a particular survey in the router. Once qualification is attempted for the targeted survey, those disqualifying may or may not be routed to attempt qualification for other surveys, based on the business rules for the router and that sample. Screening Process Screener A set of questions usually asked prior to main questionnaire content to determine or verify overall study eligibility or branching within a specific study. Generally composed of items beyond basic demographics, such as respondent or household category usage or ownership, respondent decision-making status or responsibilities (e.g., primary grocery shopper, maintainer of household vehicles, financial decision-making status), “security” items, etc. Dynamic Screener Questions asked of a respondent that are determined on-the-fly based on the currently available surveys in a router. Standard Screener A standard set of questions that are asked of all respondents entering a router. Block Screener A set of questions that contain the qualifying criteria for a survey. Generally used in serial routing. (aka Screener Block) Screener Complete / Qualifier A respondent who qualifies for a survey in the router via a screener. They may or may not complete the main survey. Screener Counter A counter used to track any of a number of items, including the number of respondents who have started a screener, the number of studies for which qualification was attempted by a single respondent, etc. When counters reach certain thresholds, logic is generally engaged. Disqualified Respondent A respondent who did not reach a survey because they did not qualify in the screener process. Pre-Screening A portion of a study screener – generally a key question(s) only. Used primarily in parallel routing where an initial determination is made about potential qualification prior to further screening. Qualified Respondent A respondent who has passed the screener but who has not yet completed the survey. Screener Question A question asked in a router before a client survey for the purposes of assessing a respondent’s qualification for the survey. Timer/Length/LOS The length of time required to complete a screener.
- 7. Survey Cost CPI Cost per Interview Average CPI (ACPI) The weighted average cost across all independently-priced sample segments (e.g., a cell, subgroup, supplier or sample source type). Default CPI (DCPI) The default CPI price for sample routed from a specific supplier. Targeted CPI (TCPI) The CPI that is agreed to between the buyer and the seller for targeted sample for a study. CPA Cost per Action. A term used by digital traffic providers to signify the manner in which they will charge for traffic delivery -- in this case, based on a predefined action such as registering or completing a survey. CPC Cost per Click. Some suppliers of online traffic to routers charge by the number of clicks into the router versus per charging per interview. Also a term used by digital traffic providers. Survey Response Complete A respondent that meets all screener qualifications, when quotas are open, then finishes a survey to completion. Over-quota The number of respondents who qualify for a survey but who cannot complete it because their quota group is already full. Redirect A URL typically assigned to a survey that returns a URL or pixel to register the respondent's final status in a survey. Return The number of respondents who qualify for and return from a survey. Termination The number of respondents who qualify for a survey but are terminated for some reason (e.g., shut their browser and closed the survey before completion). Survey Selection Hybrid A process of survey selection that incorporates both priority and random factors. Priority A process of survey selection where a specific survey is prioritized and presented based on an algorithm utilizing pre-defined business rules. This is usually determined by executive researchers and business owners, with oversight by a router administrator. Random A process of survey selection where survey selection is solely based upon random factors. Weighted Random A process of survey selection where surveys are randomly chosen via weighted factors. Selection Set The set of surveys a respondent can be directed to in a router. Targeting A process where a sample provider screens respondents, either in real time or by the use of historical data, to filter for respondents who match the target population. Traffic The volume or flow of respondents in a router, measured in a given time period. Traffic Mix The sample types or sources of respondents in a router (see: Sample – Panel, Sample – Rewards, and Sample – Social Networking)
- 8. Uplift The percentage increase in the delivery of completes that a router provides above the expected rate that would be expected without the use of the router. This metric can be used to demonstrate the efficiency of a router vs. other methods. Utilization (aka Usage Rate) A measure of router productivity, generally defined as the aggregate router qualification rate. It is described as "total qualified sample" divided by "total traffic / incoming sample." Higher may not necessarily be better, as methods to efficiently assign respondents to surveys can produce skews in study samples. This metric is used by router owners to gauge the overall respondent experience and measure router consistency.