Online Consumer Panel simulator - demo: Project Description
- 1. Computer-aided Online Consumer Panel Simulation Sketch of First Version
- 2. Introduction ● Brief description of the parameters, variables and functions (VP&F) ● Diagrams
- 3. Calculation of Main Parameters Universe Baseline Inputs: - Desired average response rate - Baseline period Preliminary assignment of response rates to Panelist instances Average Response Rate Equal to desired one for several iterations? No Yes Recalculate again Outputs: - Parameters to run successive universes so that rr is constant Exposure Rate Calculation Inputs: - Universe Baseline - “Typical” project - Expected outcome of “Typical” project Preliminary sampling/interview for a typical project assuming days in week using an Exposure rate Is there a rate for which we obtain the Expected outcome? No Yes Recalculate again Outputs: - An Exposure Rate at asymptotic behaviour
- 4. Sketch of events at each tracking iteration Recruits Purges Excluded Universe Eligible To Sampling To Interview Got a status? Sample Yes No Instance lifetime Panelist: Attribute initialization and assignment of response rate If sample still active (not end of fieldwork) and or target hasn’t been reached... (incidence, exposure, etc) (response rate) End of Exclusion
- 5. ● Exploratory demo ○ First overview of model and potential ● Pending: better version with better data... ● More info about project: ○ slideshare.net/evaristoc ○ github.com/evaristoc Remarks
- 7. VP&F: Panelist; Response Rate Class Panelist: ● Some attributes included: ○ unique identification (indid) ○ “response rate” assignment ○ status in panel: active or inactive (“purged”) ○ “response statuses” (completed survey, quotafull, screenout, etc) ○ “elimination” period Response Rate model: ● arbitrary approximation to what have been observed in field ● simple concept (assumes business rules touches/invites); fixed ● assigned to each Panelist randomly following a sketched distribution: ○ Two betas: one for “bad” respondents and another for “good” ones
- 8. VP&F: Recruits; Purging Recruits model: ● List of Panelist unique instances with assigned response rate ● Fixed amount per iteration cycle (eg 2000 initializations) Purging model: ● A priority queue: FIFO and then importance sampling ○ Active universe (see Universe) ordered by entry, the “older” ones evaluated first ○ Then purging by sampling over response rate: the higher the response rate, the higher the chance of staying ● Assumes: ○ Older ones leave first but higher response => higher Loyalty ○ Number of Purged = Number of Recruits (i.e. fixed amount)
- 9. VP&F: Universe, Eligible Universe model: ● List of active instances at each cycle of Recruits and Purgings Universe baseline model: ● Used to calculate the parameters associated to Response Rate model, particularly the relationship between the two beta distributions ○ arbitrary model for demo purposes ○ Calculated using an iterative goal-seeking method ○ Objective is to find consecutive universes which average response rates stay around a requested value (asymptotic behaviour) ■ universe’s average response rate = (current universe size)-1 * Σ (rr of active instances) Eligible model: ● The instances available for sampling at each iteration (see Exclusion) ○ It is not purging
- 10. VP&F: Sampling, Interviewing Sampling model: ● Sampled instances extracted from Eligible ● Assumes: ○ Quota Sampling: surveying stops if a fixed number of completes (“target”) is reached ○ Sample size calculation based on business rules and previous performance (eg. see http://www.slideshare.net/drkellypage/pagek-marketing-researchwk61) Interviewing model: ● Conditioned Bernoulli rules hierarchy to assign response statuses
- 11. VP&F: Exclusion Exclusion (or Elimination) model: ● Temporary inactivation for sampling (“flagging”) of an active Panelist instance ● Used in real practice to avoid recall bias ● Inactivation period usually defined following best practices rules and based on last response status ○ In this work, implemented as a reverse counter that updates per iteration and with Panelist response status as parameter ○ Arbitrary values for demonstration only
- 12. Tracking model: ● The core of the experiment; complex ● Describes typical operations to reach quota target during a determined period of time (eg. DAYS of WEEK) called fieldwork ● Assumes: ○ Quota sampling over a specific demographic (eg. “young males”) ○ Each iteration represents a fieldwork; a nested one for days ○ Target reached either with one or more samples per fieldwork ○ Any additional sample during a fieldwork is called “batch” or “extra” ○ Eg. constraints that might enter the formula for sample size calc.: ■ Incidence (measures of the product penetration in universe; usually given) ■ Exposure (see below) VP&F: Tracking
- 13. VP&F: Exposure Exposure model: ● A proposal for this project ● Measures the inability to reach the whole sample at once; supported by empirical evidence ● Also a measure of technology effect over survey access ● Hypergeometric-family dist: instances in sample without a response status will be resampled according to the exposure parameter until sample turns inactive (in DAYS of fieldwork) ● Assumes: ○ Constant ○ Example of how to estimate it: Using a referential experiment ■ iterative procedure using a goal-seeking method over a simple tracking experiment ■ simple tracking experiment should be a typical one; the one given here is arbitrary