ACUITY THEORY AND FAKING RESISTENCE(4)
- 1. Symposium/Forum TITLE AcuityTheory:A studyof twofakingresistantnon-cognitive measures ABSTRACT Thispaperdescribesanewmethodof non-cognitivemeasurementthatdoesn’tinvolve introspective self report.We manipulatedlevel of incentive(within andbetweengroupdesign)tomotivate asample of Spanishsalesincumbentstomaximize theirscores.We foundcriterion-relatedvalidityonlyunder incentivizedconditionsandshowresistance tofaking. PRESS PARAGRAPH The theoryof cognitive acuityproposesthatthe more one possessesanymentallyheldcharacteristic the more one will be sensitivetophenomenathatrepresentthatcharacteristic.Sensitivityismeasured by presentingpairsof contrastedphenomena(e.g.,calibratedwrittenstatements) which are treatedas signalstobe detected.Thispaperdescribesthismethodof non-cognitive measurementthatdoesn’t involve introspectiveself-report.We manipulatedlevel of incentive (withinandbetweengroup) to motivate asample of Spanishsalesincumbentstomaximizetheirscores.We foundcriterion-related validityonlyunderincentivizedconditionsandshow resistance tofaking. WORD COUNT 2896
- 2. The problem of faking on non-cognitive measures continues to be a concern for both researchers and practitioners alike. Studies examining the causes and consequences of faking behavior proliferate in the social sciences. At the most basic level, Snell and Sydell (1999) proposed that faking behavior is a function of 1) the applicant’s ability to fake and 2) the applicant’s motivation, tendency, or propensity to fake. These authors point out that motivation to fake may be related to individual traits, demographic characteristics, and perceptual factors. However, the motivation to distort ones responses on a measure in order obtain a more desirable score would seem to be a rational and fundamental test-taking impetus and one which all high stakes test takers would likely share especially as the rewards of test performance increase. This study presents an experimental non-cognitive testing method and manipulates the degree of test- taker-motivation (TTM) by varying the test performance reward using a monetary reward and a within and between subjects research design. We aim to criterion-validate this method and show how incentivizing test performance improves rather than degrades criterion-related validly. TTM Background Arvey, Strickland, Drauden, and Martin (1990) described TTM as, “the extent to which respondents demonstrate effort, diligence, and focus in writing a test.” These authors also describe research suggesting that that levels of TTM vary between applicants and incumbents and that those differences could be related to test validity and test performance. Avery and colleagues also report that when incumbents were compared to applicants, the applicants showed higher test-taking motivation and excreted more effort to answer correctly even after controlling for the effects of ability level. It is widely known that low-stakes testing often leads to a lower effort from the test-taker (Wise & DeMars, 2005). If participants feel that there is nothing to gain from the test, they won’t
- 3. be motivated to take the test and their scores may not serve as valid indicators of their true level of the construct of interest (Sundre, 1999; Sundre & Kitsantas, 2004; Wise & DeMars, 2005). Most of the research conducted on low-stakes testing and TTM suggests attempting to increase examinee motivation (Wise, Bhola, & Yang, 2006) and filtering out examinees with low motivation (Sundre & Moore, 2002; Wise & Kong, 2005; Wise, Wise, & Bhola, 2006) are possible ways to reduce construct-irrelevant variance. O’Neill, Goffin, & Gellatly (2010) maintained that, “motivated test-takers might be inclined to distort or ‘fake’ their responses to achieve personal ends” and suggest that higher TTM will result in higher validity. If it were possible to measure non-cognitive constructs in a similar way as cognitive constructs then we might capitalize on test takers motivation set to improve test validity. The theory of cognitive acuity The theoryof cognitive acuitypostulatesthat1) the more one possessesanymentallyheld construct (e.g.,extroversion),the more one willbe sensitivetodistinctionsinconstruct-representative phenomena(e.g.,objects,statements,events) and2) that we can measure the degree towhichone possessesanymentallyheldconstructbypresentingconstruct –representative phenomenaof various magnitudesof construct-representativesignal strengthandobservehow sensitive (i.e.,acute) the respondentistothese distinctions.The more acute one is,the more he/she islikelytopossessthe construct andmanifestitbehaviorally(e.g.,jobperformance).Forexample,because of his/hergreater exposure toextroversionphenomena(e.g.,participatinginsocial events),anextrovertisexpectedtobe more attunedto the difference betweenthe followingtwostatements—“Anextrovertenjoysloud parties”versus“Anextrovertenjoysbeingwithfriends”. Acuitytheorytreatsthese statementsas construct-relevantsignalswhichare differentiallydetectable dependingonone’spossessionof the construct. Anotherexample usingthe customerservice orientation(CSO)constructwouldbe topresent the followingtwostatements.
- 4. “A true CSOpersonthinksaboutpastinteractionswithcustomers” “A true CSOpersonputsthe customer’sneedsabove the needtomake asale” The more one possessesthe CSOconstructthe more sensitive he/she will be tothe factthat one of these statementswasratedbyconstructsaturants(those highlypossessedof theirrespective constructs,i.e.,strongextroverts)asa strongerrepresentationof the CSOconstructthan the other. Note that nointrospectioniscalledforindecidingwhichisthe strongerCSO- representative statement and thuswe minimize respondent’sreliance onintrospectionwhichisprone todistortionsandbiases (Nisbett&Wilson,1977; Pronin,& Kugler,2007). CSOand Extroversion Evidence forthe criterion-relatedvalidityof CSOmeasureshasbeenprovidedbyameta-analysis conductedbyFrei and McDaniel (1998) who,aftera review of 41 studies,reportedanunadjusted validitycoefficientof r = .24 whensupervisoryperformance ratingswere the criteria.Likewise,the personalitytraitof extroversionhasfrequentlybeenstudiedasanimportantconstructfor effective salesworkerperformance (HausknechtandLangevin,2010). Barrick and Mount(1991) conducteda metaanalysisinvolving2,316 subjectsacross22 studiesandreporta meancorrelationof r = .09 (estimatedtrue correlation=.15 ) betweenextroversionandperformance criteria. Ina 1998 meta- analyticreviewbyVinchur,Schippmann,Switzer,andRothreportan average unadjustedcriterion- correlationof r = .09 betweenextroversionandsalesperformance. Recently,Grant,(2013) examinedthe extroversion-salesperformance relationinthe extant literature andproposedthat ambiverts,thosewhodemonstrate amore flexiblepatternof talkingand listening,are more aptto be the betterperformersandpositedaninvertedU-shaped(-shaped) relationbetweenextroversionandsalesperformance.Grant(2013) wenton to demonstratedthat among340 call-centerrepresentatives,the additionof aquadraticextroversionterminthe regression
- 5. of salesrevenueonextroversionproducedasignificanteffectshowingthatthe relationisbetter modeledas -shapedfunction. Hypotheses H1 Amongthe incentivized(butnotthe un-incentivized) samples,salesassociates’sensitivityto contrastsin CSOrepresentativenessamongCSOstatementswillbe significantlyrelatedtoobservedCSO behavior(i.e.,salesperformance). H2 Amongthe incentivized(butnotthe un-incentivized) salesassociates’sensitivitytocontrastsin extroversionrepresentativenessamongextroversionstatementswill be significantlyquadratically(- shaped) relatedtoobservedextroversionbehavior(i.e.,salesperformance). Method InstrumentDevelopment Basedon a literature reviewof the respective constructors,atotal of 123 extroversion statementsand119 CSO statementswere developed.These statementswere writtentorange fromnot- at-all representativeof the respective constructstoextremelyrepresentative. Statementconstruct-calibrationsurvey developmentand translation A surveywasconstructedforthe extroversionandCSOconstructswhichwasintendedtocollect opinionsfromconstructsaturantsas to the degree towhicheachstatementwasrepresentativeof its respective construct.A five-pointscale wasusedtorate representativenessandrangedbetween1 - Not at all representative to5 - Extremelyrepresentative. We alsodevelopedaconventional self-report measure of bothCSO and extroversiontoserve acontrol measure.Basedonthe construct/measure reviewa16-itemextroversionand15-itme CSOself-reportmeasurewere developed.Nextwe identified a total of 15 construct saturantsfor boththe CSO and extroversionstatementconstruct-calibration
- 6. effort. The surveyandconventional measureswereprofessionallytranslatedintoCastilianSpanishand administeredtothe saturantsinorderto calibrate our measures.Eachof the statementsinthe construct-calibrationsurveywasratedbyitsrespective saturantandthusitwas possible toevaluatethe degree towhichanystatementgarneredweakversusstrongagreementfromamongthe saturantswho ratedit. Statementratingswere evaluatedforconsistencyusingBrown,ReaganandHauenstein’s(2005) coefficientof agreementaWG(1) (avariantof rWG ).Those response optionsproducingaWG(1).<.80 were removedfromthe analyses.The meansaturant ratingof the survivingstatementswasusedasthe estimate of thatstatement’sconstruct-representativeness.Survivingstatementsfrombothsurveys were nextpairedwithotherstatementsintheirconstructinorderto formconstruct-representativeness contrasts(i.e.,signalstobe detects) whichwouldbe usedasthe final testitemsforthe CSOand extroversionmeasures.Foreachconstructa set of 100 pairswere selectedtoformtwoinstruments whichwere translatedintoCastilianSpanish. Analysis Methods Measurementof Acuityusing Binary Logistic Regression Analysis(BLRA) To testour hypotheseswe firsthadtoestimate how sensitiveeachrespondentwastothe statementcontrastsignal magnitudes.The challenge wastomodel the respondent’spatternof binary signal detections(dependentvariables) associatedwiththe continuousstatementcontrastsignal magnitudes(independentvariables).Aftermuchresearchanddiscussionswithcolleaguesthe authors electedtouse binarylogisticregressionanalysis (BLRA)toestimate respondent’ssensitivitytothe contrast magnitudes. AcuityScore Generation
- 7. Whenappliedtoeachrespondent’ssetof binaryresponses(dependents) andcontinuous independents(pairedcontrasts),the BLRA generatestwoparameterestimateswhichare relevanttoour measurementof extroversionandCSOstatementcontrastsignal sensitivity. Logistic betaweights(Beta).The BLRA resultingBetaweight(slope) isthe natural logof the odds ratioand representsthe importance withwhichthe respondentregardedthe statementpaircontrastin his/herdecisiontoselectone optionoverthe otherandservesasan indicatorof construct-relevant sensitivity.We willdenotethese BetaweightscoresasAcuity_CSO_BetaandAcuity_Extro_Beta respectively. Logistic chi square(χ2 ).The Pearsongoodness-of-fitchi-square statisticteststhe null hypothesis that the person-specificmodel adequatelyfitsthe dataandisregardedas a measure of the respondent’ssensitivityasitcapturesthe goodnessof fitbetweenthe respondent’spatternof statementselectiondecisionsandthe patternof statementcontrastmagnitudes.We will denote these scoresas Acuity_CSO_χ2 and Acuity_Extro_χ2 . These χ2 fitstatisticsare interpretedsuchthathigh scoresindicate poorfitand hence weakrespondentacuitywhile low scoresindicate goodfitandstrong acuity. As controlswe alsocomputedraw binaryscoresfor eachrespondentbasedsimplyonthe proportionof correct versusincorrectsignal detections.Thus, Acuity_CSO_Raw_Score =the meanof the 100 binaryacuityCSOitems Acuity_Extro_Raw_Score =the meanof the 100 binaryacuityextroversionitems ConventionalScore Generation The followingadditionalcontrol scoreswere collectedbasedonthe previouslydiscussed conventional self reportmeasures.
- 8. Self_Report_CSO_Score =the mean of the 5-pointLikertscale 15-itemCSOself-reportmeasure Self_Report_Extro_Score =the mean of the 5-pointLikertscale 16-itemextroversionself-report measure Participants Sample One consistedof 285 Spanishgeneral merchandise retail store salesemployeeswho were recruitedfromeightdifferentdepartmentstores(inthreedifferentcities) withinthe company. Participantswere askedtovolunteertocomplete the predictormeasures.Of thisnumber69 were malesand213 were femaleswith3beingundisclosed.Theiraverage age was42.50 (S.D = 9.36). All incumbentscompletedthe measureswhileoncompanytime. Sample Twoconsistedof 56 incumbentemployeesincluding12 males,44 females,withan average age of 40 (S.D= 7.92). Like Sample One,thissample wasalsorecruitedvoluntarilyfromthe same general populationandcompletedall measures.However,twoweeksaftercompletingthe measures,thissubsample wasaskedtocomplete the measuresasecondtime underanincentivized conditioninordertosimulate ahighstakestestingenvironment.Inthisconditionthe participantswere askedto perform theirbestonthe testand were offeredone of three 100 Euro giftcertificatestobe awardedto the highestscorer. Performance Measures The supervisorsof eachof the participatingemployeesratedthatemployee’sperformance on three dimensionsusinga1-4 pointscale with1 = Low,2 = Normal,3 = Good, 4 = Excellent.The three dimensionswere; 1. Overall Rating:the global ratingassignedtothe salesrepresentative
- 9. 2. CSORating: "Treatkindlyandknowshow to adaptto the customer" 3. SociabilityRating:"Integrateswithpeers,maintainingagoodworkingclimate".The SociabilityRating relatestoone of the strongeraspectsof workplace-relevantextroversioninSpanishretail environments (Oviedo-García,2007). Results No estimatesof internalconsistencyreliabilitywere availableforthe acuityscoresbecause the signal detectionmethodologypresentsdifferentmagnitudesof construct-relevantsignal andsaturant agreementisperhapsthe bestmeasure of reliability. To testH1 we usedSpearmanrank ordercorrelationsbecause muchof the datawere foundto be non-normal accordingtoShapiro-Wilktestsof eachindependentanddependentvariable.Table 1 presentthe descriptive statisticsandcorrelationcoefficientsforthe conventional andacuityCSO measuresalongwiththe criterionvariables.Resultsshow thatonlyunderincentivizedconditionsdidthe Acuity_CSO_Betasignificantlycorrelatewithsupervisors’CSORating(r= .31, p < .05) butnot with Overall Rating(r= .23, NS) or SociabilityRating(r= .24, NS) althoughresultsare inthe expected direction.Again,onlyunderincentivizedconditionsdidAcuity_CSO_2 significantlycorrelate withboth supervisors’Overall Ratings(r=-.28, p < .05) and SociabilityRatings(r= -.30, p < .05) but notto CSO Ratings(r = -.18, NS).However,again,all effectswere inthe expectednegativedirection.Underthe incentivizedconditionthe Acuity_CSO_Raw_Score wassignificantlyrelatedtoSociabilityRating(r=.29, p < .05) butnot to Overall Rating(r= .00, NS) or CSO Rating(r = .10, NS). Thus H1 ispartiallysupported. ----------------------------------------------------------------------- Place Table 1 About Here ----------------------------------------------------------------------
- 10. To testhypothesis H2we useda hierarchical regressionanalysisprocedure proposedbyAiken and West(1991) and employedbyGrant(2013). For eachof the tworelevantdependentvariables (Overall,andSociabilityratings) a2-stephierarchal regressionanalyseswasconductedforeachof the twoindicators(2 andBeta) of extroversion.Table 2presentsthe descriptivestatisticsandTable 3 presentsthe resultsof thisanalysiswhichwere dividedintotwosectionsbasedonwhichof the two dependentvariableswasbeingregressed. ----------------------------------------------------------------------- Place Table 2 and 3 About Here ---------------------------------------------------------------------- In step1 the linearindependenttermwasenteredandinstep2 the quadratic term(acuityscore squared) wasenteredwithsignificantimprovementtothe regressionbetaweighttakenasevidencefor a significantcurvilineareffect.Foronlythe incentivizedcondition,andwhere OverallRatingwasthe DependentVariable,significanteffectswere observedforbothAcuity_Extro_Beta (= -1.14, p < .05) in model a andb and for Extro_ χ2 ( = -1.69, p < .05) formodel c and d. Neitherthe linearnorcurvilinear functionsof Acuity_Extro_Raw_Score orSelf_Report_Extro_Scorewere foundtobe significant contributorstosupervisors’Overall Ratingof theiremployees. Under incentivizedconditions(butnotun-incentivized) andwhere supervisors’Sociability Rating was the DependentVariable,asignificantimprovementtothe linearrelationwasobservedafterthe introductionof the quadratictermforAcuity_Extro_Beta (= -1.04, p < .05) in modelsaand b.The Extro_ χ2 indicatorwasborderline significant (= -1.40, p < .06) for modelscand d.Again,neitherthe linearnorcurvilinearfunctionsof Acuity_Extro_Raw_Score orSelf_Report_Extro_Score were foundto be significantpredictorsof supervisors’Sociabilityratingsof employees.Thus H2 ispartially supported withthree outof foursignificanteffectsandone borderlineeffectobserved.Allsignificanteffects
- 11. occurredin the incentivizedconditionwhile evidencingthe predicted -shape.Neitherthe conventional self reportmeasuresnorthe acuityraw scoresproducedsignificanteffects. Discussion Acuitytheoryholdsthatthe more one possessesaconstruct(e.g.,extroversion) the more sensitiveone willbe tothe distinctionsinconstruct-representative phenomena(e.g. statements).Inthis studywe estimatedthe degree towhichrespondentswere sensitivetodistinctionsinthe CSOand extroversion-representativenessof pairedstatements.Forexample,forthe extroversionconstruct,one statementmayread“A true extrovertenjoysbeingwithfriends”while the otherreads“A a true extrovertenjoysloudparties”. The latterispositedtobe astrongerrepresentationof the extroversion construct thanthe former. Importantly,recognizingthese distinctionsdoesnotrequire thatthe respondentengage inintrospectionandself-report(e.g.,“Ilike loudparties”).Byremovingthe introspectiveself-reportcomponent,we anticipatedandfoundthatrespondentsproducedmore criterion-validresponsesevenundermonetarilyincentivizedconditions.We suspectthatthisoccurred because respondentsexertedefforttodetectthe small differencesinconstruct-representative statementsratherthanexertingefforttowardcontrivingsociallydesirableresponses.Ourresults supportthe propositionthat,whenappliedtonon-cognitive constructs,acuitymethodsmayoffer resistance tosociallydesirableresponding(SDR) because the taskbefore the respondentisnot construct self-reportbutratherconstruct-representativenessdetection. Acuity-basedestimationmethodsshowninthisstudywere precise enoughtomodel quadratic (-shaped) relationsbetweenextroversionsensitivityandsupervisor-observedsalesjobperformance consistentwithhypothesis.These resultswereanticipatedgiventhe seminal researchbyGrant (2013) whofoundthat intermediate levelsof extroversionare optimal whileeitherveryhighorverylow levels
- 12. are sub-optimal forsalesperformance. Rather,we foundthatthose whoare intermediate intheir sensitivitytodistinctionsinextroversionrepresentativestatementsweremore likelytoreceiveahigh salesperformance ratingfromtheirsupervisor. Several importantlimitationstothisstudymustbe considered.1) The sample size of n= 56 providedverysmall statistical power.2) Ratherthan usingthe newlydevelopedmeasuresof CSOand extroversionpresentedhere,subsequentresearchmightuse instrumentswithestablished measurementproperties.3) Finally,subsequentresearchwouldbenefitfromcomparingmeasurement propertiesof statisticsobtainedfromacuitymethodsandthose derivedfromItemResponseTheory (IRT).Despite these limitations,the preliminaryfindingsreportedhere supportcontinuedresearchin applyingacuity-basedconstructestimationmethodsastheyappeartoofferresistance toSDRand may pointto newdirectionsinmental measurement. References Aiken,L.S.,& West,S. G. (1991). Multiple regression:Testingandinterpretinginteractions.Newbury Park,London,Sage. Arvey,R.D., Strickland,W.,Drauden,G.,& Martin, C. (1990). Motivational componentsof testtaking. PersonnelPsychology,43,695–716. Barrick,M.R. & Mount, M.K. (1991). The bigfive personalitydimensionsandjobperformance:A meta- analysis.Personnel Psychology,44,1-26. Brown,T. J., Mowen,J.C., Donavan,D.T., & Licata,J. W. (2002). The customerorientationof service workers:Personalitytraiteffectsonself andsupervisorperformance ratings.Journal of Marketing Research,39, 110–119. doi:10.1509/jmkr.39.1.110.18928
- 13. Grant, A. (2013). Rethinkingthe extravertedsalesideal:The ambivertadvantage,Psychological Science, 24, 1024–1030. Hausknecht,J.P.,& Langevin,A.M. (2010). Selectionforservice andsalesjobs.Retrieved4/27/2014 fromCornell University,ILR School site:http://digitalcommons.ilr.cornell.edu/articles/190 Leeds,J.P.(2012). The theoryof cognitive acuity:Extendingpsychophysicstothe measurementof situational judgment, Journalof Neuroscience,Psychology,and Economics,5,3, 106-181. Leeds,J.,P.& White,A.(2013). Estimatingcognitive acuityintwosituational judgmenttests:More lessonsfrompsychophysics.Paperpresentedatthe 25th Annual Conventionof the Associationfor Psychological Science,WashingtonDC. Nisbett,R.& Wilson,T.(1977). Tellingmore thanwe canknow:Verbal reportsonmental processes. Psychological Review,84,231-259. O’Neill,T.A.,Goffin,D.G., Gellatly,I.R.(2010). Test-takingmotivationandpersonalitytestvalidity. Journalof PersonnelPsychology,9, 3,117-125. Oviedo-García,M. A.(2007). Internal Validationof aBiodataExtraversionScale forSalespeople.Social BehaviorandPersonality,35,5, 675-691. Pronin,E.& Kugler,M.B., (2007). Valuingthoughts,ignoringbehavior:The introspectionillusionasa source of the biasblindspot. Journalof ExperimentalSocialPsychology,43,565-578. Snell,A.F.,Sydell,E.J.,&Lueke,S.B.(1999). Towardsa theoryof applicantfaking:Integratingstudiesof deception. Human ResourceManagementReview,9,219-242.
- 14. Sundre,D.,L. & Kitsantas,A.(2004). Anexplorationof the psychologyof the examinee:Canexaminee self-regulationandtest-takingmotivationpredictconsequential andnon-consequential test performance? Contemporary EducationalPsychology,29, 6–26. Sundre,D.L., & Moore, D. L., (2002). The StudentOpinionScale:A measure of examinee motivation. AssessmentUpdate,14, 8-9. Sundre,D.L (1999). Does examineemotivation moderation therelationship between testconsequences and test performance? Paperpresentedatthe annual meetingof the AmericanEducational ResearchAssociation,Montreal,Quebec,Canada. Vinchur, A. I., Schippmann, J. S., Switzer, F. S., III, & Roth, P. L. (1998). A meta-analytic review of predictors of job performance of salespeople. Journal of Applied Psychology,83,586-597. Wise,S.L., & DeMars, C. E. (2005). Low examinee effortinlow-stakesassessment:Problemsand potential solutions. EducationalAssessment,10,1-17. Wise,S.L., & Kong,X. (2005). Response time effort:A new measure of examinee motivationin Computercasedtests. Applied Measurementin Education,16,163183. Wise,S.L., Bhola,D., & Yang, S.(2006) Taking thetime to improvethe validity of lowstakestests:The effortmonitoring CBT.Paperto be presentedatthe annual meetingof the National Councilon MeasurementinEducation,SanFrancisco,CA. Wise,S.L.,Bhola,D.S.,& Yang, S. (2006). Takingthe time toimprove the validityof low-stakestests:The effort-monitoringCBT. EducationalMeasurement: Issuesand Practice,25, 21-30.
- 15. Table 1 SpearmanCorrelationAnalysesof CustomerService OrientationAcuityScoresonSales AssociatesJobPerformance WithinandBetweenSubjectsComparision UNINCENTIVISED 1. Acuity_CSO_Beta 1.38 .80 .08 -.31 1 2. Acuity_CSO_ 2 † 17.43 6.89 .75 1.82 -.53** 1 3. Acuity_CSO_Raw_Score † .73 .09 -.38 -.17 .53** -.37** 1 4. Self_Report_CSO_Score† 4.54 .34 -.68 -.24 .04 .09 -.12 .83 5. Overall Rating † 3.04 .64 -.19 -.04 .04 -.08 -.00 .06 .82 6. CSO Rating † 3.24 .61 -.19 -.56 .05 -.01 -.01 .04 .66 ** 1 7. Sociability Rating † 3.12 .76 2.36 .24 -.00 -.07 -.04 .05 .61** .54** 1 1. Acuity_CSO_Beta 1.51 .79 .25 -.73 1 2. Acuity_CSO_ 2 † 16.94 7.43 .74 .49 -.46 ** 1 3. Acuity_CSO_Raw_Score .73 .09 -.90 1.24 .34 * -.32 * 1 4. Self_Report_CSO_Score† 4.52 .32 -.30 -.72 .18 .00 -.07 .83 5. Overall Rating † 3.09 .51 .15 .86 .03 -.16 .13 -.10 .83 6. CSO Rating † 3.23 .54 .14 -.15 -.03 -.20 .08 .04 .72** 1 7. Sociability Rating † 3.07 .57 .02 .23 .02 -.08 .22 .06 .53** .41** 1 1. Acuity_CSO_Beta 1.85 .89 .20 -.35 1 2. Acuity_CSO_ 2 † 15.71 6.70 .23 -.90 -.50** 1 3. Acuity_CSO_Raw_Score .74 .10 -.69 .86 .34* -.39** .86 4. Self_Report_CSO_Score† 4.56 .31 -.15 -1.13 .15 .09 -.07 .85 5. Overall Rating † 3.09 5.14 .15 .86 .23 -.28 * .00 .13 1 6. CSO Rating † 3.23 5.39 .14 -.15 .31* -.18 .01 .10 .72** 1 7. Sociability Rating † 3.07 5.68 .02 .23 .24 -.30* .29* .07 .53** .41** 1 **. Correlation is significant at the 0.01 level (2-tailed). † Shapiro-Wilk test of departure from normality for small samples w as significant (the data are non-normal) INCENTIVISED Sample 2 - Time 2 (N =56) Sample 2 - Time 1 (N =56) UNINCENTIVISED *. Correlation is significant at the 0.05 level (2-tailed).
- 16. Table 2 Descriptive StatisitcsforExtraversionAcuityScoresonSalesAssociatesJobPerformanceWithin & BetweenSubjectsComparison Mean Std. Deviation Skewness Kurtosis UNINCENTIVISED Sample 1 (n = 285) Acuity_Extro_Beta -.04 2.24 -.30 .78 Acuity_Extro_ 2 31.16 213.97 12.27 162.19 Acuity_Extro_Raw_Score .70 .13 -.80 1.90 Self_Report_Extro_Score 2.43 .46 .07 .59 Overall Rating 30.35 6.45 -.19 -.04 CSO Rating 32.43 6.13 -.19 -.56 Sociability Rating 30.88 6.48 -.48 .83 UNINCENTIVISED Tample 2 - Time 1 (n = 56) Acuity_Extro_Beta 2.19 1.06 .32 -.18 Acuity_Extro_ 2 28.53 9.37 .36 -.14 Acuity_Extro_Raw_Score .67 .14 -.23 .88 Self_Report_Extro_Score 2.60 .34 -.29 -.16 Overall Rating 30.89 5.14 .15 .86 CSO Rating 32.32 5.39 .14 -.15 Sociability Rating 30.71 5.68 .02 .23 INCENTIVISED Sample 2 - Time 2 (n = 56) Acuity_Extro_Beta 2.28 .98 .70 1.11 Acuity_Extro_ 2 29.08 9.51 .15 -.48 Acuity_Extro_Raw_Score .71 .11 .02 -.43 Self_Report_Extro_Score 2.63 .34 .05 -.31 Overall Rating 30.89 5.14 .15 .86 CSO Rating 32.32 5.39 .14 -.15 Sociability Rating 30.71 5.68 .02 .23 CSO Rating: Supervisor rating "Treat kindly and know s how to adapt to the customer" Overall Rating = Rating by supervisor in the last year (10 = Low , 20 = Normal, 30 = Good, 40 = Excellent) Sociability Rating: Supervisor rating "Integrates w ith peers, maintaining a good w orking climate"
- 17. Table 3 Hierarchical RegressionAnalysesTestingforQuaderatic Effectsof ExtraversionAcuityScores on SalesAssociatesJobPerformance Within&BetweenSubjectsComparison Model R F Change Sig. F Change R F Change Sig. F Change R F Change Sig. F Change Shape of Relation a. Step 1 -.01 .01 .03 .86 .11 .11 .59 .45 -.12 .12 .76 .39 b. Step 2 -.10 .06 .91 .34 -.97 .28 3.85 .06 -1.14* .34 5.80 .02* c. Step 1 -.03 .03 .24 .62 -.16 .15 1.18 .28 .01 .01 .01 .92 d. Step 2 -.04 .03 .10 .75 .57 .18 .67 .42 -1.69* .31 5.68 .02* e. Step 1 .08 .07 1.59 .21 -.05 .05 .13 .72 -.10 .10 .53 .47 - f. Step 2 -.02 .08 .00 .95 .08 .05 .01 .92 .43 .11 .09 .77 g. Step 1 -.03 .03 .19 .67 -.02 .02 .01 .91 -.04 .00 .08 .78 - h. Step 2 .18 .03 .11 .74 1.07 .10 .43 .51 -1.66 .14 .94 .34 Dependent Variable: Overall Rating a. Step 1 -.05 .05 .68 .41 .12 .11 .69 .41 -.05 .05 .13 .72 b. Step 2 -.18 .12 3.28 .07 -.78 .24 2.46 .12 -1.04* .29 4.62 .04* c. Step 1 -.02 .02 .08 .78 -.13 .07 .29 .59 -.13 .13 .96 .33 d. Step 2 -.09 .05 .59 .44 -.01 .19 1.67 .20 -1.4 .29 3.83 .06 e. Step 1 .00 .00 .00 1.00 .01 .10 .53 .47 .11 .11 .66 .42 - f. Step 2 -.36 .06 1.18 .28 -.24 .11 .08 .77 -1.66 .19 1.35 .25 g. Step 1 -.04 .01 .25 .62 .01 .03 .03 .86 .01 .01 .01 .92 - h. Step 2 .13 .07 .07 .80 -.07 .10 .36 .55 -2.11 .17 1.55 .22 Dependent Variable: Sociability Rating a. Acuity_Extro_Beta b. Acuity_Extro_Beta Squared c. Acuity_Extro_ 2 d. Acuity_Extro_ 2 Squared e. Acuity_Extro_Raw_Score f. Acuity_Extro_Raw_Score Squared g. Self_Report_Extro_Score h. Self_Report_Extro_Score Squared * Significant at the 0.05 level (2-tailed). No statistically significant (p < .05) effects were found or reported where CSO_Rating was the dependent variable. † Sample size for the Conventional_Customer_Service_Raw_Score Between Subjects analysis = 168 . Sociability Rating: "Integrates w ith peers, maintaining a good w orking climate" Overall Rating is the rating assigned of the sales representative by the immediate supervisor in the last year = Standardized regression coefficients UNINCENTIVISED Tample 2 - Time 1 (n = 56) INCENTIVISED Sample 2 - Time 2 (n = 56)UNINCENTIVISED Sample 1 (n = 285)