Rule Order Manipulation and the IRAP
- 1. Department of Psychology Maynooth University, Maynooth Final Year Research Project Exploring the effect of rule order manipulation on implicit self-esteem in the Implicit Relational Assessment Procedure (IRAP) Thesis Presented in Part Fulfilment of the Requirements for the BA Honours Degree in Psychology, Maynooth University, Maynooth Submitted by: Sarah Kenehan Supervisor: Dr. Yvonne Barnes-Holmes Date February 2015.
- 2. i Table of Contents Table of Contents........................................................................................................................i Acknowledgements....................................................................................................................ii Abstract.....................................................................................................................................iii Introduction................................................................................................................................1 Method.......................................................................................................................................7 Participants.....................................................................................................................7 Setting.............................................................................................................................7 Apparatus and Materials.................................................................................................7 Measurement..................................................................................................................7 Procedure......................................................................................................................12 Ethical Considerations..................................................................................................13 Results......................................................................................................................................14 Discussion................................................................................................................................17 References................................................................................................................................23 Appendix A: Consent Form Appendix B: Rosenberg Self-Esteem Scale (RSES) Appendix C: Depression Anxiety Stress Scales (DASS)
- 3. ii Acknowledgements I would like to take this opportunity to thank my supervisor Dr. Yvonne Barnes-Holmes for her constant support, encouragement and guidance. Working with you has been a pleasure and a valuable learning experience. A special thanks to Deirdre Kennedy who kept me grounded, especially when things were not going according to plan. I wish to express my gratitude to all my participants who took time out of their busy schedules to complete my experiment, sometimes multiple times, without your participation this research would not have been possible. I would also like to thank the Psychology Department of Maynooth University, who have helped me greatly throughout my three years here. Finally, I would like to thank my family and friends for their unconditional support, love, and encouragement throughout my degree.
- 4. iii Abstract Self-esteem is an important and well-established concept particularly areas of psychopathology such as depression. In the past explicit measures were used to assess self- esteem however these measures are susceptible to different kinds of bias. In order to overcome this implicit it has become increasingly more common to use implicit measures such as the IAT (Implicit Association Test) and IRAP (Implicit Relational Assessment Procedure). As with other measurement methodologies, it is essential to assess its validity and to eliminate any variables that will confound the data. This experiment aims to evaluate the impact of rule order manipulation on the IRAP effect during a self-esteem IRAP. The experiment was conducted with 41 participants who were split into two group. In one group the “self” aspect of the rule was presented first, whilst in the other group “others” was always at the beginning of the rule. Results showed that there was a difference between the groups, especially for the self positive and others positive trial types, however these differences did not reach statistical significance. The findings suggest that implicit responding may be influenced by rule order effects.
- 5. 1 Reporting high levels of self-esteem is thought to be of great importance to the mental health and the well being of an individual (Dunning, Meyerowitz, & Holzberg, 1989). This has been found as a result of extensive research that finds that people who suffer from depression often report thinking negatively about the self (see Hagga, Dyck, & Ernst, 1991 for a review of the literature). This recurrent finding that depression is linked to negative self- evaluations and low self-esteem stimulated the cognitive behavioural theory of depression. This theory stated that it is these cognitively dysfunctional negative schemata that maintain and increase vulnerability to depressive episodes (Williams, 1997). This theory has been supported by the use of various questionnaires that demonstrate the tendency of clinically depressed patients to report lower self-esteem than controls (Ingram, Miranda & Segal, 1998). In recent years however, the use of questionnaires to evaluate self-esteem has come under increasing criticism (Young, 1994). Explicitly asking individuals about their self- esteem can be gravely affected by multiple forms of bias and external factors such as demand characteristics, social desirability, and self-representation (Ingram & Wisnicki, 1991). As well as this, according to the cognitive behavioural theory, some schemata relating to the self may not be consciously accessible to an individual so it is therefore impossible for the individual to explicitly report these cognitions (Greenwald & Banaji, 1995). These flaws in explicitly measured self-esteem have acted as a catalyst, triggering research in the measurement of implicit attitudes (Greenwald, McGhee & Schwartz, 1998).
- 6. 2 The development of the Implicit Association Test (IAT) and other implicit tests have brought about a new way to measure self-esteem and have begun to contradict already established theories of self-esteem (Greenwald et al., 2002). These new paradigms have improved the way in which underlying schema-processes are evaluated which is reflected in their predictive power (Gremar , Segal, Sagrate, & Kennedy, 2001). The IAT measures implicit attitudes by assessing the strength of associations between two concepts. It assumes that if concepts are implicitly closely related and share the same response key that participant’s response times will be quicker than those stimuli with weaker associations (Pinter, & Greenwald, 2005). DeRaedt, Schacht, Franck and DeHouwer, (2006) were one of the first researchers to use an implicit measure to assess self-esteem. They asked participants to associate positive words (such as capable, competent and good) and negative words ( for example inferior, failed and worthless) to a “self” and “not-self” category. They found that both depressed and non-depressed participants displayed positive implicit self-esteem (DeRaedt et al., 2006). These results contradicted the popular theory that depressed individuals had lower self-esteem and stimulated the creation of other implicit measures. One of the limitations of the IAT is that it only measures associations between stimuli. The direction and relationship between the stimuli cannot be assessed, overcoming this limitation was one of the main aims a newly developed implicit measure known as the Implicit Relational Assessment Procedure (IRAP; Barnes-Holmes, Barnes-Holmes, Stewart & Boles, 2010). The IRAP is based upon a recent account of human language known as Relational Frame Theory (RFT). RFT uses the concept of stimulus equivalence but hypothesises that the relations that arise as a result of stimulus equivalence are subsequently framed and functionally categorised as part of a generalised operant response (Hayes, Barnes- Holmes, & Roche, 2001). Stimulus equivalence occurs through the process of creating relations about an object through a discrimination in language (Hayes, Barnes-Holmes &
- 7. 3 Smeets, 2000). These relations may be non-arbitrary based and on the physical properties of the object, or arbitrary and formed by abstract factors which have been related in previous interactions with the object (Torneke, Luciano & Salas, 2008). Non-arbitrary relations have been found in the animal kingdom, but the arbitrary responding found in humans has yet to be found in other species. It is hypothesised that the ability to create arbitrary relations is at the core of complex human language (Barnes-Holmes et al., 2010). The IRAP measures peoples’ implicit attitudes by asking them to respond to a sample stimulus and a target item in line with a particular belief which is provided by the researcher (Barnes-Holmes, Hayden & Stewart, 2008). Similar to the IAT, the IRAP assumes that the response times will be quicker on trials where the rules being followed are consistent with the implicit beliefs of the participant. The differences seen in the response times of participants during consistent and inconsistent trials is hypothesised to be due to the participants having to respond against their more probable implicit responses during an overt responding task (O’Hora, Barnes-Holmes, Roche & Smeets, 2004). The first IRAP that was used to assess self-esteem compared two groups that were known to have different levels of self-esteem, prisoners and Irish students (Vahey, Barnes-Holmes, Barnes-Holmes, & Stewart, 2009). Students have reliably been reported to have higher self-esteem than prisoners (Gullone, Jones & Cummins, 2000). Vahey and colleagues (2009) found that students demonstrated did in fact display higher self-esteem than prisoners which began to validate the IRAP as a measure of self-esteem. Providing evidence of concurrent validity is of extreme importance to any type of new measure, but this is particularly true of implicit measures as the dependent variable is more abstract (Golijani-Moghaddam, Hart, & Dawson, 2013). The IAT is the oldest and most validated measure of implicit self-esteem and as discussed earlier when implicit self-esteem was measured using this method it was found that there was no difference between depressed patients and non-depressed controls (DeRaedt et
- 8. 4 al., 2006). This differed from the explicit measures but this was difference was hypothesised to be caused by bias and other confounding variables surrounding the explicit measure. However, researchers began to question the validity of the IAT of self-esteem. They postulated that as the IAT only assessed associations it was possible that it was measuring one of two different variables in the two participant groups, either ideal or actual self-esteem (Zentner & Renaud, 2007). In order to test this hypothesis the IRAP was used because of its ability to capture propositional behaviour rather than simply measuring associations. Reume and colleagues (2013) conducted an experiment whereby participants, who were either rated as highly dysphoric (participants who scored highly on the Beck Depression Inventory) or normal, completed two separate IRAPs; one where stimuli were related to the actual self ("I am") or the ideal self ("I would like to be"). The results of the study showed that the dysphoric group had a higher ideal self-esteem and a lower actual self-esteem when compared to the control group (Reume et al., 2013). This research highlights the importance of validating procedures and re-evaluating findings of previous research to provide the most accurate data and interpretations possible. This is of considerable importance due to the emphasis that has been put on self-esteem in relation to theories and treatment of people suffering from depression (Roberts, Gotlib, & Kassel, 1996). This process of constantly evaluating techniques used to measure psychological concepts is of extreme importance and is reflected in the amount of research being conducted in identifying and analysing variables that may influence IRAP data (see Golijani- Moghaddam, Hart, & Dawson, 2013 for a review of the literature). One that is of particular interest is the malleability of the IRAP effect (Hughes, Barnes-Holmes & Vahey, 2012). Implicit measures are extremely influenced by context and it is important that effects like this are controlled for during future research (Ferguson & Bargh, 2007). Hooper, Villatte, Neofotistou & McHugh (2010) conducted an experiment in which participants were exposed
- 9. 5 to an IRAP which assessed their level of avoidance. Upon completing the first IRAP, participants were shown a disturbing image of a dead body so that negative thoughts were induced. Subsequently the participants were exposed to a 10 minute audio file which promoted either the use of mindfulness or thought suppression to control negative thoughts. The participants then completed the original IRAP again and it was found that the individuals in the thought suppression group experienced an increase in implicit experiential avoidance compared to the mindfulness group (Hooper et al., 2010). The introduction of an intervention targeting the implicit attitude under investigation produced significant differences in the results of the experiment clearly demonstrating the malleability of implicit attitudes. This finding has been replicated in other clinically relevant populations such as people suffering from depression (Hussey & Barnes-Holmes, 2012). The malleability of implicit attitudes has also been found whereby once a context has been established implicit racist and ageist attitudes have been altered (Fazio, Jackson, Dunton, & Williams, 1995; Cullen, Barnes- Holmes & Barnes-Holmes, 2009). Implicit attitudes are not only susceptible to be change through direct measures, such as mood induction or a contextual intervention, they are also heavily influenced by more technical and indirect variables such as screen presentation (Parris, Sharma, & Weekes, 2007). As participants during an IRAP task are under strict time pressure (in order to ensure that results are due to implicit responding), what is presented on screen is of optimal importance to allow participants to respond accurately and achieve mastery criteria (Deroost & Saetens, 2006). In order to investigate the effects that screen presentation had on the IRAP effect, four different groups of participants were designated to groups where screen presentations were slightly varied (Campbell, Barnes –Holmes, Barnes-Holmes, & Stewart, 2011). They alternated sample stimuli and response options from being in a fixed position throughout the trial to switching randomly on screen. They found that the randomisation of
- 10. 6 the sample stimuli (in the presence or absence of randomisation of response options) produced the strongest IRAP effect. These results once again demonstrate the robustness of IRAP responses and shows the lack of control the participant has over their responses (McKenna, Barnes-Holmes, Barnes-Holmes, & Stewart, 2007). This process of investigating the impact of multiple variables on the IRAP is of extreme importance to improve the procedure and to validate the IRAP as a measure of implicit attitudes (Pothos, 2005). Previous technical research surrounding the IRAP has led to notable changes in its procedure, such as the decrease in the minimum response time from 300ms to 200ms (Barnes-Holmes, Murphy, Barnes-Holmes, & Stewart, 2010). Alongside the changes in screen presentation and minimum response latencies, it has been found that asking participants to respond according to a rule before each block can help to reduce attrition rates (Barnes-Holmes et al., 2010). During the IRAP trials, the order in which participants are exposed to the rules (consistent or inconsistent first) is typically reversed, however rule order is not typically manipulated. This research seeks to fill this gap in the research. To achieve this aim participants will be split into two groups where both order presentation (inconsistent versus consistent) and the rule order (self-first versus others-first) will be changed between groups. One group will be exposed to the inconsistent trials first and with the rules reversed so that the rule is “Others are good and I am bad”, while on the other IRAP the first rule will be consistent with their current beliefs and will begin with the rule “I am good and others are bad”. Our hypothesis predicts that there will be a significant difference between the groups with the reversal of the anti-other effect in the others-first group.
- 11. 7 Method Participants This experiment consisted of 41 undergraduate students attending Maynooth University (21 females and 20 males) aged between 18 and 26 (M=19.61, SD=2.02). The participants were selected using random convenience sampling. The groups did not differ significantly with regard to age. All participants were briefed on the nature of the experiment and gave their written consent before taking part in any aspect of the research. Setting The current study took place in the experimental lab in the Department of Psychology at Maynooth University. Each participant completed the experiment individually in a separate cubicle containing no distractions. The experimenter was only present during the instructional phase and practice blocks of the IRAP and vacated the room while the participants were completing the test blocks and questionnaire measures. Apparatus and Materials All aspects of the experiment (including questionnaires) were completed on a DELL desktop computer with a Pentium 4 processor with a 15” screen and a standard QWERTY keyboard. The IRAP procedure was delivered via a programme called PsychoPy 1.81.03
- 12. 8 which controlled all aspects of stimulus presentation and the recording of participant responses. For the purposes of statistical analysis a combination of SPSS 2.0, Microsoft Excel 2007 and StatView were used. Measurement Depression Anxiety Stress Scales. The Depression Anxiety Stress Scales 21 (DASS 21) consists of three different self-report scales which are used to evaluate the three negative emotional states of depression, anxiety and stress. The scale consists of three main sections pertaining to each of the emotions separately, with each of these sections containing 7 items. Subjects are asked to indicate the severity/frequency to which they have experienced the negative symptoms of these emotions over the past week. To calculate the scores for each of the three emotional states the researcher simply adds the scores for the relevant items. The DASS has been shown to have high internal consistency in a variety of settings and with numerous populations and has been repeatedly validated and shown to be reliable (Brown, Chorpita, Korotitsch & Barlowm 1997). Rosenberg Self-Esteem Scale. The Rosenberg Self-Esteem Scale (RSES) scale is a 10 item questionnaire used to measure self-esteem. Participants are asked to indicate on a four point scale the extent to which they agree or disagree with a statement. An individual’s score is calculated by totalling the score of each item after the negatively worded items have been reversed. The RSES has been shown to be extremely reliable with a test-retest reliability correlation of 0.85. This scale has also shown strong concurrent, construct and predicative validity suggesting that it is a useful explicit measure of self-esteem (Robins, Hendin, & Trzesniewski, 2001). IRAP Self-esteemmeasures. Each IRAP trial presented one of two sample sentence stimuli (“I AM” and “OTHER PEOPLE ARE”) which appeared on the top of the computer
- 13. 9 screen. This was accompanied with one of the 6 positive or 6 negative target stimuli(see Table 1 for stimulus arrangements). At the bottom of the screen were the two response options, TRUE and FALSE. To select TRUE the participant pressed the “d” key and to select FALSE the participant pressed the “k” key. Table 1. The response combinations deemed consistent in the self-esteem IRAP Sample 1 Positive Targets Sample 2 Negative Targets Sample 1 Negative Targets Sample 2 Positive Targets Loyal Manipulative Manipulative Loyal Trustworthy Dishonest Dishonest Trustworthy I am Kind Others are Cruel I am Cruel Others are Kind Moral Horrible Horrible Moral Generous Selfish Selfish Generous Friendly Heartless Heartless Friendly Response Option 1 Response Option 2 True False During the IRAP participants would see one of the sample stimuli (“I AM” or “OTHERS ARE”) together with a target stimulus such as “LOYAL”. The participant would then be required to respond either “TRUE” or “FALSE” based on the rule they were given at the start of the block of trials. During the consistent trials participants will be instructed to respond as if “I am good and others are bad” or “Others are bad and I am good”, depending on the group they have been randomly assigned to. In order to respond correctly (i.e., according to the rule) participants must select the “TRUE” response whenever the sample stimulus “I AM” is paired with a positive target and select the “FALSE” response whenever the sample stimulus “I AM” is paired with a negative target (See figure 1 for trial types).The converse is deemed correct when the sample stimulus “OTHERS ARE” is paired with a positive or negative target stimulus. During the inconsistent trials, responses that expressed
- 14. 10 positivity towards the self were deemed to be “FALSE”, whilst target stimuli paired with positivity towards others should emit the “TRUE” response. The main assumption behind the IRAP is that participants perform better on trials that are consistent with how they typically relate the relevant stimuli. For example, if an individual generally responded quicker to “I AM” and “FRIENDLY” with “TRUE” rather than “FALSE” they would be said to have a self-is-positive implicit bias. . Figure 1: Examples of the four trial types employed in the self-esteem IRAP. Building on previous IRAP research findings and recommendations (see Drake et al., 2010 for a review) a maximum of four practice blocks had to be completed by the participant before they progressed to the test blocks. Once the participant reached the mastery criteria (80% accuracy and 2000ms response latency) they progressed onto the six tests blocks. In I Am Loyal Select “d” Select “f” For True For False Consistent Inconsistent Others Are Manipulative Select “d” Select “f” For True For False Consistent I Am Manipulative Select “d” Select “f” For True For False InconsistentConsistent Others Are Loyal Select “d” Select “f” For True For False Inconsistent Consistent Inconsistent Inconsistent Consistent
- 15. 11 line with previous research (Campbell, Barnes-Holmes, Barnes-Holmes & Stewart, 2011) surrounding screen presentation IRAP effects, response options were fixed so participants always pressed “d” for true and “f” for false. The format of all blocks during the IRAP trials (both practice and test) were identical, with all stimuli appearing simultaneously. If the participant responded correctly (i.e., in line with the rule presented at the start of the trial), the response was followed by a blank screen for 400ms which was followed by the presentation of the next pairing task. If the response was not in accordance with the current rule a red “X” would appear on the screen and the participant could not move onto the next trial until the correct response had been recorded. During the trials latency feedback was also presented to the individual. If the participant went beyond their allocated 2,000ms response latency a red exclamation mark would appear at the bottom of the screen, informing them that they were required to respond quicker. Upon completing the 24 IRAP trials the participant was presented with two types of feedback for the previous block: the percentage of correct responses and the median response latency. Once the participant pressed the spacebar to continue onto the next test block a screen appeared displaying the rule for the next set of blocks. The IRAP alternated between consistent and inconsistent blocks, meaning that participants were required to switch their response patterns after each block. As this study aimed to investigate rule order manipulation during the IRAP, participants were randomly assigned to one of two IRAPs with rules presented in the following way: IRAP 1 1. “I am good and others are bad” 2. “I am bad and others are good” IRAP 2
- 16. 12 1. “Others are good and I am bad” 2. “Others are bad and I am good” In this way half of the participants were exposed to consistent trial first and the other half began with inconsistent. The IRAPs differed regarding both the order of the rules (self-first or others-first) and in the presentation of the consistent or inconsistent rule first. The end of the 6th block marked the end of the IRAP task and a screen displayed the message “This task has been completed. Please contact the researcher”. Procedure The procedure for the experiment was identical for participants assigned to either IRAP. When the participants arrived they were briefed on the nature of the experiment after which they read and signed a consent form (See Appendix A). Each participant was subsequently assigned an identification number which served to protect their anonymity. Once the participant was satisfied to commence the experiment the IRAP was explained to them. The experimenter emphasised to the participant that the task did not involve the participant giving their own opinion but it was asking for the participant to respond to a given rule. It was also made clear that it was crucial to respond accurately to the task and that speed would come with practice. Once the experimenter was satisfied that the participant understood the nature of the task the IRAP began. The experimenter interacted directly with the participant during the practice blocks and drew their attention to their feedback so they would achieve the mastery criteria to move onto the test blocks. The experimenter left the room once the participant had progressed to the test blocks. Once the IRAP was complete the participants completed two online questionnaires; the Depression Anxiety Stress Scales (DASS; See Appendix B) and the Rosenberg Self- Esteem Scale (RSES; See Appendix C). The experimenter left the room while the
- 17. 13 questionnaires were being completed for the privacy of the participant. Once the questionnaires had been completed the participants were debriefed and thanked for their participation. They were told to contact the experimenter if they had any questions or problems regarding the study. Ethical Considerations This research was deemed to be ethical by the Ethics Department of Maynooth University. All participants who took part in the current research were over the ages of 18 and gave their informed consent after being briefed by the researcher on exactly what would be expected of them during the experiment. Participants were informed that if they wished to withdraw their data they could do so. In order to provide anonymity to the participants each person was randomly allocated an identification number. Upon the completion of the experiment participants were fully debriefed and given contact information if they required more information about the study.
- 18. 14 Results The main dependent variable during an experiment that uses an IRAP is the response latency, which is a measurement of the time taken between the onset of a trial and the length of time taken for a correct response to be emitted. Although the accuracy of the participants is also recorded, this information is used only in order to identify participants who display a low accuracy score so that they can be emitted from the data analysis. If the participants did not meet the criteria of 80% accuracy in under 2000ms they were excluded from analysis. These response latencies were then converted into D-IRAP scores using an adaption of the D- algorithm (Barnes-Holmes, Barnes-Holmes, Stewart, I., & Boles, 2011). The data from participants’ practice blocks were not included in the practice blocks. A repeated measures ANOVA was carried out to compare the D-scores of each of the four trial types. The participants were split into two groups depending on the IRAP they were exposed to (self-rule first or other-rule first). IRAP type was defined as the between participant variable. There was no significant main effect for group on D-IRAP scores (F (1, 114)= 0.52, p= 0.8211). There was also no significant interaction effect found (F (3, 114)= 0.917, p=0.4350). Figure 2 shows the inverted means for each trial type across both conditions. From the graph it can be seen that there was a positive bias for the self positive trial for participants who were exposed to the self rule first, with all other trials in this condition reporting a
- 19. 15 negative bias towards the self and others. In contrast to this the others-first group demonstrated a positive bias towards both self and others across all trial types. Figure 2. Graph showing the inverted means and response bias direction for each of the trial types. In total eight one-sample t-test were conducted to establish if the D-IRAP scores for any of the individual trials types were significant from 0. From the participants who were exposed to the self-rule first, only the self positive trial reached statistical significance ( t (19)= 4.447 p=0.003). For the participants who were presented with the other-rule first, it was found that the only trail type that was significantly different from zero was also the self positive trial (t (19) = 4.074 p= 0.006), which mirrored the results found in the self-first group. -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 Self Positive Self Negative Others Positive Others Negative Mean Trial Type Self Rule 1st Other Rule 1st Positive Bias Negative Bias
- 20. 16 Twelve dependent t-tests were then conducted to see if trial types were significantly different from each other in each of the groups. Statistically significant differences were found in the self-rule first group between the self positive and self negative (t(19)= 4.111, p=0.006), self positive and others positive (t(19)= 3.379, p=0.0031) and the self positive and others negative (t(19)= 2.581, p=0.0183). Similarly to the self-rule first group, the other-rule first group also displayed statistically significant differences between the self positive and self negative (t(19)= 2.809, p=0.0112), the self positive and other positive (t(19)= 3.572, p=0.002) and the self positive and others negative trial types (t (19)= 2.997 p= 0.0074). Following on from this analysis, four independent t-tests were conducted to see if there were a difference between the groups on each trial type. None of the t-tests reached statistical significance. Although these differences did not reach statistical significance, it must be noted that there does exist a distinct difference between the groups (See Figure 2). These differences can be seen on the self positive trial type with the other-rule first demonstrating a stronger self-positive bias (M=0.440, SD=0.484) than the self-rule first group (M=0.329, SD= 0.330). A notable difference was also found between the groups on the others positive trial, with the others-rule first showing a positive bias (M=0.147, SD=0.458) and the self-first trial showing a negative bias (M=-0.004, SD=0.382). These differences did not reach statistical significance, which means that caution must be exercised when interpreting these results, however these differences must be noted. To investigate the relationships between the implicit measures (IRAP) and the explicit measures (DASS and RSES) a correlation matrix was produced. There was only one relationship that showed a moderate correlation between variables. This was the relationship between the stress score of the participants and the results of the others positive trial (r=-
- 21. 17 0.339). The relationship displayed a moderate negative relationship with higher D-IRAP scores on this trial being associated with low levels of stress. Discussion From the current research it was found that the only significant differences lay within comparison of the trial types within the groups. The differences between the groups in these t-tests is similar to other self-esteem research and highlights the strong self-positive bias that has been found time and time again in implicit research (Reume et al., 2013; Vahey et al., 2009). Although these were the only tests that reached statistical significance, differences between the groups were also noted. The most substantial differences were found between the self-positive and others-positive trial types with the others-first group showing a stronger self-positive and others-positive bias. This would suggest that there is a slight reversal of the anti-others bias in the others-first group which was mediated by the manipulation of the order of the rules. The correlation matrix did not show any significant correlations, with only one moderate correlation being found between stress and the others-positive trial type. This finding highlights the potential differences between explicit and implicit measures of self- esteem. The present research was conducted to investigate the influence of rule order on implicit responding in the IRAP. In previous studies it has been found that small alterations to screen presentations (Campbell et al., 2011) can influence the IRAP effect and it would hold that as rules govern responding, the order in which they are presented would have a major impact on implicit attitudes. Although the differences between the groups did not differ
- 22. 18 significantly, it is clear that rule order has an effect on the results of experiment. For both of the self-positive trials there was a considerable positive bias seen in both groups which is mirrored in all previous IRAP research concerning self-esteem (Remue, et al., 2013; Vahey et al., 2009). This trend is not surprising due to how commonplace it has become in implicit research concerning self-esteem (Franck, De Raedt, Dereu, Van den Abbelle, 2007; De Raedt, Schacht, Franck, & DeHouwer, 2006; ),what is most interesting in the current research is the difference that exists between the groups on the others positive trial (See figure 2). This difference did not reach statistical significance so caution must be exercised when interpreting results, but with this in mind it is hard to deny a clear difference does exist. The lack of significance could be mediated by the small size of the sample (n=41) and it would not be reckless to assume that if the population size was to increase the two means would be pulled further apart making the statistic significant. Our cautious optimism about future results is reinforced by previous research surrounding the malleability of implicit attitudes (Cullen, Barnes-Holmes, & Barnes-Holmes, 2009). As in previous research, we also found that implicit attitudes as measured by the IRAP are susceptible to modification depending on variables or interventions altered by the researcher (Hussey & Barnes-Holmes, 2012). Cullen and colleagues (2009) conducted an IRAP study to investigate the malleability of ageist attitudes. In this research two experiments were conducted, in the first experiment participants were asked to complete an IRAP where they were asked to respond to stimuli as similar or opposite to young and old people. However, in this experiment the experimenters purposefully used negative words that are typically associated with old people (for example, tired, and weary) and positive words (for example, energetic, and enthusiastic) that are associated with young people. In this way they primed participants to associate the negative stimuli with old people. This priming effect was seen in the results which found that participants were faster on consistent trials (Similar-
- 23. 19 Positive-Young People) compared to inconsistent trials (Similar-Positive-Old People). As in our experiment it was found that the results of the IRAP are malleable and are heavily influenced by the wording of the IRAP. This highlights the importance of perfecting the IRAP procedure before the research is applied in clinical settings or before forming theories of psychopathological conditions such as depression. The use of stereotypical words during the IRAP trials provided a reference point for participants which arose from their history of responding and the surrounding context. The above research contained a second experiment which investigated the direct influence of context on implicit responding. To achieve this, participants were split into a pro-old or a pro- young group. In the pro-old group the participants were presented with a booklet containing images of admired old people and disliked young people, a booklet containing images of admired young people and disliked old people was also presented to the pro-young group. The subjects then completed an IRAP that was identical to the first experiment (still containing the stereotypical words). The researchers found that although the results did not reach statistical significance, there was a difference between the two groups after the exemplar training, particularly in the pro-old group who showed a more positive older people bias after the intervention (Cullen et al., 2009). The researchers stated this reversal in the overall IRAP effect in the pro-old condition was not due to the creation of an anti-young bias, but a reversal in the anti-old effect. From the above study it is clear that placing a stimulus, in this case an old person, in context can change responding in the IRAP. These results strongly resonate with the current research. As in our experiment the priming of a pro-others attitude through the presentation of the others-first rule reversed the anti-others bias while simultaneously increasing the self-positive bias. The malleability of relational responses within the IRAP procedure has also been seen in other experiments. One particular experiment investigated the effect of a mood
- 24. 20 induction procedure performance of depressed and non-depressed individuals in IRAP trials (Hussey & Barnes-Holmes, 2012). During the experiment participants depressive symptoms were measured using the Depression Anxiety and Stress Scale (DASS) and they were then split into groups of high and normal levels of depressive symptoms. The researchers found that after the mood induction procedure, only the high depression group showed a decrease in the positivity of their emotional biases (Hussey & Barnes-Holmes, 2012). The experimenters postulate that the results of this experiment demonstrate that mildly depressed individuals are increasingly more susceptible to changes caused by mood state than non-depressed counterparts and have postulated that this could be at the root of depression. Although the IRAPs have been found to have strong predictive effects (Steinberg, Karpinski & Alloy, 2007; Carpenter, Martinez, Vadham, Barnes-Holmes, & Nunes, 2012; Nocks et al., 2010), it has also been found that the results of the experiments are extremely malleable and can be influenced by many different variables (Campbell et al., 2011). Making slight changes in what is presented on screen can increase or decrease the IRAP effect, even in relation to stimuli which should have been consistently categorised as negative (abuse) or positive (peace) in an individual’s responding history (Campbell et al., 2011). Just as screen presentation influenced the IRAP effect, so too did the rule order in our current research (albeit not significantly), once again demonstrating the malleability of implicit responding. Self-esteem is a concept that is thought to be well-established due to the multiple exemplars that an individual would have come across in their responding history and the impact of altering rule-order on implicit self-esteem is stark. In previous studies it was found that depressed individuals have been found to have a larger difference between self-esteem on implicit actual and ideal self-esteem when compared to normal controls and this was hypothesised to be one of the catalysts for their depressive symptoms (Reume et al., 2013). Although this hypothesis could be true, the research on implicit is still in its youth and we
- 25. 21 must be very careful before making any grand claims about its utility. From this small piece of research it is clear that the effects of the IRAP can be manipulated by the smallest changes in the procedure. Before grand conclusions are drawn we must polish, perfect and eliminate variables (such as rule order effects) which interfere with the results of the IRAP. The robustness of IRAP responses is clear from the above evidence. Through direct methods such as including some kind of intervention during the procedure, or through indirect measures such as changing screen presentation or rule order, implicit attitudes are extremely sensitive to minor changes (Campbell et al., 2011). The research becomes even more interesting when you consider experiments that have found that the IRAP cannot be readily faked. McKenna and colleagues (2007) found that when participants are asked explicitly to slow down on consistent trials in order to fake the IRAP their results showed no evidence of faking. Finding that implicit attitudes are more malleable after a small manipulation of the rule order or screen presentation than when participants are explicitly asked to alter responding, demonstrates the robust nature of IRAP effects and their immunity to change as a result of the participant’s manipulations (Vahey et al., 2009). The necessity of future research in the area of rule order manipulation has been highlighted from the current research. Before implicit research delves into the treatment of psychopathology, particularly depression, it must concern itself with the impact of rule order on arbitrary and experimentally controlled relations. This could be achieved by conducting IRAP research using stimuli such as shapes or nonsense syllables and investigating the effect of the manipulation of rule order on these stimuli. Perfecting this method of research is of the utmost importance if the research is to be applied in clinical settings. Although this current research could act as a catalyst for future research, it is not without limitations. Firstly, the study contained a small sample size of 41 and cannot be readily generalised to the general population. This experiment, although demonstrating a difference, did not reach statistical
- 26. 22 significance and thus the results must be treated cautiously. It would also be conducive to have both the groups start on a consistent or inconsistent trial types to ensure that this was not a confounding factor in the research. However these limitations can be easily overcome by increasing the sample size and altering procedures which should lead to the differences between the groups being drawn out and reaching statistical significance. Research in this field is important for both the validity of the IRAP and possibly for its application in clinical settings. In conclusion, although the differences between the groups did not differ significantly there is enough evidence to suggest that further research in this area is warranted. The manipulation of rule order could significantly influence implicit responding and in this way it is extremely important that future research addresses this question in order to expose participants to the most reliable and valid measure of implicit attitudes. As IRAP research has been shown to have a predictive quality in matters such as suicide attempts (Nocks et al., 2010) the importance of weeding out technical issues, such as rule order, cannot be overemphasised.
- 27. 23 References Barnes-Holmes, Y., Barnes-Holmes, D., & Roche, B. (2001). Exemplar training and a derived transformation of function in accordance with symmetry: II. The Psychological Record, 51, 589-603. Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2011). A sketch of the implicit relational assessment procedure (IRAP) and the relational elaboration and coherence (REC) model. The Psychological Record, 60, 527-542. Barnes-Holmes, D., Hayden, E., Barnes-Holmes, Y., & Stewart, I. (2008). The Implicit Relational Assessment Procedure (IRAP) as a response-time and event-related potentials methodology for testing natural verbal relations: A preliminary study. The Psychological Record, 58, 497-515. Barnes-Holmes, D., Murphy, A., Barnes-Holmes, Y., & Stewart, I. (2010). The Implicit Relational Assessment Procedure (IRAP): Exploring the impact of private versus public contexts and the response latency criterion on pro-white and anti-black stereotyping among whit Irish individuals. The Psychological Record, 60, 57-66. Brown, T. A., Chorpita, B. F., Korotitsch, W., & Barlow, D. H. (1997). Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples. Behaviour Research and Therapy, 35(1), 79-89.
- 28. 24 Campbell, C., Barnes-Holmes, Y., Barnes-Holmes, D., & Stewart, I. (2011). Exploring screen presentations in the Implicit Relational Assessment Procedure (IRAP). International Journal of Psychology and Psychological Therapy, 11(3), 377-388. Catania, A. C. (1999). Learning, 4th Edition. Upper Saddle River, New Jersey: Prentice-Hall. Carpenter, K. M., Martinez, D., Vadham, N. L., Barnes-Holmes, D., & Nunes, V. E. (2012). Performance based measures of attentional bias and relational responding are associated with behavioural treatment outcome for cocaine dependence. The American Journal of Drug and Alcohol Abuse, 38, 146-154. Cullen, C., Barnes-Holmes, D., Barnes-Holmes, Y. (2009). The implicit relational assessment procedure (IRAP) and the malleability of ageist attitudes. The Psychological Record, 59, 591-620. De Raedt, Schacht, R., Franck, E., & De Houwer, J. (2006). Self-esteem and depression revisited: Implicit positive self-esteem in depressed patients. Behaviour Research and Therapy, 44(7), 1017-1028. Deroost, N., & Saetens, E. (2006). Spatial processing and perceptual sequence learning in SRT tasks. Experimental Psychology, 53, 16-30. Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1998). Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. Journal of Personality and Social Psychology, 57, 77-85. Drake, C. E., Kellum, K. K., Wilson, K. G., Luoma, J. B., Weinstein, J. H., & Adams, C., H. (2010). Examining the implicit relational assessment procedure: Four preliminary studies. Psychological Record, 60(1), 81-100. Ferguson, M. J., & Bargh, J. A. (2007). Beyond the attitude object: automatic attitudes spring from object-centred-contexts. In B. Wittenbrink & N. Schwarz (Eds.), Implicit
- 29. 25 Measures of Attitudes. Guilford: New York pp.216-246. Franck, E., De Raedt, R., Dereu, M., Van den Abbeele, D. (2007). Implicit and explicit self- esteem in currently depressed individuals with and without suicidal ideation. Journal of Behaviour Therapy and Experimental Psychiatry, 38, 75-85. Golijani-Moghaddam, N., Hart, A., & Dawson, D. L. (2013). The Implicit Relational Assessment Procedure: Emerging reliability and validity data. Journal of Contextual Behavioural Science, 2(3), 105-119. Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem and stereotypes. Psychological Review, 102, 4-27. Greenwald, A. G., Banaji, M. R., Rudmanm L. A., Farnham, S. D., Nosek, B. A., & Mellot, D. S. (2002). A unified theory of implicit attitudes, stereotypes, self-esteem and self- concept. Psychological Review, 109, 2-25. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74(6), 1464. Gremar, M. C., Segal, Z. V., Sagrati, S., & Kennedy, S. J. (2001). Mood-induced changes on the implicit association test in recovered depressed patients. Journal of Abnormal Psychology, 110, 282-289. Gullone, E., Jones, T., & Cummins, R. (2000). Coping styles and prison experiences as predictors of prychological well-being in male prisoners. Psychiatry, Psychology and Law, 7, 170-181. Hagga, D., Dyck, M., & Ernst, D. (1991). Empirical status of cognitive therapy of depression. Psychological Bulletin, 110, 215-236.
- 30. 26 Hayes, S. C., Barnes-Holmes, D., Roche, B. (2001). Relational Frame Theory : A Post Skinnerian Account of Human Language and Cognition. Hingham, MA: Kluwer Academic Pulishers. Healy, O., Barnes-Holmes, D., & Smeets, P. (2000). Derived generalised responding as a generalised operant behaviour. Journal of Experimental Analysis of Behaviour, 74, 207-227. Hooper, N., Villatte, M., Neofotistou & McHugh, L. (2010). The effects of mindfulness versus thought suppression on implicit and explicit measures of experiential avoidance. International Journal of Behavioural Consultation and Therapy, 6(3), 233-244. Hughes, S., Barnes-Holmes, D., & Vahey, N. (2012). Holding on to our functional roots when exploring new intellectual islands: A voyage through implicit cognition research. Journal of Contextual Behavioural Science, 1, 17-30. Hussey, I., & Barnes-Holmes, D. (2012). Implicit Relational Assessment Procedure as a measure of implicit depression and the role of psychological flexibility. Cognitive and Behavioural Practice, doi:10.1016/j.chpra.2012.03.0202. Ingram, R. E., Miranda, J., & Segal, Z. V. (1998). Cognitive Vulnerability to Depression. New York: Guilford Press. Ingram, R. E., & Wisnicki, K. (1991). Cognition and depression. In P. A. Margo (Ed.), Annual Review of Psychopathology, Vol. 1 (pp.187-230). Luciano, C., Valdivia, S., Cabello, F., & Hernandez, M. (2004). Developing self-directed rules. In R. Rehlfeld & Y. Barnes-Holmes (Eds). Applying Relational Frame Theory to the Language Disabled. Oakland, CA: New Harbinger.
- 31. 27 McKenna, I., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2007). Testing the fake- ability of the Implicit Relational Assessment Procedure (IRAP): The first study. International Journal of Psychology and Psychological Therapy, 7(2), 253-268. Nock, M. K., Park, J. M., Finn, C. T., Deliberto, T. L., Dour, H. J., & Banaji, M. R. (2010). Measuring the suicidal mind: Implicit cognition predicts suicidal behaviour. Psychological Science, 21, 511-517. O’Hora, D., Barnes-Holmes, D., Roche, B., & Smeets, P. M. (2004). Derived relational networks and control by novel instructions: A possible model of generative verbal responding. The Psychological Record, 54, 437-460. Oser, C. B. (2006). The criminal offending-self-esteem nexus: Which version of the self- esteem theory is supported? The Prison Journal, 86, 344-460. Parris, B. A., Sharma, D., & Weekes, D. (2007). An optimal viewing position effect in the Stroop task when only one letter is the colour carrier. Experimental Psychology, 54, 273-280. Pinter, B., & Greenwald, A. G. (2005). Clarifying the role of the “other” category in the self- esteem IAT. Experimental Psychology, 52(1), 74-79. Pothos, E. M. (2005). Expectations about stimulus structure in implicit learning. Memory and Cognition, 33, 171-181. Remue, J., De Houwer, J., Barnes-Holmes, D., Vanderhasselt, A., & De Raedt, R. (2013). Self-esteem revisited: Performance on implicit relational assessment procedure as a measure of self-versus ideal self-related cognitions in dysphoria. Cognition and Emotion, 27(8), 1441-1449.
- 32. 28 Roberts, J. E., Gotlib, I. H., & Kassel, J. D. (1996). Adult attachment security and symptoms of depression: the mediating roles of dysfunctional attitudes and low self-esteem. Journal of personality and social psychology, 70(2), 310-320. Robins, R. W., Hendin, H. M., & Trezeniewski, K. H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 27(2), 151-161. Torneke, N., Luciano, C., & Salas, S. V. (2008). Rule-governed behaviour and psychological problems. International Journal of Psychology and Psychological Therapy, 8, 141- 156. Steinberg, J. A., Karpinski, A., & Alloy,L. B. (2007). The exploration of implicit aspects of self-esteem in vulnerability- Stress models of depression. Self and Identity, 6, 101- 117. Vahey, N. A., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2009). A first test of the implicit relational assessment procedure as a measure of self-esteem: Irish prisoner groups and university students. The Psychological Record, 59, 371-388. Williams, J. M. G. (1997). Depression. In D. M Clark, & C. G. Fairburn (Eds.), Science and Practice of Cognitive Behaviour Therapy (pp.259-283). Oxford: Oxford University Press. Young, J. E. (1994). Cognitive Therapy for Personality Disorders: A Schema-Focused Approach. Sarasota: Professional Resource Press. Zentner, M., & Renaud, O. (2007). Origins of adolescents’ ideal self: An intergenerational perspective. Journal of Personality and Social Psychology, 92(3), 557-574.
- 33. Appendices Appendix A: Consent Form INFORMED CONSENT FORM FOR STUDENT RESEARCH In agreeing to participate in this research I understand the following: This research is being conducted by Sarah Kenehan, an undergraduate student at the Department of Psychology, National University of Ireland, Maynooth. The method proposed for this research project has been approved by the Departmental Ethics Committee, which means that the Committee does not have concerns about the procedure itself as detailed by the student. It is, however, the above-named student’s responsibility to adhere to ethical guidelines in their dealings with participants and the collection and handling of data. If I have any concerns about participation I understand that I may refuse to participate or withdraw at any stage. I have been informed as to the general nature of the study and agree voluntarily to participate. I will complete a number of questionnaires, some of which will ask about symptoms of psychological suffering, as well as a number of computer based pairing tasks where I will be asked to pairs words related to self esteem . There are no known expected risks associated with any aspect of participation. All data from the study will be treated confidentially. The data from all participants will be irrevocably anonymised, compiled, analysed, and submitted in a report to the Psychology Department. The data will be retained for approximately 5 years before being destroyed. No participant’s data will be identified by name at any stage of the data analysis or in the final report. At the conclusion of my participation, any questions or concerns I have will be fully addressed. I may withdraw from this study at any time without giving a reason, and may withdraw my data at the conclusion of my participation if I still have concerns.
- 34. Participant’s signature Researcher’s signature Participant’S name (PLEASE PRINT) Date Should you be in distress experiencing any form of mental health complaint we encourage you to contact the NUIM student counseling service. This professional, free, and confidential counseling service can be contacted on 01-7083554 or counselling.nuim.ie to schedule an appointment.
- 35. Appendix B: Rosenberg Self-Esteem Scale (RSES) RSES Belowisa listof statementsdealingwithyourgeneral feelingsaboutyourself. Please rate how much youagree with each statementbycirclinganumbernexttoit. Use the scale below tomake your choice. Strongly Agree Agree Disagree Strongly Disagree 1 2 3 4 1. I feel thatI am a personof worth,at leaston an equal plane withothers. 1 2 3 4 2. I feel thatI have a numberof good qualities. 1 2 3 4 3. All inall,I am inclinedtofeel thatIam a failure. 1 2 3 4 4. I am able to do thingsaswell asmost otherpeople. 1 2 3 4 5. I feel Ido nothave much to be proudof. 1 2 3 4 6. I take a positive attitude towardmyself. 1 2 3 4 7. On the whole,Iam satisfiedwithmyself. 1 2 3 4 8. I wishI couldhave more respectformyself. 1 2 3 4 9. I certainlyfeel uselessattimes. 1 2 3 4 10. At timesIthinkI am no goodat all. 1 2 3 4
- 36. Appendix C: DepressionAnxiety Stress Scales 21 (DASS)