Action Effect Negativity
- 1. Biological Psychology 82 (2009) 211–218 Contents lists available at ScienceDirect Biological Psychology journal homepage: www.elsevier.com/locate/biopsycho Action-effect negativity: Irrelevant action effects are monitored like relevant feedback Guido P.H. Band a,b,*, Henk van Steenbergen a,b, K. Richard Ridderinkhof c, Michael Falkenstein d, Bernhard Hommel a,b a Leiden Institute for Brain and Cognition, Wassenaarseweg 52, 2333 AK Leiden, The Netherlands b Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333 AK Leiden, The Netherlands c Amsterdam Center for the Study of Adaptive Control in Brain and Behavior (Acacia), Amsterdam, The Netherlands d Leibniz Research Centre for Working Environment and Human Factors (IfADo), University of Dortmund, Dortmund, Germany A R T I C L E I N F O A B S T R A C T Article history: Goal-directed action presupposes the previous integration of actions and their perceptual consequences Received 22 May 2008 (action-effect binding). One function of action-effect bindings is to select actions by anticipating their Accepted 28 June 2009 consequences. Another, not yet well understood function is the prediction of action-contingent feedback. Available online 7 August 2009 We used a probabilistic learning task and ERP analyses to compare the processing of explicit, performance-related feedback with the processing of task-irrelevant response-contingent stimuli. Keywords: Replicating earlier findings, we found that negative performance feedback produced a feedback-related Performance monitoring negativity (NFB), presumably related to response outcome evaluation. Interestingly, low-probability but Error-related negativity NE/ERN task-irrelevant action effects elicited a signal similar to the NFB, even though it had a shorter duration. N2 Response delays on trials following negative feedback and following low-probability action effects were Ideomotor theory correlated with one another. These observations suggest that automatically acquired action-effect Reinforcement learning relations are exploited for anticipating upcoming events. Like task-relevant performance feedback, task- irrelevant action effects serve as a basis for action monitoring processes, presumably mediated by medial frontal cortex. ß 2009 Elsevier B.V. All rights reserved. 1. Introduction co-occurrence, presumably through some kind of Hebbian learning process (Elsner and Hommel, 2001, 2004). As a consequence, Human action is goal-directed and thus likely to be driven by neural representations of actions and effects become associated in anticipations of the intended action effects (e.g., rewards or other a bidirectional fashion, so that activation between representations incentives that are worth the effort). Anticipated action effects can can spread in two directions. For one, this allows for the intentional be assumed to subserve at least two important processes in action selection of actions as envisioned by ideomotor theory (Hommel control: to allow for the selection of actions that are suitable to et al., 2001; James, 1890; Lotze, 1852): ‘‘thinking of’’ the intended generate the consequences the agent is interested in (action action consequences (i.e., endogenously activating the codes of the selection: e.g., Elsner and Hommel, 2001) and to represent the intended action effect) primes and selects the associated action. In intended outcome of an action against which the actual outcome fact, neuroimaging studies have shown that presenting a familiar can be compared (action monitoring/evaluation: e.g., Blakemore action effect activates brain areas associated with episodic et al., 1999). Adaptive action control thus requires the acquisition memory and response selection (Elsner et al., 2002; Melcher of knowledge about the relationship between actions and their et al., 2008). For another, bidirectional action-effect associations consequences. support action monitoring (Hommel, 1998): activating the representation of a particular action spreads activation to the representations of its effects, which amounts to anticipating or 2. Action-effect integration predicting them (e.g., Blakemore et al., 1999). For example, in experienced piano players, depressing a particular piano key will Actions and their effects are thought to be integrated into what induce the anticipation of the tone by priming the action-tone one may call action concepts (Hommel, 1997) as a result of frequent representation, and this will help to detect errors rapidly. While the role of action-effect bindings in action selection is * Corresponding author at: Leiden University, Institute of Psychology, Wasse- naarseweg 52, 2333 AK Leiden, The Netherlands. widely supported, the role in action monitoring is still largely E-mail address: band@fsw.leidenuniv.nl (Guido P.H. Band). unexplored. If representations of action effects are involved in 0301-0511/$ – see front matter ß 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.biopsycho.2009.06.011
- 2. 212 G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 action monitoring, one would expect that the processing of action- primates found sources of activity scattered over the medio-frontal effect information is similar to the processing of other types of cortex (MFC), with the highest density of estimated sources in the action feedback. This expectation was tested with behavioral rostral cingulate zone (Ridderinkhof et al., 2004). measures and event-related potentials (ERPs). Several studies have recently tried to link the NE/ERN or NFB Note that even though we often contrast ‘‘feedback’’ with with components in the N2 family with a maximum amplitude ‘‘action effects’’ in the present paper, this is not to say that feedback over the fronto-central cortex (e.g., Folstein and Van Petten, 2008; is not an action effect or that action effects cannot provide Nieuwenhuis et al., 2003; Yeung et al., 2004). The control-related feedback. The main distinction we intend to draw is between N2 is sensitive to the presence of conflicting response tendencies information about an action that is relevant for the given task and and the requirement to suppress incorrect responses. Some have the target of optimization processes, and therefore presumably argued that this component reflects the same process as the NE/ attended and explicitly represented (this is what we in the present ERN and NFB, but that the timing varies depending on when the context call ‘‘feedback’’) and other action-contingent information participant finds out whether the response tendency was correct that is not relevant for the task, presumably not considered for (e.g., Nieuwenhuis et al., 2003; Yeung et al., 2004). strategic optimization, and probably not attended or explicitly Alternatively, it has been argued that the NFB may actually be an represented (this is what we call ‘‘action effects’’). oddball N2, a negative waveform that is larger on low-probability than high-probability stimuli (Holroyd, 2004), even if the 3. Action monitoring and feedback processing probability refers to an endogenously anticipated stimulus rather than the actual frequency of a stimulus throughout an experiment 3.1. Feedback-related negativity (Folstein and Van Petten, 2008). Holroyd et al. (2008) performed principle components analysis on the ERPs of an oddball task and a Numerous studies support the idea that the brain has a dedicated task with feedback and found that the ERP difference following mechanism for monitoring performance: people often detect their negative versus positive feedback is not caused by differences in own errors and correct them immediately (e.g., Band and Kok, 2000; the N2, but by differences in an overlapping positive waveform. Rabbitt, 1968; Yeung et al., 2004), and tend to be more cautious in Regardless of the way the NFB can best be classified, the current trials following an error (e.g., Band and Kok, 2000; Rabbitt, 1968; study is aimed at testing whether irrelevant low-probability action Ridderinkhof, 2002; Smith and Brewer, 1995). For a more direct effects, as compared to high-probability action effects, elicit an indication, ERPs synchronized to the onset of an incorrect response action-effect negativity (NAE) with the same properties as the NFB. show a negative deflection relative to correct trials, a component It is hypothesized that a low-probability action effect, because it known as the error negativity (NE; Falkenstein et al., 1990) or error- does not match a learned association, signals that a different action related negativity (ERN; Gehring et al., 1993). The NE/ERN than planned may have been performed. Therefore, it will rapidly immediately follows the response and is usually largest over trigger medio-frontal cortex activity associated with performance fronto-central scalp locations. The functional interpretation of the monitoring, which will be visible in a negative potential difference NE/ERN has developed over the years since its first discovery because compared to a high-probability action effect. other events than overt errors also elicit a component similar to the NE/ERN. A broader interpretation of these waveforms is in terms of 3.2. Feedback-related positivity conflict monitoring, which is assumed to form the basis for detecting both errors and risk of errors (Yeung et al., 2004). Response-feedback does not only induce an NFB but also a The interpretation of the NE/ERN reflecting performance positive deflection relative to correct trials around 300–600 ms monitoring processes is further supported by the observation of a after the feedback, with a centro-parietal maximum. The inter- very similar ERP component after participants receive feedback pretation of this component is still preliminary, but it has been about their performance (Miltner et al., 1997): ERPs go more argued that unexpected feedback is a rare event and evokes a P3b negative over fronto-central midline brain areas, between 200 and (Overbeek et al., 2005). In the current study feedback, action 400 ms following negative as compared to positive feedback, effects, or both may elicit a positive waveform (non-theoretically regardless of whether performance was actually incorrect. This labeled as PFB and PAE). It is hypothesized that low-probability waveform has been dubbed the feedback-related negativity (NFB). action effects, because they are irrelevant for the task at hand and We will adopt this name as a non-theoretical descriptive label for the may even escape awareness, will not evoke a PAE, whereas relevant negativity following feedback. The relationship between NE/ERN negative feedback will evoke a PFB. and NFB has been elucidated by Holroyd and Coles (2002), who varied the validity of performance feedback in a probabilistic 4. Present study feedback study. NE/ERN and NFB were both sensitive to the valence of the eliciting event as well as its information value, i.e., whether the Can low-probability action effects that are task-irrelevant and event clarified whether an error was made. Based on these and other thus unlikely to be attended evoke ERP components that are findings, Holroyd and Coles (2002) presented a unified account for comparable to those reported in studies of intentional feedback the NE/ERN and NFB. They argued that these components reflect the processing? We tested this by recording ERPs for both stimulus transmission of a negative reinforcement signal from the mesence- events in the same experiment. To assess the processing of phalic dopamine system to the ACC, which in turn initiates adaptive negative feedback, we included a standard manipulation that is processes that affect the probability of giving a specific response to a known to produce an NFB—a slightly adjusted version of the stimulus. Accordingly, the system responsible for the NE/ERN and probabilistic learning task used by Holroyd and Coles (2002). Our NFB is involved in reinforcement learning: stimulus–response (S–R) participants had to classify visual stimuli by speeded two-choice combinations leading to unexpected absence of reward lose responses and the correct stimulus–response mapping had to be associative strength, whereas combinations leading to unexpected discovered by using the negative or positive feedback. Positive reward gain associative strength. Consistent with this approach, feedback was associated with a monetary bonus and negative source analyses and imaging studies have indicated that the same feedback was associated with monetary loss. The validity of the part of the ACC is responsible for both the NE/ERN and NFB feedback was fixed at 80%. In line with previous studies, the component (Holroyd et al., 2004; Miltner et al., 1997) and a meta- prediction was that negative, as compared to positive feedback analysis of studies on performance monitoring in humans and other evokes an NFB and possibly a PFB.
- 3. G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 213 The theoretically more interesting manipulation was that each 3.08). Three more participants were excluded from subsequent analyses because they had earned a negative bonus, suggesting that either they did not understand key press produced a task-irrelevant but action-contingent tone— the instructions or did not put enough effort in the task. The experiment was the events that we call ‘‘action effects’’. Numerous ideomotor conducted in accordance with relevant laws and institutional guidelines and was studies have shown that this kind of manipulation is sufficient to approved by the local ethics committee from the Faculty of Social Sciences. All acquire bidirectional associations between the key press and the students read and signed informed consent. tone representations, provided that the delay between the key press and the tone is not too long (about 1 s or less) and the key- 5.2. Stimuli and apparatus tone contingency is nonrandom (e.g., Elsner and Hommel, 2001, Participants sat in front of a monitor in a dimly lit, sound-attenuated room. 2004; Hommel and Elsner, 2000). Visual stimuli were presented on a 19 in. CRT monitor. Auditory stimuli were We used action effects that were entirely irrelevant for the presented through Etymotic1 air pulse ear phones. Responses were recorded with classification of visual stimuli. They consisted of vowel sounds that keys mounted on the armrests below the index and middle finger of both hands. Experimental trials started with the presentation of a fixation cross during were triggered by pressing a particular key. Within each block of 500 ms. Next, a random choice out of two visual stimuli from the Snodgrass trials one of the two responses was followed by one vowel on 80% standardized set (Snodgrass and Vanderwart, 1980) was presented as a black line and by another vowel on 20% of the trials, thus creating one drawing on a white background. The image subtended a visual angle of about 28. dominant action-effect association for this key and making the These imperative stimuli required a speeded response with the index or middle finger of the same hand. If a key press was made within 800 ms, it was followed irrelevant action effect relatively predictable. The other response after 14 ms by an auditory stimulus of 270 ms duration and an intensity of was followed by four possible vowels, occurring with frequencies approximately 70 dB. During response omissions no sound was presented. of 40%, 40%, 10%, and 10%, thus making the action effect of this key Following the auditory stimulus the visual stimulus remained on the screen for virtually unpredictable. The unpredictable conditions served 930 ms after which the fixation cross for the next trial was presented. mainly as filler conditions to allow presenting all action effects The auditory stimulus was a merger of an action effect and a feedback element. The action-effect element was one of four different Japanese vowels /æ/, /i:/, /o/, and with the same probability across the experiment and to keep the /a/ (http://www.ymec.com/hp/signal2/voice5.htm). These action-effect stimuli will overall predictability of action effects at a minimum. The main be referred to as E1, E2, E3, and E4. The contingency of E1–E4 on the use of the four interest was in the contrast between the conditions with high- response buttons is clarified in Fig. 1 for one of the balancing conditions. Of the four probability (80%) and low-probability (20%) action effects. response buttons (R1–R4), R1 presses were unpredictably followed by E1 on 80% and E2 on 20% of the trials. R4 (right middle finger) was mapped in the same way to If our assumption is correct that irrelevant action effects are E3 and E4. To provide an equal distribution of action effects, R2 and R3-presses (left processed the same way as task-relevant, attended feedback, we and right index fingers) were associated with all possible action effects, with E1 and would expect that a low-probability action effect produces an NAE E3 presented in 10% of the trials, and E2 and E4 in 40% of the trials. This procedure comparable to the NFB that can be observed for negative feedback. was chosen so that even in the unlikely situation that participant would figure out Some evidence that acquired action effects can modulate ERP the contingency they would be able to predict the action effect correct on only 60% of the trials. This was meant to contribute to the overall experience that action components has been provided by Waszak and Herwig (2007). Their effects were not predictable, despite the fact that in the 80%/20% conditions they participants acquired associations between left and right key were more predictable. The procedure also ensured that each action effect occurred presses and tones of different pitch before participating in an with the same probability throughout the session. auditory oddball task (with numerous standards and infrequent The feedback element in the auditory stimulus consisted of the presence or absence of white noise, indicating a correct or incorrect response with 80% validity. deviants). Auditory deviants produced a P3 that was more That is, correct responses were followed by positive feedback on 80% and negative pronounced when it was preceded by the response that was feedback in 20% of the trials. After incorrect responses, there was negative feedback associated with the standard, suggesting that action-tone associa- on 80% and positive feedback on 20% of the trials. The assignment was balanced tions affected tone processing. The observation of an NAE in the across participants, and the action effect and feedback elements varied current study would support the idea of a shared mechanism for the independently. The negative and positive feedback corresponded to a loss of two cents or a gain of one cent to their bonus. Omissions led to a loss of four cents. evaluation of both action-relevant and merely incidental conse- Participants began the experiment with a 100 Euro cents bonus. The current bonus quences of an action and would suggest interesting theoretical score was displayed after each block. connections between views on reinforcement learning and on Note that the current design, as most other designs of probabilistic learning studies, ideomotor action control. In contrast, we assume that negative actually confounds the validity, predictability, and valence of feedback. This lies in the logic of learning: valid feedback about a correct response is necessarily positive, and feedback, which in our study needed to be consciously monitored, making this feedback less predictable necessarily affects the frequency of valid and might evoke a PFB. However, because action effects were not relevant positive feedback in the context of correct responses. Given that analyses (of RTs and for the classification task and were not likely to be attended, no PAE ERPs) are typically restricted to trials with correct responses, some degree of confound was predicted to occur in response to low-probability action effects. is unavoidable and we flagged them in the present paper wherever appropriate. Observing an NFB after irrelevant but deviant action effects Interestingly, however, the similarity between the effects of positive versus negative feedback (where the confounds are present) and high-probability versus low- would have theoretical implications, not only for the role of action probability action effects (where they are not) that we will demonstrate in the effects in performance monitoring, but also for the functional following suggests that expectation is actually the most relevant variable. interpretation of the NFB. In particular, current theories of the NFB implicitly assume that the eliciting stimuli are all task relevant (see 5.3. Procedure e.g., Ridderinkhof et al., 2007 for a review). In particular, all models Participants were familiarized with the auditory stimuli in advance. Noisy and of the NE/ERN and NFB consider the role that the current outcome of clear feedback sounds were presented before the start of the experiment, and their an action has in attaining the current goal. If the current study meaning was explained, including the fact that the feedback signal ‘‘was often but would show an NAE on irrelevant action outcomes, this would point not always reliable’’. In contrast, the meaning of the different vowels was not at a more generic role of the MFC in deviance detection, of which explained in advance although participants were instructed to ‘‘look for a pattern between responses and subsequent vowels’’. This instruction was given to draw the response conflict, overt errors, unfavorable feedback, and per- participant’s attention to the vowel sounds at the onset, and thereby trigger at least ceived risk of errors would be the cases already known, and a minimum amount of vowel processing.1 However, the instruction was not possibly also the more salient ones. 1 Although in most studies integration of actions and effects seems to happen 5. Method automatically, that is without attention (Eenshuistra et al., submitted for 5.1. Participants publication; Hommel et al., 2003; Kunde, 2004), there also seems to be a modulation by the salience of the action effect. Dutzi and Hommel (2009) found Eleven female and eight male students from Leiden University participated in the that while integration occurs automatically with task-irrelevant auditory action experiment. The age range was 18–25 years (M = 21) and three participants were effects, for visual task-irrelevant stimuli to be integrated, they need to be salient. left handed. They were paid s 12 or course credit for participating in the 2-h This uncertainty about the automaticity of integration was reason to briefly bring experiment and could earn a monetary bonus up to s 5 for high accuracy (M = s the variability of the vowels to the participants’ attention through an instruction.
- 4. 214 G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 395–710 ms for PFB. An analogous PAE was not observed. Differences in onsets and offsets were tested using the jackknife-based method proposed by Miller et al. (1998). 6. Results 6.1. Behavioral performance For reaction time (RT) analyses, trials with button presses preceding the stimulus, RT > 1000 or an incorrect response were excluded. Mean RT was 403 ms and mean accuracy was 76.1%. Performance to stimuli and responses associated with an 80%/20% versus a 40%/40%/10%/10% action-effect schedule was compared by means of an ANOVA with the factor action-effect schedule. Unsurprisingly, there were neither effects on RT nor on accuracy, Fs < 1, which rules out that the frequency of confirmation of the anticipated action effect as such played a role. Given our interest in Fig. 1. An example of the response–action-effect mapping as used in this the contrast between processes following high-probability versus experiment. R1 and R4 were always associated with the middle fingers of the low-probability action effects under otherwise identical condi- left and right hand. After a particular response button was used, in a random 80% of tions, further analyses were restricted to trials with the 80%/20% the trials this was followed by one particular action-effect sound (e.g., /æ/) while action-effect schedule. infrequently (20%) followed by another sound (e.g., /o/). All four different sounds were associated with buttons R2 and R3 to balance the frequency of different If we assume that registering negative feedback or low- sounds presented to the participant. probability action effects induces conflict between the codes representing the expected and the actual events, it is possible that this affects subsequent trials. To exclude this possibility, we repeated or referred to throughout the rest of the session. In response to informal inquiry with the participants following the experiment, none reported any analyzed only those trials on which both the current and the knowledge of the contingencies. previous response were correct and from the 80%/20% action-effect After instructions, 32 blocks of 40 trials each had to be carried out, interrupted by schedule, which implies that these were all stimulus repetitions. short self-paced breaks. In each block only one hand was used, with left and right Mean RT did not differ between trials following negative and hand use alternating each block. On every block, participants were told which hand positive feedback, F = 1. Then, the data were further filtered to and buttons to use and were instructed to find the responses belonging to each of the two stimuli used through trial and error, making use of the probabilistic include only trials with positive feedback on the previous trial, and feedback. Each block of 40 trials used two new visual stimuli, which were the effect of action-effect history was tested. Mean RT was 8 ms equiprobable. longer after low-probability than after high-probability action effects, F(1,18) = 8.8, MSe = 62, p < .01, h2 ¼ 0:329, which may p 5.4. Electrophysiological recordings indeed point to a conflict between (implicitly) expected and actual Electroencephalographic (EEG) activity was recorded over positions C3, C4, Fz, action effects. Also of interest, the measures of feedback-related FCz, Cz, Pz and Oz of the 10/20 standard. Horizontal eye movements were calculated and action-effect-related slowing (RT following negative feedback by bipolar derivations of electro-oculogram (EOG) signals over the left and right minus RT following positive feedback, and RT following low- outer canthus. Vertical eye movements were calculated by bipolar derivations of probability effects minus RT following high-probability effects) signals above and below the left eye. Monopolar recordings were referenced to the common mode sensor (CMS) and drift was corrected with a driven right leg (DRL) were correlated, r(19) = 0.52, p < .02. electrode (for details see http://www.biosemi.com/faq/cms&drl.htm). In order to Analogous analyses were performed on response accuracy. All re-reference the data off-line, two electrodes were placed at the left and right trial inclusion criteria were the same as for RTs, except for the mastoid. Signals were DC amplified and digitized with a BioSemi ActiveTwo system accuracy of the current response. Accuracy was higher following at a sampling rate of 256 Hz positive than negative feedback (99.0 vs. 84.3%), F(1,18) = 117.9, MSe = 17.5, p < .001, h2 ¼ 0:868. Accuracy did not differ between p 5.5. Analysis responses following low-probability as compared to high-prob- Off-line analyses were performed with Brain Vision Analyzer. After re- ability action effects, F < 1. referencing the channels to the average mastoid, data were smoothed with a high-pass filter of 0.1 Hz and a low-pass filter of 12 Hz (both 24 dB/oct). Correction 6.2. Event-related potentials for ocular artifacts was applied using the standard Gratton et al. (1983) method. EEG and EOG segments with absolute amplitudes exceeding 1000 mV, as well as EEG segments with amplitude changes smaller than 0.50 mV or amplitude changes For the NFB component only correct response trials were used, larger than 100 mV were rejected as artifacts (10% of the trials). EEG data were first comparing between all positive and negative feedback trials. To baselined to the mean amplitude in the 200-ms interval prior to the presentation of investigate ERPs in relation to action effects, the analysis was the imperative stimulus. From these baselined EEG data, feedback-locked segments were taken with a window from 1200 ms before to 1000 ms after the feedback, restricted to responses that were followed by high-probability which were later averaged. (80%) or low-probability action effects (20%). Only those trials were Analyses of the effect of feedback and action effect were performed as analyses of included that had a correct response as well as positive feedback. variance (ANOVA) on the mean amplitude of the NFB, NAE and PFB using the factors Fig. 2 (left panel) shows the grand average ERPs on correct trials, electrode and signal probability. Greenhouse–Geisser correction was applied to the following positive and negative feedback. The negative amplitude dfs, but original dfs are reported. To calculate the onsets and offsets of the NFB and NAE, the first step in the analysis difference following negative as compared to positive feedback was to compute two difference waves; one for low-probability versus high- had a diffuse scalp distribution and was largest over Pz, followed probability action effects, and one for incorrect versus correct feedback. The onsets by Fz, FCz and Cz. On average, the NFB for these four electrodes and offsets in these difference waves were defined as the first and last sample in the exceeded the 3 SD criterion from 173 to 354 ms and peaked at first 500 ms following feedback on which the amplitude was more negative than À3 times the SD of the activity in the window preceding the feedback. This was from 268 ms following the feedback. An ANOVA of the mean amplitude 173 to 354 ms for NFB and 176 to 234 ms for NAE. Likewise, onsets and offsets of the between 173 and 354 ms, with the factors feedback (negative vs. PFB was defined by as the first and last sample in the window of 300–1000 ms positive) and electrode (Fz, FCz, Cz, Pz) showed a main effect of following feedback on which the amplitude was more positive than 3 SD. This was feedback, F(1,18) = 33,3, MSe = 3.4, p < .001, h2 ¼ 0:649, a marginal p
- 5. G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 215 Fig. 2. Average ERPs time-locked to the auditory stimulus (time zero), which immediately followed the response and carried relevant feedback as well as an irrelevant action effect. All ERPs are based on trials with correct responses. Left panel: ERPs to positive and negative feedback and the difference wave. Right panel: ERPs to the high-probability (80%) and low-probability (20%) action effects, and the difference wave. effect of electrode, F(3,54) = 2.9, MSe = 7.6, p < .10, h2 ¼ 0:138, and p low) and electrode (FCz, Cz, Pz). Apart from a first order effect of no interaction (F < 1). information, F(1,18) = 5.1, MSe = 2.3, p < .05, h2 ¼ 0:221, which p Fig. 2 (right panel) shows that there was also a negative indicated that the negative peak was larger to relevant feedback amplitude difference following trials with low-probability action than to action effects, there were no effects involving the factor effects relative to high-probability action effects. This difference information Fs < 1. had a diffuse scalp distribution and was largest over Pz, followed The timing of the NFB the NAE was compared using the jackknife by Cz and FCz. On average, the NAE for these three electrodes procedure (Miller et al., 1998) limited to the Pz. The onset exceeded the 3 SD criterion from 176 to 234 ms and peaked at difference between the NFB (148 ms) and the NAE (167 ms) was not 201 ms following the feedback. An ANOVA of the mean amplitude significant, t(18) = 0.3, p > .1, but the offset was later for the NFB between 176 and 234 ms, with the factors action effect (low- (386 ms) than for the NAE (257 ms), t1-tailed(18) = 1.8, p < .05. probability vs. high-probability) and electrode (Pz, FCz, Cz) showed Both difference waves had diffuse scalp topographies, with no a main effect of action effect, F(1,18) = 16.4, MSe = 4.4, p < .001, significant differences between amplitudes across midline elec- h2 ¼ 0:477, a marginal effect of electrode, F(2,36) = 2.7, MSe = 8.4, p trodes, so the finding that they both reached a parietal maximum is p < .10, h2 ¼ 0:129, and no interaction (F < 1). p insufficient support for a topographic similarity of the waves. In Following the NFB, there was a positive amplitude difference on fact, given the small differences in amplitude, a voltage map or trials following negative relative to positive feedback which we current–source density map might be misleading. Therefore, the refer to as a PFB. It was largest over Cz, followed by FCz and Pz. It correlation between the mean amplitudes of the two grand exceeded the 3 SD criterion from 395 to 710 ms for all three average difference waves in the interval 176–234 ms was electrodes. An ANOVA of the mean amplitude in this window, with calculated, using electrodes as cases (cf. Holroyd et al., 2008). the factors feedback and electrode (Cz, FCz, Pz) showed a main The Pearson correlation was r(7) = .93, p < .002. effect of feedback, F(1,18) = 23.0, MSe = 5.7, p < .001, h2 ¼ 0:561, a p Finally, it was tested whether individual differences in the size main effect of electrode, F(2,36) = 19.8, MSe = 3.6, p < .001, of the NAE were associated with individual differences in the size of h2 ¼ 0:524, but no interaction of feedback and electrode, F < 1. p the NFB. For this purpose, mean amplitudes were calculated in the The comparison of low- and high-probability action effects did not windows with significant activity. The correlation of both reveal a PAE. amplitudes across participants over Pz was significant, r(19) = .43, p < .05. 6.3. Comparison between negative waveforms 7. Discussion A direct comparison of the negative differences following feedback and action effects consisted of three parts; the amplitude, In this study we used a probabilistic learning task with the timing, and the scalp topography. Mean amplitudes between irrelevant action effects to study possible similarities between 176 and 234 ms, the window in which both difference waves were the intentional processing of feedback and the automatic proces- significant, were submitted to an ANOVA with the factors sing of action effects. In particular, we were interested to see information (feedback vs. action effect), probability (high vs. whether low-probability action-contingent signals would induce
- 6. 216 G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 the same ERP components as negative feedback information does. correlated. The onset latencies did not differ, but the NAE decayed In view of the mean accuracy of about 76%, it seems safe to sooner than the NFB. The amplitude difference of the negative wave conclude that our participants were able to use the provided elicited by feedback compared to action effects did not interact feedback to acquire the appropriate stimulus–response rules and with electrode or stimulus probability, suggesting that the two that they were not too much distracted by the manipulation of waves were equally sensitive to these experimental factors. With action-effect contingencies. regard to the unity of the source, converging evidence might be Our manipulation of feedback validity replicated previous sought by source localization studies or the combination of EEG findings in showing a pronounced NFB for negative feedback over a and fMRI. Finally, while the similarity in onset, eliciting events and substantial part of the midline, peaking between 173 and 354 ms scalp topography already strongly suggests that NFB and NAE were after feedback onset (Holroyd and Coles, 2002; Nieuwenhuis et al., the same component, there was also a significant correlation 2005). The behavioral data provide evidence that performance in between individual amplitudes differences in NFB and NAE. trials following the processing of negative feedback is impaired (in terms of accuracy, not RT). Clearly, the mismatch between 7.1. Comparison between ERP components expected and actual feedback creates conflict. The presence of conflict has been claimed to trigger conflict-resolution operations, A remaining question is whether the NFB and NAE obtained in such as updating the task representation (Egner and Hirsch, 2005), the current study should be interpreted as NE/ERNs or, rather as which arguably uses up resources that otherwise could be spent on N2s. N2s are fronto-central negativities peaking around 250 ms processing the subsequent target. Even though such post-conflict that show up in several versions (see Folstein and Van Petten, 2008 impairments are more commonly reported for RTs than they are for a review). The control-related N2, which has been argued to be for error rates (e.g., Rabbitt, 1968), our observations fit with the same as the NE/ERN (e.g., Nieuwenhuis et al., 2003) follows stimuli general idea that dealing with conflict takes time and resources— that either induce response conflict, or require response inhibition be it because of dedicated conflict-resolution processes or simply (Folstein and Van Petten, 2008). The NFB and NAE do not seem to be because the conflict extends into the subsequent trial and control N2s because this component only occurs if auditory stimuli increases cognitive noise. In any case, the observed impact of are very similar (Falkenstein et al., 1995; Nieuwenhuis et al., 2004), feedback expectancy provides clear evidence for the detection of which was not the case in the current study. Furthermore, it is feedback information that deviates from the expectations. difficult to imagine how relevant feedback could give rise to Particularly interesting for our purposes, violations of expecta- response conflict or why it would trigger inhibition, given that the tions related to action effects produced very similar outcomes. response is already elicited before the feedback. First, low-probability action effects also impaired performance in It is also unlikely that the NFB and NAE were a version of the the subsequent trial. Here it was RTs but not error rates that were ¨¨ ¨ mismatch negativity (MMN, Naatanen and Alho, 1997); an reliably affected. The significant correlation between feedback- auditory oddball N2 (as suggested by Holroyd et al., 2008). The and action-effect-related slowing suggests that the two expecta- MMN is evoked only if there is a substantial difference between the tion effects followed similar patterns. Apparently, processing an overall probability of a standard versus a deviant auditory action effect that does not fit with one’s (presumably implicit) stimulus, which was not the case here because none of the expectations induces a conflict similar to what we concluded for merged sounds was presented more frequently than on 36% of the the intentional processing of feedback. If so, action effects must trials.2 In other words, there were more deviant than standard have been processed, encoded together with the corresponding sounds. response, and stored—as expected from an ideomotor perspective Most important, however, the alternative interpretations on action control (Elsner and Hommel, 2001; Hommel et al., 2001). mentioned here in terms of a link between the NAE/NFB and the The absence of slowing following negative feedback may be due N2 would all be consistent with the main idea of this paper that as to S–R relations changing from block to block. As a result, negative much as the NFB signifies feedback monitoring, the NAE signifies the and positive feedback were equally informative to the subject in monitoring of action effects. That is, action effects can only elicit an the process of finding out the rules and this may explain why it did oddball N2, a control N2, or a short-lived NFB if the contingency of not differentially affect response speed on subsequent trials. In an action effect on a specific button press is encoded as a template contrast, action-effect contingencies remained the same through- for the upcoming sound, which is the same as monitoring whether out the session, and thereby participants had the chance to develop the action effect deviates from the expectation. The mismatch strong action-effect anticipations. This may explain why low- between high-probability and actual action effects could only be probability action effects were followed by a change in RT. registered if indeed the brain was able to anticipate on the most Importantly, our findings suggest that even task-irrelevant likely action effect. In sum, regardless of the exact classification of action-effect representations are used to predict the most likely the NAE, it supports the view that irrelevant action effects play a outcome of a given action and are matched against the actual role in monitoring for potentially erroneous actions. outcome (Wolpert et al., 1995). This confirms that action effects do The observation that the NAE was short-lived in comparison not only play a role in action selection, the process most ideomotor with the NFB might suggest that feedback was processed more theories focus on, but also in action monitoring and/or evaluation. elaborately and to a more advanced level than action effects were Given that the manipulated action effects were entirely irrelevant (see Krigolson and Holroyd, 2007, for a comparable differentiation for the task at hand, finding evidence for a predictive role of action- of error processing levels). The distinction between processing of effect representations provides support for approaches claiming feedback and action effect information receives further support that the human cognitive system is fundamentally anticipatory from the finding that there was a PFB but not an analogous positive and continuously trying to generate the best predictions possible peak to low-probability action effects (PAE). Given that the action based on the available information (e.g., Neisser, 1974; Schubotz, effects in the present study were irrelevant, this dissociation is in 2007; Zacks et al., 2007). line with our expectations and the idea that a PFB is more tightly A second reason to consider the detection of low-probability action effects and negative feedback equivalent is that both types 2 The highest probability of a particular vowel was 80% in half of the trials and of violations of expectations evoked a highly similar negative 10% in the other half. This was orthogonally combined with an 80% probability of waveform. Both the NFB and the NAE showed a maximum effect valid positive feedback. As a result, the highest probability within each block of a over Pz and the topographies of NFB and NAE were highly specific vowel/noise combination was (80 + 10)/2  80% = 36%.
- 7. G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 217 related to conscious processes than the NFB. The dissociation may performance through irrelevant action effects? The most obvious be a manifestation of the relatively early selection of relevant advantage relates to the development of action control. The information. Indeed, one may speculate that the longer-lasting the behavior of newborns and young infants is often driven by stimuli electrophysiological response to unexpected signals the greater and reflexes, but by systematically exploring their environment their informational value, and vice versa. and their own body they very soon acquire the ability to carry out goal-directed, intentional actions. A monitoring system that 7.2. The role of attention automatically picks up contingencies between involuntary move- ments and action effects would be of utmost importance in this In view of the conclusion that irrelevant action effects were used ongoing struggle for gaining control over one’s motor system to support performance monitoring, it might be argued that (Hommel and Elsner, 2009). participants were looking for regularities between buttons presses But the monitoring of action effects has advantages for the adult and the vowels, either by instruction or by their own initiative. A valid as well, especially if action effects provide useful information that question then is whether this rendered the action effects relevant, is not otherwise available, that is neither through explicit feedback, and whether they were attended. The primary task for the participant nor as part of the attended task. For one thing, the short temporal was to make speeded classifications and find out S–R relations on the difference between the action and the outcome is beneficial for the basis of feedback. The action effect was irrelevant in this regard associative strength. For another, the information may become because it was not part of the speeded task and was useless in finding evident over time, as an emergent property of memory storage of out S–R relations, but we do not want to argue that participants paid events, possibly under the restriction that actions and their effects no attention to it. After all, one of the instructions the participants had occur close enough in time (Elsner and Hommel, 2001; Hommel, received at the beginning of the experiment was to look for a pattern. 2005). However, it is not very likely that this instruction was held active What initially seems to be irrelevant information can turn out throughout a 2-h experiment, because it was neither reiterated, nor to be a more useful indicator of performance than the feedback that relevant for the task at hand, nor evaluated. In contrast, correct is provided by the task. Obviously, the usefulness of an action effect performance on the probabilistic learning task continuously required in monitoring will become evident most rapidly if it immediately maintenance of the goal to monitor relevant feedback and the follows the action and comes from the same source as the action, instruction and outcome were repeated on every block. such as when a tennis player hears the sound of poor stroke. In this The possibility that participants continued to pay attention to sense, the current design was a safe choice in providing immediate the action effects throughout the task was even further reduced by action effects that were unmistakably triggered by the button its difficulty. That is, looking for action-effect patterns was not press. It is an open question, however, to what extent mismatches inherently rewarding. When asked after the experiment, most in contingencies will also be picked up if actions and their effects participants reported that they were not aware of any contingency are infrequent or remote in space, such as when turning up the between buttons and sounds. This makes sense because overall the home thermostat does not cause the sound of the heating system action effects were only 60% predictable. If we consider that even switching on. 100% contingency between events can be ignored by about every The idea that mismatches between expected and actual action second participant (Alonso et al., 2006) and that response- effects drive the monitoring process implies that the nature of the feedback contingencies of 70% can go unnoticed entirely (Haruno deviating action effect as such is not critical, as long as there is a and Kawato, 2006), it seems safe to conclude that awareness was detectable mismatch from the action effect template. Given a low- not a crucial mediator in the present study. probability action effect, no difference is predicted between Finally, treating the action effect as relevant would be monitoring processes evoked by familiar versus novel action incompatible with the finding of an NAE without a PAE. As argued effects, although obviously, the mismatch with the expected action above, the PFB could be interpreted as either reflecting the effect should be sufficiently salient. The distinction that is made is conscious processing or feedback, or as processing of relevant not between one action effect or another, but between presence and infrequent information. The absence of a positivity to low- versus absence of the expected action effect. This assumption is probability action effects, as opposed to the presence of a positivity consistent with a dissociation that has been demonstrated to occur to feedback indicates that either the action effect did not reach following positive versus non-positive feedback (Hajcak et al., awareness, or was not processed as a relevant stimulus. It is hard to 2006; Holroyd et al., 2006): the NFB following neutral feedback is as imagine how a goal to look for action-effect contingencies can be large as the NFB following negative feedback, in contrast with the maintained as relevant if salient deviations of the action-effect view implicit in the traditional reinforcement learning theory of contingency do not reach awareness. the NFB (Holroyd and Coles, 2002); that the size of the NFB is Interestingly, previous research has shown that the acquisition proportional to the magnitude of negative feedback. Holroyd et al. of action-effect associations does not depend on (and may even be (2006) concluded from this pattern that feedback evaluation works hindered by) conscious perception or task relevance of the in a binary manner: the evaluative system either concludes that relationship between actions and effects (Eenshuistra et al., the goal is achieved (cf. that the action effect is as anticipated, the submitted for publication; Hommel et al., 2003; Kunde, 2004; feedback is positive) or that it is not. Holroyd et al.’s (2006) version although see Dutzi and Hommel, 2009, for an exception). Indeed, of the theory takes into account that a subjective evaluation that ideomotor theory assumes that the acquisition of action-effect the goal is not achieved can cause any deviation from the associations precedes and actually enables the performance of anticipated outcome to trigger an NFB (see also Hajcak et al., 2006). outcome-eliciting voluntary action, which implies that the If the action outcome is worse than anticipated, this triggers a acquisition itself occurs rather automatically (Elsner and Hommel, phasic decrease in dopamine that is transferred to the anterior 2001; Hommel, 1997). cingulate cortex, which translates it into an NFB. The current study seems to suggest that a similar process may also underlie the NAE. 7.3. The functional role of action-effect monitoring In sum, our findings are consistent with the prediction from the ideomotor approach to action control that action-contingent In sum, it is safe to conclude that while action effects were not signals are automatically integrated with the action they treated as relevant stimuli, they did play a significant role in accompany (Elsner and Hommel, 2001). The novel contribution monitoring performance. But why would the brain monitor of the present study is that it provides evidence that action-effect
- 8. 218 G.P.H. Band et al. / Biological Psychology 82 (2009) 211–218 associations do not only play a role in action selection but also in Holroyd, C.B., Pakzad-Vaezi, K.L., Krigolson, O.E., 2008. The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected posi- the monitoring of actions. Apparently, these associations give rise tive feedback. Psychophysiology 45, 688–697. to anticipation of task-relevant and irrelevant perceptual con- Hommel, B., 1997. Toward an action-concept model of stimulus–response compat- sequences of a given action. Mismatches with respect to both types ibility. In: Hommel, B., Prinz, W. (Eds.), Theoretical issues in stimulus–response compatibility. North-Holland, Amsterdam, pp. 281–320. of consequences are detected automatically and result in an NAE, Hommel, B., 1998. Perceiving one’s own action—and what it leads to. In: Jordan, J.S. analogous to the NFB Although more research on this issue is (Ed.), Systems Theory and Apriori Aspects of Perception. North-Holland, certainly needed, these observations seem to corroborate the idea Amsterdam, pp. 143–179. Hommel, B., 2005. How much attention does an event file need? Journal of that performance monitoring does not rely on explicit feedback but Experimental Psychology: Human Perception and Performance 31, 1067–1082. can exploit any action-contingent information available—a possi- Hommel, B., Alonso, D., Fuentes, L.J., 2003. Acquisition and generalization of action bility that seems to go beyond the assumptions of available action- effects. Visual Cognition 10, 965–986. Hommel, B., Elsner, B., 2000. Action as stimulus control. In: Schick, A., Meis, M., monitoring approaches. Reckhardt, C. (Eds.), Contributions to psychological acoustics: results of the 8th ¨ Oldenburg Symposium on Psychological Acoustics. Universitat Oldenburg, Acknowledgement Oldenburg, pp. 403–424.. Hommel, B., Elsner, B., 2009. Acquisition, representation, and control of action. In: Morsella, E., Bargh, J.A., Gollwitzer, P.M. (Eds.), Oxford Handbook of Human This research is supported by a fellowship from the Royal Action. Oxford University Press, New York, pp. 371–398. Netherlands Academy of Arts and Sciences to the first author. We ¨ Hommel, B., Musseler, J., Aschersleben, G., Prinz, W., 2001. The theory of event ´ are grateful to Gayle Arendsz, Ellahe Armbruster, and Tineke van coding (TEC): a framework for perception and action planning. Behavioral and Brain Sciences 24, 849–878. der Linde for their help in testing. James, W., 1890. The Principles of Psychology. Dover Publications, New York. Krigolson, O.E., Holroyd, C.B., 2007. Hierarchical error processing: different errors, References different systems. Brain Research 1155, 70–80. Kunde, W., 2004. Response priming by supraliminal and subliminal action effects. Alonso, D., Fuentes, L.J., Hommel, B., 2006. Unconscious symmetrical inferences: a Psychological Research 68, 91–96. role of consciousness in event integration. Consciousness and Cognition 15, Lotze, R.H., 1852. Medicinische Psychologie oder die Physiologie der Seele. Weid- 386–396. mann’sche Buchhandlung, Leipzig. Band, G.P.H., Kok, A., 2000. Age effects on response monitoring in a mental-rotation Melcher, T., Weidema, M., Eenshuistra, R.M., Hommel, B., Gruber, O., 2008. The task. Biological Psychology 51, 201–221. neural substrate of the ideomotor principle: an event-related fMRI analysis. Blakemore, S.-J., Frith, C.D., Wolpert, D.W., 1999. Spatiotemporal prediction mod- Neuroimage 39, 1274–1288. ulates the perception of self-produced stimuli. Journal of Cognitive Neu- Miller, J., Patterson, T., Ulrich, R., 1998. Jackknife-based method for measuring LRP roscience 11, 551–559. onset latency differences. Psychophysiology 35, 99–115. Dutzi, I.B., Hommel, B., 2009. The microgenesis of action-effect binding. Psycholo- Miltner, W.H.R., Braun, C.H., Coles, M.G.H., 1997. Event-related potentials following gical Research 73, 425–435. incorrect feedback in a time-estimation task: evidence for a ‘generic’ neural Eenshuistra, R.M., Verschoor, S., Weidema, M.A., Hommel, B. When words (and system for error detection. Journal of Cognitive Neuroscience 9, 788–798. thoughts) speak louder than actions: explicit task representation suppresses ¨¨ ¨ Naatanen, R., Alho, K., 1997. Mismatch negativity – The measure for central sound implicit action-effect associations, submitted for publication. representation accuracy. Audiology and Neuro-otology 2, 341–353. Egner, T., Hirsch, J., 2005. Cognitive control mechanisms resolve conflict through Neisser, U., 1974. Cognitive Psychology. Prentice-Hall, Englewood Cliffs, NJ. cortical amplification of task-relevant information. Nature Neuroscience 8, Nieuwenhuis, S., Slagter, H.A., Alting von Geusau, N.J., Heslenfeld, D.J., Holroyd, C.B., 1784–1790. 2005. Knowing good from bad: differential activation of human cortical areas by Elsner, B., Hommel, B., 2001. Effect anticipation and action control. Journal of positive and negative outcomes. European Journal of Neuroscience 21, 3161– Experimental Psychology: Human Perception and Performance 27, 229–240. 3168. Elsner, B., Hommel, B., 2004. Contiguity and contingency in the acquisition of action Nieuwenhuis, S., Yeung, N., Cohen, J.D., 2004. Stimulus modality, perceptual over- effects. Psychological Research 68, 138–154. lap, and the Go/NoGo N2. Psychophysiology 41, 157–160. Elsner, B., Hommel, B., Mentschel, C., Drzezga, A., Prinz, W., Conrad, B., Siebner, H., Nieuwenhuis, S., Yeung, N., Van den Wildenberg, W., Ridderinkhof, K.R., 2003. 2002. Linking actions and their perceivable consequences in the human brain. Electrophysiological correlates of anterior cingulate function in a Go/No-Go Neuroimage 17, 364–372. task: effects of response conflict and trial-type frequency. Cognitive, Affective Falkenstein, M., Hohnsbein, J., Hoorman, J., Blanke, L., 1990. Effects of errors in and Behavioral Neuroscience 3, 17–26. choice reaction tasks on the ERP under focused and divided attention. In: Overbeek, T.J.M., Nieuwenhuis, S., Ridderinkhof, K.R., 2005. Dissociable components Brunia, C., Gaillard, A., Kok, A. (Eds.), Psychophysiological Brain Research. of error processing: The functional significance of the Pe vis-a-vis the ERN/Ne. Tilburg University Press, Tilburg, The Netherlands, pp. 192–195. Journal of Psychophysiology 19, 319–329. Falkenstein, M., Koshlykova, N.A., Kiroj, V.N., Hoormann, J., Hohnsbein, J., 1995. Late Rabbitt, P.M.A., 1968. Three kinds of error-signaling responses in a serial choice ERP components in visual and auditory go/no-go tasks. Evoked Potentials, task. Quarterly Journal of Experimental Psychology 20, 179–188. Electroencephalography and Clinical Neurophysiology 96, 36–43. Ridderinkhof, K.R., 2002. Activation and suppression in conflict tasks: empirical Folstein, J.R., Van Petten, C., 2008. Influence of cognitive control and mismatch on clarification through distributional analyses. In: Prinz, W., Hommel, B. (Eds.), the N2 component of the ERP: a review. Psychophysiology 45, 152–170. Common Mechanisms in Perception and Action. Attention & Performance, vol. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E., Donchin, E., 1993. A neural system XIX. Oxford University Press, Oxford, pp. 494–519. for error detection and compensation. Psychological Science 4, 385–390. Ridderinkhof, K.R., Nieuwenhuis, S., Braver, T.S., 2007. Medial frontal cortex func- Gratton, G., Coles, M.G.H., Donchin, E., 1983. A new method for off-line removal of tion: an overview and introduction. Cognitive, Affective, and Behavioral Neu- ocular artifacts. Electroencephalography and Clinical Neurophysiology 55, 468– roscience 7, 261–265. 484. Ridderinkhof, K.R., Ullsperger, M., Crone, E.A., Nieuwenhuis, S., 2004. The role of Hajcak, G., Moser, J.S., Holroyd, C.B., Simons, R.F., 2006. The feedback-related medial frontal cortex in cognitive control. Science 306, 443–447. negativity reflects the binary evaluation of good versus bad outcomes. Biolo- Schubotz, R.I., 2007. Prediction of external events with our motor system: towards a gical Psychology 71, 148–154. new framework. Trends in Cognitive Sciences 11, 211–218. Haruno, M., Kawato, M., 2006. Different neural correlates of reward expectation and Smith, G.A., Brewer, N., 1995. Slowness and age – speed accuracy mechanisms. reward expectation error in the putamen and caudate nucleus during stimulus- Psychology and Aging 10, 238–247. action-reward association learning. Journal of Neurophysiology 95, 948–959. Snodgrass, J.G., Vanderwart, M., 1980. A standardized set of 260 pictures: norms for Holroyd, C.B., 2004. A note on the N200 and the feedback ERN. In: Ullsperger, M., name agreement, image agreement, familiarity, and visual complexity. Journal Falkenstein, M. (Eds.), Errors, Conflicts, and the Brain: Current Opinions on of Experimental Psychology: Human Learning and Memory 6, 174–215. Performance Monitoring. MPI of Cognitive Neuroscience, Leipzig. Waszak, F., Herwig, A., 2007. Effect anticipation modulates deviance processing in Holroyd, C.B., Coles, M.G.H., 2002. The neural basis of human error processing: the brain. Brain Research 1183, 74–82. reinforcement learning, dopamine, and the error-related negativity. Psycholo- Wolpert, Ghahramani, Z., Jordan, M.I., 1995. An internal model for sensorimotor gical Review 109, 679–709. integration. Science 269, 1880–1882. Holroyd, C.B., Nieuwenhuis, S., Yeung, N., Nystrom, L., Mars, R.B., Coles, M.G.H., Yeung, N., Botvinick, M.M., Cohen, J.D., 2004. The neural basis of error detection: Cohen, J.D., 2004. Dorsal anterior cingulate cortex shows fMRI response to conflict monitoring and the error-related negativity. Psychological Review 111, internal and external error signals. Nature Neuroscience 7, 497–498. 931–959. Holroyd, C.B., Hajcak, G., Larsen, J.T., 2006. The good, the bad and the neutral: Zacks, J.M., Speer, N.K., Swallow, K.M., Braver, T.S., Reynolds, J.R., 2007. Event electrophysiological responses to feedback stimuli. Brain Research 1105, 93–101. perception: a mind/brain perspective. Psychological Bulletin 133, 273–293.