4. Serving Two Masters

Customer loyalty: a case study

BAYER From 1898 through to 1910 Bayer marketed a “non-addictive morphine substitute” and “cough medicine” for children.

“ That will keep them coming”

And with this, we conclude our case study on customer loyalty.

Let’s briefly review of the course, and have a first thesis proposal

On Computers & Understanding The Capyblanca project De Groot & Simon & Gobet & Chase

After a 5sec presentation, subjects are asked to reconstruct the position.

The Capyblanca Project Emergence Small, independent processes produce the final result Priming Perception of subtle cues guide future processing; processing is NOT pre-programmed Understanding The system obtains a synthesis of the situation, in an information-efficient level

FARG, 1.0 The ability interpret ambiguous, confusing, isolated cues into a meaningful, coherent, situation. This is what FARG already does now. It's what CAPYBLANCA does. But we want to replicate these, using a single framework, at least in NUMBO and Copycat.

FARG 2.0: Next stopping point The ability to do it again, now faster. How can the system learn and self-organize such that, in the future, things get more and more efficient? One idea here that goes beyond FARG is that of (positive or negative) feedback changing future behavior; the other idea is the emergence of preference, in which the IOWA gambling task, or perhaps the N-armed bandit problem can help in addressing.

However, things need to get faster. But is this of any quality?

Ron “Suki” King, a checkers world champion 385 opponents—simultaneously He beat them all 2 seconds per move  12.30mins response

Capablanca’s remark “ I see only one move. The best one.” Jos é Raul Capablanca

Matheu 6:24 " No one can serve two masters; for either he will hate the one and love the other, or he will be devoted to one and despise the other. ”

But I think we all serve two masters

Two Masters Master #1: The Hedonic Feedback system pain and pleasure Master #2: The Attentional Feedback system Boredom & excitement

Master #1. Pain & Pleasure The Hedonic Feedback System Hedonic tonalities Regulates frequency of actions Objective of the system: generate preferences and values for the system What do I want from the world?

A thesis proposal How do these two feedback systems interact with each other? Hedonic system mostly works on the background, unconsciously. Hedonic system “cancels out” after some time (things originally intense become “boring, expected, normal”—perhaps because the attentional system has no more memory trace of them?) Attentional system triggers hedonic system

They usually go together Heroin Boredom, expected events, nothing new Excitement, unexpected change or anticipation of change Pain Pleasure Child playing on power outlet “Blind driving” Girl feasting on box of ice cream Sinus

But sometimes intense feelings become boring

Imagine if you had to live next to a monster that screamed in your ear and threatened you every 5 minutes.

After some time, you would get used to it, and think it is just “normal life”.

I am not talking about marriage

Youtube description by user woodwardcars “ This is where your job went. Notice the safety requirements: 60 ton press coming your way, no stop button, either "duck" or "move aside" when press comes down. If you are not on your "A" game today you'll lose an arm at best, crushed skull at worst. No problem, there's 20 guys standing outside waiting to take your position.”

To me, this sounds just like a description of the US troops in Iraq. “ press[ure] coming your way, no stop button, either "duck" or "move aside" when press[ure] comes down. If you are not on your "A" game today you'll lose an arm at best, crushed skull at worst. No problem, there's 20 guys standing outside waiting to take your position.” [Or at least that’s what Bush is asking for]

Think about… Chinese textile “docile girls” (who are ‘never going back to the farms’) Concentration camp survivors Doctors in an emergency room Garbage collectors Married people

If, sometimes, intense feelings become boring, what does this mean? That there are two feedback systems.

What do scientific experiments show?

IOWA Gambling task 2 sets of red cards 2 sets of blue cards Players get $2.000 and instructions to get rich, but none of the following information. Game: get $100 with each red card get $50 with each blue card Penalties for red cards go up to $1250 Penalties for blue cards are less than $100 NOW, WHAT HAPPENS?

Results After 50 cards, most people have a feeling about the good and the bad cards After 80 cards, something has clicked, and people can explain what’s going on. They have created a hypothesis, “tested it”, and it seems good. They have figured out the game. After 10 (!) cards, people start to develop a stress response to the bad cards, 40 steps before these are recognized as bad.

The IOWA gambling task Tells us about where this type of feedback happens in the brain Tells us that judgements are constantly made unconsciously Tells us that emotions are needed for decision-making And gives us an estimate of the time, in terms of impressions, needed for judgements to arise.

The Hefferline experiment Experimental setting: the desired response was a tiny thumb-twitch so small as to be subliminal, that is, the subjects could control it but only unconsciously (it was referred to as an ‘invisible thumb-twitch’ since the subjects had no knowledge of their controlling of it). The response was recorded by electrodes placed on the palmar base of the left thumb and on the medial edge of the left hand. Subjects heard music through headphones, and there was an aversive noise that would be mute for 15 seconds whenever the response was obtained, or, in case of it already being mute at the time of response, being then postponed for more 15 seconds.

The Hefferline experiment Group 4. Recording electrodes and a meter that informed them of response. Knew their behavior would stop the sound. Straightforwardly developed the response. Group 3. Recording electrodes, no meter. Knew their behavior would stop the sound. Some subjects attempted to produce the response voluntarily, which never occurred.

The Hefferline experiment Group 2. Recording electrodes PLUS some dummy electrodes, so they could not know where to attempt to respond to the noise. All subjects developed the response. Group 1. Recording and dummy electrodes. They were not informed that had any control on the disturbing sound. Instead, they were told that the study was about “body tension of noise superimposed on music”. Thus, they were simply told to “listen through earphones, and, otherwise, do nothing”.

The Hefferline experiment Impulsivity: Involuntary automatic unconscious processing connecting perception and (unconscious micro) action HEFFERLINE, R.F. — KEENAN, B.—HARFORD, R.A.: Escape and avoidance conditioning in human subjects without their observation of the response. Science, vol. 130, 1959, pp. 1338-1339.

The Hefferline experiment One might be led to believe that group 1 was a control group, since subjects lacked any knowledge whatsoever of the required task, but this is far from the truth. Group 1 gradually developed the desired response, as their brains were constantly scanning the senses for associations, and, in less than an hour, they were fully conditioned to stop the noise. The immense subtlety involved in perceiving a causal connection between an ‘invisible’ thumb-twitch and the annoying sound cannot be overstated, especially when we consider the massive sensorial information constantly bombarding the subjects. After the experiments were finished, subjects in group 1 were shocked to know they had been in control of the noise all along, as they still believed to have had been passive listeners throughout the whole experiment.

This experiment tells us That the brain constantly associates perception with [motor] action. That it quickly “picks up” the right association.

Master #1. Pain & Pleasure The Hedonic feedback system Hedonic tonalities regulate frequency of actions Objective of the system: generate preferences and values for the system What do I want from the world ? HOW DOES THIS SYSTEM WORK?

How can we implement pain or pleasure? Computers with pain or pleasure? Far from a mere technicality; this is a deep philosophical question Fortunately…

What do pain & pleasure feel like? (System shutdown; global update)

To be executed Previously executed CODERACK

To be executed Previously executed Pain Signal !!!! Interrupts processing for a global system update CODERACK

To be executed Previously executed Pain Signal !!!! Interrupts processing for a global system update 2. Lowers the Probability that these sequences will happen again in the future CODERACK

Codelets (or associated nodes in the slipnet) are observers of what what’s going on on the hedonic feedback system.

Not only the hedonic feedback system throws someone away from “ immediately painful behavior”, but it also improves system speed, by changing the probabilities in a directed manner.

“ immediately painful behavior”? Sometimes behavior is painful in short-term, but pleasurable/desirable in the long-run. Think, for example, of a vaccine. Sometimes, of course, the opposite occurs. Think about heroin.

Trainspotting “ I chose not to choose life. I chose something else. And the reasons? There are no reasons. Who needs reasons when you’ve got heroin?”

A study with mice Button #1. a sexy female mouse appears Button #2. Tasty food Button #3. activates the brain’s pleasure center What do they do?

Who needs reasons when you’ve got a button linked to the pleasure center of the brain?

Master #2. Boredom & excitement The attentional feedback system Regulates memory storage Blind driving September 11 th Objective of the system: discover how the world works and predict its behavior How does the world work and how is it going to change ? HOW DOES IT WORK? Top-down, expectation-driven, subcognitive processes Links incoming from the brain to sensory neurons

The shape of things to come How do semantic networks evolve, reorganize, grow, and learn over time? How to avoid the handcoding problem? When does semantic memory touch episodic memory? Can a node compute its own correlation to others as it receives their signals, without global association mechanisms? How do new nodes "pop"out? How do abstract chunks come to life to enable analogy-making? Can there ever be composite command codelets? Can there ever be "general codelets", to handle come what may? How do feedback systems and the credit assigment problem fit into this picture? Finally, what does the N-armed bandit problem or the IOWA gambling task have to do with all of this?

4. Serving Two Masters

More Related Content

Similar to 4. Serving Two Masters (20)

More from Alexandre Linhares (20)

Recently uploaded (20)

4. Serving Two Masters