Learning: classical, instrumental and observational

Where learning sits on the test

Learning is one of four Cognitive subareas, with 6–8 questions on classical conditioning, instrumental conditioning, observational learning, and the theories and applications built on them (ETS, The GRE Psychology Test fact sheet). The ideas also appear in clinical questions (exposure therapy), developmental questions and social questions (modeling of aggression). Learning here means a relatively lasting change in behavior or knowledge that comes from experience.

Classical conditioning

Ivan Pavlov, a Russian physiologist who had won a Nobel Prize for work on digestion, noticed that his dogs salivated before food reached their mouths. He then studied this systematically: a neutral stimulus paired with food eventually produced salivation on its own.

Term	Meaning	Pavlov's example
Unconditioned stimulus (US)	Produces a response without learning	Food in the mouth
Unconditioned response (UR)	The unlearned response to the US	Salivation to food
Conditioned stimulus (CS)	Originally neutral; comes to produce a response after pairing	A tone
Conditioned response (CR)	The learned response to the CS	Salivation to the tone

The CR often resembles the UR but need not be identical. It can be weaker, or even opposite in direction (see drug tolerance below).

Worked example: label the parts

A child gets a painful shot from a nurse in a green uniform. Later the child cries on seeing anyone in green scrubs.

• US: the injection (pain without learning). UR: crying at the pain.
• CS: the green uniform. CR: crying at the uniform.
• Crying at green scrubs on other people is stimulus generalization.

Key phenomena

• Acquisition: the CR strengthens over pairings. It is usually strongest when the CS comes shortly before the US (delay conditioning, CS still on when the US arrives). Trace conditioning (a gap between CS offset and US) is usually weaker. Simultaneous pairing is weaker still, and backward conditioning (US before CS) usually produces little excitatory conditioning.
• Extinction: the CS is presented repeatedly without the US, and the CR declines. Extinction is new learning, not erasure. Three findings show this:
  • Spontaneous recovery: after a rest, the extinguished CR returns, though weaker.
  • Renewal: the CR returns when you test outside the extinction context.
  • Reinstatement: the CR returns after the US is presented alone.
• Generalization and discrimination: stimuli similar to the CS also elicit the CR, in a gradient. Training in which one stimulus is paired with the US and another is not produces discrimination.
• Higher-order (second-order) conditioning: a well-established CS1 is paired with a new CS2, and CS2 comes to elicit the CR without ever being paired with the US.
• Latent inhibition: a stimulus pre-exposed many times without consequence is later slower to become a CS.

Contingency, blocking and prediction error

Early accounts held that contiguity (pairing close in time) was enough. Three lines of work showed otherwise:

1. Contingency (Rescorla, 1968). What matters is whether the CS predicts the US, that is, whether the US is more likely with the CS than without it. If shocks are as likely without the tone as with it, the tone acquires little fear even though tone and shock are often paired.
2. Blocking (Kamin, late 1960s). First, a light is paired with shock until it is a strong CS. Then light plus tone are paired with the same shock. Later, the tone alone produces little fear. The light already predicted the shock, so the tone added no new information.
3. The Rescorla–Wagner model (1972) formalized this. Learning on a trial is proportional to the prediction error, the difference between the US that occurs and the US that all present cues predict. When the prediction is already perfect, there is no error, so no learning. That is blocking.

$$\Delta V = \alpha\beta(\lambda - \Sigma V)$$

Here $\lambda$ is the maximum the US supports, $\Sigma V$ is the summed strength of all cues present, and $\alpha$ and $\beta$ are learning rates for the CS and US. You do not need to calculate with it. Know that learning stops when $\Sigma V = \lambda$.

Biological preparedness and taste aversion

Garcia and Koelling (1966) gave rats "bright, noisy, tasty" water. Rats made ill afterward learned to avoid the taste but not the light and noise; rats shocked afterward learned to avoid the light and noise but not the taste. Associations are not equally easy for all stimulus pairs: animals are prepared to link tastes with illness and external cues with pain. Taste aversion also forms in a single trial and despite delays of hours between taste and illness, two departures from classic contiguity rules.

Applications

• Fear learning. Watson and Rayner (1920) reported conditioning fear of a white rat in an infant, "Little Albert," by pairing the rat with a loud noise. It is historically important but ethically unacceptable today. It was a single case, poorly controlled, and the fear's generalization and persistence are less certain than textbook retellings suggest.
• Exposure therapies rely on extinction. NIMH describes exposure therapy as a type of CBT in which a person learns to tolerate the distress of feared items or situations in a supportive setting until, over time, the fear goes down (NIMH, Psychotherapies). Systematic desensitization pairs graded exposure with relaxation, which is counterconditioning.
• Drug cues. Shepard Siegel's research proposed that cues regularly present at drug use come to elicit compensatory CRs opposite to the drug's effect, which contributes to tolerance. On this account, taking a usual dose in an unfamiliar setting can be more dangerous. NIDA likewise describes how cues linked to drug use can trigger strong cravings long after use stops (NIDA, Drugs and the Brain).

Instrumental (operant) conditioning

In instrumental conditioning the consequence of a behavior changes how often the behavior occurs.

Thorndike's law of effect. Around 1900 Edward Thorndike timed cats escaping from puzzle boxes. Escape times fell gradually, not suddenly, and he concluded that responses followed by satisfying consequences are "stamped in."

B. F. Skinner studied freely emitted behavior in the operant chamber ("Skinner box"), recording response rate.

The four consequences

"Positive" means a stimulus is added; "negative" means one is removed. Reinforcement increases behavior; punishment decreases it.

Effect on behavior	Stimulus added	Stimulus removed
Behavior increases	Positive reinforcement (praise after homework)	Negative reinforcement (seatbelt chime stops when you buckle)
Behavior decreases	Positive punishment (a fine for speeding)	Negative punishment (phone taken away; "time-out" from reinforcement)

Escape (ending an aversive stimulus) and avoidance (preventing it) are negative reinforcement. Mowrer's two-factor theory explains avoidance: classical conditioning makes a warning signal frightening, and the avoidance response is then negatively reinforced by fear reduction. This helps explain why avoidance-maintained fears resist extinction: the person never stays long enough to learn the feared outcome does not happen.

Schedules of reinforcement

Schedule	Rule	Typical pattern
Fixed ratio (FR)	Reinforce every nth response	High rate with a pause after each reinforcer
Variable ratio (VR)	Reinforce after an unpredictable number of responses (an average of n)	High, steady rate; very resistant to extinction (slot machines)
Fixed interval (FI)	Reinforce the first response after a fixed time	"Scallop": little responding early, accelerating as the time nears
Variable interval (VI)	Reinforce the first response after an unpredictable time	Moderate, steady rate (checking email)

Partial-reinforcement extinction effect: behavior reinforced intermittently extinguishes more slowly than behavior reinforced every time (continuous reinforcement), because nonreinforcement is harder to detect.

Other key ideas

• Primary reinforcers satisfy biological needs (food). Secondary (conditioned) reinforcers gain value through association (money, tokens). A token economy applies this.
• Shaping reinforces successive approximations; chaining links steps.
• Premack principle: a more probable behavior can reinforce a less probable one ("homework, then video games").
• Matching law (Herrnstein): with two options on concurrent schedules, the proportion of responses tends to match the proportion of reinforcement each yields.
• Discriminative stimulus ($S^D$): a cue signaling that a response will be reinforced; behavior comes under stimulus control.
• Superstitious behavior: reinforcement that happens to follow a behavior by chance can strengthen it.
• Instinctive drift (Breland and Breland): trained animals drift back toward species-typical behaviors (a raccoon "washing" coins instead of depositing them), another biological constraint.
• Learned helplessness (Seligman and Maier, 1967): dogs given inescapable shocks later failed to escape shock when escape became possible. The finding shaped cognitive models of depression. Later work emphasized that perceived control is the key variable.
• Punishment suppresses behavior best when it is immediate, consistent and paired with reinforcement of an alternative. It can also produce fear, avoidance of the punisher, and modeling of aggression.

Cognition in learning

• Latent learning (Tolman and Honzik, 1930). Rats that explored a maze without reward made few errors once food was introduced, quickly matching rats rewarded all along. Learning had occurred without reinforcement and showed up in performance only when there was a reason. Tolman argued that rats form cognitive maps.
• Insight (Köhler, 1910s). Chimpanzees solved problems (stacking boxes to reach bananas) suddenly after a pause, not by gradual trial and error.

Observational learning

Albert Bandura and colleagues (1961) showed that children who watched an adult behave aggressively toward an inflatable "Bobo" doll later imitated specific aggressive acts. In a later study (1965) the model was rewarded, punished or neither. Children who saw the model punished imitated less. But when all children were later offered incentives to reproduce the model's actions, the groups did about equally well. Punishment of the model had suppressed performance, not learning.

Bandura described four processes: attention, retention, reproduction and motivation. Vicarious reinforcement (seeing a model rewarded) raises performance. These ideas fed into social-cognitive theory and the concept of self-efficacy (see the Personality lesson).

Common traps

• Negative reinforcement is not punishment. If the behavior increases, it is reinforcement, even if something unpleasant is involved (it is being removed).
• Extinction vs forgetting. Extinction needs CS-alone trials (or responses without reinforcement). Forgetting is loss over time without such trials. Spontaneous recovery shows extinction did not erase the original learning.
• Generalization vs discrimination. Generalization is responding to similar stimuli. Discrimination is responding differently to them.
• The CR is not always like the UR. Compensatory drug CRs run opposite to the drug effect.
• "Pairing" is not the whole story. Pick answers about prediction, contingency or information when a question involves blocking or random control.
• Schedules. Ratio schedules depend on responses, interval schedules on time. The VR schedule gives the highest steady rate and the most resistance to extinction.

Check yourself

• In a taste-aversion scenario, which stimulus is the CS, and which two classic rules does taste aversion bend?
• Why does a light that already predicts shock block conditioning to a tone added later?
• Classify: a child stops whining because a parent ignores it.
• Which schedule produces a scalloped cumulative record?
• What did offering incentives to all children reveal in Bandura's 1965 study?