December 21, 2011

Rewind: The Path from Behaviorism to Mental Maps [update: other hpc functions added; video at bottom]

All students agree as to the facts. They disagree, however, on theory and explanation
 Tolman, 1948

Animals DO think!!
source
Occasionally, I like to venture back to older papers and see what they found, how they interpreted it, and how the data would be interpreted today (okay, I just stumble upon them usually).  With the guise of a scientific model/framework of how a certain region of the brain works, some (much?) older data is disregarded, because it does not fit the mold of the current understanding.  This disregarded data may be legitimate and it may not be legitimately disregarded.  It may just be disregarded because of selective "cherry picking" of what fits the model (confirmation bias).  Some of the data are still relevant and maybe more so today, as it gives pause to the person and makes them question the model, if ever so briefly.

One extraordinary example of this comes from the field of adult neurogenesis.  It was discovered in the rat (and confirmed, re-confirmed a few years later) by Altman & Das (1965).  Post-natal neurogenesis was disregarded, because it did not 'fit' the current model of how the brain operates (e.g. brain only generates new neurons before or soon-after birth).  After 40 years, the phenomenon was re-examined, has subsequently been confirmed, and now has a large literature dedicated to it.  This example shows the importance of thinking about our current models of interpretation, always question them, and also raises our consciousness to the fact that scientific findings are not under a monolithic interpretation.  The hippocampus is thought largely today to be the neural equivalent of the 'cognitive map', which was formulated in experimental psychology by Tolman.

*note: I've decided to cover a wider spectrum of information here in order to thrust towards a larger point: Question.

read on for a brief history of behaviorism, latent learning, vicarious trial and error, Tolman and the cognitive map, and where we are today... and video!

A [very] Brief History of Time... errr Experimental Psychology

Much research in the mid/late-1800s through early-1900s (as far as experimental psychology) was influenced by behaviorism.  Behaviorism was popularized by a list of eminent experimental psychologists, including Thorndike, Watson, Pavlov, and Skinner.  Very briefly, behaviorism is classicaly known as a  'black box model' of simple association; one such type is stimulus-response learning, whereby a stimulus elicits a behavioral response from the animal.

Stimulus-Response behavioral learning is when a stimulus produces an action (response).   For a brief overview of the major components of applied behavior learning, see here; also see bottom of post for short video on operant conditioning)


Vicarious Trial & Error: 
They CAN Think!

source
Tolman found that simple S-R learning was cumbersome and that place learning was a far easier strategy for the rodent.  In experiments that tried to 'tap into' the learning beyond simple S-R, the cognitive map theory evolved.  Tolman believed that rats learned by Vicarious Trial & Error (originally coined by Muenzinger), first forming hypotheses before making a conclusion.  This VT&E refers to the slight hesitation that rodents (and humans) exhibit at the choice-point where they must decide.  To test this, Tolman et al. had rats placed in front the discrimination doors on a 'jumping stand' (fig to right).  The doors, described following, are discriminated against for 'correct' vs. 'incorrect'.  They trained rats to discriminate between 3 stimuli on a gradient of difficulty: First, between a white and black door (easy); second, between a white and medium-grey door (moderate); third, between a white and light-grey door (hard).  The rat learned that one door was the correct one (e.g. always the lighter door); after deciding which one is correct, the rat jumps to the door, it falls open, and the rat receives food.  If the rat picks wrong, well, they fall into a locked door and are captured by a net.  As you can see from the diagram to the right, there is no "ledge" under the two doors.







To really show VT&E, Tolman used a similar set up but had a 'landing platform' in front of the doors.  The rat could jump to a door, land on the ledge, change their mind, jump back to the' jumping stand', and jump to the correct door.  Okay, a priori predictions are that one might expect that, the 'harder' discrimination (e.g. light-grey vs. white door), the more VT&E is necessary to make a correct choice.  This is, in fact, exactly opposite of what Tolman et al. found!


With the same visual discrimination test, Tolman found that during easy discriminations (e.g. black vs. white door), rat learned the discrimination task faster (expected) and used more VT&E (not expected; fig to left - line labeled 'BLACK' vs line 'LIGHT GREY').  In rats that showed exceptional learning within the 'BLACK' group, more VT&E occurred.  What does one make of this finding?  Well, Tolman gives a great explanation for what is most likely going on....  looking 'under the obvious' is so fun to do!

So, with humans, we know exactly what to do, because we have language and can hear instructions on what to do in the task when we first encounter it (or tell yourself to choose the lighter door).  The rat, however, has no clue what the instructions of the task are at the beginning!  For all they know, we plop them down in front of two doors and are supposed to stare very intensely to burn the door down (? pyrokinesis).  If you consider this, most of a rats attention at the beginning of the task is lent to trying to figure out these instructions (pick lighter door for reward).  An increase in VT&E occurs when they first start to catch on to the rules of the game.  In the graph above, you can see the rats in the beginning don't understand what to do. Once they sufficiently understand the task and are quite sure of what the instructions are for a correct trial, you see VT&E decrease (days ~13 for 'BLACK', at least; easier if you look at %correct instead but did not post that graph here).  In the 'LIGHT GREY' group, they have trouble figuring out the instructions of the task to begin with!  Correct choices for this group is consistently at chance level (graph not shown).  So, they don't understand and do not really participate in VT&E.


 In follow-up work, they did test this interpretation and does seem to be the case.  After the rats had learned the instructions of the task (indicated by a plateau in %correct reached in any of the groups), VT&E vs. Correct Choice was examined (graph below).  The figure below shows 3 different sections of data after each of the discrimination groups had learned the instructions.  The first represents easy discrimination of WHITE vs BLACK.  The dotted line represents VT&E and solid black line represents correct choices.  If they only looked at VT&E after the rats had sufficiently 'learned' the instructions, it was in fact that case that the more difficult the discrimination, the more VT&E and sampling occurred.  The data just needed to broken down into "Learning the Instructions" and "Post-Instructional Learning".  The medium discrimination (light grey vs. black in this case), showed overall increase in VT&E that eventually decrease.  The hardest discrimination, section 3, shows that VT&E occurred throughout the testing (and correct choice was overall lower than the other two discriminations.






data examined after all groups learned the rule of the task










Latent Learning & the Cognitive Map


So, where does Stimulus-Stimulus behavioral learning come in to play?
Edward Tolman
Source
Despite Tolman using behaviorist applications in his methods, he was not a strict behaviorist, like that of Pavlov and BF Skinner, and was more of a blend between classical behaviorists (parts-based, S-R) psychology and Gestalt (unifying, whole) psychology.  Tolman believed that rodents could learn a stimulus-stimulus response that did not require an 'event' or stimulus to make learning possible.  He also thought about the same, for example, of a cat escaping a puzzle box, gorilla trying to survive by locating food, or man in conversation with another human; these are all characterized as goal-directed (mental) behaviors and make up the majority, if not all, of what non-human and human behavior consists.  This is referred to as purposive behaviorism.  Tolman was fond of using stimulus-stimulus learning by being exposed to the environment and frequently used one first described by Blodgett (1929).  In this maze procedure, there was an absence of any obvious reward/motivation to learn the maze.  This early work (and subsequent work from Tolman) started psychologists thinking, 'hey, maybe non-humans do have some sort of ability to think'.  The knowledge gleaned from this maze procedure sparked (or placed a glowing-hot ember of) doubt into the mind of researchers and eventually lead to the cognitive 'revolution' of psychology that we are so fond of, now, along with several other experiments that led the way.

maze used by Blodgett
source


I will describe the original finding from Blodgett but Tolman followed this with several different behavioral paradigms and found similar results...

Blodgett had 3 groups of rats:

(1) those who ran the alley maze and continually found food at the end after solving the maze (control group),

(2) those who received food in their homecage for the first six days, and, starting on the seventh day, received food in the maze after successful navigation to the reward.

(3) those that ran the maze for two days and received food in their homecage, but, on the third and subsequent days, the mice received food at the end of successfully running the maze

Error curves (aka wrong entries/wrong
moves to the reward) for all 3 groups.  The solid line represents
group (1) with almost-immediate learning occurring, signified by
steep slope.  Group (2) is the dark, dashed line and shows an 'x'
at day 7 (start of in-maze reward), when in-maze reward began, and
shows a very steep slope starting at
day 7.  Group (3) is the perforated line and shows an 'x' at
day 3 (start of in-maze reward) and shows a fairly steep slope.
source
The findings were striking (fig. 5; to your right): as you would expect, the time taken for group (1) to navigate the maze and eat their reward decreased very quickly (represented by error score in the figure) - a simple stimulus-response.  Group (3) that received in-maze rewards on the third+ day showed fairly quick learning of the maze, a somewhat intermediate stimulus-response learning, and group (2) appeared to never learn the maze, until they started to get in-maze rewards on 7+ days.  Why is this interesting?  Because, it appeared in both groups (2) and (3) that no learning was occurring while in-maze rewards were absen; however, in group (2), in particular, there is a quick learning curve, such that the rat knows almost exactly where to go to get food but had no motivation to run a successful trial while they were not getting rewarded for the participation on days 1-6.  Once the rats were exposed to the in-maze reward once (on day 7), they went almost straight to the source, faster than those in group (1), where there was an obvious learning curve.  These results were interpreted by Tolman to be such that latent learning was occurring in the rats -- they were forming their 'cognitive maps' (or became aware and started to "internalize" their surrounding surfaces of the environment) but did not utilize this information because no motivation was present.  Once motivation to use this information/internal field map was present, the rats showed a remarkable ability in 'learning' (or remembering/utilizing their field map).



The moral of the story?  Just from looking at these two experiments, we can clearly see that rats "think" and try to understand things.  It is not just a simple stimulus input--response output situation.  It is not a 'black box' model - the picture is more complicated.

Okay, so, where do we stand today with the cognitive map?


The Map & The Hippocampus

After this experiment, several other studies started to look at the 'cognitive' aspect of the rat.  What is the prevalent theory today of the hippocampus?  Keeping in mind the Field/Cognitive map from Tolman, this theory largely took off after observing specific pyramidal cells that fire when the rat is in a specific location of the room, termed place cells (O'Keefe & Nadel).  There was, of course, other evidence that led these two researchers to posit a cognitive map theory of the hippocampus (e.g. the case of HM), and this theory has been strengthened by findings of various cells, especially those that appear to be spatial, like the place cells,  Head-direction cells, heading-vector cells, border cells, grid cells, theta rhymicity and navigation, and other electrophysiological and behavior/lesion evidence.  After all this evidence started to accumulate, it is now thought that the hippocampus is a major player in spatial navigation and memory.  There is, of course other such strong evidence from other behavioral studies not mentioned here.

In addition, as I will not belabor the point and draw out the post (maybe in another post), there have been several accounts of other hippocampal deficits that today are not well known (buried in the depths of history from the 1800-mid 1900s).  I wrote a few down that I thought about and some that I came across..

  Hippocampal lesions in rats (mid 1800 - mid 1900's)
(please let me know other things to add)

-perseverative responding during extinction
-excessive responding in conditioning (even when an increase in pressing a lever is associated with decrease reward - e.g. switch from CRF to DRL schedule in conditioning; not impaired at learning of no switch occurs, just straight DRL schedule)









-superior to controls in learning to press in sequence of 2 or 3 (for reward)







-no deficit in simultaneous discrimination task (black vs white arm discrim)







-deficit in successive discrimination (e.g. both arms black, left correct)







-deficit in go/no-go discrimination










-deficit with mazes which have multiple choice points










 (e.g. Hebb-Williams Maze, multiple U maze, elevated maze, Lashley III maze)





-show irregular navigational abilities in traversing a maze
-(human) intrusive and incorrect memories, lack of remembrance of recent episodes
-decreased anxiety-related behaviors in anxiogenic situations
-difficulty in reversal learning of choice tasks- persist to respond to two- or three-choice situation when discriminative cues are reversed
-deficit in attending to magnitude of the reward (less of negative contrast effect (they fail to reduce response rates when %reward probability is decreased; controls do this naturally; and usually the still perform even when reward is completely absent & don’t slow responding rates!))
-after learning lever press for reward and switch to rewarding for slower response, hippocampectomized (hpcx) rats don’t slow down (discontinuous negatively correlated reward)
                 -difficulty stopping an on-going motor process
-hpcx does affect motor function (increases it) when animal is aroused (via stimuli, conditioning for reward/punishment, etc).
-deficits in one-way passive avoidance learning ; superior in two-way passive avoidance    
-inhibition of escape responding in shuttle box paradigm
-in circular runway, have superior learning and reaction to escaping an on-coming stimulus that gives them a shock
-show non-spatial 'place fields,' such as those described by Vinogradova and the nictitating membrane in a DRL paradigm,
-when threatened by cat, normal rats froze where hpcx showed more escape responses (highly arousing situation)
-impaired at spontaneous alteration
-impaired dietary intake control and other motivational functions
-become indecisive and 'freeze' when there are two or more options that are both highly relevant to the animal at that time

*if interested in learning more, I can provide you with relevant material!


Interesting video on operant conditioning...  seeing things in video makes it more 'real', ya?


References:
Altman, J. & Das, G. (1965).  Autoradiographic and histological evidence of postnatal hippocampal neurogenesis in rats.  J Comp Neurol, 124, 319-336.

Cohen, N. J. & Eichenbaum H. (1991).  The theory that wouldn't die: A critical look at the spatial mapping theory of hippocampal functional. Hippocampus, 1(3), 265-268.

Tolman, E. C. (1948).  Cognitive maps in rats and men.  Psychological Reviews55(4), 189-208.


some articles of interest on hippocampus / neurogenesis



No comments:

Post a Comment