Decision-making Code

  • Click here for work on a decision-making model.
  • Learning and Solving Associative Cognitive Tasks

    Any thoughtful action takes into account prior knowledge and context. I am implementing computational models to explain how cortical and hippocampal networks can solve a range of tasks that require choices based on learning and context.

    I am using computational models to address the following questions:

    (1) Why is the hippocampus necessary for solving some associative tasks, but not other, similar ones?

    (2) Is there a trade-off between persistent activity following an input, as needed for short-term memory, and the ability to associate one input with another?

    (3) Does homeostasis explain the loss of ability to solve a previously learned task upon removal of hippocampus, when the task can be solved more rapidly without the hippocampus?

    (4) Can associations formed in the hippocampus bias a decision-making network, enabling correct performance in a set of contextual tasks?

    (5) Can the same associations in the hippocampus slow learning of tasks that do not require hippocampal associations to form?

    Parametric Working Memory and Sequential Discrimination

    Working memory is the ability to hold information temporarily, `on-line' in preparation for its imminent use in a decision. The mechanism of this short-term memory is either through recurrent firing of neurons, particularly in the prefrontal cortex, or through persistent currents which have been measured in the entorhinal currents.

    The task that I model requires monkeys to make a decision by comparing two stimuli that are separated by a delay. Ranulfo Romo and co-workers at UNAM in Mexico City train monkeys to differentiate the vibrational frequency of stimuli to their finger tips. The discrimination is sequential, with a delay between the two stimuli (hence the need for working memory). Importantly the two stimuli are to the same finger, and activate the primary sensory cortical neurons in the same manner, just at different times. A model of the cortical network hence requires a response based on whether the second stimulus is of greater or lesser frequency than the first. As such, it must perform a subtraction across time.

    I am carrying out computer simulations of networks of neurons which maintain persistent activity in response to a cue stimulus. As such, they exhibit the critical features of working memory. The memory circuit is equivalent to an integrator in the mathematical sense, as a greater frequency of vibration for a fixed duration leads to greater activity in a memory store. Similarly, an integrator does not change when its input ends, so exhibits persistent activity during any delay between stimuli. Much research is ongoing to address whether neuronal integrators are essentially continuous, or more discrete, allowing for robustness.

    In my model of the discrimination part of the task, I use a robust integrator to hold the memory, but assume inhibitory connections from the integrator to its inputs. This method of integral feedback control is common in engineering, and allows for a robust cancellation of the input during the first stimulus, leading to discrimination of the difference between two stimuli spaced in time.

  • Click here for computer code of the discrimination model.
  • Molecular basis of long-term memory

    John Lisman and coworkers have put forward a model for long-term memory based on the autophosphorylation of CaMKII holoenzymes and their dephosphorylation by PP1.Influx of calcium into the synapse can cause the process of autophosphorylation to dominate, leaving the CaMKII holoenzymes in a highly phosphorylated "up" state. The system is bistable, so that at resting calcium an unphosphorylated "down" state is stable, as well as the "up" state. Bistability occurs, because (a) autophosphorylation is cooperative within a holoenzyme and slow to get started, while (b) dephosphorylation saturates, so the per molecule rate is slower in the "up" state than "down" state.

    The location of CaMKII molecules that can take part in long-term potentiation (the strengthening of a synapse believed to underly long-term memory) is the post-synaptic density. The number of CaMKII holoenzymes in the postsynaptic density is between 5 and 30, depending on its size. As in reality, all reactions occur stochastically, it is a significant question as to how stable any memory storage can be. Fluctuations will eventually cause a switch of the system between "up" and "down" states or vice versa. Any such spontaneous switching is disruptive of memory, so it is important to know the timescale of such behavior. I have carried out both detailed simulations and analytical calculations for the system of enzymes, verifying that the timescale for spontaneous transitions rises exponentially with the number of enzymes present. In a reasonable range of parameters, 20 holoenzymes are sufficient to maintain memories on the timescale of human lifetimes, even when the individual proteins are being replaced by turnover randomly on a timescale of 30 hours.

    Last modified November 2, 2006

    Paul Miller,

  • Back to Paul Miller's home page