Math 56a (Stochastic processes)
Brandeis Math Department
Spring 2008

Updated: 5/6/08, 4:45pm
What is new: answers to HW8.
Jump to: Notes, Quizzes, Homework.




due date
Answers Old HW (2006) with answers Comments
  new HW 0
  due Jan 28  
  answers to HW 0
  old HW 1 with answers   Matrices should be multiplied out and answers should resemble the answer sheet from 2006 (without the red stuff).
  new HW 1a (corrected)
  due Feb 4  
  answers to HW 1a
  old HW 2 with answers   Be prepared to answer similar questions on a half hour quiz.
  new HW 1b
  due Feb 14  
  answers to HW 1b
  old HW 3 with answers  
new HW2:
Ch2, #1,2,14,16  
  answers to HW 2
  old Chap 2 HW answers  
new HW3
  answers to HW 3
  old Chap 3 HW answers  
  new HW4
  due Wed 3/19
  answers to HW 4   old Chap 4 HW answers   4.6(c) should say: Find the largest alpha.
  new HW5
  due Wed 3/26
  answers to HW 5
  old Chap 5 HW answers  
  new HW6
  due Thurs 4/3
  answers to HW 6
  old Chap 6 HW answers  
  NO HW7  
  old Chap 7 HW answers  
  new HW8
due 4/30
  answers to HW 8
  old Chap 8 HW answers   Please give HW8 directly to the grader (JongHyunKim)

Date Quiz and answers Description Old Quiz (2006) with answers Comments
  Feb 4     Practice Quiz 1
  with answers 0
  Linear recurrence and Markov chains
  Feb 7     Quiz 1
  with answers 1
  Markov chains: basics, based on HW1a
  old Practice Quiz 1 with answers  
  Mar 17     Practice Quiz 2 answers
  Markov chains: Chap 1,2,3
  Mar 19     Practice Quiz 2b
  transience/recurrence and explosion
  Mar 20     Quiz 2 with answers
  For which p do we have transience, +/0-recurrence, explosion?
  due Apr 10     Quiz 3   on martingales (Chapter 5)
  due May 14 unless you are a senior.     Take Home Final Exam   on everything

Lecture notes
Title Lecture notes 2008 Description Old version (2006) Other
  Chapter 0   Chap 0 all
(link fixed)
  Complete lectures notes for Chap 0, with HW. jump to details   old notes for Chapter 0.  
  Chapter 1
  Chap 1 all
  Finite Markov Chains. p. 21-56. jump to details   old notes for Chapter 1.   worksheet 1, answers 1
worksheet 2, answers 2
  Chapter 2
  Chap 2 all
  Countable Markov Chains.jump to details   old notes for Chapter 2.  
  Chapter 3
  Chap 3 all   Continuous Markov Chains. Poisson processes, explosion and birth death   old notes for Chapter 3.  
  Chapter 4
  Chap 4 all   Optimal Stopping Time. basic case, with cost, with discount   old notes for Chapter 4.   Worksheet 3 (convex functions) with answers 3
  Chapter 5
  Chap 5 all   Martingales Definitions,Conditional expectation, Integrability   old notes for Chapter 5.   Chap 5 source file, figure
  Chapter 6
  Chap 6 all   Renewal. Concepts, Age of process, Convolution and queueing   old notes for Chapter 6.  
  Chapter 7
  (I will write this later)   Reversible Markov Chains.   old notes for Chapter 7.  
  Chapter 8
    Brownian Motion.   old notes for Chapter 8.  
  sec 8.1
  Introduction   Mathematical definition and approximation by random walk.    
  sec 8.2
  Reflection principle   strong Markov property, the reflection principle, Chapmann-Kolmogorov, return probability.    
  sec 8.3
  Dimension of zero set   Fractal nature of the zero set, box dimension.    
  sec 8.4
  Heat equation   Brownian motion in Rd. backward time equation.    
  sec 8.5
  Recurrence and transience   Brownian motion in Rd, probability of going to infinity.    
  Chapter 9
    Stochastic Integration.   old notes for Chapter 9.  
  sec 9.0,9.1
  Discrete stochastic integration   Concept of stochastic integral, Ito's formula, quadratic variation and discrete versions of these.    
  sec 9.2
  Integration wrt Wt   Definition of stochastic integral for simple processes and in general (as an L2 limit).    
  sec 9.3
  Ito's formula   Proof of Ito's formula and Levy's theorem.    
  sec 9.4
  Ito's other formulas   covariation, product rule, multivariable Ito formula.    
  sec 9.5
  Black-Scholes   Black-Scholes equation for the value of stock options.    

Details for Chapter 0
Title/date Lecture notes 2008 Description Old version (2006) Other
  First lecture
  detailed lecture schedule,
  population extinction example
  Chap 0, sec 1
  Linear Diffeqs
  linear differential equations in one variable
  notes from the first half of the second lecture with proofs added
  SIR model
  Kermack-McKendrick epidemic model with graphs. This is the deterministic version given by differential equations.    See exercise at end of notes.
  Chap 0, sec 2
  Matrix equations
  First order linear differential equations in several variable take the form of matrix equations and the solution is a matrix exponential.   Here are the old notes for this lecture.   You can practice finding eigenvalues and eigenvectors in your homework.
  Chap 0, sec 3
  Difference equations
  Linear difference equations are also known as linear recurrences.     We didn't quite finish this in class. I'll do the two example tomorrow.

Details for Chapter 1
Title/date Lecture notes 2008 Description Old version (2006) Other
  Chap 1, intro
 Markov chains, intro
  Markov Chains: overview, definition, examples.   old notes for this chapter.   Page 20 is HW0, page 21 has the table of contents of Chap 1.
  Chap 1, sec 2
  The concept of periodicity, communication classes, recurrent, transient.     Here is worksheet 1 with answers 1.
  Chap 1, sec 3,4
 Invariant distribution
  Invariant probability distribution, transient states.    
  Chap 1, sec 5
 Canonical form of P
correction: new page 44
  Larger transient states, canonical form for P, instructions for worksheet 2.     Here is worksheet 2 with answers 2
  Chap 1, sec 6.1
 Substochastic matrix Q, part 1   Details of the Mouse-Cat-Cheese example.    
  Chap 1, sec 6.2
 Substochastic matrix, part 2   Calculation of expected time.    
  Chap 1, sec 6.3
  Part 3: Leontief model   Lecture on the Leontief economic model.    
  Chap 1, sec 7
  Review   Review of differences between transient and recurrent.    

Details for Chapter 2
Title/date Lecture notes 2008 Description Old version (2006) Other
  Chapter 2
  Intro and extinction (revised)
  Introduction to Countable Markov Chains,
  detailed explanation of extinction probability.
  Chapter 2
  Random walk
  Transient and recurrent, criterion for transience,
  details for random walk on Z n.
  Chapter 2
  Transient,recurrent,null recurrent
  Guest lecture by Alan Haynes: Review of transient/recurrent.
  Definition of null recurrent, positive recurrent.