Math 56a (Stochastic processes)
Brandeis Math Department
Spring 2008


Updated: 5/6/08, 4:45pm
What is new: answers to HW8.
Jump to: Notes, Quizzes, Homework.


Information


Instructor


TA's


Homework
 
Homework
due date		 Answers Old HW (2006) with answers Comments
  new HW 0
  due Jan 28   		  answers to HW 0
  old HW 1 with answers   Matrices should be multiplied out and answers should resemble the answer sheet from 2006 (without the red stuff).
  new HW 1a (corrected)
  due Feb 4   		  answers to HW 1a
  old HW 2 with answers   Be prepared to answer similar questions on a half hour quiz.
  new HW 1b
  due Feb 14   		  answers to HW 1b
  old HW 3 with answers  
new HW2:
Ch2, #1,2,14,16   		  answers to HW 2
  old Chap 2 HW answers  
new HW3
  		  answers to HW 3
  old Chap 3 HW answers  
  new HW4
  due Wed 3/19 		  answers to HW 4   old Chap 4 HW answers   4.6(c) should say: Find the largest alpha.
  new HW5
  due Wed 3/26 		  answers to HW 5
   old Chap 5 HW answers  
  new HW6
  due Thurs 4/3 		  answers to HW 6
   old Chap 6 HW answers  
  NO HW7  
  old Chap 7 HW answers  
  new HW8
due 4/30 		  answers to HW 8
   old Chap 8 HW answers   Please give HW8 directly to the grader (JongHyunKim)


Quizzes
 
Date Quiz and answers Description Old Quiz (2006) with answers Comments
  Feb 4     Practice Quiz 1
  with answers 0		   Linear recurrence and Markov chains
    
  Feb 7     Quiz 1
  with answers 1 		  Markov chains: basics, based on HW1a
  old Practice Quiz 1 with answers  
  Mar 17     Practice Quiz 2 answers
  Markov chains: Chap 1,2,3
   
  Mar 19     Practice Quiz 2b
answers 		  transience/recurrence and explosion
   
  Mar 20     Quiz 2 with answers
   For which p do we have transience, +/0-recurrence, explosion?
   
  due Apr 10     Quiz 3   on martingales (Chapter 5)
   
  due May 14 unless you are a senior.     Take Home Final Exam   on everything
    


Lecture notes
 
Title Lecture notes 2008 Description Old version (2006) Other
  Chapter 0   Chap 0 all
(link fixed)		   Complete lectures notes for Chap 0, with HW. jump to details   old notes for Chapter 0.  
  Chapter 1
  Chap 1 all
  Finite Markov Chains. p. 21-56. jump to details   old notes for Chapter 1.   worksheet 1, answers 1
worksheet 2, answers 2
  Chapter 2
  Chap 2 all
   Countable Markov Chains.jump to details   old notes for Chapter 2.  
  Chapter 3
  Chap 3 all   Continuous Markov Chains. Poisson processes, explosion and birth death   old notes for Chapter 3.  
  Chapter 4
  Chap 4 all   Optimal Stopping Time. basic case, with cost, with discount   old notes for Chapter 4.   Worksheet 3 (convex functions) with answers 3
  Chapter 5
  Chap 5 all   Martingales Definitions,Conditional expectation, Integrability   old notes for Chapter 5.   Chap 5 source file, figure
  Chapter 6
  Chap 6 all   Renewal. Concepts, Age of process, Convolution and queueing   old notes for Chapter 6.  
  Chapter 7
  (I will write this later)   Reversible Markov Chains.   old notes for Chapter 7.  
  Chapter 8
    Brownian Motion.   old notes for Chapter 8.  
  sec 8.1
  Introduction   Mathematical definition and approximation by random walk.    
  sec 8.2
  Reflection principle   strong Markov property, the reflection principle, Chapmann-Kolmogorov, return probability.    
  sec 8.3
  Dimension of zero set   Fractal nature of the zero set, box dimension.    
  sec 8.4
  Heat equation   Brownian motion in Rd. backward time equation.    
  sec 8.5
  Recurrence and transience   Brownian motion in Rd, probability of going to infinity.    
  Chapter 9
    Stochastic Integration.   old notes for Chapter 9.  
  sec 9.0,9.1
  Discrete stochastic integration   Concept of stochastic integral, Ito's formula, quadratic variation and discrete versions of these.    
  sec 9.2
  Integration wrt Wt   Definition of stochastic integral for simple processes and in general (as an L2 limit).    
  sec 9.3
  Ito's formula   Proof of Ito's formula and Levy's theorem.    
  sec 9.4
  Ito's other formulas   covariation, product rule, multivariable Ito formula.    
  sec 9.5
  Black-Scholes   Black-Scholes equation for the value of stock options.    


Details for Chapter 0
 
Title/date Lecture notes 2008 Description Old version (2006) Other
  First lecture
  1/16   		  Introduction
  detailed lecture schedule,
  population extinction example		    
  Chap 0, sec 1
  1/17   		  Linear Diffeqs
  linear differential equations in one variable
  notes from the first half of the second lecture with proofs added 		   
  Example
  1/17 		  SIR model
  Kermack-McKendrick epidemic model with graphs. This is the deterministic version given by differential equations.    See exercise at end of notes.
  Chap 0, sec 2
  1/23 		  Matrix equations
  First order linear differential equations in several variable take the form of matrix equations and the solution is a matrix exponential.   Here are the old notes for this lecture.   You can practice finding eigenvalues and eigenvectors in your homework.
  Chap 0, sec 3
  1/23 		  Difference equations
  Linear difference equations are also known as linear recurrences.     We didn't quite finish this in class. I'll do the two example tomorrow.


Details for Chapter 1
 
Title/date Lecture notes 2008 Description Old version (2006) Other
  Chap 1, intro
  1/24 		 Markov chains, intro
  Markov Chains: overview, definition, examples.   old notes for this chapter.   Page 20 is HW0, page 21 has the table of contents of Chap 1.
  Chap 1, sec 2
  1/28 		 overview
  The concept of periodicity, communication classes, recurrent, transient.     Here is worksheet 1 with answers 1.
  Chap 1, sec 3,4
  1/30 		 Invariant distribution
  Invariant probability distribution, transient states.    
  Chap 1, sec 5
  1/31 		 Canonical form of P
correction: new page 44		   Larger transient states, canonical form for P, instructions for worksheet 2.     Here is worksheet 2 with answers 2
  Chap 1, sec 6.1
  2/4 		 Substochastic matrix Q, part 1   Details of the Mouse-Cat-Cheese example.    
  Chap 1, sec 6.2
  2/6 		 Substochastic matrix, part 2   Calculation of expected time.    
  Chap 1, sec 6.3
  2/6 		  Part 3: Leontief model   Lecture on the Leontief economic model.    
  Chap 1, sec 7
  2/7 		  Review   Review of differences between transient and recurrent.    


Details for Chapter 2
 
Title/date Lecture notes 2008 Description Old version (2006) Other
  Chapter 2
  2/11 		  Intro and extinction (revised)
   Introduction to Countable Markov Chains,
  detailed explanation of extinction probability.		    
  Chapter 2
  2/13 		  Random walk
   Transient and recurrent, criterion for transience,
  details for random walk on Z n.		    
  Chapter 2
  2/14 		  Transient,recurrent,null recurrent
   Guest lecture by Alan Haynes: Review of transient/recurrent.
  Definition of null recurrent, positive recurrent.		    


links