Math 56a (Stochastic processes)
Brandeis Math Department
Spring 2008
Updated: 5/6/08, 4:45pm
What is new: answers to HW8.
Jump to: Notes, Quizzes, Homework.
Information
Instructor
TA's
Homework due date 
Answers  Old HW (2006) with answers  Comments 

new HW 0 due Jan 28 
answers to HW 0 
old HW 1 with answers  Matrices should be multiplied out and answers should resemble the answer sheet from 2006 (without the red stuff). 
new HW 1a (corrected) due Feb 4 
answers to HW 1a 
old HW 2 with answers  Be prepared to answer similar questions on a half hour quiz. 
new HW 1b due Feb 14 
answers to HW 1b 
old HW 3 with answers  
new HW2: Ch2, #1,2,14,16 
answers to HW 2 
old Chap 2 HW answers  
new HW3 
answers to HW 3 
old Chap 3 HW answers  
new HW4 due Wed 3/19 
answers to HW 4  old Chap 4 HW answers  4.6(c) should say: Find the largest alpha. 
new HW5 due Wed 3/26 
answers to HW 5 
old Chap 5 HW answers  
new HW6 due Thurs 4/3 
answers to HW 6 
old Chap 6 HW answers  
NO HW7  
old Chap 7 HW answers  
new HW8 due 4/30 
answers to HW 8 
old Chap 8 HW answers  Please give HW8 directly to the grader (JongHyunKim) 
Date  Quiz and answers  Description  Old Quiz (2006) with answers  Comments 

Feb 4  Practice Quiz 1 with answers 0 
Linear recurrence and Markov chains 

Feb 7  Quiz 1 with answers 1 
Markov chains: basics, based on HW1a 
old Practice Quiz 1 with answers  
Mar 17  Practice Quiz 2 answers 
Markov chains: Chap 1,2,3 

Mar 19  Practice Quiz 2b answers 
transience/recurrence and explosion 

Mar 20  Quiz 2 with answers 
For which p do we have transience, +/0recurrence, explosion? 

due Apr 10  Quiz 3  on martingales (Chapter 5) 

due May 14 unless you are a senior.  Take Home Final Exam  on everything 
Title  Lecture notes 2008  Description  Old version (2006)  Other 

Chapter 0  Chap 0 all (link fixed) 
Complete lectures notes for Chap 0, with HW. jump to details  old notes for Chapter 0.  
Chapter 1 
Chap 1 all 
Finite Markov Chains. p. 2156. jump to details  old notes for Chapter 1.  worksheet 1, answers 1 worksheet 2, answers 2 
Chapter 2 
Chap 2 all 
Countable Markov Chains.jump to details  old notes for Chapter 2.  
Chapter 3 
Chap 3 all  Continuous Markov Chains. Poisson processes, explosion and birth death  old notes for Chapter 3.  
Chapter 4 
Chap 4 all  Optimal Stopping Time. basic case, with cost, with discount  old notes for Chapter 4.  Worksheet 3 (convex functions) with answers 3 
Chapter 5 
Chap 5 all  Martingales Definitions,Conditional expectation, Integrability  old notes for Chapter 5.  Chap 5 source file, figure 
Chapter 6 
Chap 6 all  Renewal. Concepts, Age of process, Convolution and queueing  old notes for Chapter 6.  
Chapter 7 
(I will write this later)  Reversible Markov Chains.  old notes for Chapter 7.  
Chapter 8 
Brownian Motion.  old notes for Chapter 8.  

sec 8.1 
Introduction  Mathematical definition and approximation by random walk.  

sec 8.2 
Reflection principle  strong Markov property, the reflection principle, ChapmannKolmogorov, return probability.  

sec 8.3 
Dimension of zero set  Fractal nature of the zero set, box dimension.  

sec 8.4 
Heat equation  Brownian motion in R^{d}. backward time equation.  

sec 8.5 
Recurrence and transience  Brownian motion in R^{d}, probability of going to infinity.  

Chapter 9 
Stochastic Integration.  old notes for Chapter 9.  

sec 9.0,9.1 
Discrete stochastic integration  Concept of stochastic integral, Ito's formula, quadratic variation and discrete versions of these.  

sec 9.2 
Integration wrt W_{t}  Definition of stochastic integral for simple processes and in general (as an L^{2} limit).  

sec 9.3 
Ito's formula  Proof of Ito's formula and Levy's theorem.  

sec 9.4 
Ito's other formulas  covariation, product rule, multivariable Ito formula.  

sec 9.5 
BlackScholes  BlackScholes equation for the value of stock options.  
Title/date  Lecture notes 2008  Description  Old version (2006)  Other 

First lecture 1/16 
Introduction 
detailed lecture schedule, population extinction example 


Chap 0, sec 1 1/17 
Linear Diffeqs 
linear differential equations in one variable notes from the first half of the second lecture with proofs added 


Example 1/17 
SIR model 
KermackMcKendrick epidemic model with graphs. This is the deterministic version given by differential equations.  See exercise at end of notes.  
Chap 0, sec 2 1/23 
Matrix equations 
First order linear differential equations in several variable take the form of matrix equations and the solution is a matrix exponential.  Here are the old notes for this lecture.  You can practice finding eigenvalues and eigenvectors in your homework. 
Chap 0, sec 3 1/23 
Difference equations 
Linear difference equations are also known as linear recurrences.  We didn't quite finish this in class. I'll do the two example tomorrow. 
Title/date  Lecture notes 2008  Description  Old version (2006)  Other 

Chap 1, intro 1/24 
Markov chains, intro 
Markov Chains: overview, definition, examples.  old notes for this chapter.  Page 20 is HW0, page 21 has the table of contents of Chap 1. 
Chap 1, sec 2 1/28 
overview 
The concept of periodicity, communication classes, recurrent, transient.  Here is worksheet 1 with answers 1. 

Chap 1, sec 3,4 1/30 
Invariant distribution 
Invariant probability distribution, transient states.  

Chap 1, sec 5 1/31 
Canonical form of P correction: new page 44 
Larger transient states, canonical form for P, instructions for worksheet 2.  Here is worksheet 2 with answers 2 

Chap 1, sec 6.1 2/4 
Substochastic matrix Q, part 1  Details of the MouseCatCheese example.  

Chap 1, sec 6.2 2/6 
Substochastic matrix, part 2  Calculation of expected time.  

Chap 1, sec 6.3 2/6 
Part 3: Leontief model  Lecture on the Leontief economic model.  

Chap 1, sec 7 2/7 
Review  Review of differences between transient and recurrent.  
Title/date  Lecture notes 2008  Description  Old version (2006)  Other 

Chapter 2 2/11 
Intro and extinction (revised) 
Introduction to Countable Markov Chains, detailed explanation of extinction probability. 


Chapter 2 2/13 
Random walk 
Transient and recurrent, criterion for transience, details for random walk on Z^{ n}. 


Chapter 2 2/14 
Transient,recurrent,null recurrent 
Guest lecture by Alan Haynes: Review of transient/recurrent. Definition of null recurrent, positive recurrent. 

links