Math 56a (Stochastic processes)
Brandeis Math Department
Spring 2008

Updated: 5/6/08, 4:45pm
What is new: answers to HW8.

Information

• Our class has moved to room 317.
• Here is the syllabus for the course. See the first lecture notes for detailed schedule of lectures.

Instructor

• Kiyoshi Igusa
• Goldsmith 305
• Office hours: MW 2-3 (after class), Thursday 11-12.
• Phone 63062.
• standard email: lastname@brandeis.edu

TA's

• Jim Tseng
• Goldsmith 104
• Office hours: 2:30-3:30 on Mondays and Wednesdays.
• grader for HW1a, HW2, etc
• Jong-Hyun Kim
• Goldsmith 311
• Office hours: Tuesday 3-4, Thursday 5-6.
• grader for HW0, HW1b, etc.

Homework

Homework
due date
new HW 0
due Jan 28
old HW 1 with answers   Matrices should be multiplied out and answers should resemble the answer sheet from 2006 (without the red stuff).
new HW 1a (corrected)
due Feb 4
old HW 2 with answers   Be prepared to answer similar questions on a half hour quiz.
new HW 1b
due Feb 14
new HW2:
Ch2, #1,2,14,16
new HW3

new HW4
due Wed 3/19
answers to HW 4   old Chap 4 HW answers   4.6(c) should say: Find the largest alpha.
new HW5
due Wed 3/26
new HW6
due Thurs 4/3
NO HW7
new HW8
due 4/30

Quizzes

Feb 4     Practice Quiz 1
Linear recurrence and Markov chains

Feb 7     Quiz 1
Markov chains: basics, based on HW1a
old Practice Quiz 1 with answers
Mar 17     Practice Quiz 2 answers
Markov chains: Chap 1,2,3

Mar 19     Practice Quiz 2b
transience/recurrence and explosion

Mar 20     Quiz 2 with answers
For which p do we have transience, +/0-recurrence, explosion?

due Apr 10     Quiz 3   on martingales (Chapter 5)

due May 14 unless you are a senior.     Take Home Final Exam   on everything

Lecture notes

Title Lecture notes 2008 Description Old version (2006) Other
Chapter 0   Chap 0 all
Complete lectures notes for Chap 0, with HW. jump to details   old notes for Chapter 0.
Chapter 1
Chap 1 all
Finite Markov Chains. p. 21-56. jump to details   old notes for Chapter 1.   worksheet 1, answers 1
Chapter 2
Chap 2 all
Chapter 3
Chap 3 all   Continuous Markov Chains. Poisson processes, explosion and birth death   old notes for Chapter 3.
Chapter 4
Chap 4 all   Optimal Stopping Time. basic case, with cost, with discount   old notes for Chapter 4.   Worksheet 3 (convex functions) with answers 3
Chapter 5
Chap 5 all   Martingales Definitions,Conditional expectation, Integrability   old notes for Chapter 5.   Chap 5 source file, figure
Chapter 6
Chap 6 all   Renewal. Concepts, Age of process, Convolution and queueing   old notes for Chapter 6.
Chapter 7
(I will write this later)   Reversible Markov Chains.   old notes for Chapter 7.
Chapter 8
Brownian Motion.   old notes for Chapter 8.
sec 8.1
Introduction   Mathematical definition and approximation by random walk.
sec 8.2
Reflection principle   strong Markov property, the reflection principle, Chapmann-Kolmogorov, return probability.
sec 8.3
Dimension of zero set   Fractal nature of the zero set, box dimension.
sec 8.4
Heat equation   Brownian motion in Rd. backward time equation.
sec 8.5
Recurrence and transience   Brownian motion in Rd, probability of going to infinity.
Chapter 9
Stochastic Integration.   old notes for Chapter 9.
sec 9.0,9.1
Discrete stochastic integration   Concept of stochastic integral, Ito's formula, quadratic variation and discrete versions of these.
sec 9.2
Integration wrt Wt   Definition of stochastic integral for simple processes and in general (as an L2 limit).
sec 9.3
Ito's formula   Proof of Ito's formula and Levy's theorem.
sec 9.4
Ito's other formulas   covariation, product rule, multivariable Ito formula.
sec 9.5
Black-Scholes   Black-Scholes equation for the value of stock options.

Details for Chapter 0

Title/date Lecture notes 2008 Description Old version (2006) Other
First lecture
1/16
Introduction
detailed lecture schedule,
population extinction example

Chap 0, sec 1
1/17
Linear Diffeqs
linear differential equations in one variable
notes from the first half of the second lecture with proofs added

Example
1/17
SIR model
Kermack-McKendrick epidemic model with graphs. This is the deterministic version given by differential equations.    See exercise at end of notes.
Chap 0, sec 2
1/23
Matrix equations
First order linear differential equations in several variable take the form of matrix equations and the solution is a matrix exponential.   Here are the old notes for this lecture.   You can practice finding eigenvalues and eigenvectors in your homework.
Chap 0, sec 3
1/23
Difference equations
Linear difference equations are also known as linear recurrences.     We didn't quite finish this in class. I'll do the two example tomorrow.

Details for Chapter 1

Title/date Lecture notes 2008 Description Old version (2006) Other
Chap 1, intro
1/24
Markov chains, intro
Markov Chains: overview, definition, examples.   old notes for this chapter.   Page 20 is HW0, page 21 has the table of contents of Chap 1.
Chap 1, sec 2
1/28
overview
The concept of periodicity, communication classes, recurrent, transient.     Here is worksheet 1 with answers 1.
Chap 1, sec 3,4
1/30
Invariant distribution
Invariant probability distribution, transient states.
Chap 1, sec 5
1/31
Canonical form of P
correction: new page 44
Larger transient states, canonical form for P, instructions for worksheet 2.     Here is worksheet 2 with answers 2
Chap 1, sec 6.1
2/4
Substochastic matrix Q, part 1   Details of the Mouse-Cat-Cheese example.
Chap 1, sec 6.2
2/6
Substochastic matrix, part 2   Calculation of expected time.
Chap 1, sec 6.3
2/6
Part 3: Leontief model   Lecture on the Leontief economic model.
Chap 1, sec 7
2/7
Review   Review of differences between transient and recurrent.

Details for Chapter 2

Title/date Lecture notes 2008 Description Old version (2006) Other
Chapter 2
2/11
Intro and extinction (revised)
Introduction to Countable Markov Chains,
detailed explanation of extinction probability.

Chapter 2
2/13
Random walk
Transient and recurrent, criterion for transience,
details for random walk on Z n.

Chapter 2
2/14
Transient,recurrent,null recurrent
Guest lecture by Alan Haynes: Review of transient/recurrent.
Definition of null recurrent, positive recurrent.