# Stochastic Processes Course Notes

**By: Anton Wakolbinger**

**Link to material:** http://www.math.uni-frankfurt.de/~stoch/wakolbinger/SS2001/StochProc01new_1.pdf

**Contents**

**1 Discrete Markov chains**

1.1 Random paths; stochastic and Markovian dynamics

1.2 Excursions from a state; recurrence and transience

1.3 Renewal chains

1.4 Equilibrium distributions

1.5 The ergodic theorem for Markov chains

1.6 Convergence to equilibrium

1.7 Optimal Stopping

1.8 Renewal chains revisited

**2 Renewal processes**

2.1 The renewal points and the residual lifetime process

2.2 Stationary renewal processes

2.3 Convergence to equilibrium

2.4 Homogeneous Poisson processes on the line

**3 Poisson processes**

3.1 Heuristics

3.2 Characterization

3.3 Construction

3.4 Independent labelling and thinning

3.5 Poisson integrals, subordinators

and Levy processes

**4 Markov chains in continuous time**

4.1 Jump rates

4.2 The minimal process and its transition semigroup

4.3 Backward and forward equations

4.4 Revival after explosion

4.5 Standard transition semigroups and their Q-matrices

**5 Conditional Expectation**

**6 Martingales**

6.1 Basic concepts

6.2 The supermartingale convergence theorem

6.3 Doob's submartingale inequalities

6.4 Stopping times

6.5 Stopped supermartingales

**7 The Wiener Process**

7.1 Heuristics and basics

7.2 Levy's construction of W

7.3 Quadratic variation of Wiener paths

7.4 Intermezzo: Filtrations and stopping in continuous time

7.5 The Ito-integral for simple integrands

7.6 Integrators of locally finite variation

7.7 Continuous local martingales as integrators

7.8 Stochastic calculus for continuous local semimartingales

7.9 Levy's characterisation of W

7.10 Reweighting the probability = changing the drift

7.11 A strategy for (almost) all cases

**Link to material:** http://www.math.uni-frankfurt.de/~stoch/wakolbinger/SS2001/StochProc01new_1.pdf