States and deterministic environments

Introduction

States

State evolution

For environment with no actors, transition model is matrix (for finite state).

Can multiply matrix n times to get state in n periods.

Transition matrix is perumation matrix. 1 or 0 only.

Static and dynamic environments

Does time stop to allow decisions. eg chess v driving.

Discrete and continuous environments

Episodic and sequential environments