Sensors
and Partially Observable Markov Decision Processes (POMDP)
Partially observable
environments
Hidden Markov Model (HMM)
recap
Belief states
Inc initial knowledge of environment, page on storing
observations
eg if position is right or left, at start belief state is both, if we
try to move right, we are definitely right.
what if actions differ between states? can consider only actions
available to all states if cost of illegal action is high
goal test: need to test all belief states
Partially
Observable Markov Decision Processes
Partially
Observable Markov Decision Processes (POMDP)
Policies for POMDPs
Value functions for POMDPs
max of linear terms?
Belief Markov Decision
Processes
Identifying policies
Value iteration for POMDPs
Exponential in complexity for actions, observations?
Policy iteration for POMDPs
POMDPs with infinite states
POMDPs with infinite states