Recitation 8 - Ex1
roy (guest) 29 Jun 2018 17:10
In Recitation 8 Ex1 we are asked to compute $v_{t}^{\lambda}$.
which is defined: $v_{t}^{\lambda} = (1 - \lambda)\sum_{n=1}^{\infty} \lambda^{n-1} v_{t}^{(n)}$
In class we defined that for n larger then the episode length we pad with zeros.
However in the recitation wev'e simply set $v_{t}^{(n)}$ to 0 for all $n$ larger then the episode.
The 2 approaches give different solutions.
(1 - Recitation)
$v_{1}^{\lambda} = 0.5\cdot[1\cdot(-1)+0.5\cdot0+0.25\cdot0.25]=-\frac{15}{32}$
(2 - considering all n up to infinity)
$v_{1}^{\lambda} = (1-\lambda)\sum_{n=1}^{\infty}\lambda^{n-1}v_{1}^{(n)} = (1-\lambda) [-1+0.5\cdot0+\sum_{n=3}^{\infty}\lambda^{n-1}0.25] = -\frac{7}{16}$
which is correct?
Thanks