Hi,
In lecture 7 we defined the n step return as:
$R_t^{(n)}(s_t) = \sum_{i=0}^{n-1}\gamma^i*r_{t+i} + \gamma^nV(s_{t+n})$
And in recitations 7 and 8 the n step return was defined as:
$R_t^{(n)}(s_t) = \sum_{i=1}^{n}\gamma^{i-1}*r_{t+i} + \gamma^nV(s_{t+n})$
(for example in part 3 of exercise 1 in recitation 8)
Is the definition in the recitation is wrong? if not, why are we skipping $r_t$?
Also I don't understand the calculations in section 4 of exercise 1 in recitation 8 for $v_1^\lambda$. I get the following result:
$v_1^\lambda = 0.5*([r_1+0.5*V(s_2)] + [r_1 +r_2*0.5 + 0.5^2*V(r_3)] + [r_1 + 0.5*r_2 + 0.5^2*r_3 + V(r_4)] =$
$= 0.5*([0+0.5*(-2)] + [0 +0*0.5 + 0.5^2*0] + [0 + 0.5*0 + 0.5^2*1 + 0] =$
$= 0.5(-1+0+0.5^2)$
Where is my mistake?
Thanks