Q5

We are given two data points with 2 different timestamps.

At the timestamp t = 1, we have a data point (x1, x2, y) where (x1, x2) = (0.3, 0.6) and y = 0.2.

At the timestamp t = 2, we have a data point (x1, x2, y) where (x1, x2) = (0.1, 1.0) and y = 0.4.

Here, x1 and x2 are 2 input variables. y is the output variable.

(a) Consider the traditional LSTM model. Initially, we have the following internal weight vectors and bias variables as follows.

In the model, we have the following status variables. For each t = 1, 2, ….

1. forget gate variable ft

2. input gate variable it

3. input activation variable at

4. internal state variable st

5. output gate variable ot

6. final output variable yt

Suppose that y0 = 0 and s0 = 0.

Consider the input forward propagation step only.

(i) What are the values of the above status variables when t = 1 and when t = 2? Please show each answer up to 4 decimal places.

(ii) What are the errors of the final output variables when t = 1 and when t = 2? Please show each answer up to 4 decimal places.

(b) Consider the GRU model. Initially, we have the following internal weight vectors and bias variables as follows.

In the model, we have the following status variables. For each t = 1, 2, ….

1. reset gate variable rt

2. input activation variable at

3. update gate variable ut

4. final output variable yt

Suppose that y0 = 0.

Consider the input forward propagation step only.

(i) What are the values of the above status variables when t = 1 and when t = 2? Please show each answer up to 4 decimal places.

(ii) What are the errors of the final output variables when t = 1 and when t = 2? Please show each answer up to 4 decimal places.

(c) What is the major disadvantage of the traditional neural network model compared with the recurrent neural network model?