Q10 – Backpropagation

Contributed by Matthew Zak.

Create a very simple graph(circuit) given with $f(x_1,x_2,x_3,x_4)=x_1x_2+x_3x_4$ and compute all the derivatives of f with respect to inputs $(\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\frac{\partial f}{\partial x_3},\frac{\partial f}{\partial x_4})$ using a chain rule ( $\frac{\partial f}{\partial x}=\frac{\partial f}{\partial q}\frac{\partial q}{\partial x}$ ).
Show how will the gradient of $f$ with respect to $x_1$ change when we increase the input $x_2$ by $\Delta h$ .
Having a function $g(f(x1, x2, x3, x4))$ where $f$ is given by the function above and $g(t) = \sigma(t)$ is is a sigmoid function, compute the derivative of $g$ with respect to input $x_1(\frac{\partial g}{\partial x_1})$ .

4 thoughts on “Q10 – Backpropagation”

Théo Rubenach says:

April 10, 2017 at 3:56 pm

I am not sure about the meaning of the word “circuit”, as I do not see how to build a graph with a closed loop from f. Does someone have an explanation ?

LikeLike

Reply
gagnonlg says:

April 11, 2017 at 11:42 am

For 2., do you mean the _derivative_ of f with respect to x1, or the gradient of f with respect to _x_?

LikeLike

Reply
- julianzaidi says:
  
  April 11, 2017 at 6:17 pm
  
  I think he talks about the component x1 so I considered the derivative of f w.r.t x1
  
  LikeLike
  
  Reply
- Philippe Lacaille says:
  
  April 12, 2017 at 10:35 pm
  
  Since the gradient of $f$ with regards of $x_1$ is a function of $x_2$ . I think he means what is the impact on the gradient.
  
  I think the answer is $\Delta h$ .
  
  LikeLiked by 1 person
  
  Reply