Q10 – Backpropagation

Contributed by Matthew Zak.

  1. Create a very simple graph(circuit) given with f(x_1,x_2,x_3,x_4)=x_1x_2+x_3x_4 and compute all the derivatives of f with respect to inputs (\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\frac{\partial f}{\partial x_3},\frac{\partial f}{\partial x_4}) using a chain rule (\frac{\partial f}{\partial x}=\frac{\partial f}{\partial q}\frac{\partial q}{\partial x}).
  2. Show how will the gradient of f with respect to x_1 change when we increase the input x_2 by \Delta h.
  3. Having a function g(f(x1, x2, x3, x4)) where f is given by the function above and g(t) = \sigma(t) is is a sigmoid function, compute the derivative of g with respect to input x_1(\frac{\partial g}{\partial x_1}).

4 thoughts on “Q10 – Backpropagation

  1. I am not sure about the meaning of the word “circuit”, as I do not see how to build a graph with a closed loop from f. Does someone have an explanation ?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s