Contributed by Pulkit Khandelwal.
Consider a neural network as shown in the Figure below. The network has linear activation functions. Let the various weights be defined as shown in the figure and also the output of each unit is multiplied by some constant k.
Answer the following questions:
- Re-design the neural network to compute the same function without using any hidden units. Express the new weights in terms of the old weights. Draw the obtained perceptron.
- Can the space of functions that is represented by the above artificial neural network also be represented by linear regression?
- Is it always possible to express a neural network made up of only linear units without a hidden layer? Give a brief justification.
- Let the hidden units use sigmoid activation functions and let the output unit use a threshold activation function. Find weights which cause this network to compute the XOR of and for binary-valued and . Assume that there are no bias terms.