# Q12 – Function Representation and Network Capacity

Contributed by Pulkit Khandelwal.

Let us say that we are given two types of activation functions: linear and a hard threshold function as stated below:

• Linear:  $y = w_{0} + \sum_{i}w_{i}x_{i}$
• Hard Threshold:  $y=\left\{ \begin{array}{@{}ll@{}} 1, & \text{if}\ w_{0} + \sum_{i}w_{i}x_{i} \geq 0 \\ 0, & \text{otherwise} \end{array}\right.$

Which of the following can be exactly represented by a neural network with one hidden layer? You can use linear and/or threshold activation functions. Justify your answer with a brief explanation.

1. polynomials of degree 2
2. polynomials of degree 1
3. hinge loss
4. piecewise constant functions