Feed Forward Neural Network with code

3 min readAug 7, 2019

similarly, convert all images into 1-D vector

let's take a simple predefine simple neurons where x1,x2,.. represent the input value to model and w, bias is the weight and bias value respectively. h1 and h2 are the output of first layers neurons and input for the second layer. cap y is the output of second layer neurons. so overall network structure is shown in the above figure. this type of network can be used for radial classification of given data shown below.

let's take some terminology of simple three-layer neural network shown below...

How to choose an output layer(MCC)?

the above figure shows the Multiclass classification(MCC) problem and regression problem

For MCC problem we use softmax to calculate final probability where each softmax value always greater than 0. we are not choosing linear in this case because, for negative input, linear can give a negative value which is not possible in MCC problem.

but when the dimension is very high (more than 3), it is not possible to visualize the data so in this case problem is how to decide the non-linearity of the dataset. this possible by doing hyperparameter tuning by plotting the loss value of each model. the model shows minimum loss chosen for this case shown below...

Notes: for binary class classification, we require only one output probability because other is (1-first) so we don't need softmax in this case.

Loss calculation for classification problem

-> Let's compute the loss of Binary class classification

here predicted y is near to 1 which is desired condition so loss value is less. here we don't need softmax function to calculate final probability value.

> Let’s compute the loss of Multiclass classification(MCC).