Activation functions: Softmax and Sigmoid Compared
What are activation functions ? Activation functions are the key to Neural Networks (NN). They are the recipe to learn non-linear relationships in data, without them NNs would just be linear in nature. If you know the basics of logistic regression, you’ll understand the statement I made. I am planning to write another blog to cover activation functions from basics, therefore the focus here is just to compare softmax and sigmoid activation functions....