Neural Nets Cheat Sheet by khouloudch

An Artificial Neural Network (ANN) is an inform­ation processing paradigm that is inspired by the way biological nervous systems, such as the brain, process inform­ation.

ANNs, like people, learn by example. An ANN is configured for a specific applic­ation, such as pattern recogn­ition or data classi­fic­ation, through a learning process. Learning in biological systems involves adjust­ments to the synaptic connec­tions that exist between the neurones. This is true of ANNs as well.

Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experi­ence.

Real Time Operation: ANN comput­ations may be carried out in parallel, and special hardware devices are being designed and manufa­ctured which take advantage of this capabi­lity.

The activation function is used to introduce non-li­nearity in the modeling capabi­lities of the network

to estimate the loss (or error) and to compare and measure how good/bad our prediction result was in relation to the correct result (remember that we are in a supervised learning enviro­nment and we have the label that tells us the expected value). Ideally, we want our cost to be zero

Once the loss has been calcul­ated, this inform­ation is propagated backwards.