A Sigmoid or logistic function is also relatively popular, as shown here:

func sigmoid(x){
    return 1 / (1 + Exp(-x))
}

The output is as follows:

Sigmoid has a property that is also useful: it can map any real number back down to a range between 0 and 1. This can be very useful for producing models that prefer an output between 0 and 1 (for example, a model for predicting the probability of something).

It also has most of the properties we are looking for, as listed here:

• It is nonlinear. Therefore, stacking several layers of these will not necessarily result in being the same as one layer.
• It is differentiable. Therefore, it works with backpropagation.
• It is monotonic.

However, one drawback is that it is more costly to compute compared to ReLU, and therefore, it will take longer overall to train a model with this.