Close

Sigmoid function

alexanderAlexander wrote 07/03/2018 at 05:04 • 2 min read • Like

I recently rediscovered the sigmoid function. It is incredibly useful in many areas of programming, but most recently has seen intense use in the neural network and machine learning software development communities. It takes any number as input, and scales it to between 0 and 1. This allows you to easily get a "true" or "false" out of a floating point number.

The equation itself is very simple:

For those of you not as familiar with mathematics, here is the article on e. It's equivalent to about 2.71828 (50 digits in the above figure). For most use cases a shorter approximation is fine, but sometimes you may need many more significant digits. The function has a very characteristic S-shaped curve (hence the name sigmoid), which looks like this:

Implementing this equation is quite easy in many languages. Let's take a look at an implementation in C:

#include <math.h>

double sigmoid(double x) {
     double result;
     result = 1 / (1 + exp(-x));
     return result;
}

This is very easy as the C standard math library includes the exp() function, which returns e raised to the number given. We could create our own exp() function if we didn't want to use the math library, although we would also have to create our own pow() function as well.

Why don't we take a look at sigmoid in Python? It's just as simple as the C example.

import numpy as np
def sigmoid(x):
    return (1 / (1 + np.exp(-x)))

Like the C example, this Python depends on the function exp from the numpy library. You can install numpy using pip.  Pretty awesome, right? Try using the function in some code! There are other variations on sigmoid in case you want the 'linear' region to be steeper or shallower, as well as many other tweaks. These are all listed in the Wikipedia article I linked earlier.

Let me know if you've ever used sigmoid in the comments!

Like

Discussions