Making logic gates following McCulloch-Pitts Neuron or the Perceptron is a rather simple task as it deals with a simple if-else logic, wherein the branching is done by a threshold value set against the output of the neural network, which is an activation function applied over the cumulative sum of the product of the inputs and weights.
If we were to devise a perceptron for any logic gate, we would require two inputs (a combination of 2 digit binaries, which result in one binary output), their corresponding weights, and the bias with the yielded output.
However, if you were to train them, you’ll have to account for changes in the weights, which is what I’ll be covering here. The change in weights is usually defined as an additional value that is added to them, which comes to be the product of the learning rate, expected/target value, and the corresponding input. Do keep in mind that higher learning rates do result in inaccurate predictions, so keeping the value of alpha less than 1 is a recommended practice.
Included below is my quick C++ implementation for one round of perceptron training for an OR gate with weights initialized to zero, which will eventually be adjusted to comply with the target values.
Note: Jekyll compilation gives an error for double curly braces having unfamiliar syntax inside them (For more information, read this), which is why I separated the braces including the 2D vector’s contents into separate lines, ensuring that no line has two consecutive opening and closing curly braces to overcome Jekyll injected specs.
#include <iostream>
#include <vector>
int main()
{
// Initializing 'Inputs' vector with Binary input combinations of two digits:
std::vector<std::vector<float>> Inputs{
{0, 0}, {0, 1}, {1, 0}, {1, 1}
};
// Initializing 'Weights' vector with zero-valued weights, which will be trained to (1, 1) ordinally with training:
std::vector<float> Weights{0, 0};
// Initializing 'Target' vector which accounts for OR gate output(s) for Inputs taken in order from 0,0 <-> 1,1:
std::vector<float> Target{0, 1, 1, 1};
// Initializing the bias, the learning rate (alpha), cumulative sum of inputs Yin = Σ(input * weight) and Yout:
float Bias = 0, Alpha = 0.5, Yin, Yout;
// Traversing through the 'Inputs' vector:
for(int i = 0; i < Inputs.size(); i++)
{
// Defining Yin as the sum of inputs times weights plus the bias:
Yin = Inputs[i][0] * Weights[0] + Inputs[i][1] * Weights[1] + Bias;
// Following Perceptron branch (either-or) logic, with Yout = 1 for Yin > 0 & Yout = 0 for Yin <= 0:
Yout = (Yin > 0) ? 1 : 0;
// If Yout matches my target value, the weights should be left unchanged (signifying that I reached my goal)
// If not, then there has to be changing weights in proportion to the learning rate, with the changes being reflected in the bias as well:
if(Yout != Target[i])
{
Weights[0] += Alpha * Target[i] * Inputs[i][0];
Weights[1] += Alpha * Target[i] * Inputs[i][1];
Bias += Alpha * Target[i];
}
// Output statement:
std::cout << "Values of: Inputs = (" << Inputs[i][0] << ", " << Inputs[i][1] << "), Yin = " << Yin << ", Yout = " << Yout << ", Weight 1 = " << Weights[0] << ", Weight 2 = " << Weights[1] << std::endl;
}
return 0;
}
After executing the above program, you’ll observe that the inputs (1, 0) and (1, 1) yield a correct output of 1, with the second weight being initialized to 0.5. However, (0, 1) yields 0 instead of 1, which is because the perceptron isn’t fully trained yet. In order to train it in its entirety, one can emplace the Inputs’ vector traversal loop inside another loop which breaks out when all the output values match up against their corresponding target values, as shown below:
#include <iostream>
#include <vector>
#define float F
int main()
{
std::vector<std::vector<float>> Inputs{
{0, 0}, {0, 1}, {1, 0}, {1, 1}
};
std::vector<F> Weights{0, 0}, Target{0, 1, 1, 1};
F Bias = 0, Alpha = 0.5, Yin, Yout;
int count;
// Looping till the perceptron is fully trained: (i.e., running indefinitely till it explicitly breaks on my condition)
while(1)
{ // Initializing count to 0 at the beginning of every iteration to avoid it possessing the value(s) of previous iterations:
count = 0;
for(int i = 0; i < Inputs.size(); i++)
{
Yin = Inputs[i][0] * Weights[0] + Inputs[i][1] * Weights[1] + Bias;
Yout = (Yin > 0) ? 1 : 0;
count = (Yout == Target[i]) ? 1 : 0;
// Changing weights in proportion to the learning rate, with changes reflecting in the bias as well:
else
{
Weights[0] += Alpha * Target[i] * Inputs[i][0];
Weights[1] += Alpha * Target[i] * Inputs[i][1];
Bias += Alpha * Target[i];
}
std::cout << "Values of: Inputs = (" << Inputs[i][0] << ", " << Inputs[i][1] << "), Yin = " << Yin << ", Yout = " << Yout << ", Weight 1 = " << Weights[0] << ", Weight 2 = " << Weights[1] << std::endl;
// When the value of count reaches four (size of our Target vector), I should have accurate values for my weights and hence I exit the loop:
if(count == Target.size())
{
std::cout << "Perceptron model for OR gate is fully trained with correct weights.";
exit(0);
}
}
}
return 0;
}
This brings the values of my weights to (1, 1) which satisfies (Input_one) * (weight_one) + (Input_two) * (weight_two) = Yout for my OR gate:
| Input_one | Input_two | Yout |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
| Anirban | 02/29/2020 |