unit 1.3 - XNOR in neural netsο
We will take a closer look at the XNOR problem in neural networks. This was a suggested homework from some of the lectures in this module! This is the solution.
What is XNOR?ο
The XNOR boolean function is implemented by the following truth table:
X1 |
x2 |
x1 XNOR x2 |
|---|---|---|
0 |
0 |
1 |
0 |
1 |
0 |
1 |
0 |
0 |
1 |
1 |
1 |
How do we implement this with neural networks?ο
When neural networks were first invented, people used single neurons to implement the AND, OR, NOT boolean functions, bu they had trouble implementing the XOR And XNOR functions. Why?
In order to implement this function, let us see where the XNOR 1s are⦠we can express this XNOR using NOT, AND, OR:
\(XNOR(x1,x2) = x1*x2 + NOT(x1)*NOT(x2)\)
Note that \(x1*x2\) = AND(x1,x2) and that \(NOT(x1)*NOT(x2)\) = AND(NOT(x1), NOT(x2))
Now: we know how to implement AND and OR functions with single neurons. How can we implement the entire XNOR?
We will need 2 layers - one implementing both \(x1*x2\) (AND) and \(NOT(x1)*NOT(x2)\), and one final layer to combine them (OR them).
The number in parentheses near the neurons are the bias values. All neurons use the ReLU non-linearity. Inputs are 0 or 1 values. Output are 0 (logic 0) or > 0 (logic 1).
These are the equation from each of the neurons:
Neuron 1 (N1) - \(AND(x1,x2)\): \(N1_{output} = ReLu(x1*1 + x2*1 - 1.5)\)
Neuron 2 (N2) - \(NOT(x1)*NOT(x2)\): \(N2_{output} = ReLu(x1*-1 + x2*-1 + 0.5)\)
Neuron 3- OR: \(OR_{output} = ReLu(N1_{output}*1 + N2_{output}*1 - 0.5)\)
HOMEWORKο
Try to implement the XOR function.