One-Pixel Attack

As A.I. becomes more ubiquitous in our lives, it’s important to learn about its limitations and vulnerabilities. A.I. systems can be hacked.

Today, I learned about a particularly devious attack.

It’s possible to change just one pixel in an image in a way that is difficult for humans to detect; however, this one-pixel change can cause deep neural networks to misclassify the entire image.

Here’s a link to the paper that shows how it works. While published in 2019, this is the first I’ve learned about it.

The approach uses a Differential Evolution (DE) algorithm to find the smallest change needed to fool a deep neural network that classifies images.

Evolution in nature depends on random mutations to our DNA. When a mutation, or a series of mutations, leads to a competitive advantage, it is naturally selected.

The researchers do something similar. They use a “fitness function” to determine the efficacy of an attack that changes the RGB values of a pixel. If a random change makes the image more difficult to classify, it “survives”. Calculus lets them find the minimum change needed to the image to fool the deep neural network.

I can imagine a disturbing future war in which drones are used to destroy enemy targets. If the enemy manages to hack the drone’s image-detecting software, the targets could become invisible to the drone.

Refer to the following article for another explanation of the phenomenon.