ArXiv Pre-Print “On Mitigating Random and Adversarial Bit Errors”

Deep neural network (DNN) accelerators are specialized hardware for inference and have received considerable attention in the past years. Here, in order to reduce energy consumption, these accelerators are often operated at low voltage which causes the included accelerator memory to become unreliable. Additionally, recent work demonstrated attacks targeting individual bits in memory. The induced bit errors in both cases can cause significantly reduced accuracy of DNNs. In this paper, we tackle both random (due to low-voltage) and adversarial bit errors in DNNs. By explicitly taking such errors into account during training, wecan improve robustness significantly.


Figure 1: Left: Exemplary SRAM bit error rate $p$ and normalized energy per SRAM access when scaling voltage below $V_{\text{min}}$, the minimum voltage for reliable operation. Middle: Original weight value, quantized with 16 bits in $[-0.15, 0.15]$ on CIFAR10, plotted against the weight with random bit errors of rate $p = 1\%$. 5.5M weights are shown. Color indicates error magnitude (from violet, zero to roughly 0.225 in yellow). Right: Impact of bit errors on test error for CIFAR10. Reducing the quantization range to $[-0.15,0.15]$ (clipping) and random bit error training (RandBET) clearly improves robustness.

The design of deep neural network (DNN) accelerators, i.e., specialized hardware for inference, has received considerable attention in past years due to saved cost, area, and energy compared to mainstream hardware. We consider the problem of random and adversarial bit errors in quantized DNN weights stored on accelerator memory. Random bit errors arise when optimizing accelerators for energy efficiency by operating at low voltage. Here, the bit error rate increases exponentially with voltage reduction, causing devastating accuracy drops in DNNs. Additionally, recent work demonstrates attacks on voltage controllers to adversarially reduce voltage. Adversarial bit errors have been shown to be realistic through attacks targeting individual bits in accelerator memory. Besides describing these error models in detail, we make first steps towards DNNs robust to random and adversarial bit errors by explicitly taking bit errors into account during training. Our random or adversarial bit error training improves robustness significantly, potentially leading to more energy-efficient and secure DNN accelerators.

Paper on ArXiv
    author    = {David Stutz and Nandhini Chandramoorthy and Matthias Hein and Bernt Schiele},
    title     = {On Mitigating Random and Adversarial Bit Errors},
    journal   = {CoRR},
    volume    = {abs/2006.13977},
    year      = {2020}
What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.