IAM

DAVIDSTUTZ

I am looking for full-time (applied) research opportunities in industry, involving (trustworthy and robust) machine learning or (3D) computer vision, starting early 2022. Check out my CV and get in touch on LinkedIn!

ARTICLE

ArXiv Pre-Print “Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators”

Deep neural network (DNN) accelerators are popular due to reduced cost and energy compared to GPUs. To further reduce energy consumption, the operating voltage of the on-chip memory can be reduced. However, this injects random bit errors, directly impacting the (quantized) DNN weights. As result, improving DNN robustness against these bit errors can significantly improve energy efficiency. Similarly, these chips are subject to bit-level hardware- or software-based attacks. In this case, robustness against adversarial bit errors is required to improve security of DNN accelerators. Our paper presented in this article addresses both problems.

Abstract

Figure 1: Left: Average bit error rate $p$ (blue, lef y-axis) measured on 32 SRAM arrays of size $512\times 64$ and energy (ref, right y-axis) vs. voltage (x-axis). Reducing voltage reduces energy consumption, bit introduces exponentially increases rates of bit errors. Middle: Robust test error RErr after injecting random bit errors plotted against bit error rate. Our robust quantization (red), combined with (per-layer) weight clipping (blue) and random bit error training (violet) improves robustness considerably. Right: RErr against up to $320$ adversarial bit errors, with adversarial bit error training reducing RErr from above $90\%$ to $26.22\%$.

Deep neural network (DNN) accelerators received considerable attention in recent years due to the potential to save energy compared to mainstream hardware. Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights. Furthermore, DNN accelerators have been shown to be vulnerable to adversarial attacks on voltage controllers or individual bits. In this paper, we show that a combination of robust fixed-point quantization, weight clipping, as well as random bit error training (RandBET) or adversarial bit error training (AdvBET) improves robustness against random or adversarial bit errors in quantized DNN weights significantly. This leads not only to high energy savings for low-voltage operation as well as low-precision quantization, but also improves security of DNN accelerators. Our approach generalizes across operating voltages and accelerators, as demonstrated on bit errors from profiled SRAM arrays, and achieves robustness against both targeted and untargeted bit-level attacks. Without losing more than 0.8%/2% in test accuracy, we can reduce energy consumption on CIFAR10 by 20%/30% for 8/4-bit quantization using RandBET. Allowing up to 320 adversarial bit errors, AdvBET reduces test error from above 90% (chance level) to 26.22% on CIFAR10.

Paper on ArXiv
@article{Stutz2021ARXIV,
    author    = {David Stutz and Nandhini Chandramoorthy and Matthias Hein and Bernt Schiele},
    title     = {Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators},
    journal   = {CoRR},
    volume    = {abs/2104.08323},
    year      = {2021}
}

What is your opinion on this article? Did you find it interesting or useful? Let me know your thoughts in the comments below: