Cormac Herley, Paul C. van Oorschot. SoK: Science, Security and the Elusive Goal of Security as a Scientific Pursuit. IEEE Symposium on Security and Privacy 2017.
Herley and van Oorschot explore how to make security research more scientific. In particular, they discuss different historic notions of what “scientific” means and related these insights to current practices in security research. I want to discuss only two points that I found very insightful. First, there seems to be a misalignment between formal methods, and empirical methods. While some researchers argue for more mathematically verifiable security methods, others claim that attackers do not care about mathematical proofs – and even provably secure systems can be implemented insecurely. And second, security is often based on unfalsifiable claims. This is problematic, as research findings that cannot be refuted by any observable event are generally assumed to be “unscientific”. In security, however, it can easily be shown if a system/method is insecure, while there is no possible observation allowing to determine security.
Herley and van Oorschot explore how to make security research more scientific. In particular, they discuss different historic notions of what “scientific” means and related these insights to current practices in security research. I want to discuss only two points that I found very insightful. First, there seems to be a misalignment between formal methods, and empirical methods. While some researchers argue for more mathematically verifiable security methods, others claim that attackers do not care about mathematical proofs – and even provably secure systems can be implemented insecurely. And second, security is often based on unfalsifiable claims. This is problematic, as research findings that cannot be refuted by any observable event are generally assumed to be “unscientific”. In security, however, it can easily be shown if a system/method is insecure, while there is no possible observation allowing to determine security.