Evidence-based cybersecurity policy? A meta-review of security control effectiveness

Figures & data

Table 1. The studies we collected via literature search.

Study (Year)	Outcome variable	Sample window		Type of Measurement
		Start	End
Industry
Marsh (2023)	claims	2010	2023	questionnaire
GallagherRe (2022)	claims	2021	2023	scans
AtBay (2023)	claims	June 2018	June 2022	mixed
Coalition (2023)	claims	January 2022	January 2023	scans
BitSight and Marsh (2022)	claims	2018	2021	scans
BitSight (2023)	incidents	Nov 2018	June 2021	scans
SecurityScorecard and Marsh (2022)	incidents	2019	2021	scans
Cisco (2021)	questionnaire	2021	2021	questionnaire
Arete and Cyentia (2022)	incidents	May 2019	December 2021	investigation
Meyer et al. (2023)^†	inbox activity	April 2022	September 2022	system data
IBM (2023)	incident cost	2022	2022	questionnaire
Academic
Gandal et al. (2023)	questionnaire	July 2020	March 2021	questionnaire
Doerfler et al. (2019)^‡	inbox activity	June 2017	May 2018	system data
Nagle, Ransbotham, and Westerman (2017)	security events	July 2014	April 2015	scans
Edwards, Jacobs, and Forrest (2019)	botnet infections	January 2015	January 2016	scans
Tajalizadehkhoob et al. (2017)	network abuse	March 2016	March 2016	scans
Bilge, Han, and Dell’Amico (2017)	device infection	Mid 2015	End of 2015	AV data
DeKoven et al. (2019)	device infection	June 2018	December 2018	network flow

Note: ^†all authors are Microsoft employees. ^‡two of the seven authors are Google employees.

Marsh. 2023. “Using Data to Prioritize Cybersecurity Investments.” Accessed 6 June 2023. https://www.marsh.com/us/services/cyber-risk/insights/using-cybersecurity-analytics-to-prioritize-cybersecurity-investments.html.

 Google Scholar

GallagherRe. 2022. “External Scanning for Insurance.” Accessed 6 June 2023. https://www.actuaries.org.uk/system/files/field/document/B1.

 Google Scholar

AtBay. 2023. “Ranking Email Security Solutions a Data Analysis of Cyber Insurance Claims.” Accessed 27 February 2023. https://www.at-bay.com/ranking-email-security-solutions.

 Google Scholar

Coalition. 2023. 2023 Cyber Claims Report. Accessed 6 June 2023. https://info.coalitioninc.com/download-2023- cyber-claims-report.html.

 Google Scholar

BitSight and Marsh. 2022. “Make Better Cybersecurity Decisions with Trusted Data Analytics.” Accessed 6 June 2023. https://www.bitsight.com/press-releases/study-finds-significant-correlation-between- bitsight-analytics-and-cybersecurity.

 Google Scholar

BitSight. 2023. “Evidence-based Strategies to Lower Your Risk of Becoming a Ransomware Victim.” Accessed 6 June 2023. https://www.bitsight.com/blog/ransomware-prevention.

 Google Scholar

SecurityScorecard and Marsh. 2022. “Reduce Cyber Risk with the Predictive Power of Security Ratings.” Accessed 6 June 2023. https://resources.securityscorecard.com/cyber-insurance/reduce-cyber-risk-marsh- mclennanpage=1.

 Google Scholar

Cisco. 2021. Security Outcomes Study, Vol. 1. Accessed 6 June 2023. https://www.cisco.com/products/security/security- outcomes-report-vol-1.html.

 Google Scholar

Arete and Cyentia. 2022. “Reining in Ransomware.” Accessed 6 June 2023. https://www.cyentia.com/wp- content/uploads/Arete-Reining-In-Ransomware-1.pdf.

 Google Scholar

Meyer, L. A., S. Romero, G. Bertoli, T. Burt, A. Weinert, and J. L. Ferres. 2023. How Effective is Multifactor Authentication at Deterring Cyberattacks?” arXiv preprint arXiv:2305.00945.

 Google Scholar

IBM. 2023. “Cost of a Data Breach Report.” Acceessed 27 February 2023. https://www.ibm.com/reports/data-breach.

 Google Scholar

Gandal, N., T. Moore, M. Riordan, and N. Barnir. 2023. “Empirically Evaluating the Effect of Security Precautions on Cyber Incidents.” Computers and Security 103380.

 Google Scholar

Doerfler, P., K. Thomas, M. Marincenko, J. Ranieri, Y. Jiang, A. Moscicki, and D. McCoy. 2019. “Evaluating Login Challenges as a Defense Against Account Takeover.” The World Wide Web Conference, 372–382. https://doi.org/10.1145/3308558.3313481.

 Google Scholar

Nagle, F., S. Ransbotham, and G. Westerman. 2017. “The Effects of Security Management on Security Events.” In Workshop on the Econ. of Information Security.

 Google Scholar

Edwards, B., J. Jacobs, and S. Forrest. 2019. “Risky Business: Assessing Security with External Measurements.” arXiv. http://arxiv.org/abs/1904.11052.

 Google Scholar

Tajalizadehkhoob, S., T. Van Goethem, M. Korczynski, A. Noroozian, R. Böhme, T. Moore, W. Joosen, and M. van Eeten. 2017. “Herding Vulnerable Cats: A Statistical Approach to Disentangle Joint Responsibility for Web Security in Shared Hosting.” In Proceedings of th Conference on Computer and Communications Security, 553–567. ACM.

 Google Scholar

Bilge, L., Y. Han, and M. Dell’Amico. 2017. “Riskteller: Predicting the Risk of Cyber Incidents.” In Proceedings of the Conference on Computer and Communications Security, 1299–1311. ACM.

 Google Scholar

DeKoven, L. F., A. Randall, A. Mirian, G. Akiwate, A. Blume, L. K. Saul, A. Schulman, G. M. Voelker, and S. Savage. 2019. “Measuring Security Practices and how They Impact Security.” In Proceedings of the Internet Measurement Conference, 36–49. ACM.

 Google Scholar