Quantifying the Value of Bug Bounty Programs: ROI, ROM, or Both?

HackerOne customers consistently factor in cost savings when measuring the success of their security engagements, with 59% valuing the estimated savings of reputational or customer-related incidents and 54% valuing the financial savings estimated from avoiding risk. However, quantifying the ROI for security control testing can be challenging due to the intangible nature of cybersecurity benefits. How do you measure the value of preventing something from happening?

Supplementing ROI With ROM

Traditional ROI calculations often fall short in capturing the full value of security investments. Gaining traction as an alternative, and in most cases, complementary, assessment mechanism is Return on Mitigation (ROM), which compares the anticipated costs of a security breach with the costs of implementing mitigation strategies. It provides a more nuanced understanding of the qualitative and quantitative benefits of proactive security measures. ROM factors in various potential costs, including:

• Restoring compromised systems
• Lost revenue due to downtime
• Legal and regulatory penalties
• Damage to public trust and reputation

By assessing the effectiveness of mitigation or prevention strategies in terms of potential financial consequences, ROM offers a practical framework for stakeholders to evaluate the tangible and intangible value of security investments. It also shifts the focus from immediate cost savings to long-term resilience, with a magnifying glass on risk management.

“The bug bounty program is the highest ROI across all of our spend. It’s really hard to show ROI, but with bug bounty, I have a baseline. I can say, ‘This vulnerability was able to be found by someone outside the organization. Someone that was not authorized to access this system was able to access it.’ Even with vulnerabilities that are not within our program, bug bounty allows me to put a price tag on them. I can explain this business case and our stakeholders are able to prioritize bug bounty higher than other tools that also generate ROI.”
— Eric Kieling, Head of Application Security, Booking.com

ROI and ROM Calculations

To illustrate the practical benefits of human-powered security testing and calculations of ROI and ROM, consider a case study from the financial services sector. A major financial institution implemented a bug bounty program alongside its existing red teaming efforts. Over the course of a year, the program identified several critical vulnerabilities that had been overlooked by previous tests.

Scenario

• Initial security investment: The institution invested $200,000 in the bug bounty program and an additional $100,000 in red teaming exercises.
• Potential breach costs: A potential breach was estimated to cost the institution $5 million, including costs associated with restoring compromised systems, lost revenue, legal penalties, and reputational damage.

Return on Investment (ROI)

A simple ROI calculation looks at the return of $300,000 against a potential $5 million breach.

• Breach prevention: By identifying and mitigating vulnerabilities, the institution avoided a potential $5 million breach.
• Cost of testing: The total investment in proactive security testing was $300,000.

Using Traditional ROI Calculations

Traditional ROI or cost-benefit analyses yield approximately $15.67 in ROI.

If we look at ROM, we compare the cost of implementing security measures against the anticipated breach cost.

In this scenario, the ROM indicates that for every dollar spent on mitigation, the organization potentially saves $16.67 in breach costs. For the sake of this case example, we kept these costs simple. However, it is important to remember that breach costs these days include much more than just a simple flat dollar amount. They also include potential ransom payments, compliance requirements, regulatory fines, legal fees, brand damage, and much more. Breaches in the financial services sector, for example, cost an average of $6.08 million.

Real-World ROM

According to the 7th Annual Hacker-Powered Security Report, the median price of a bug on the HackerOne platform is $500, up from $400 in 2022. The average bounty in the 90th percentile is up from $2,500 to $3,000. But here’s the dramatic reality: the cost of these vulnerabilities going unnoticed and being exploited in the wild is an overwhelming 1,600 times more than the cost of the bounty — $4.88M on average.

“Since 2019, Zoom has worked with 900 hackers, of which 300 have submitted vulnerabilities that we have had to quickly move on. We’ve paid out over $7 million. It’s a substantial investment but the returns are worth it: we find world-class talent to find real-world solutions before it’s a real-world problem.”
— Michael Adams, CISO, Zoom

Deliver Strategic Value From Security Initiatives With HackerOne

At HackerOne, we’re not only the leader in high-quality, repeatable security engagements — we’re also the experts in helping partners quantify and qualify the value of those engagements for more robust security budgets and successful programs. To learn more about ROI and ROM and the best ways to express the value of proactive security to stakeholders with human-powered security, download the SANS White Paper: Human-Powered Security Testing.