HackerOne Reputation – Increasing Valid Bug Submissions and Signals

Edited on 12/11/2015 to reflect the latest Reputation implementation.

One of the primary challenges when running a vulnerability coordination program is distinguishing signal from noise. Our former colleagues at Facebook evaluate over 20 invalid submissions for each valid report – that’s only 4.6% signal! The programs hosted at HackerOne have fared a bit better: on average 19% of reports are valid, but some outliers deal with as low as 6%. This noise is undesirable for everyone, driving up response time, introducing unnecessary latency in resolving security issues, and increasing the likelihood that valuable signal will get lost.

The HackerOne vulnerability coordination platform provides security teams with the tools to overcome these challenges. Today, we’re turning a beta feature live for everyone: a new reputation system that makes running a program even easier. This system gives additional recognition to the best researchers while more quickly surfacing quality reports to security teams.

The vast majority of security researchers generate reports of consistently high quality. Our analysis shows that the remaining noise stems from a classic tragedy of the commons: a minority of researchers with low confidence submissions hoping to stumble upon a success. But many of these researchers learn and improve with each failed attempt. The challenge for security teams is that past performance is not indicative of future results.

This new reputation system will help provide security teams with the means to more effectively act upon the invaluable information shared by the security community. As a researcher submitting vulnerabilities through the HackerOne platform, your reputation measures how likely your finding is to be immediately relevant and actionable. Reputation is gained or lost based exclusively upon your track record as a researcher.

How it works

You gain reputation when:

• Your report is Closed as Resolved: +7
• Your report is Closed as Duplicate (Resolved): +2. Only applied if reported before the original was closed.
• You are awarded a bounty. The amount is based on standard deviation from the program’s mean:
  • +50: $ >= µ + 1σ
  • +25: $ > µ
  • +15: $ >= µ – 1σ
  • +10: $
• Your report is Closed as Informative: 0
• Your report is Closed as Duplicate (Informative): 0

You lose reputation when:

• Your report is Closed as Not Applicable: -5
• Your report is Closed as Duplicate (Not Applicable): -5
• Your report is Closed as Duplicate (Resolved and Public at time of submission): -5

Other details:

• We believe everyone deserves the benefit of the doubt. Researchers begin with a reputation of 100. Reputation cannot decrease below 0.
• You’ll always have access to a detailed log of reputation history.
• Reputation will never be necessary to access core functionality on the platform to ensure it remains accessible to new or anonymous users.
• Our approach draws inspiration from two communities we’re fond of.

The most visible manifestation of the new reputation system will be its usage as a ranking mechanism for the many Thanks pages on the platform. In the near future, we’ll also be announcing a number of privileges that are gained by maintaining a high reputation, such as becoming eligible to receive invitations to private bounty programs.

Finally, and most importantly, we’re now tying our rate limiting system directly to reputation. Should your reputation decrease, the system will gradually reduce the number of submissions allowed in a given time period. We believe it is critical to this community that response teams be afforded a high-signal environment so that they can focus on providing a quality response to researchers who turn in the best vulnerabilities.

HackerOne is committed to empowering the world to build a safer Internet, and building the most useful platform for vulnerability coordination is central to that mission. We welcome feedback on this new reputation system, and we hope response teams and researchers will enjoy the benefits of even higher quality reports and faster response times.

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!