Expanding Reputation: Introducing Signal and Impact

Edited on 4/8/2016 to reflect the latest Signal and Impact implementations.

We first introduced Reputation in October 2014 to provide additional recognition to the best researchers, and to highlight quality reports to security teams. Each researcher has a Reputation based on their track record on HackerOne. Reputation has been continuously improved since its introduction and has become integral to the platform.

To build upon its usefulness as a ranking system, we launched two new dimensions to better show how each researcher’s Reputation has been achieved: Signal and Impact.

Why We’re Improving Reputation

The existing Reputation system is effective at measuring a consistent track record of valid results, serving as an accurate ranking function. However, it occasionally obscured quality over quantity. The introduction of Signal and Impact helps further highlight quality performance.

New Dimensions: Signal and Impact

Signal is the average Reputation per report.
Reputation is gained or lost each time a report is closed, making Signal an aggregate representation of report validity.

Impact is the average Reputation per bounty.
Reputation is gained based on the relative size of the awarded bounty, making Impact an aggregate representation of report severity.

• Signal and Impact are only calculated when there are more than 3 closed reports or 3 bounties, respectively.
• Signal is measured on a scale from -10 to 7, corresponding to Reputation changes for triage states ranging from “Spam” to “Resolved.” Self closed reports and Duplicate reports are excluded.
• Impact is measured on a scale from 0 to 50, corresponding to Reputation gains calculated by bounty levels awarded.
• Impact is only calculated for reports submitted to programs that offer bounties. We also display the percentile rank for Signal and Impact, relative to other researchers on the platform.

Examples

Alice and Bob have the exact same amount of Reputation (241). But Alice has a higher Signal (4.12 > 1.35), indicating a better track record of submitting valid reports. Alice also has a higher Impact (15.83 > 9.50) and therefore, a higher chance of submitting an important report.

This is valuable information for both of them. Alice will see that she is doing quite well with her current trajectory, while Bob will see that others are more accurate about their reports, and can adjust his reporting processes accordingly.

Conclusions

For a response team, Signal provides a means of identifying researchers who have had consistently valid reports while Impact highlights those with the greatest severity. For researchers, we hope both Signal and Impact will help you benefit through an improved understanding of your performance relative to your peers. In the coming months, we’ll be incorporating Signal and Impact into other visible areas of HackerOne, such as the invitation system. We hope you’ll find them useful and as always, we’d love to hear your thoughts.

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!