Why is pentesting still a customer requirement?

It always bothered me that pentesting often shows up as a security requirement on software contracts. Why? Because rather than focusing on a desired outcome, it mandates an output. It's like enforcing the amount of hours your employees work rather than focusing on productivity and work quality.

As with most things, the answer as to whether or not a pentest makes sense depends on your goals and motivations. From the perspective of a business-aligned security program that optimizes your available resources, it's crucial to start by defining the primary goal of why you're testing.

Many companies conduct pentests simply because of the aforementioned customer requirement rather than because of any risk-driven motivation. While certainly a valid reason, doing it to check a box often leads to a different outcome than if you are trying to actually assess the security of your platform.

Pentests shine when you have new or reworked services where vulnerabilities have the potential to be disastrous. In this case, the assurances from having a highly skilled team digging into a focused area can be invaluable. Reworked authorization models, functionality to run user provided code or new complex business logic are all areas where vulnerabilities could easily spiral into disaster, making a thorough test easily justifiable. These targeted pentests can provide strong assurances around the security of the new system before you set it loose into the world.

On the other hand, many pentests end up as very different engagements from the above. The average “checkbox” pentest sees the customer give the testers access and they'll poke at the platform for some amount of time once a year, delivering your annual report with some token informational and low severity findings that keep everyone happy. While this approach will satisfy a contractual requirement, it fails to fulfill what was likely the desired outcome of the original agreement - to ensure active testing of the software’s security posture.

A complementary alternative

Given the sheer size and complexity of modern cloud delivered software, the realities of available time to test means that generalized pentests aren’t as likely to find critical vulnerabilities compared to continuous and/or targeted testing. This is where well managed bug bounty programs can shine.

While they require more time and involvement from internal teams, the model of paying for verified vulnerabilities in your predefined scope puts the incentives in the right place to drive results. The ongoing nature of bounties then provides far more assurances that you're getting a clear picture of your application's security posture over time. Bug bounty programs tap into the collective expertise of security researchers worldwide, providing continuous testing against the latest threats and techniques, occurring over a far longer span of time. This allows testers to become much more familiar with the quirks and nuances of your platform and enabling higher quality results as time goes on.

For the types of outcomes that contracts are actually looking for - finding and fixing vulnerabilities on your platform - bounty programs tend to be both more thorough as well as more cost-effective. With a pentest, you're paying for some amount of hours, regardless of the results. With a bug bounty program, you pay for verified vulnerabilities, ensuring that with every additional payout, the needle is moved. Funded sufficiently, bounty programs incentivize researchers to dig deeper and find the more complex vulnerabilities that might be missed in a time-boxed pentest engagement. You also have the option to raise bounties for a specific type of findings or in a specific area to further align incentives with desired outcomes.

You'll likely still need the pentest to close deals, but by focusing on desired outcomes rather than outputs, rather than spending significant budget on a large test from a big name, you can get a much smaller scoped test from a lesser-known vendor for a fraction of the cost. This lets you check the compliance box, allocating the budget difference towards bounties, providing thorough, ongoing testing of your platform's security rather than a point in time snapshot.

Used wisely, pentests and bug bounty programs are complementary but for most software companies looking for a generalized security test of their platform, it's hard to beat the value a well structured bounty program brings. By balancing targeted pentests with a robust bug bounty program, you’re able to achieve a comprehensive and cost-effective security testing strategy that uses the right tool for the right problem, staying ahead of emerging threats and maintaining a strong security posture over time.