Penthera and 25 free security tools, on four public benchmarks
Penthera and 25 free open-source security tools, scored the same way on four public datasets other people built. On the sealed half of each, 2nd of 10 on leaked secrets; 1st of 6 on malicious npm packages; 1st of 8 on malicious pypi packages; 1st of 8 (within the margin of error) on injection bugs in javascript libraries. Measured on 25 October 2026.
How does Penthera rank against the free tools?
| Benchmark | Penthera's rank | Penthera's score | Top other free tool | Scored by |
|---|---|---|---|---|
| Leaked secrets Samsung CredData | 2nd of 10 | 0.74 | deepsecrets 0.78 | F1 |
| Malicious npm packages DataDog's dataset | 1st of 6 | 0.71 | malcontent 0.63 | caught minus false alarms |
| Malicious PyPI packages DataDog's dataset | 1st of 8 | 0.82 | GuardDog 0.74 | caught minus false alarms |
| Injection bugs in JavaScript libraries SecBench.js | 1st of 8, within the margin of error | 0.70 | odgen 0.68 | share found in the right file |
- Free secret scanners on Samsung CredData: 10 tools rankedPenthera 2.0.0 ranked 2nd of 10 free tools by F1 (0.74), behind deepsecrets (0.78).
- Free npm malware detectors on DataDog's dataset: 6 tools rankedPenthera 2.0.0 ranked 1st of 6 free tools by caught minus false alarms (0.71); next, malcontent (0.63).
- Free PyPI malware detectors on DataDog's dataset: 8 tools rankedPenthera 2.0.0 ranked 1st of 8 free tools by caught minus false alarms (0.82); next, GuardDog (0.74).
- Free JavaScript static analyzers on SecBench.js injection bugs: 8 tools rankedPenthera 2.0.0 ranked 1st of 8 free tools by share found in the right file (0.70), within the margin of error of odgen (0.68).
How the benchmarks were run
Each dataset was split in half before any tuning. Penthera was improved on one half, the dev half, and scored once on the other, sealed half. The scoring and ranking rules were registered before that run: the protocol. Every tool ran offline, on its defaults. A rank is given only where at least five other free tools were measured.
Paid tools (GitGuardian, Socket, Snyk, Phylum, Semgrep Pro) are not here. None publishes results on these benchmarks; we cannot measure them (cost and licence terms).
Check it yourself
Every number on these pages is read from published.json when the site is built. The results file has the method, the leakage audit and per-family detail; the runner's README says how to reproduce each run. The bundled benchmark in bench/RESULTS.md is different: small labs we wrote ourselves, which catch regressions and do not measure accuracy.
Measured on . Sealed test half, scored once (E11b § 10).
penthera.org · Benchmarks · Blog · Live check · Terms · Trust · Source · MIT license