# Free npm malware detectors on DataDog's dataset: 6 tools ranked

> Which free tool catches the most malicious npm packages, with the fewest false alarms?

On a sealed half of DataDog's malicious npm packages (112 with a payload) and 150 popular benign packages, Penthera 2.0.0 ranked 1st of 6 free tools, with caught minus false alarms 0.71. Next was malcontent with 0.63. Measured on 25 October 2026.

## Read this before the numbers

- GuardDog's authors helped build this dataset, so GuardDog's catch rate is inflated by construction.
- osv-scanner looks packages up in a list of reports. It is shown, not ranked: a lookup of known packages is not detection.
- The benign side is popular packages, which are the easiest to keep quiet on. Your dependencies may be less tidy.
- The free tools ran on their defaults. Penthera was tuned on the other half of the dataset, then scored once on this half.
- A benchmark is not your code. Accuracy on real projects is experiment E1, pre-registered and not yet run.

## Every tool, with 95% intervals

Scored by caught (packages with a payload) minus false alarms (benign packages): Youden's J. Counts are found over total; intervals are 95% Wilson.

| Rank | Tool | Caught | False alarms | caught minus false alarms |
|---|---|---|---|---|
| 1 | **Penthera 2.0.0** | 0.71 [0.62, 0.78] (79/112) | 0.00 [0.00, 0.02] (0/150) | 0.71 |
| 2 | malcontent 1.27.1 (bf280ad6), yara-x 1.21.0 | 0.68 [0.59, 0.76] (76/112) | 0.05 [0.03, 0.10] (8/150) | 0.63 |
| 3 | GuardDog 3.2.0 | 0.62 [0.52, 0.70] (69/112) | 0.03 [0.01, 0.07] (4/150) | 0.59 |
| 4 | ossgadget 0.2.20 (414b6a59) | 0.94 [0.88, 0.97] (105/112) | 0.89 [0.83, 0.93] (133/150) | 0.05 |
| 5 | clamav 1.5.4 | 0.01 [0.00, 0.05] (1/112) | 0.00 [0.00, 0.02] (0/150) | 0.01 |
| 6 | yara yara-python 4.5.4 (YARA 4.5), YARA-Forge core 20261004 | 0.00 [0.00, 0.03] (0/112) | 0.00 [0.00, 0.02] (0/150) | 0.00 |
| – | Penthera 1.9.0 | 0.57 [0.48, 0.66] (64/112) | 0.00 [0.00, 0.02] (0/150) | 0.57 |
| – | osv-scanner 2.6.0 | 0.90 [0.83, 0.94] (101/112) | 0.01 [0.00, 0.04] (1/150) | 0.90 |

Not ranked: Penthera 1.9.0 (the previous release, for comparison); osv-scanner 2.6.0 (lookup of already-reported packages, not detection).

## Where another tool does better than Penthera

- Caught: ossgadget 0.94, Penthera 0.71. Overall, ossgadget scored 0.05.

## Sources

- Dataset: [DataDog malicious software packages dataset](https://github.com/DataDog/malicious-software-packages-dataset)
- Numbers: https://github.com/danoszz/penthera/blob/main/bench/public/published.json (measured 2026-10-25; Sealed test half, scored once (E11b § 10).)
- Protocol: https://github.com/danoszz/penthera/blob/main/docs/research/prereg/E11b-improvement-study.md
- All four benchmarks: https://penthera.org/benchmarks.md