 |
| from here |
I imagine I've already done something like this before, but if people keep repeating the mistake of studying AV effectiveness by using VirusTotal then I guess this needs repeating too.
For those who aren't yet aware: VirusTotal is a service that allows you to submit suspected malware samples and have them scanned with over 40 different anti-malware products to see if any of those products thinks your sample really is something bad. Unfortunately, it's not feasible for VirusTotal to use all the detective capabilities of those anti-malware products, so if you try to study the effectiveness of anti-malware products by using VirusTotal your results are going to be profoundly misleading.
2 comments:
Profoundly? That's taking it a bit far, in my opinion. Most enterprises don't turn on many of those additional detection engines because of the high false positive rate. I work for Bit9, and part of our success thrives on the consistent failures of AV technologies, including the other detection features they have added. Our customers use Virus Total results as confirmation that a particular malware is not detected by their vendor, but has a high number of other vendors detecting it. If someone was trying to show one AV is better than another, then you're not doing it right. But if you index every file in your environment, look it up on VT, your eyes will be open to how porous AV technology is.
What's this? An employee of a company whose business model revolves around taking potshots at the entire AV industry (even as it subsumes them) thinks VirusTotal isn't all that bad a representation of AV detection capabilities?
Wow, what a surprise.
Wait, did you actually call them "additional detection engines"? I'm sorry, are you from the past? You do realize that a number of AV products now include functionality that is effectively a whitelist, much like the product your company peddles, don't you? AV products these days include preventative technologies that haven't the slightest thing to do with "detection".
Tell me something, though. If I look up all the files in my environment on VirusTotal as you suggest, how exactly is that going to show how porous AV technology is? The files in my environment are non-malicious. Most files in anyone's environment are non-malicious. VirusTotal isn't likely to report otherwise, and if it did it wouldn't be demonstrating porosity, quite the opposite in fact. You can't demonstrate porosity of detection when non-detection is the expected result.
This post by David Harley (http://www.welivesecurity.com/2013/01/03/imperva-virustotal-and-whether-av-is-useful/) probably has the most comprehensive list of articles debunking a VirusTotal-based study I've seen, including a link to a paper David co-authored with Julio Canto of VirusTotal. I encourage people to check it out.
Post a Comment