Home > Uncategorized > A Kudo for NSS Labs

A Kudo for NSS Labs

It is likely that you already know about NSS Labs. If you don’t, you can find out about them by visiting www.nsslabs.com. This organization was created nearly five years ago to perform independent evaluation and certification of information security products. Products that meet NSS Labs’ standard criteria are approved, and ones that meet even higher criteria are awarded the NSS Labs Gold status. NSS Labs is not the only entity that does testing of this nature. What is different about this organization is that the testing is truly independent; it does not cost anything to participate in the testing. The vast majority of NSS Labs’ revenue is instead from vendor certification, although because group testing started just last year, their revenue related to testing (e.g., having vendors pay for rights to distribute the NSS group test executive Summaries) is likely to grow substantially. NSS Labs also makes revenue from evaluation reports that it produces after testing for a particular type of product is complete. NSS Labs then sells these reports.

I admit that I am biased in that I know and very much like the NSS Labs CEO and President, Rick Moy. I got to know him when I consulted for High Tower Software (before I was hired as the CTO there). Rick was the product manager, and I thought he did a very fine job in this role. He not only wrote a product requirements document that clearly stated the functionality that the High Tower SIEM tool needed, but along with Antonio Bianco also had a major hand in designing the user interface. Over the years, this product’s user interface was one of the best in any security product, and Rick had a lot to do with this accomplishment.
NSS Labs’ testing has produced some extremely useful results. Recently NSS Labs found that only one of the anti-virus software products included in the testing was able to discover and eradicate a new mutation of a virus. I’m not surprised by these results—I lost my confidence in anti-virus software a long time ago (even though I still run it on all my Windows hosts). NSS Labs also tested major intrusion prevention tools both in a tuned and untuned state. Regardless of state, Sourcefire outperformed all other products with a block rate of approximately 90 percent of all malicious traffic when tuned. What shocked me is that other, better known and more highly touted products had a rather dismal block rate—one made by a very well known vendor scored only approximately 17 percent, regardless of whether it was tuned. In fact the overall block rate for all intrusion prevention products tested was not really all that impressive. Anyone who uses or is considering using intrusion prevention technology ought to read NSS Labs’ report—it certainly does not (with Sourcefire excepted) bolster my confidence in this technology.

NSS Labs also tested how effective browsers are in resisting phishing-based malware infections. Surprisingly, Internet Explorer 8 turned out to be the best with an 85 percent effectiveness score. Second best was Mozilla Firefox with a rating of only 29 percent. Other browsers fared even worse. Microsoft must have jumped for joy—deservedly, especially considering that earlier versions of IE were as riddled with holes as Swiss cheese! The bad news here once again is that most products tested did not score very well.

I’ve worked in the security software arena and have seen and heard many audacious and unbacked claims. Gartner does not really help in discovering the truth (or lack thereof) concerning these claims. Gartner does not even test products, and worse yet, it bases its Magic Quadrant placements for products on only two criteria—market penetration and visionary approach. Many times I have seen virtually worthless products placed in the top quadrant simply because the vendor is large enough to virtually give the product away when someone buys other of this vendor’s products and because some product manager for this vendor pulls the proverbial wool over a Gartner analyst’s eyes concerning mundane features that are only by a stretch of one’s imagination visionary. In contrast, NSS Labs provides an invaluable service by objectively and systematically testing products according to sound criteria and then making the results, good or bad, available to everyone.

Keep it up, Rick, and keep it up NSS Labs. You are doing what the security community really needs and you are doing it very well!

Categories: Uncategorized Tags:
  1. No comments yet.
  1. No trackbacks yet.
You must be logged in to post a comment.