Penetration Testing: Part 2
In my previous blog entry I pointed out differences between vulnerability scanning and penetration testing and explained why penetration testing is a potentially more useful and valuable activity to an information security practice. But alas, some penetration tests are done better than others in that some are more systematic and thorough. One of the problems with penetration testing today is that there are no widely accepted standards for conducting penetration tests. Some individuals and organizations that conduct such tests fall far short of being very systematic and complete. One of the most common shortfalls in penetration testing is conducting penetration tests that target only certain vulnerabilities, e.g., vulnerabilities that tools such as an out-of-the-box version of Metasploit will target, but not others that could have been targeted had additional scripts been added. Penetration testers may find that a number of exploit scripts work, but will not even attempt to determine whether other vulnerabilities are present and can be exploited. In this case the write-up of the penetration test results are likely to indicate that certain vulnerabilities can be exploited and even that unauthorized root access has been obtained because of them. The tester has in all likelihood done a great disservice to the customer, however, in that no mention of the tests that were not conducted, but that should have been conducted will appear in the write-up that follows testing. The customer will probably mitigate the most serious vulnerabilities and then assume that all is well as far as vulnerability management goes.
Another problem with penetration testing occurs when the penetration tester is unsuccessful in exploiting any vulnerability, so this person turns to social engineering as a last resort. I do not really have an objection if social engineering is specified as an allowable or even required activity, i.e., the statement of work requires that social engineering also be conducted as part of penetration testing activity. But being unable to exploit any vulnerabilities through technical means and then without advance permission tricking one or more users into doing or saying something that compromises security is hardly any kind of triumph at all. Remember, research studies indicate that somewhere around 70 percent of users succumb to social engineering attacks. So all a customer who orders a penetration test in which penetration testers are unable to successfully exploit any technical vulnerabilities, but then is told that one or more users fell for a social engineering ploy has paid good money to learn the obvious. The customer would have done better to use part of the money used in conducting the penetration test to instead provide anti-social engineering training for users.
Another all-too-frequent problem associated with penetration tests is poorly written write-ups after testing has been completed. Let’s face it–a fairly large percentage of the most brilliant technical staff cannot write very well. Write-ups may thus be incomplete and/or difficult to understand. Some write-ups also suggest only one mitigation measure for each exploited vulnerability, again a great disservice to the customer. An effective write-up will propose alternative solutions that are available, and will provide recommendations concerning the effectiveness and cost-benefit ratio of each solution.
All penetration testing is not equal. Some providers go farther than others, yielding information and output that is of genuine value to customers. Once again, you get what you pay for.