At White Oak Security, we partner with companies across a wide range of industries, sizes, locations, etc. Many of these organizations have very mature information security programs, but many of them do not. That’s where we come in, we do not discriminate – if you are trying to improve your organization’s security and lower your risk, we are happy to work with you and your team to move things forward. But first, here’s some important info on some key points regarding the world of security testing.
This post is part one of an overview, non-technical series, where our goal is to help prepare you and your company to make decisions about what type of testing you need, how to optimize working with your security testing partner efficiently, what to expect during testing, and how to most effectively address post-testing items. We will cover the different types of security testing, defining common terms, and where the different types fit into your security program. I’ll try not to rant too much about some product vendors and low-end testing providers misleading clients on what a penetration test really is…
Types Of Security Testing
The first thing that we’re going to talk about is testing ‘types’ – these types just designate different breakpoints in what’s really a continuous testing model. In other words, all of the testing types we’re going to be discussing here are built on top of each other – each including and then improving on the one that came before…
Here’s my quick, but very important rant!
Pay attention, not just to the labels that any testing partner uses with their services, but to the descriptions of what’s involved in those services. The term ‘Penetration Test’ or ‘Pentest’ is now often used to describe something that’s just not a penetration test. You end up getting skewered when it comes time to provide your client, auditor, or board with the results of a penetration test, only to find out that what you got was a scan. Worse – you never realize your mistake and have a very false sense of security… This happens A LOT. It’s extremely frustrating. So please ask questions! Thanks. Rant over.
SCANS
A ‘Scan’ is a test that is performed using an automated scanning solution. There are automated tools that you (and testing firms) can utilize to scan your network, your applications, or your cloud environments. There are many tools on the market (and many open-source tools as well) and more are being added constantly.
Scans, due to the fully automated nature of the exercise, have their uses but are also very limited in what they can find and how nuanced they can be. This significantly limits their effectiveness. This is the first stage and lowest rung of the testing hierarchy, but it certainly has its place in a security program due to its low cost and ability to be automated.
Scanning Pros & Cons
Pros
- Inexpensive
- Automated
- Finds common vulnerabilities
- Required for some compliance regimes
Cons
- Usually specific to an asset class (web apps, network, cloud, etc.)
- Doesn’t identify less common (but potentially important) vulnerabilities
- Doesn’t find business logic issues (this is a big one)
- Auto-generated reports are often terrible to work with for teams tasked with remediation
- Significant numbers of false-positives
- Scanners differ in their findings – some find certain vulnerabilities well and others find different vulnerabilities well…
- Won’t be accepted as a ‘penetration test’ (because it’s not)
Where scanning fits really well into a mature information security program is for compliance reasons (i.e. it’s required), for continuous monitoring, and for gap testing between more rigorous testing exercises. It’s just generally impractical for organizations to perform deeper testing exercises on their network or on a deployed and stable application in a constant manner. Scanning can help you watch your assets in between more in-depth penetration testing. It can also help you discover what others at your organization are deploying – maybe without going through a proper security review…
VULNERABILITY ASSESSMENTS
A ‘Vulnerability Assessment’ is the next step up the hierarchy. A vulnerability assessment builds on (and typically extends) a scanning exercise by incorporating multiple scanners (with differing strengths) and manual validation. Vulnerability assessments may include some checks for business logic issues or technical issues that automated tools can miss, but it’s dependent on the organization doing the assessment.
Vulnerabilities should be reported using a format and structure that is significantly more useful and effective than auto-generated scanner reports.
An additional step should also be completed here, but often isn’t. Vulnerabilities are risks. Each risk needs to be properly understood and put into the right context so that someone at your organization can make a decision about remediation priorities. The team doing your vulnerability assessment (whether an internal team or an external partner) should try to understand this context as best as possible and should actively work to include that context where they are able.
Vulnerability Assessment Pros & Cons
Pros
- Better results than just scanning
- Some business logic issues identified
- Cleaner vulnerability data (fewer false positives)
- Some contextual risk information (hopefully)
Cons
- More expensive than scanning
- Requires expertise and experience to do it effectively
- Effective reporting requires an understanding of both technical and business issues (and how to communicate those issues)
- Still won’t be accepted as a ‘penetration test’ (because it’s not)
Vulnerability assessments are useful in taking your testing to a level of insight that should go well beyond basic scanning. It’s particularly useful in situations where you want to avoid testing risk to an asset or environment. As vulnerability assessments do not include exploit of vulnerabilities, it can be a safer option (than a full penetration test) if you are concerned about the target asset’s stability. That doesn’t mean that a scan or a vulnerability assessment can’t take down systems – they can.
PENETRATION TESTING
Penetration Testing or ‘Pentesting’ is the next step up. It builds on scanning & vulnerability assessments to provide additional, very useful, information regarding the target asset or environment.
In a penetration test, a wide variety of automated scanners are typically utilized first to find the easy-to-identify vulnerabilities and to gather information for the manual work. Then the steps of a vulnerability assessment are performed to test for manual vulnerabilities, business logic issues, and other problems that the scanners won’t find. Vulnerabilities are also manually verified to ensure that false positives won’t be an issue.
Once those steps are completed, vulnerabilities are analyzed to better understand their potential as ‘entry points’. These are vulnerabilities that might conceivably provide an attacker a way to penetrate your environment or application. Once these potential entry points are identified the penetration tester(s) will attempt to safely exploit these issues.
This ‘safe exploit’ exercise is extremely important, as it allows an organization to understand their potential vulnerability risks to a much higher degree and creates a situation that allows you to prioritize remediation far more effectively.
However, safely exploiting vulnerabilities does come with risks and requires that your penetration testers have both significant expertise and experience. A good penetration tester understands not only how to exploit vulnerabilities, but how to do so safely and how to translate their results into business-relevant information.
Penetration Testing Pros & Cons
Pros
- Much more insight into vulnerabilities and risks
- Will be accepted as a penetration test (because it is!)
- A better understanding of remediation prioritization
- More complete results than vulnerability assessments
- Business logic issues identified
- Cleaner vulnerability and remediation data (no false positives)
Cons
- More expensive than scanning or vulnerability assessments
- Required significantly more experience and expertise than either scanning or vulnerability assessments (if done properly)
- Requires more involvement from the security team to ensure that things go smoothly
Penetration testing will provide you with more useful, more actionable, and more relevant data than either scanning or vulnerability assessments – if it’s done correctly. The risks involved with penetration testing can make this sort of testing problematic if done poorly or without giving the pentesting team information that they need to do their work effectively and safely. However, if working with a strong and experienced team, a penetration testing exercise is an extremely effective way to better understand the technical vulnerabilities and the business logic issues (and their proper priority) in critical assets and environments.
Scanning vs. Vulnerability Assessment vs. Penetration Testing
So that’s the primer on the different types of security testing and the differences between scanning, vulnerability assessments, and penetration testing. Now we have a favor to ask,if you are just now starting to look for a penetration testing partner please, please, please ask questions. Challenge your potential partners on how they do pentesting. What tools do they use? What methodologies do they follow? What is their experience in the industry? How do they rate vulnerabilities discovered? How do they address false positives? (btw – I’m going to do another post on what to look for in a security testing partner.)
‘Pentesting’ is a term that many people use incorrectly and, far too often, it’s used incorrectly on purpose…to mislead… Information security is a growth industry and it’s pretty easy to download a free scanner, play with it a bit, and start charging people for ‘penetration tests’. This is something that we run into quite often. It’s really, really frustrating… Real pentesting takes time, expertise, a great deal of experience, a strong process, technical ability, and the right tools. It costs actual money but can be extremely valuable if done correctly.
Thanks for taking the time to read this, if you have any comments or questions please let us know via the contact form and we’ll get back to you!
