Blog: Vendor Management

"You need a pen test. This is a vulnerability assessment. Have you considered Red Team testing?"

You might have been told this by a regulatory examiner, your IT vendor, a senior partner, a Board member, or read it in an article. It's a common statement in the penetration testing space these days and with good reason. The scope and methodology of penetration tests is not standardized or regulated, so each provider can create their pen test service as they see fit. This puts more responsibility on you, the customer, to determine if they are meeting your needs.

We covered these topics in our webinar Choosing Pen Tests & Real-Life Horror Stories. In this webinar we discussed:

• The different types of penetration tests available
• The pros and cons of different penetration tests
• How to choose the best penetration test for your organization
• Stories of real-world exploits
• How to make the most of your penetration testing budget

What is Penetration Testing?

Many terms are used to describe penetration testing…

Red Team, Physical Intrusion, Gray Box Testing, Phishing, Reconnaissance, Privilege Escalation, Precision Strike, War Dialing, Black Box Testing, Social Engineering, Capture the Flag, Web Application Testing, Purple Team, White Box Testing, Blue Team, Pivoting, Internal Testing, External Testing

They overlap, they conflict, they can be misleading.

So, what is pen testing?

Whether it's physical, logical, or human, pen testing shows you how an attacker would look at your organization. They look for holes and possibilities to disrupt the way you work.

Penetration testers look for holes and vulnerabilities that could disrupt the way you work.

According to the FFIEC IT Exam Handbook for financial institutions, there are many types of penetration tests . . . and management should determine the level and types of tests employed to ensure effective and comprehensive coverage.* The regulators are giving you the responsibility for figuring out what level and type of pen test you need.

So, what terms mean something, and what is terminology fluff? What should you look for when determining the pen test you need and what vendor to use?

Choosing and understanding penetration test vendors and knowing the terminology is important. Our aim in this article is to help remove some of the mystery around pen testing and help you feel confident in determine the best solution for your company.

What type of Pen Test do you need?

What is the type or level of penetration testing that you need?
Let's go back to the regulation:
"A penetration test subjects a system to real-world attacks selected and conducted by the testers." - FFIEC IT Exam Handbook, Information Security Booklet, Sep 2016 (emphasis added)

Essentially you are choosing what you want tested, you are not telling them how you want them to test it. That should be up to the penetration tester. The reason is, the penetration tester should have the expertise to know what the attackers are doing, and they are going to choose the real-world attacks that the attackers would choose. They have done the research, gone through the certifications and the training to know what types of methods to use. So it's important to know you're not telling them how to do the test, you're telling them what you want tested.

You figure out WHAT you need tested and the penetration tester will figure out HOW to test it.

Here is a simple definition of penetration testing that doesn't use any of the fancy marketing terms. Let's start here.

At its core, penetration testing should show you how an attacker would see you and describe specific attacks they would choose to use against you. That means you don't need to tell the pen testers HOW to do the testing. You just need to tell them WHAT to test and ensure their METHODOLOGY meets your needs and keeps you safe.

So, how do you determine WHAT you need?

Start with a Risk Assessment

When done correctly, a risk assessment will inform you of what you need to do for security testing.

Let's use an example to illustrate. Here is a simple risk assessment.

Simple Risk Assessment

• What attacks do we hear about from IT, in the news, etc.?
  • Phishing!
  • Ransomware
  • Website attacks
• What assets do those attacks target for us?
  • Employees
  • Corporate email and perimeter defenses
  • Web servers
• What testing do we need?
  • Email social engineering for ALL employees
  • Internet perimeter testing for ALL of our public IP addresses

In this sample risk assessment, we are showing that as a company that offers services via the Internet our top concern is cybersecurity; but more specifically, we are concerned with phishing, ransomware, and website attacks.

So how do those threats apply to us? Phishing relies on our employees to click a link or open an attachment, so testing our people is important. Ransomware is typically introduced from the outside, so our corporate email defenses and perimeter defenses, such as firewalls, are important. Website attacks target our webservers directly, so those are also important targets.

Translate your Risk Assessment

After you perform a risk assessment, you will need to translate the results to answer the questions "What" to test and "Why" test it. The penetration test company will figure out "How" to test it.

For our sample risk assessment illustrated above, here is what we would choose:

1. What to test? — Internet exposed systems
   Why test it? — These are our most exposed systems. They are what our attackers are going to hit first.
2. What to test? — Employee responses to social engineering, because our employees are the ones who will protect us from phishing attacks.
   Why test it? — Phishing attacks are frequent and successful. They are hurting other people and happen all the time.

TIP: Test against common attacks. Attackers stick with what works. We're tempted to say "It's so common, let's look for the next big thing instead." Attackers will stick with what works as long as it works. So if you're hearing about things in the news, like ransomware and you're worried about it, then that is a valid threat to test against. It will continue to happen until it's no longer profitable for the attacker.

Test against common attacks. Attackers stick with what works as long as it works.

Whether your risk assessment is a formal or informal process is not as important as the results. You need to identify the areas where you are exposed, where your most critical assets are, and where attacks are most likely to occur. These decisions come from knowing your organization and knowing what types of attacks are common. We see these attacks in the news and other places all the time.

Determine the testing you need

Going back to our simple risk assessment example, we can take the assets we identified in the first step and translate them into the type of testing we need. In this case, we decided on email social engineering for all employees and Internet perimeter testing for all of our public IP addresses.

Be specific!

It is important to be very specific when communicating to your penetration testing company what type of testing you want performed. In a minute, we will look at what could happen if we don't communicate clearly.

Choosing a Pen Test Provider

Now that we know what we want to test, we need to define the scope of the testing. Do we want to test just our web servers or our entire Internet exposure? Do we want to test a specific group of employees or are we specifically concerned about a certain department that performs risky tasks – maybe they initiate wire transfers for our company.

In this example, we will define the scope as:

• All public IP addresses
• All employees

TIP: Include ALL external IP addresses, active and inactive. You never know when an inactive IP address might become active. Mistakes and misconfigurations happen and attackers are looking for them.

This scope can be modified over time. Maybe we want to test all employees the first time and then focus on a risky department the next time.

Set the Rules of Engagement

The final step in our pen test selection process is to set the boundaries for our pen testers to follow. Remember that we get to choose WHAT they target, but they get to choose the attacks that would be most effective – what the real attacker might choose. By settings boundaries on those attacks or rules of engagement, we can ensure the pen test won't go too far and cause harm.

In this step you will agree on:

• What the pen testers WILL attempt
• What the pen testers WON'T attempt

"The test mimics a threat source's search for and exploitation of vulnerabilities to demonstrate a potential for loss." - FFIEC IT Exam Handbook, Information Security Booklet, Sep 2016 (emphasis added)

Going back to our regulatory reference, these boundaries are stated as "demonstrating the potential for loss." Notice it doesn't say "demonstrating the loss," but rather the potential for loss. This is an important word to highlight because it identifies the need for pen test boundaries. We want our pen test firm to show us how an attack could happen and how it could negatively impact our organization, but we don't want them to go so far as to actually cause loss or have a significant negative impact.

Here's an example set of the rules of engagement that CoNetrix will use. Because we typically test live, operational systems, we put certain rules into place to make sure the testing stays within bounds.

These are the rules of engagement we practice here at CoNetrix when performing penetration tests.

Evaluate a Pen Test Provider

So let's wrap up how to choose a pen test provider with some ways you can evaluate pen testers and then we'll move on to the fun stuff – the exploits!

1. Know what you need tested and clearly communicate that in the project scoping and quoting process. We talked about identifying what you need tested. Make sure the pen test company's expertise matches with what you need tested. As we saw earlier, some pen testers excel in physical security testing. Others excel at Internet-based testing. Some are industry-specific such as SCADA pen testers. Make sure you find out what their strengths are.
2. Make sure their report is understandable and well-organized. Many pen testers are great at successfully demonstrating attacks. After all, that's the fun part – the part that gets put into the movies. Buy many don't put as much effort in communicating what they did and how you can improve your defenses. Make sure the format of the report and the way information is communicated in the report is going to help you improve your security. Is it actionable for you and your IT/security staff?
3. Check out their certifications on top of experience. Certifications are a great way of making sure a pen test company has the knowledge and skills necessary. Some examples include Certified Ethical Hacker (CEH), Offensive Security Certified Professional (OSCP), and any certification that shows they have knowledge of the area you need tested.
4. Make sure the penetration test company will help you understand the report. We believe security testing is most useful when it is a partnership, not a stand-alone service. Ask how the pen test company will help you understand what the issue is and what you can do about it. Do they review their results with you? Do they provide remediation recommendations? Or do they drop a bunch of results on you and wish you good luck.

Phases of Pen Testing

So now let's talk about the fun side of pen testing—exploitation. It's good to know that exploitation is part of the process. Many times it is because of this process that a penetration test report may look like a vulnerability assessment. Every pen test process follows a similar process—identify how the target is exposed—identify vulnerabilities that could be attacked—exploit those vulnerabilities—report both the vulnerabilities and any successful exploits.
But what if no exploits were possible—at least not within the rules of engagement? In that case, the report would only identify vulnerabilities and could look a lot like a vulnerability assessment. But through your pen test selection process, you made sure the firm would exploit vulnerabilities when safe and possible and their documented methodology should reflect that approach.
One of the best things you can do when evaluating a pen test company is ask them for examples of what they have done. See how they did it and whether you're comfortable with the things they are capable of doing.

One of the best ways to evaluate a pen test company is to ask for some examples of their successful exploits.

_{*FFIEC IT Exam Handbook, Information Security Booklet, Sep 2016
**This blog post is for information purposes only. Evaluate your risks before acting on any ideas presented in this article.}