Our CEO and founder, Christopher Emerson, is regularly asked to provide his expertise, opinion, and insight on a variety of cybersecurity topics. Often our clients, industry contacts, or community request feedback or direction from someone who is not only experienced, but takes a positive approach to creating strong security programs.
This happens enough that we realized it may be helpful to bring this information to a broader audience. Below is part one of an interview series conducted by White Oak Security. Over the years we often hear many of the same questions repeated, some of the same topics discussed, and several of the same pain points resolved.
Application Security Program Roadmap
What are the first steps that an organization should take in building an application security program from scratch?
I think a lot of it starts with getting an inventory. There’s a lot of great discussion these days about asset discovery, asset tracking, and maintaining a good CMDB, but in addition to that, when you are starting an application security program make sure that you’re including people, their skills, your existing processes, and policies.
The more information you have, the more you can review and see what you have to work with – what kind of baselines do we have? What is the foundation that is already here? Take that and try to understand what is working and what isn’t working…. The best way to get that feedback is talking with the people who are doing the work, or are impacted by it today.
If you are fortunate enough to already have team members who have been doing some of this work (regardless of the type of security testing) or if you have the opportunity to talk with some of the developers or product team members, reach out to them and ask:
- What’s working for you?
- What’s not working for you?
- What are your pain points?
- How do you suggest changing things?
- Are your results inconsistent?
- Are your recommendations not applicable to the environment?
- Are your findings not repeatable?
- Do your customers understand what they are being told?
Try to get your initial baseline on what you have and then poll your stakeholders to find out what works for them. I believe that’s where you have to start.
Tools vs. Training
I’m going to take the consultant way out and say ‘it depends’. But it depends on who you are targeting with your application security program.
Tools have more inherent scalability, so if you have a large group of people you are looking to enable (i.e. multiple product teams), tools are a great way to start. They have enough stuff on their plate with delivery, meeting deadlines, getting their stories completed, getting their QA testing done… But if you can help them by providing them easy tools that they can work into their current delivery process (waterfall, agile, etc.) easily you’re going to see return.
If you’re targeting your security team, then I would start asking them:
- What do they feel they need?
- Where do they think they need help?
- Do they not have tools to do their jobs effectively?
That may be true regardless, but they may say ‘hey, this particular training is going to give me more job satisfaction than this tool that will increase my throughput 10%.’
So getting that feedback from your team, I think that’s a big part of leadership – you have to delegate appropriately and then trust your team members to provide that feedback. These decisions will ultimately vary from organization to organization, but it has to be part of a conversation and dialogue with those impacted stakeholders. Their input should help guide those decisions.
What are the first metrics that you would recommend tracking to show adoption and/or program progress?
When you start a program from scratch a lot of it depends on what you want to deliver, what is tracked, and what will be delivered.
There is a book called “Measure What Matters,” by John Doerr that discusses Objectives and Key Results (OKRs), and I feel that is a great way to look at your metrics. They need to be able to support what you are trying to achieve (Objectives) and be measurable (Key Results), and ultimately, they need to help you and your company make decisions.
In application security, some of the key metrics I prefer to start with are:
- Number of Applications
- Percentage of Applications undergoing security review (this can be broken down into types)
- Number of security defects identified per application (broken down by severity)
- Number of security defects identified categorized by OWASP Top 10 Category
- Average time to remediate security defects per application (broken down by severity)
- Percentage of developers & engineers who have undergone secure development training
Aggregating this data and watching it trend over time can help identify opportunities to make strategic decisions that will impact the overall security of an organization’s applications. For example, if you notice that the largest quantity of security defects is in the the OWASP A1 – Injection category, you can start to identify if, as a company, you should provide the developers more training around preventing that issue in their code, or if there are technical controls you could enable to help remediate that for all development.
The trending data will then let you identify if your changes have been successful, if you see less and less Injection defects being identified over time.
How can you successfully promote the application security program or getting others outside the program to see its value?
So that’s a tough one… As much as those in security try not to increase the workload of other teams, that’s basically what you are doing – your job is generating additional work and overhead for other teams. That can sometimes be a tough pill to swallow… It’s up to you to try to find effective ways to communicate and build a culture where these teams feel empowered and enabled to fix these issues and have a desire to do so – that they are properly incentivized.
Some of the things that I’ve seen work really, really well in that space are having security champions – essentially deputizing people who have interest in security from each of the product teams to get them to be that first line of security contact. As developers are working through their stories and grooming the backlog, that person can speak up (and feel confident enough to speak up) and say ‘that is something that has security implications – let’s make sure that we’re doing the appropriate security stories as part of that.’ They also need to feel confident and comfortable coming to the security team and saying ‘this has reached a point where I’ve reached my level of knowledge on this topic and I think the team should bring you in for additional feedback.’ So also trying to empower people.
One of the ways that I’ve found to get people to want to participate as security champions would be something that we’ve called ‘ride-alongs.’ As we’re going through and doing a security test, whether it’s source code review or dynamic testing or a full-blown penetration test, setting aside some time on the schedule to have a developer who has interest to sit with you and understand your process – ‘here’s how we go through and find indicators of potential vulnerabilities, here’s how we start performing the research to identify whether these are true security defects or are they false positives, here’s how to create exploits for these types of vulnerabilities to show the true risk to the environment in question.’
We’ve gotten really great feedback from the engineers and developers who sit with us – it helps to open their eyes – we hear things like ‘I never thought this could happen in my code’, ‘now that I’m seeing it I understand how you guys are doing it.’ It kind of gets the security bug in them where they start wanting to do some of this on their own. Then it’s up to the security people to empower them – provide them with tools, guidance, and education to allow them to do more on their own.
Thanks for reading part one, we hope you learned more about application security programs. Please don’t hesitate to reach out with any questions!
