How to Run a Purple Team Exercise When Nobody Has Done One Before

Most purple team advice is written for teams that have done several. The advice assumes that the team has a red team, a blue team, a process, a vocabulary, and a history of working together. The advice does not apply to the team that is about to do the first one.

Here is the version for the team that is about to do the first one, in order, with the parts that most programmes get wrong.

The first decision: what are you testing

The first decision is the most important. The first decision is the decision about what the purple team exercise is testing. Most programmes skip the first decision. Most programmes start the exercise without a clear answer to the question. Most programmes produce exercises that test nothing in particular.

The first decision has three options. The first is the detection capability. The exercise tests whether the blue team can detect the specific threats that the red team is emulating. The first decision is the most common. The first decision is the one that most programmes should start with.

The second is the response capability. The exercise tests whether the blue team can respond to the specific incidents that the red team is generating. The second decision is harder than the first. The second decision requires the blue team to have a response procedure to test against. The second decision is the one that the team should move to after the first.

The third is the operational capability. The exercise tests whether the operations team can operate under the stress of a real incident. The third decision is the hardest. The third decision requires the operations team to have a baseline of operational capability to test against. The third decision is the one that the team should move to after the second.

The first exercise is almost always the first decision. The first exercise tests the detection capability. The first exercise is the foundation for the second and third decisions.

The second decision: what are you emulating

The second decision is the decision about what the red team is going to emulate. Most programmes make this decision badly. Most programmes make this decision by emulating the most sophisticated threat they can think of. Most programmes make this decision by emulating the threat that is most famous. Most programmes make this decision without reference to what the blue team is actually likely to face.

The second decision should be made by reference to the threat intelligence. The threat intelligence should tell the team what threats are credible to the environment. The threat intelligence should tell the team what threats the environment is actually likely to face. The threat intelligence should be the input to the second decision.

If the team does not have a threat intelligence function, the second decision should be made by reference to the industry. The team should pick a threat that is credible to the industry. The team should pick a threat that the environment is likely to face. The team should not pick a threat that is exotic. The team should not pick a threat that is famous. The team should pick a threat that is real.

The second decision is the most important technical decision. The second decision shapes the exercise. The second decision shapes the value of the exercise. The second decision should be made with care.

The third decision: what is the scope

The third decision is the decision about the scope of the exercise. The third decision is the decision about what is in scope and what is out of scope. The third decision is the decision about the boundaries of the exercise.

The third decision is the decision that most programmes get wrong. The third decision is the decision that most programmes make by trying to test everything. The third decision is the decision that most programmes make by setting the scope to be the entire environment. The third decision is the decision that most programmes make by saying "anything in the environment is in scope."

The third decision should be made by reference to the first decision. If the first decision is the detection capability, the scope is the detection capability. The scope is the systems that the blue team is responsible for. The scope is the threats that the red team is emulating. The scope is the boundaries of the exercise.

The third decision should also be made with the consent of the leadership. The leadership is the one who is going to be told what the exercise found. The leadership is the one who is going to make the changes. The leadership should be part of the third decision.

The fourth decision: who is on the team

The fourth decision is the decision about who is on the team. The fourth decision is the decision about the red team. The fourth decision is the decision about the blue team. The fourth decision is the decision about the facilitator.

The red team should be small. Two to three people is the right size for a first exercise. The red team should be experienced. The red team should be able to emulate the threat credibly. The red team should not be the same people who built the environment. The red team should not be the same people who are the SOC.

The blue team should be the people who are the SOC. The blue team should be the people who are going to be on shift when the exercise runs. The blue team should not be a special team. The blue team should not be the best team. The blue team should be the regular team.

The facilitator is the most important person. The facilitator is the person who runs the exercise. The facilitator is the person who is not the red team and not the blue team. The facilitator is the person who is responsible for the scope being respected. The facilitator is the person who is responsible for the timing being respected. The facilitator is the person who is responsible for the debrief being held.

The facilitator should be experienced. The facilitator should have run purple team exercises before. The facilitator should be trusted by both the red team and the blue team. The facilitator should be able to hold the space for the exercise to be honest.

The fifth decision: how long is the exercise

The fifth decision is the decision about the length of the exercise. The fifth decision is the decision about how much time the exercise takes.

The first exercise should be short. Two weeks is the right length for a first exercise. Two weeks is enough time for the red team to do the work. Two weeks is not so much time that the blue team is overwhelmed. Two weeks is the right length.

The first exercise should also have a clear end. The exercise should have a specific end date. The exercise should have a specific time on the end date. The exercise should have a specific time on the start date. The exercise should not be open-ended. The exercise should not be "until we have learned enough." The exercise should have a specific length.

The sixth decision: what is the debrief

The sixth decision is the decision about the debrief. The sixth decision is the most important operational decision. The sixth decision is the decision that determines whether the exercise is useful.

The debrief should happen on the day the exercise ends. The debrief should happen within hours of the exercise ending, not days. The debrief should happen while the lessons are fresh.

The debrief should have all the people in the room. The red team. The blue team. The SOC manager. The detection engineering lead. The CISO. The facilitator. The debrief is the meeting where the lessons are captured.

The debrief should be structured. The four-section debrief is the structure. The kill chain. The detection gap. The response gap. The action. The four-section debrief is the most useful structure.

The debrief should be written down. The debrief should be a document. The debrief should be sent to the leadership within 24 hours. The debrief should be the start of the change, not the end of the exercise.

The seventh decision: what are the changes

The seventh decision is the decision about the changes. The seventh decision is the decision that the leadership signs. The seventh decision is the decision that the leadership commits to.

The changes should be specific. The changes should have owners. The changes should have dates. The changes should be tracked. The changes should be reviewed at the next exercise.

The changes should be the changes that the exercise drove. The changes should not be the changes that the leadership was going to make anyway. The changes should be the specific changes that the exercise showed were needed.

The seventh decision is the decision that makes the exercise useful. The seventh decision is the decision that the leadership is measured on. The seventh decision is the decision that the next exercise is built around.

The honest answer

The honest answer is that the first exercise is hard. The honest answer is that the first exercise will not be perfect. The honest answer is that the first exercise will teach the team things the team did not expect to learn.

The honest answer is that the first exercise is worth doing badly. The first exercise is the foundation. The first exercise is the baseline. The first exercise is the thing that the next exercise is built around.

If you are about to do your first purple team exercise, do it. If you are about to do your first purple team exercise, do it badly. If you are about to do your first purple team exercise, do it badly and learn from it. The first exercise is the foundation. The first exercise is the work.