Threat modeling for those with paws

For the proactive identification of such issues, organizations and communities like NIST and OWASP recommend using threat modeling within their guidelines and frameworks. In our practice, this is a rather creative process that requires understanding both the product and technical sides.

At Avito, we have scaled this process to 2500+ engineers, and today I will tell you how we got here — what challenges we faced, what framework we developed, and how we adapted it to the real needs of product teams.

Essence of the process and practical value

Threat modeling is the search for and analysis of threats and methods of their prevention to protect some value (asset). In IT systems, the value is almost always user data that the system stores and processes.

Threat modeling is usually conducted in the format of a brainstorming session. A team of experts discusses the software system model and answers the question "What could go wrong?" The main thing is not just to find potential problems, but to plan in advance how to avoid them. In other words, the essence of the process is as follows:

1. You need to build a system model

2. Identify vulnerabilities

3. Determine what needs to be fixed and in what order.

The model helps to discard unnecessary details and focus on what is important. In information security (InfoSec), the focus is on data flows between system components and the ways an attacker can steal, tamper with, or make them unavailable, i.e., violate their confidentiality, integrity, or availability.

Analogy

Imagine you want to make a chest of drawers with a compartment for storing documents. The dimensions or color of the furniture do not affect the security level, but the following are important:

• the location of the drawer (so that it cannot be pulled out from above);

• the type of lock (so that it cannot be broken off);

• the material of the chest of drawers (so that the walls cannot be broken through);

• fastening to the floor or wall (so that the chest of drawers cannot be carried away).

Most of these items can be taken into account right away at the stage of designing the chest of drawers.

The same happens during software development: at the technical design stage, to assess information security threats, all you need is a sheet of paper and a couple of colored pencils — it is cheap and does not require a test bench.

Example of the threat modeling process

Let's imagine that we are buying a house with a plot of land around it and want to make sure that it will be a truly comfortable place to live with minimal risks for us.

In this case, the threat modeling process will look like this:

1. Identify the main values on the plot and in the house.

For example, these are the house itself, the people in it, the car in the garage, and the safe in the bedroom.

2. Let's outline potential threat vectors, without paying attention to existing protective measures for now.

Threats can be both very likely (for example, trespassing by deception, fence destruction, natural disasters) and very unlikely (for example, a tunnel or a shark falling on the roof). It is worth remembering that "unlikely" does not mean impossible.

3. Let's especially highlight potential threats to our valuables.

For example, to break into a house, you can break a window, and to break into a safe, you can crack it, blow it up, or carry it out of the house to open it in a calm environment.

4. We prioritize threats we want to protect against (since resources are limited).

For example, it makes sense to immediately replace all locks in the house, install an automatic fire suppression and alarm system, as well as get a dog to guard the property. We will hide the safe right away so it is not in plain sight, while purchasing a more secure safe can be postponed for later, just like installing a higher and sturdier fence. For the most unlikely threats, we will simply accept the risks as they are, but we will not forget about them and will review them in the future.

So we have completed full threat modeling for our new property. It looks quite simple — this is the approach we took as the foundation of our methodology.

Why threat modeling is useful and necessary at Avito

Delegation of responsibility. With thousands of microservices deployed to production almost daily, the information security department cannot assess the risk level of every change. The threat modeling framework allows part of the work to be delegated to teams that know their domain area better, and together we make our products more secure.

Information security section in TDR. At Avito, there is a TDR (Technical Design Review) process: teams defend their proposed architecture, receive feedback, and align on decisions. This is the best time to assess information security threats: the code has not been written yet, and adjustments cost less.

Engineering ladder requirements. At Avito, feature leads are valued — high-grade engineers capable of bringing a new service or product from idea to launch. A senior engineer needs to be able to account for information security (InfoSec) threats during design and development — this is directly related to threat modeling.

Threat modeling is beneficial for everyone:

• for Avito — a convenient and scalable way to reduce InfoSec risks;

• for the engineer — competency growth and a step toward the role of feature lead.

Threat modeling methodologies and international practices

If you think threat modeling in software development is complex and artificial, take a look at the experience of global leaders:

• For decades, Microsoft has been promoting its approach to information security threat assessment called STRIDE:

• the American institute NIST began standardizing threat modeling in 2015–2016 and continuously expands set of recommendations;

• the OWASP project has a separate stream on threat modeling;

• threat modeling is already required at the first level of information security maturity in the BSIMM and OWASP SAMM;

• Computer Science guru Martin Fowler and his followers described in great detail the implementation of this practice in Agile development in a guide.

Threat modeling is enshrined in recommendations from both NIST and OWASP. However, neither of these sources regulates the process itself, as it can vary significantly depending on the subject area, level of detail elaboration, and the team's expertise. Nevertheless, the community has developed methodologies and tools of varying degrees of universality designed to assist in conducting threat modeling.

Methodologies and Tools

MITRE ATT&CK®

This is a comprehensive framework that allows you to focus on implementation details thanks to a well-developed classification of potential attacks. This approach is rightfully called the "security professional's Bible" by some: all attacks are cataloged and organized in accordance with the sequence of actions during an attack — from reconnaissance and information gathering to the use of destructive actions in the target system.

However, this advantage is also the framework's drawback: for an unprepared person, it will be difficult to master, will require a large amount of technical knowledge and time costs in attempts to "fit" the entire attack library to their solution.

Elevation of Privilege

This is a card game that makes the threat modeling process more interactive and creative. It requires a moderator-host who organizes game rounds.

The essence of the game is simple: all session participants are dealt cards with possible attack vectors, and players take turns playing these cards from their hand, coming up with attacks on their area under consideration.

The cards are organized in the form of a poker deck of five suits, divided into attack groups from the STRIDE methodology — the higher the card rank, the more critical the attack. Any session participant can join in and propose the same played attack in a different context of the area under consideration, which enables collaborative co-creation.

Thus, the threat modeling process becomes far more dynamic and collaborative, and attack descriptions become far more understandable for product development teams. However, this approach also has its downsides. Cards from the deck may not be applicable to a specific team's context (for example, physical security when working with cloud services), and the limited session duration and random drawing of cards from the deck introduce additional uncertainty into the results: it may turn out that the most critical vulnerabilities simply do not end up in the players' hands, and instead, teams will pull attacks that are of little practical use to them out of thin air.

Difficulties we encountered when choosing a methodology

The most obvious problems include:

• the need to integrate threat modeling into existing development processes, complementing them rather than redesigning them from scratch;

• systematization of the process;

• the ability to conduct retrospective threat modeling alongside proactive modeling;

• the required entry threshold for the process.

At the same time, there is a temptation to postpone threat modeling until later, when more data is available. However, this would contradict the shift-left approach, where we strive to implement our processes at the earliest possible stages of design and development. Especially since even with such limited information, we still have plenty to consider.

For example, does our designed solution affect external users? Are we hosting any resources outside of Avito's internal infrastructure? Are we planning to send any information to external vendors? What kind of information will that be — personal or payment data? Do we plan to use authentication or authorization for users and service accounts? These are all fairly general questions, but they play a major role in designing a secure solution and serve as a kind of signal to think through the implementation and discuss it with the information security team.

Developing the Solution

First and foremost, we answered the questions: when is it worth conducting threat modeling and who is responsible for it? Then we built a framework that is understandable for engineers on the one hand, and on the other hand is not too labor-intensive and formalized. We get a sort of checklist that is clear to engineers, but at the same time does not turn into a formality.

It is also important to integrate this process into existing development processes so that it complements them, rather than creating additional difficulties. In our case, this is primarily the technical design document (TDR) review stage. Teams work in the tools they are used to and enrich the context with information security aspects.

Key questions and abuse cases

One of the questions is worth highlighting separately: «What could happen if the solution is used for unintended purposes?» (abuse cases).

Let's imagine a scenario: to reduce response time and the person-hours spent on development, a team rolled out a service that is accessible without authorization, but only when connected to the corporate network. It sounds a bit dubious at first, but it is acceptable. But another team learned about the existence of this service and decided to repurpose it for their own needs. They tack on public API endpoints that call this service, and now we end up with a publicly accessible, unauthorized resource with a load different from what was planned. Who is at fault in this situation? The team that failed to keep track of unauthorized connections to its resources? The team that simply found an already running internal service and decided to repurpose it?

To avoid similar situations, it is worth thinking during the design phase about how our solution could be used differently — not in the way it was originally intended?

For example, these could be external users who set a link to prohibited resources as their avatar instead of their own image. These could also be internal employees who have not had their access restricted. Perhaps through not fully closed-off services, someone will be able to reach resources they are not supposed to have access to?

Threat prioritization and artifacts

By answering all these questions, the team will get a list of threats. Some of these threats can be mitigated right away during the development phase; some will be taken into account later, and we will be able to log corresponding tasks for rework; some risks we will take responsibility for.

It is time to sort out which attacks we will mitigate right away, and which measures we will take at some point later. If the team decides that some attacks are not worth paying attention to at all and the stakeholders are ready to accept the risks, they need to be moved to the corresponding section.

As a result of threat modeling, the team must have:

• diagram or feature description with identification of potential threats linked to their possible occurrence points;

• documented threats that have been accepted or postponed until better times;

• tasks in Jira with DoD specified for the most critical threats.

Attack library

The attack library consists of several sections:

1. Backend Attack Section. Here we discuss attacks on insufficiently isolated microservices; attacks related to input data processing (various injections, retrieval of insecure data and its subsequent execution on the server, and other validation checks) and attacks whose primary vector is unauthorized access to various operational and confidential information.

2. Frontend Component Attack Section. Some of them are again focused on working with externally provided data; some are dedicated to retrieving and displaying excessive information. In addition, this section covers attacks at the DOM tree level, client-side script execution, cross-domain requests, and other automation that allows attackers to redirect data of interest to third-party resources and mislead users.

3. Business Logic Section. The possibility of brute-forcing access to data, insufficient or missing authentication and authorization, logging and process monitoring issues (both their lack and excess) are highlighted; process-specific features are examined (for example, whether a user can perform an action twice to bypass business restrictions, or top up an account with a non-existent payment by increasing the amount in request parameters), as well as the presence of external integrations that consume our resources.

4. Mobile Platform Attacks. Typically, threats to mobile applications lie at the intersection of visibility boundaries and automatically executed code, i.e. in JS, WebView and interprocess communication. The use of non-standard components and data redirection between applications expand the potential attack surface compared to the more familiar frontend that can also be run from a mobile browser.

5. Attempts to Extract Confidential Information or Perform Actions in Violation of Applicable Law. This category includes all attacks that affect not only technical systems, but also aim to cause informational, reputational and legal harm, for example, posting illegal content that can put the entire platform at risk, exploitation of vulnerabilities arising from violations of personal and user payment data storage and processing rules, as well as bypassing moderation controls.

Process Improvement

Here, we simply could not do without gathering feedback to improve the process in order to address all the above-mentioned difficulties. As a result of the feedback received, we formed a specific list of criteria for the applicability of threat modeling and introduced an information security section into the TDR template. With the deeper integration of AI into our processes, we have revised the templates multiple times so that, on the one hand, they reflect the information security status in the TDR, and on the other hand, allow filtering and highlighting potential issues using automation and LLM. At the same time, it is important to find a balance: the clearer the text is for automation, the less understandable it is for the person who reads and validates it. Also, during the process testing, we identified the problem of varying focus in TDR development — from high-level design to point improvements in specific services.

Process Adaptation for Specific Cases

The standard approach to threat modeling adopted at Avito, although it covers most cases, sometimes there are situations where it feels more like unnecessary formalism. Or it seems that in the current case it does not fully solve the set task of finding potential threats and ways to mitigate them. We will try to review the most typical complications and deviations from the framework, as well as give some tips on adapting the process to your needs.

Cross-team Collaboration

If colleagues happen to find any potential issues and questions, they can safely be left in the threat model and must be passed on to the respective owners.

A fairly similar story happens when participants reach a component that falls outside the scope of their context, with threats unknown to them. In this case, it would not be superfluous to at least ask the team responsible for the component how they handle a particular situation coming to them from your team: after all, the most common and critical vulnerabilities arise precisely at the intersection of technologies, contexts, and business processes. If they have not yet conducted threat modeling, it makes sense to plan it.

Mobile Application

If a team is only developing a mobile application, almost all items from the attack library in the template will be irrelevant for it. At the same time, you shouldn't rush to discard items related to business logic and the backend: even with a "thin client", you can cause a lot of trouble if you don't take into account the specifics of the backend implementation and the messages it returns.

We have outlined the standard threat modeling approaches adopted at Avito. Note that this is just a general framework, so you can freely make your own modifications based on your needs.

Process Regularity

You shouldn't limit yourself to conducting threat modeling just once per service, the process should be regular. Moreover, after working with the proposed framework a couple of times and getting used to the approach, the team may no longer follow the template unconditionally: for example, they can take known threats, tweak the context a bit, think about how to abuse the solution (for instance, if someone other than the intended recipient gets access to a payment link); finally, go through a checklist that may spark new ideas.

Instead of Conclusions

We scaled this solution to 2500+ engineers by focusing on the flexibility of the approach, integration with other processes, and training. We analyzed key common mistakes and vulnerabilities, and taught teams how to identify them. As a result, we shifted about 15% of the effort spent on threat modeling to product development teams — and lowered the entry barrier for the process so much that it became accessible even to people with no prior experience.

Don't be afraid to adapt the approach to your needs, you need to try and find the most suitable approach for your team, and most importantly — don't forget to document all your findings so that you can refer to them at any time and revisit them.