About the AIOps profession: trendy, youthful, lucrative, but not yet taught

What is AIOps

In essence, the term AIOps (Artificial Intelligence for IT Operations) refers to the use of advanced analytics tools, such as artificial intelligence (AI) and machine learning (ML), to quickly and efficiently automate IT tasks. AI in this context implies a wide range of technologies that allow simulating cognitive functions, such as pattern and speech recognition, as well as performing complex AI tasks for event prediction and decision-making based on large volumes of data.

At the current stage of technology, instead of using AIOps to replace DevOps engineers, it is mainly about expanding the competencies of DevOps. These specialists use AIOps methods to manage, monitor, and resolve complex issues related to digital platforms and tools.

AIOps also provides IT specialists with the ability to analyze large volumes of machine data produced by numerous digital enterprise platforms. This allows for faster resolution of infrastructure management issues and maintaining business continuity. Additionally, in some cases, it can address potential alerts even before they occur, using ML to detect anomalies, predict potential failures, and automate responses to them.

Types of AIOps according to AWS

Although the AWS website (Amazon Web Services) does not yet offer "turnkey AIOps" solutions, the service notes that there are two types of AIOps solutions, each addressing different needs.

Domain-centric AIOps are AI-based tools that target specific areas. For example, operations teams use such platforms to monitor the performance of networks, applications, or cloud services. These tools are specifically designed to address tasks in a particular area, helping to better manage local aspects of IT infrastructure within the company.

Local AIOps are more versatile solutions that IT teams can use to scale predictive analytics and AI automation across the entire organization. These applications collect event data from various sources, integrate them, and provide valuable insights covering all aspects of corporate operations.

Now a bit about how AIOps can help DevOps teams support the digital transformation of business in the company:

1. Application Performance Monitoring (APM). Used as a method of collecting metrics when exchanging data between microservices, APIs, and data lakes. DevOps engineers turn to ML/AI services for monitoring and collecting performance metrics.

2. Root Cause Analysis. AI and machine learning technologies allow for the rapid processing of large volumes of data and correlating various causes of failures. When AIOps is applied correctly, it is possible not only to receive alerts about failures but also to get to the root of problems affecting IT system performance.

3. Anomaly Detection in Data. Anomalies are deviations from the normal state of data that may indicate problems in the IT system. AIOps is used as a tool for real-time assessment and forecasting, allowing for the quick identification of such deviations and taking action.

4. Cloud Automation and Optimization. AIOps can help in migrating IT infrastructure to the cloud (so-called "cloud business transformation"), providing observability and process automation. AIOps can be used to allocate and scale IaaS/PaaS resources as needed. Although cloud providers (including AWS, Azure, etc.) have tools for automatically scaling data center resources under client load, they are usually designed for the SaaS level, and if an organization uses a data center as bare metal, scaling must be done independently, possibly with the help of AIOps.

5. Application Development Support. Today, DevOps engineers are beginning to use AIOps tools to improve code quality. These tools allow for the automation of code review, the implementation of best practices for application building, and the identification of errors at early stages of development. For example, Atlassian uses Amazon CodeGuru to reduce the time spent investigating application behavior anomalies from several days (manual analysis) to a few minutes.

This is likely what the profession of an AIOps engineer will look like

The profession of an AIOps engineer involves a specialist who combines knowledge in IT operations with skills in AI and ML. As envisioned in 2024, the main tasks of an AIOps engineer will involve using AI and ML technologies to automate processes of monitoring, data analysis, and incident resolution in IT infrastructure.

By the way, existing CI/CD (Continuous Integration/Continuous Deployment or Continuous Delivery) tools handle automated build and patch assembly today. These include Jenkins, GitLab CI/CD, CircleCI, Travis CI, TeamCity, Bamboo, Gradle, Maven, and others. CI/CD are software development tools that automate the process of building, testing, and deploying code. The general purpose of CI/CD tools is to speed up and improve the quality of the software development process.

Key responsibilities of an AIOps engineer may include:

• Setting up and managing AIOps tools that collect and analyze data from various digital sources.

• Developing custom or selecting ready-made ML models capable of predicting and preventing failures in the organization's IT system.

• Automating the processes of detecting and resolving incidents to reduce downtime of critical systems.

• Analyzing big data and creating reports for making operational technical and managerial decisions at the CTO/CEO level.

It is assumed that the work of an AIOps engineer will be closely related to other IT specialists in the company. For example, working in the DevOps department, AIOps engineers can help better automate the software development and delivery process. In addition, AIOps engineers, together with DataOps engineers, can work on optimizing data processing and analysis processes to help management solve business problems more quickly.

Also, interaction with Site Reliability Engineers (SRE) will help combine efforts to improve customer service quality and increase the reliability of front-end systems.

In summary, an AIOps engineer is a profession that will combine several roles, making such specialists in demand and well-paid in the context of the modern digital transformation of enterprises and government agencies.

AIOps courses? There are none yet

At the time of writing this article, none of the online learning platforms in Russia trained AIOps engineers. And this is a rather important marker that the concept of AIOps has not yet formed into a profession.

IT courses in the Russian Federation, as you know, are good at “keeping their finger on the pulse” and trends, since these are mass-market service enterprises. They don't really care if you become a developer, QA, DevOps, or someone else in the IT industry. This is simply a response to the demand for vertical social mobility, which is so poor in the Russian Federation. Working in IT has become one of the substitutes for social mobility, allowing you to raise your income and improve your standard of living (although this does not always work for everyone). That is why there is so much advertising, hype, and even surveys of women about partner preferences around IT professions in Russia (CNews: 72% of Russian women want to marry an IT specialist). And only in our country are IT specialists equated with large families in terms of benefits, providing preferential mortgages.

But back to AIOps. If you look at the huge American marketplace for educational courses Coursera, you will not find any training courses on AIOps there. Yes, there is a page What Is AIOps? Definition, Examples, and Use Cases, but it offers a set of related courses, such as Google IT Support, Google AI Essentials, AI Product Management, and Machine Learning. In principle, these and other AI courses can be recommended to tekkix visitors, as most courses on Coursera are free.

How is AIOps different from other related technologies?

Although AIOps is the most commonly used term to describe the application of AI, machine learning, and big data to improve IT processes, other similar terms can also be encountered. Let's see how AIOps differs from similarly named technologies.

AIOps vs DevOps

DevOps is a methodology that automates workflows in software development and maintenance. This includes automation in the release of builds and patches, setting up continuous integration and delivery (CI/CD) processes, managing cloud infrastructure, and ensuring the smooth operation of microservice applications. Overall, a DevOps engineer helps development teams make changes to the product faster and solve user problems.

AIOps, in turn, helps use AI to support existing IT processes. DevOps teams can use AIOps practices and tools to improve code quality and reduce the time it takes for software to be ready for production.

AIOps vs MLOps

MLOps are practices that help developers integrate machine learning models into digital products (mainly products for external clients). This includes model selection, data preparation, training, evaluation, and deployment of the model into real work.

AIOps, on the other hand, uses machine learning models to extract useful information and increase the efficiency of the company's own IT systems.

AIOps vs SRE

Site Reliability Engineering (SRE) is an approach where teams automate system operations and conduct checks using software tools. Instead of manual management, SRE helps to automatically find and fix issues with page visibility and plugin operation, indexing, increasing frontend reliability and improving customer experience.

The goals of AIOps and SRE are similar, but in the latter case, it is a narrower field of application. SRE uses data and predictive analytics based on machine learning to reduce the time to resolve website incidents.

AIOps vs DataOps

DataOps is a set of practices that help organizations optimize data operations for business analytics. With DataOps, so-called "data pipelines" are created, allowing data to be received, transformed, and transmitted from various sources to support the business. An example is the widely publicized "Sberbank credit pipeline," which helps the bank evaluate borrowers in an automated mode.

AIOps, unlike DataOps, uses data provided by DataOps to detect, analyze, and resolve issues in the company's own IT systems.

Conclusion

Most likely, learning "from scratch" to become an AIOps engineer will be even more difficult than DevOps. There will also be a lot of responsibility and a very tight workload compared to the same developers. In addition, working in AIOps will require significant investments, including financial ones, in your own competencies and maintaining them at a high level. Perhaps the first full-fledged AIOps engineers in the labor market will appear as a result of the transition to this profession of specialists from DevOps and Data Science.
We will observe the process of becoming an AIOps engineer 🙂.

Let's add a drop of advertising from our blog: SSP SOFT invites QA and DevOps engineers, system analysts, data analysts, Java, JS, React and Python developers, 1C — see the page on hh.ru. If your specialty is not in the current vacancies, still send your resume, as new positions in the teams open weekly (write to Telegram or email [email protected]).

Good luck in mastering the professions and practices of DevOps and AIOps!