Sergey Markov: We are in the midst of the summer of artificial intelligence

Let's talk about the development of Data Science in Russia. Over the past 20 years, there have been fantastic changes in this field. Everything that neural networks create becomes the number one news. This is probably the most hyped topic because it is about our common future. About technologies that make our lives easier, more comfortable, more interesting, or threaten unemployment and other problems. And it only depends on us whether this world will become better or not. At least, this is the opinion of Sergey Markov, Managing Director, Head of the Department of Experimental Machine Learning Systems of the General Services Division "Salut".

— Hello, Sergey! Let's start with the hottest topic. What are the main trends in the development of DS in Russia? And how do they differ from global ones?

— In general, the development of data science in our country follows global trends. Of course, you can find some specifics here. For example, we live in conditions of some shortage of computing power. So far, the supply of the most advanced GPUs to Russia and China is prohibited. Therefore, special foreign economic magic is needed to build powerful tensor clusters.

On the other hand, we have the "asset" of the Soviet mathematical school, a developed IT industry, and a modern ecosystem with information technology infrastructure. And this implies a developed culture of IT development, the availability of specialized personnel, large and well-structured data sets.

— So the issue of resources is no longer so relevant?

Of course, the issue of resources is relevant, both human and computational. It is unlikely that we should expect the appearance of a magic wand that will allow us to create advanced machine learning models without computational costs and specialists. Of course, machine learning algorithms are improving. Their specific implementations are being optimized, so training models with a conditionally identical "level of intelligence" already requires significantly fewer GPU hours.

In the sensational article by former OpenAI employee Leopold Aschenbrenner "Situational Awareness: The Coming Decade," there is an estimate. In my opinion, it is somewhat optimistic. According to him, progress in this area in the next 2 years will allow training models of the same "power" on the same equipment 10 times faster. Approximately the same effect is given by the development of hardware platforms. Additional effects allow more to be "squeezed" out of already trained models. The methods that allow this to be done, Aschenbrenner combines under the umbrella term "unhobbling." Literally: "unhobbling," liberation. This is the application of a multi-agent approach, reasoning modeling: chains, trees, and graphs of reasoning. And also scaffolding: creating an algorithmic "framework" around a large language model.

But it is important to understand that over time the effect of all these improvements will decrease. Exponents of this kind cannot continue indefinitely. It would be interesting to talk about the reasons for this in more detail, but this will not fit into the format of the interview.

— Let's return to the question of what to do in these difficult conditions in which we find ourselves, regarding resources?

— I consider two directions to be the most important. The first is the study of the possibilities of creating alternative computing platforms. Up to the abandonment of classical electronics and the search for other physical substrates for computing. And the second is the creation of highly efficient research pipelines. This will allow us to use our computing resources most effectively. For this, in essence, it is necessary to "industrialize" research, creating hardware and software platforms that allow quickly testing ideas on small-scale models, and then scaling the most promising ones.

— Such research requires a very powerful base. How did you come to Data Science yourself? Where did you study, what did you do before that?

I am a hereditary programmer)) My father was a mathematical engineer, and my mother was a university lecturer. I graduated from Oryol State Technical University with a degree in "Information Systems in Economics", as well as from the Oryol Regional Academy of Public Administration with a degree in "State and Municipal Administration", then I completed postgraduate studies at OryolSTU (now OGU) at the Department of Higher Mathematics.

Started programming at the age of 7. Participated in school and student olympiads in computer science and programming. In 2002, with his team, he reached the finals of the ACM International Collegiate Programming Contest. He became interested in artificial intelligence while studying at the university. He started with programs for chess and other board games. Therefore, the development of AI systems for games became his first profession — in the early 2000s, he and his friends were engaged in mobile development (his first work project was the Ultimate Brain Games package for the GameBoy Advance platform). At the same time, he did not abandon hobby projects — his chess program SmarThink won the Russian championship in 2005, and the CIS championship in 2006.

— So you have been involved in artificial intelligence from the very beginning?

— In general, yes. In the late 2000s, I was already developing information systems and AI models for the financial sector. Since 2012, I have been working at Sberbank. For 12 years now, I have been developing and leading research groups and projects related to the development and application of AI systems. With my participation, ruGPT-3, ruGPT-3.5, GigaChat, ruDALL-E Malevich, Kandinsky, and mGPT were created.

In addition, I try to popularize science. Together with friends, I founded the popular science portal "XX2 Century" about science and technology. I publish scientific articles in domestic and international scientific press and give popular science lectures on AI. I released the book "Hunting for Electric Sheep. The Big Book of Artificial Intelligence", co-authored the book "Strong Artificial Intelligence: Approaching Superintelligence", and was the scientific editor of several translated editions on AI and machine learning.

I try to maintain a broad outlook and continue to develop.

— How else can a Data Science specialist develop in our time?

— I would say that the set of tools is the same as in other IT fields. It is worth reading modern literature, primarily scientific articles and reviews. It is important to practice regularly. Work and hobby projects, competitions like those published on the Kaggle platform are suitable. In general, as everywhere, you need to keep your finger on the pulse and improve your "weak sides". This can be done by reading, listening to lectures, communicating in the professional community, participating in conferences and hackathons.

— There is a scare story that soon neural networks will take away the jobs of many specialists, including in the IT field. How do you feel about this, do you consider such fears real?

— Due to the development of AI technologies, there will be not so much elimination as transformation of professions. For example, yesterday a doctor could rely on the knowledge he received at the university, but today this is not enough. You need to constantly be aware of ongoing changes. And for this, you need to be able to use specialized search engines and constantly learn new things.

The work of a programmer has been like this from the very beginning. Libraries, frameworks, and entire programming languages become obsolete within a few years. Of course, we no longer need to punch holes in punch cards ourselves or debug code manually. Advanced software development tools have increased programmer productivity. Compared to the 1940s-50s, probably by a hundred times. In the late forties, there were about a hundred programmers in the world. If the increase in productivity had simply replaced people with machines, then by now there would be only one programmer left in the world, and the work of the remaining 99% would have been taken over by "smart tools". But we know very well that this is not the case — scientific and technological progress is not a "zero-sum game". Thanks to the reduction in the cost of creating information systems, their scope of application has significantly expanded — therefore today there are not one, but tens, and possibly even hundreds of millions of IT specialists in the world.

We know that the development of new technologies also leads to the emergence of new jobs. Over the past 200 years, the unemployment rate has changed little overall. For example, thanks to progress in the field of IT, the computer game industry has emerged. It has created jobs not only for IT specialists, but also for designers, marketers, managers, and HR staff.

Of course, there are a small number of professions that will transform significantly, and may even disappear altogether. This is what happened to coachmen with the advent of automobiles. Call center operators are already being actively replaced by robots. And not only due to the increase in the number of robot operators, but also because more and more people are abandoning voice communications in favor of specialized applications and chats. Therefore, the question of the social consequences of automation is very important - it cannot be ignored. Although on the scale of the economy in the long term we are unlikely to face mass unemployment, for some people the question of the need to change professions may become very acute. Therefore, we need mechanisms for retraining specialists and social support for those people whose skills may become unclaimed in the new world.

The problem with new technologies is not that people have nowhere to apply themselves. Society will benefit from additional labor. For example, if the number of teachers is increased, school classes can be reduced to five to seven students, and young mothers could use the help of qualified nannies. Road construction and additional housing, exploration of the world's oceans, engineering, science - additional workers will be beneficial everywhere. The only problem is how to ensure the retraining of a large number of people in a relatively short time. How to direct the savings of resources resulting from automation to the benefit of society as a whole, and not to individual companies and a handful of super-rich people.

— What areas of application for neural networks do you see in the future?

— Listing the areas of application for neural networks is akin to listing the ships sent by the Greeks against Troy. It will take a lot of time before we even get to the middle. It is much harder to list the areas where neural network models will not be applied.

Not everyone realizes how closely artificial intelligence technologies have entered our lives. We use neural networks when we take photos on a smartphone, send a query to a search engine, browse social media feeds, use voice commands, machine translation systems, and when we apply for a loan at a bank. The use of AI technologies will only expand.

— What is the future of Data Science? Where do you think everything will develop?

— IDC, with the sponsorship of Seagate, conducted a study on the dynamics of the volume of the "digital universe." If in 2018 there were 33 zettabytes of data, by 2025 it is forecasted to be 175 zettabytes. If all this is recorded on DVDs with the maximum available recording density today, it will result in more than 10 trillion disks. If we stack them together, we will get a pile more than 12 million kilometers high. This is about 30 times the distance from Earth to the Moon. According to IDC, the "datasphere" of humanity will increase by approximately 1.27 times per year.

Computing power available to us is growing at a rapid pace. Its growth chart at the current stage of development also resembles an exponential curve. Significant progress is also being made in improving machine learning methods. In short, we are in the midst of the summer of artificial intelligence, which means that data science is in a phase of rapid development. I am not a futurist, not a science fiction writer, and not a magician to fit a picture of the supposed future into a couple of paragraphs of text. But it can be confidently said that under the influence of today's developing data analysis and processing technologies, the world will change radically. And whether this world will be better depends on people, on us. After all, the same technologies can be used both for the benefit of society and to its detriment. That is why the era of rapid technological development places increased demands on our human qualities.

— Who is currently developing the rules for the use of neural networks? Are there communities and how are they developing?

— In our country, this is done by the Alliance in the field of artificial intelligence. This is a self-regulating organization that brings together leading technology companies that develop and apply AI technologies. Its members have developed the "Code of Ethics in the field of AI". You can find out more on their official website.

In general, this topic is very relevant. Many people talk about it. For example, the issues of AI application are actively discussed in the academic community and at the state level. This needs to be discussed and rules need to be formed to help control the development of AI and its areas of application. And for this, professional communities are needed to help with expertise. They still need to be created, as there are few of them so far. Therefore, everything is in our hands. Now everyone can participate in this process and make their contribution. For example, come to the largest AI conference in Russia, AI Conf 2024, on September 26 in Moscow. You can listen to speakers, and industry experts, and discuss industry trends and their practical application.