Artificial Intelligence in Data Science: Tools and Limits of Possibilities

Neural networks have not yet become a mandatory requirement for employers, but in practice, many data scientists are already using LLMs in their work: generating code in Cursor, writing SQL queries in DataGrip, and asking for advice from ChatGPT.

tekkix, hello! My name is Vyacheslav Demin, and I have been working in the field of Data Science for over five years. Currently, I am the head of data analytics at Sber and an expert in the course “Data Science Specialist” at Yandex Practicum. I started with the same course in 2020, after which I worked in the insurance and petrochemical industries.

In this material, I will explain what tasks in Data Science I use AI for and why it is still not worth relying on it 100%.

Tasks in Data Science that use neural networks

Code generation and maintenance

The most popular application of AI in development is working with code: autocompletion, generating snippets, explaining logic, refactoring, and generating tests. A neural network can easily explain the functionality of an unclear section of code, translate a snippet from one language to another, continue an unfinished line, or construct a docstring for the developer — a description of a function, class, or element.

My main tool for these tasks is GigaCode, a coding assistant based on GigaChat. It is similar to GitHub Copilot in user scenarios: it helps complete code, explain snippets, and speed up routine actions. The difference is that Copilot usually integrates deeper into the usual IDE stack through extensions, while GigaCode requires separate connection and setup.

There are applications, such as Cursor, where you can customize prompts for writing code. It is a full-fledged environment where you can provide instructions and context for code generation (for example, describing style, requirements, or tasks). Such tools work with the project context — they analyze open files and relevant parts of the code. Theoretically, you can even give them a task to refactor the entire project, although in practice, such a request would require step-by-step control and review.

SQL assistants, which help generate and optimize queries, can also be classified in this category, providing suggestions on how to construct them in a convenient and readable format. It is particularly important to monitor the AI's work here — a data scientist should understand the logic behind how columns, fields, and rows are assembled, and what filtering is used. Neural networks often miss something.

For SQL tasks, both universal AI assistants and specialized tools can be used. For example, DataGrip is convenient for working with queries, refactoring, and analyzing execution plans, while platforms like Microsoft Fabric complement the AI's capabilities at the level of analytical environments. I especially like DataGrip with its query refactoring capabilities, meaning it optimizes already written code. Additionally, it allows you to view the execution plan and understand how the DBMS will execute the query without additional queries and manual checks.

Writing Documentation

AI performs excellently when working with texts. For example, it generates basic files for a project — README and docstrings for all functions.

Also, most modern models can help with summarizing correspondence and documents with client requirements—so that they can be conveniently attached to a ticket in the CRM.

Creating Test Cases

AI complements developers well in creating test cases—especially in generating edge cases and non-obvious scenarios. It is difficult for developers to predict all the situations in which something might go wrong with the code. It doesn't matter whether it concerns development, model creation, or pipelines—AI can set more conditions and discover hidden vulnerabilities and bugs.

Writing Simple Services

Modern LLMs, such as ChatGPT, Claude, Alice AI, GigaChat, Qwen Chat, or DeepSeek, are suitable for quick solutions to small tasks, such as prototyping simple services. This is especially useful if the data scientist's knowledge in this area is theoretical and they lack practical experience.

Using Agents

A separate class of solutions is agent-based pipelines, where LLMs not only respond to requests but also choose the next step, invoke tools, and pass the results further down the chain. They are most often built from scratch using LangChain and LangGraph—resulting in ready-to-use AI assistants for any task.

For example, a graph of some orchestration. You ask the agent a question, and it determines the type of task and directs you to a specific section, graph, or node with a prompt that will work with that topic. Most often, agents are used this way, for routing—that is, to build a chain with automatic action selection after processing a request.

Why Neural Networks Cannot Be Trusted 100%

There are several reasons why neural networks have not yet replaced—and are unlikely to replace—live data scientists; all AI tools have certain limitations.

Do Not Consider All Context

AI performs well in local tasks: describing a function, optimizing a fragment of code, suggesting a continuation of a specific line. But it cannot "keep in mind" the entire model of the system that the developer sees: it does not know all the requirements from the client and the non-functional constraints that AI considers significantly worse.

Offer Non-Standard Solutions (Not Always the Best)

AI does not solve problems from scratch — it relies on patterns from training data. Sometimes this is good, sometimes not so much. There may be hidden bugs in the code and simply non-standard solutions that will be difficult to explain to the team or modify after release.

An example from my experience: once in a fragment of Python code, Copilot suggested a result with an else condition for exiting a loop. Such a construct is technically correct but rarely used and often hard to read, so it is usually avoided in production code.

A developer needs to be prepared for the model to suggest a working but non-standard solution. It is usually better to replace it with a more familiar alternative, or at least try to figure out why the neural network decided to write the code that way.

They produce errors

A common problem: the code seems to work, but is logically incorrect.

When generating in Python, AI may incorrectly set a condition in the if line, add something unnecessary, incorrectly calculate a list of properties from another service via API, or raise an exception that was not in the instructions.

Another option: when AI focuses on a specific part of the request, ignoring other task conditions. For instance, your metric is 0.7, but you need it to be 0.95. The neural network outputs all 0.98, but in reality, it breaks the code: in one of the features, it duplicates the target (target leakage), which causes everything to work in tests, but fails in production after one or two runs.

They treat sources loosely

If we talk about Data Science, neural networks make another unforgivable mistake for us — they can refer to unreliable or unrepresentative sources, using data without confirming statistical significance or scientific explanation. For example, in response to a general request like "tell me about research in this area," the model might provide you with information from a small paragraph in a material as general conclusions. Moreover, the material may not even be a study but simply an article on the internet.

They accumulate errors

A conditional Copilot can generate a function that will work, but the code may not be written optimally. The developer may not notice the flaw and leave it in the code. Over time, such solutions begin to proliferate within the team or project — and become a local "standard," even if they are not optimal. Like an intern who is taught exclusively with bad examples.

How to work with neural networks without losing quality of results

To avoid typical traps when working with AI, it is important to adhere to several good habits.

Decompose tasks

The simpler the request, the more reliably AI responds. That is, adding a couple of lines to the code is easy for it, but performing a multi-component task is more difficult — this approach can lead not only to lengthy deliberations but also to the gradual accumulation of errors during the process and incorrect results.

Challenge AI in data handling

When checking the statistics and conclusions of AI, it is essential not to take its answers at face value. It is better to draw its attention to non-obvious points and ask: why do you think that? Why is this necessary? And where did you get this from?

Before the advent of AI, data scientists always used mathematical approaches and statistical estimates to assess the plausibility of experimental results. These should not be abandoned.

Verify the results of one LLM using another

When working with code, it makes sense to use different models. An LLM may be less critical of solutions built on familiar patterns. To eliminate the intern effect ("I tried, and now the code works, so everything is fine"), it makes sense to consult another model to evaluate the work more critically.

Manually validate the result

When dealing with critical processes, it is important to check the work of LLM not only with another LLM but also with the participation of a live developer. This is true for all tasks related to security, cryptography, and access rights — here, any imagination can lead to incorrect authorization. Or, for example, when working with generating queries on long-designed base storefronts. The nesting in such structures can be infinite — an error in one will result in the final storefront simply not being assembled. AI in such tasks can only act as a suggester.

Furthermore, the developer must understand the significance of each line of the generated code in any case. Suppose the result looks and works correctly, but what if it uses specific libraries or functions that won't work on another machine? What if it is just not optimized?

Even if you configure your LLM for your type of tasks and in two out of three cases it does not hallucinate, this does not guarantee that it won't make a mistake the third time — simply because the neural network worked differently. So validation of AI's work by a live developer is mandatory, at least for now.

Check result reproducibility in a clean environment

Any code needs to be tested on several devices. Usually, we send the project via Git — to GitHub, GitLab, Bitbucket, and other services. If the project lacks requirements, tests, or other configurations that allow reproducing the environment on another device, this will almost certainly lead to problems. In such a case, the code may not run on a neighboring laptop.

This is especially critical when working with AI: without clear instructions and settings, it may generate a solution that cannot be correctly reproduced in another environment.

Set limitations

Perhaps the most important thing: in 2026, the key skill is not just the ability to write good prompts. Yes, it is important to define the role, write the instructions, set limitations, and provide examples. But even more important is the ability to check and limit the result.

For example, in LangChain, there is a convenient formalized approach. You set a system prompt and can supplement it with a "mini-prompt," which increases the likelihood of a correct answer.

This is usually arranged as follows:

• a prompt and parameters are created,

• all this is sent to the LLM,

• the output result is processed by the parser.

The parser defines a clear response format. You specify in advance what data you expect — there may be more or less of it, but the structure is always fixed. Because of this, the result becomes much more stable.

An alternative way to obtain a predictable response is to use the built-in structured output in LangChain through with_structured_output(). In this approach, we do not just ask the model to "respond in JSON," but we describe the expected result schema in advance — for example, through a Pydantic model or JSON Schema. After that, LangChain wraps the model call so that the output is not arbitrary text, but an object with known fields and types in advance. This reduces the number of parsing errors, simplifies subsequent result processing in code, and makes the behavior of the LLM more stable in production scenarios.

Why it's better for juniors to complete tasks independently

I have not yet seen a single job vacancy where knowledge and use of AI was a mandatory requirement. It is certainly a useful skill, but one can get into Data Science without it.

Moreover, neural networks can complicate the solution of new and still complex tasks for beginner data scientists. Suppose we have two juniors who are solving the same task. The first works according to the classical scheme, while the second uses AI.

The first junior researches documentation, collects code from hints on Stack Overflow, and consults a mid-level colleague from their team when problems arise. As a result, after a week, they come back with code that works correctly and is even partially validated by a senior colleague.

The second produces the result — a ready task — in just two days. However, tests show that the code does not meet the specifications, and the junior is poorly oriented in what they wrote. The task has to be redone — in the end, it takes even more time than for the data scientist without AI assistants.

This is an exaggerated example, but it conveys the essence correctly: AI accelerates some processes but can slow down others and increase the path to the final result. Therefore, it can and should be used, but only with an understanding of how generation works and with a willingness to check and refine the result manually.

In summary: Copilot and other tools speed up work but do not replace thinking. AI is an assistant that should be assigned simple tasks, carefully directed, and results checked. And remember: the responsibility for the outcome of LLM work always remains with the person.

P.S. And now tell me, where is the real screenshot here :-)