30 seconds instead of 30 minutes: how we automated the generation of streaming processing configurations using RAG and A2A

Imagine a typical situation: you need to set up another stream data processor. You open the documentation, which is over 200 pages long, describing dozens of source types, transformations, filters, and error handling policies. You search for the needed parameters in sections about Kafka, RabbitMQ/ArtemisMQ, GRPC, PostgreSQL, and so on. You recall the DSL syntax — and it differs for different operation types. You copy a similar configuration from a previous project, found in a corporate Git repository, and adapt it to the new requirements. You switch between browser tabs with documentation and the IDE.

This takes from 30 minutes to several hours, and in the end, you can still make a syntax error, misconfigure the data source, or miss an important security parameter.

But what if we could shorten this process to 30 seconds? Simply describe the task in natural language — "Set up a processor to filter events from Kafka, keep only records with user_id greater than 1000 and high priority, send the result to a new topic" — and get a ready-made configuration adapted to the specifics of our product. Not an abstract YAML or CONF template that still needs to be adjusted for the product, but a configuration with correct parameters, valid DSL syntax for transformations, and adherence to the product's architectural patterns, ready for review and use in a production environment.

In this article, I will tell you how we created a system for automatically generating configurations for one of our product's components, using RAG (Retrieval-Augmented Generation), vector databases, and multi-agent interaction via the A2A protocol.

On the Problem Statement

Engineers working with data stream processing systems face the need to create configurations for various systems on a daily basis. These include data handlers, complex multi-step transformations, error handling policies, and event routing rules between services. At the same time, they have to study extensive documentation that spans hundreds of pages, where each component of the system is described in a separate section.

Developers must remember the syntax of multiple formats: YAML/JSON for main configurations and specific formats for describing security policies. However, the most challenging part is working with proprietary DSL for describing data transformations. Creating a single configuration of average complexity takes anywhere from 30 minutes to several hours. During this time, engineers switch between tasks, lose context, and get distracted by searching for information.

The classic approach - copying from previous projects with manual adjustments - is not only slow but also prone to errors. Every typo in a parameter name, incorrect indentation in YAML, forgotten mandatory parameter, or outdated syntax version from an old configuration turns into lost time spent on debugging. And if it concerns a production environment, the cost of an error can be very high.

Moreover, documentation is constantly updated. New parameters appear, some become deprecated, and recommendations change. An engineer might not be aware of the latest changes and use outdated approaches. This problem is particularly acute with proprietary DSLs: unlike standard formats like YAML or JSON, for which there are many examples on the internet, the specific syntax of transformations is only described in corporate documentation.

Specifics of DSL for Transformations

Platform V Synapse Streaming Event Processing uses a proprietary DSL to describe transformations of streaming data - tr-files. This is a declarative language specially designed to describe transformations visually without the need to write imperative code in Java or Python.

The DSL allows describing field mapping — transforming the structure of an incoming event into the required structure of the output, with renaming fields, extracting nested values from JSON, combining multiple fields into one. Event filtering is implemented through expressions with support for complex conditions, logical operators, functions for working with dates, strings, numbers, and regular expressions. There is also data enrichment from external sources. The system supports working with aggregation windows, grouping by keys, and applying aggregate functions.

Here is an example of a simple transformation in this DSL:

define isFirst = true
for ($data: in) {
    if ($isFirst) {
        out[>] = $data
        headers[>].TargetKey = generateId()
        define isFirst = false
    } else {
        out[+>] = $data
        headers[+>].TargetKey = generateId()
    }
    INFO("RESULT", out, "TEST.OUT", "-")
}

For an LLM, this DSL is non-standard; it was not present in the training dataset of GigaChat or other public models. The model cannot simply "recall" the syntax, as it could with Python or SQL. An attempt to generate a tr-file without additional context leads to hallucinations: the model might invent non-existent functions, use syntax from other similar languages, or confuse the order of sections.

This is precisely why RAG becomes not just an improvement, but a critically necessary component of the system. Without access to up-to-date documentation and examples, it is impossible to correctly generate configurations in a proprietary DSL.

From Idea to Ready Configuration in 30 Seconds

We have created a system that transforms natural language descriptions of requirements into a correct configuration in seconds. The key idea is to use the product's own documentation as a knowledge source for the AI, rather than trying to train a model from scratch on a specific DSL.

The process begins with a preparatory stage: loading all product documentation into a vector database. We take the official documentation in Markdown format; a special service splits the documents into logical fragments (chunks) and converts them into vector representations — embeddings. This process needs to be performed only once during the initial system setup; the vector database is then updated only when the documentation changes.

An engineer describes the configuration requirements in natural language, and the magic begins.

The request can be formulated simply and clearly: "Create a handler that reads events from the user-actions topic, filters only events of the purchase type with an amount greater than 1000 rubles, enriches it with user data from an additional topic, and sends the result to the high-value-purchases topic." The system does not require knowledge of the exact syntax or parameter names - it will find the necessary information in the documentation itself.

The system analyzes the request and, through the RAG mechanism, finds the relevant context in the documentation. This is not a simple full-text search by keywords, but a semantic search by meaning. The system understands that "reads events from a topic" is related to the documentation section on Kafka sources, "filters only events of the purchase type" requires filter configuration, and "sends the result to a topic" requires destination configuration.

The LLM receives the found documentation fragments and generates a ready-to-use configuration, taking into account the context. The model does not simply generate text randomly - it relies on specific examples from the documentation, uses the correct parameter names, and follows the syntax.

The result is a configuration that can be applied immediately without further refinement; all mandatory parameters are filled with correct values, the requirements for the structure of the product's configuration files according to the current version are met, and the correct DSL syntax is applied for transformations (if required).

This means that the engineer does not need to spend time searching for documentation on product versions, remembering specific parameter names, or understanding the intricacies of DSL syntax: the system does this automatically based on up-to-date documentation.

But generation is only half the story.

During configuration creation, the agent-generator using the A2A protocol calls an agent-validator. The validator receives the generated configuration and the original user request, then checks whether the configuration actually solves the stated task? Are all mandatory parameters specified? Are there no logical errors or potential performance issues? The validator can respond with "configuration is valid" or request corrections: "missing required parameter field_name in the filters section" or "for production environment, I recommend adding SSL connection."

Agents interact iteratively until a valid configuration is obtained. This may take several cycles: the generator creates the first version, the validator finds issues, the generator fixes them based on feedback, and the validator checks again. Usually, one to three iterations are required. This process is fully automatic and takes seconds, not hours of manual debugging.

After the configuration passes the AI check, it can be further verified using static methods. The Config Validator checks that YAML is syntactically correct, JSON schemas are valid, and if the configuration is intended for deployment in Kubernetes, it verifies compliance with the manifest specification.

Custom checks can be configured via regular expressions: for example, to ensure topic names comply with corporate naming conventions. The final result can be immediately applied to the cluster via the built-in deployment mechanism or copied and used in an existing CI/CD pipeline.

Vector databases: how AI understands documentation

Before moving to architecture, let's explore the system's foundation—vector databases and the embedding mechanism. This is the key technology enabling semantic search and contextual generation.

What is an embedding

An embedding is a way to represent text as a numerical vector of fixed length, typically from 384 to 1536 dimensions depending on the model.

Each number in this vector is a coordinate in a multidimensional space, where the point's location reflects the semantic meaning of the text. The key feature of embeddings is that semantically similar texts have close vectors, and texts can be found by computing the distance or cosine similarity between vectors.

For example, consider three phrases from documentation: "Kafka source" can be represented by the vector [0.82, -0.33, 0.15, 0.67, ...], "Kafka input" by the vector [0.81, -0.35, 0.14, 0.65, ...], and "read from the message queue" by [0.77, -0.33, 0.14, 0.63, ...].

All three phrases describe a similar concept of retrieving data from a message broker, and their vector representations will be close to each other in the multidimensional space. The distance between the first and second vectors will be small, while the distance between the first and a completely unrelated phrase like "configuring network policies" will be much larger.

It's important to understand that embeddings don't simply count common words. The model that creates embeddings is trained on vast amounts of text and understands context, synonyms, and relationships between concepts. Therefore, "Kafka source" and "read from a topic" will have similar embeddings even if they share no common words.

Documentation Indexing

When a document is loaded into the system, several important processing stages occur.

First, the document is divided into semantic fragments — chunks. This is a critical step because the quality of the splitting directly impacts search quality. Chunks that are too small lose context; chunks that are too large become non-specific and may contain several unrelated topics.

We adjust the chunk size depending on the documentation's specifics. For reference documentation with short parameter descriptions, the optimal size is 256–512 tokens. We use overlap between chunks of 10–20% of the chunk size. This means the last few sentences of one chunk are repeated at the beginning of the next. Why? To preserve context at the boundaries. If the description of a concept falls on the boundary between chunks, the overlap ensures that the description is presented in full within one of the chunks.

We convert each chunk into a vector using Sber's Embeddings model. We chose it because it is specifically trained on Russian language and creates high-quality semantic representations of Russian-language technical texts. The model understands specific terminology, abbreviations, and technical jargon.

Vector representations are stored in Platform V Vector DB — a specialized vector database designed for high-performance semantic search. Along with the vectors, we store the original fragment text and metadata: document source, section, update date, tags (e.g., kafka, filters, security). This allows for precise search with filters — for example, only materials for Kafka or only sections related to security.

For example, a document with network policy configuration rules can be split like this: the chunk "Deny all incoming traffic by default" will get a vector [0.76, -0.21, 0.94, ...] and metadata {source: "network-policy.yaml", section: "ingress-rules", tags: ["security", "network"]}, while the next chunk "Allow port 80 only from the frontend namespace" will get its own vector [0.68, -0.19, 0.91, ...] and similar metadata. When searching with the query "configure access to the service only for the frontend," the system will find both of these chunks as relevant, with the second one scoring higher.

RAG: Retrieve-Augment-Generate

Let's understand why simply using an LLM is insufficient for our task. Modern language models like GigaChat, GPT, or Claude have impressive text generation and context understanding capabilities. But they have fundamental limitations.

First, the model may not know the specifics of your documentation. GigaChat is trained on public data from the internet, but our internal documentation on Platform V Synapse Streaming Event Processing, proprietary DSL, and corporate standards was not in the training set.

Second, the model's knowledge is limited by its training date. If the last training was six months ago and three product updates with new parameters have been released since then, the model won't know about them.

Third, the model can "hallucinate": invent non-existent parameters, mix syntax from different versions, and generate configurations that sound plausible but are incorrect.

RAG (Retrieval-Augmented Generation) is an architectural pattern where a model doesn't solely rely on the knowledge it gained during training. Instead, it first retrieves relevant context from an external source - in our case, from a vector database containing documentation. This fundamentally changes the approach: the model becomes not a source of knowledge, but an interpreter of documentation that understands the user's query and correctly applies information from the retrieved fragments.

Three stages of RAG

Retrieve (Search). When a user submits a query, the system doesn't immediately send it to the LLM. First, the query is vectorized - transformed into an embedding using the same Embeddings model that was used for indexing the documentation. This results in a query vector. Then, the system searches the vector database for chunks with the most similar vectors by calculating the cosine similarity between the query vector and the vectors of all chunks in the database.

This is a semantic search: we're not looking for exact word matches, but for meaning. For example, a query like "configure event filtering by timestamp field greater than yesterday's date" will find relevant fragments even if the documentation uses different wording like "time filtering", "time cutoff", "filter by time field", or examples with other field names. The model understands that all these phrases are semantically close.

The system returns the top-K most relevant chunks, typically K=3-5. More isn't always better: too much context can confuse the model, increase the tokens in the prompt (and thus, cost and processing time), and may include less relevant information. We've experimentally determined the optimal value for our system.

Augment (Enrichment). The retrieved chunks aren't simply concatenated; they're structured and formatted. A prompt is formed, consisting of several sections. The system_prompt sets the role of the agent and rules of behavior; here's part of the system prompt:

You are a generator of configuration files and DSL transformations for Cloud Event Processing, operating strictly according to the specification and provided context.

Main requirements:

Use only the information from this prompt.

Generate only syntactically and semantically valid configurations and DSL files according to the specification or provided context.

All mandatory fields must be filled. If it is impossible to correctly fill a mandatory field, do not generate the file.

Do not use constructs, blocks, parameters, functions, or syntax not described in the specification or provided context. Any deviation is an ERROR.

Do not add explanations, descriptions, comments, or fictional elements to any of the files.

If multiple files are needed (e.g., config and transform), output each with its name and extension as separate blocks.

To improve generation quality, use the information provided in the context field.

The file name and extension should be the first line of the block. Duplicate names are not allowed.

Context includes relevant fragments from documentation, specially formatted. Each fragment is preceded by a header indicating the source: "From the section 'Kafka Sources Configuration':", "Example from documentation:", "Best practice:". This helps the model understand where the information comes from and how to interpret it. If code examples are found, they are included in full with comments.

User_request is the original user request, possibly slightly rephrased for clarity. For example, if the user wrote "make the handler like last time, but for a different topic", the system may ask to clarify details.

Generate (Generation). The fully formed prompt is sent to GigaChat. An important nuance: we use specific generation parameters. Temperature is set to a relatively low value around 0.5, which makes generation more deterministic and predictable, which is critical for code and configuration generation. High temperature (0.8-1.0) is good for creative tasks, but in our case it can lead to "creative interpretations" of the syntax.

LLM generates a response based on both its basic knowledge of YAML/JSON formats and general configuration principles, and the provided context from the documentation. This ensures that the configuration will comply with the current product specification, use the correct parameter names, adhere to the API version, and follow best corporate practices.

Microservice architecture of the solution

The system is built as a set of independent microservices, which provides flexibility, scalability, and the ability to update individual components without stopping the entire system.

Each service is a separate Docker container with its own resource configuration, restart policies, and health check endpoints. Interaction between services occurs via REST.

UI Service

This is a single entry point for engineers. The functionality includes managing the full cycle of working with configurations. This is just one possible implementation; the system is designed so that other services can be accessed directly via REST API, integrating them into existing CI/CD processes.

For example, you can set up a Jenkins pipeline that calls the Config Generator API to automatically generate configurations based on parameters from a Git repository, checks the result via Config Validator, and applies it through Config Deployer. The interface is a presentation layer for interactive operation but not a mandatory link.

DB Service

This is the central metadata repository of the system and a REST API for working with the DBMS. The database stores system prompts for various configuration types. Each configuration type has its own optimized prompt with instructions for the model, examples of the desired output format, and lists of mandatory and optional parameters.

Validation rules are stored in a structured way: regular expression patterns for checking names, ranges for numeric parameters, and lists of allowed values for fields with enumerations. Deployment templates are Jinja2 templates for Kubernetes manifests with placeholders for inserting generated configurations.

Vector database metadata includes the database name, description, creation date, number of chunks, used embedding model, chunking parameters (size, overlap), status (active/inactive), and tags for filtering.

Generation history is fully journaled: request timestamp, user_id, query text, configuration type, used vector databases, found context, generated configuration, validation results, deployment status (if applied). This data is invaluable for analyzing system performance quality and its improvement.

Vector Generator

This is an ETL pipeline for documentation. It is responsible for creating and populating vector knowledge bases, transforming unstructured text into vector representations that can be searched (searchable embeddings).

The process starts with creating vector databases: initializing a new database (collection) in Platform V Vector DB with specified parameters. Then document processing follows: loading files of various formats with automatic format detection and parsing.

The next step is chunking, where the chunk size is configured individually for each vector database—256 or 512 tokens. Overlap is also configurable: typically 10-20% of the chunk size, but for technical documentation with many cross-referenced examples, it can be increased to 30%. After this, embeddings are generated.

Metadata synchronization is the final stage, where information about the vector database is registered in the DB Service for further use by other components. The Vector Generator can also incrementally update existing databases: add new documents, update changed ones (based on content hash), and delete obsolete ones. This allows maintaining up-to-date vector databases without full reindexing.

Vector Loader

A key component for implementing RAG, the bridge between the user query and the documentation. This service can handle many concurrent requests in parallel.

It starts by querying active vector databases from the DB Service. You can filter which databases to use for specific configuration types—for example, when generating a Kafka source, search only in databases tagged with "kafka" and "sources," ignoring irrelevant ones.

Context formation involves the top 5 chunks by similarity score with the original query.

The result is an ordered list of chunks with metadata, ready to be inserted into the prompt.

Config Generator

The heart of the system, the orchestrator of the entire generation process. This is an asynchronous service for working with the GigaChat API. When it receives a request from a user (via the interface or directly through the API), a workflow is launched. First, the Vector Loader is automatically called to obtain relevant context—this is a synchronous call; the Config Generator waits for the result. Then, a system prompt specific to the type of configuration being generated is loaded from the DB Service.

The full prompt is assembled, combining the system prompt, context, and user query. Order matters here: first, the system prompt establishes the role and rules, then the context provides knowledge, and only then does the user query specify the concrete task. This helps the model properly prioritize information.

The prompt is sent to GigaChat with generation parameter configurations.

After generating the configuration, the Config Generator does not immediately return it to the user. Instead, an automatic validation process is triggered. In this process, the Config Generator acts as a coordinator in a multi-agent system, facilitating interaction via the A2A protocol to coordinate with the AI Validation Agent. This is an asynchronous callback process: the Config Generator sends the configuration for validation and subscribes to notifications about the result.

AI Validation Agent

This is the second AI agent in the system, specializing in validation. An important aspect: it is a separate service with its own prompt and model, not part of the Config Generator. Why? Because generation and validation tasks require different approaches, different prompts, and different model parameters.

The agent receives a validation request via the A2A protocol containing the generated configuration, the original user query, and context from the documentation (the same chunks used during generation). This is critical: the validator must see the same context to evaluate correctness against the current documentation, not abstract notions of "correctness."

Several types of analysis are performed. Correctness analysis checks whether the configuration actually solves the stated task—if the user asked to "filter by user_id > 1000," is there a corresponding filter in the configuration? Are the parameters correctly specified? Completeness analysis ensures all required parameters are provided: for each configuration type, there is a schema with mandatory fields, and the validator cross-references it.

Style analysis checks adherence to best practices and company standards—for example, whether recommended default values are used, whether all necessary fields for cybersecurity requirements are set, and whether logging is enabled. This is a straightforward check: the configuration may be technically correct but still fail to meet company standards. The validator flags this as a warning, not an error.

Security analysis—one of the most critical—identifies potential issues: whether TLS is used for data transmission, whether secrets are hardcoded in the configuration. The validator is trained to recognize patterns of insecure configurations.

The agent returns a structured response: an overall verdict (valid, invalid, warning), a confidence score (how confident the agent is in the assessment, from 0 to 100), a list of issues (each with a priority indication: error, warning, info, location in the configuration, and a description of the issue), and general recommendations for improvement. If errors are found, Config Generator can automatically request regeneration, including a description of the issues found in the prompt ("the previous version had error X, fix it").

Config Validator

This is a level of protection against syntax errors and schema violations, which works in parallel with the AI Validation Agent. A classic validator without AI, but no less important.

Checking YAML/JSON formats uses yamllint and built-in Python parsers with detailed error messages — not just "invalid YAML", but "line 15, column 3: expected indentation of 2 spaces but found 4". Checking Kubernetes manifests verifies compliance with the Kubernetes API specification: whether apiVersion is correctly specified, whether such a kind exists, and whether all required fields are present in metadata/spec.

Regex checks for specific patterns: you can set rules like "topic names must match the pattern ^[a-z0-9-]+$", "ports must be in the range 1024-65535", "namespace cannot be 'default' for production". Dry-run in Kubernetes — a final check before actual application: sending the manifest to the Kubernetes API with the dryRun=true flag, the API checks correctness without actually creating resources.

The results of all checks are combined into a single report with categorization by types of remarks and recommendations for correction.

Config Deployer

The final link in the chain, responsible for the safe deployment of configurations in Kubernetes.

The system supports two modes of operation. In direct manifest generation mode, the model immediately creates a ready-to-use Kubernetes manifest with a full Deployment, Service, ConfigMap specification, and all necessary fields. This is faster but less flexible. In templating mode, pre-prepared manifest templates with corporate standards are used: labels for monitoring, annotations for service mesh, imagePullSecrets, resource requests/limits, liveness/readiness probes. The LLM generates only the business logic of the configuration (e.g., application settings), which is substituted into the template.

The second approach ensures consistency of deployments, compliance with corporate policies, and integration with the existing infrastructure. The template creation mechanism is based on Jinja2 with custom filters and functions for handling specific cases.

Full lifecycle of the system

Let's follow a real example to track the detailed path from request to applied configuration.

User — engineer Alexey — received a task: he needs to configure a handler to filter events from a Kafka source, leaving only events from users with user_id greater than 1000 and priority high, and send the result to a new topic for further processing. Previously, Alexey would open the documentation, find examples of Kafka source configuration, then sections on filters and destination, and assemble it all together.

Now Alexey simply opens the interface of our system, selects the Cloud Event Processing configuration type, and enters into the text field: "Configure a handler to filter events from Kafka source topic user-events, leave only events with user_id > 1000 and priority == 'high', send the result to topic high-priority-users". He clicks "Run generation".

Vector Loader receives the request, vectorizes it through Embeddings, and searches in vector databases. It finds top-5 chunks: "Kafka source configuration with an example of specifying bootstrap servers and topic", "Syntax of filter expression for numeric fields", "Syntax of filter expression for string fields and enum values", "Example of combining multiple filters", "Destination configuration for writing to Kafka topic". These chunks are returned to Config Generator with similarity coefficients 0.89, 0.87, 0.85, 0.82, 0.79 respectively.

Config Generator receives a system prompt from DB Service, which contains the instruction: "Generate configuration in CONF format. Structure: source, aggregationStep, transformStep, destination. Use exact syntax from examples. Do not invent non-existent parameters". It combines the prompt, context, and user request to form a complete prompt.

Sends to the GigaChat API with carefully selected generation parameters. Temperature=0.5, why this particular value was chosen was discussed earlier. However, we also pass another parameter to the model — max_tokens with a value of 1500, which limits the response length. This is sufficient for medium-complexity configurations: a typical Cloud Event Processing configuration takes about 300 tokens. The buffer is needed for comments and possible explanations from the model. A too small limit (500-800) would truncate complex configurations in the middle, while a too large one (3000+) would allow the model to generate redundant content: additional examples, explanations, alternative options that would complicate parsing the result.

The Config Generator via the A2A protocol automatically sends this configuration to the AI Validation Agent. The validator analyzes it in the context of the original request and the documentation found. Checks: is there a filter for user_id > 1000? Yes. For priority == 'high'? Yes. Is the destination set to the correct topic? Yes. Are all required fields present? Checks source (type, config.bootstrap_servers, and config.topic are present), transformStep (structure is correct), destination (type and config are present) — everything is in place.

The validator notes that a placeholder ${KAFKA_BOOTSTRAP} is used instead of a fixed address — this complies with company standards and is marked as positive. Checks security: are credentials exposed in plain text? No. Is group_id used to avoid duplicate processing? Yes. The confidence score is calculated as 0.94 (very confident).

The validator returns a verdict: {status: "valid", confidence: 0.94, issues: [], recommendations: ["Consider adding an error handling policy for failure messages"]}.

The Config Generator receives a positive verdict, launches the Config Validator for static checking. The validator checks the generated CONF file using schema-based static validation.

In the interface, Alexey sees the generated configuration with a green checkmark "Validated" and recommendations in the form of informational hints. The configuration looks correct. Alexey clicks "Install configuration to namespace".

Config Deployer receives a request, loads a Kubernetes manifest template for the Cloud Event Processing component from DB Service, substitutes the generated configuration into the template for deployment. It performs a dry-run via Kubernetes API. The API returns success — the manifest is correct and will be accepted. Deployer applies the configuration. Kubernetes creates a resource, and a Pod with the Cloud Event Processing component is launched. Config Deployer monitors the state: it waits until the Pod transitions to the Running state and checks the readiness probe. After 15 seconds, the Pod is ready, and the health check is green. Deployer saves a record to DB Service: applied successfully, timestamp, user: Alexey, namespace: dev, as well as the ready manifest.

In the interface, Alexey sees "Deployed successfully to dev namespace", a green indicator, and a link to the Pod in the Kubernetes dashboard. It took 30 seconds from the request to the working configuration in the cluster. Alexey looks at the monitor with a smile.

How the system works with proprietary DSL

Let's return to the question of working with tr-files — a proprietary DSL for transformations, which we mentioned at the beginning of the article. The technical implementation of vectorizing DSL examples has its own peculiarities.

In the vector database, we store not just the text of documentation, but specially structured examples: complete working tr-files, broken down into semantic blocks with annotations (e.g., "example of aggregation with a time window"), comments explaining non-obvious constructs, and typical usage patterns with variations. Each example is vectorized as a whole, without breaking it down into small chunks — this is crucial for preserving the syntactic integrity of the code.

When a user requests the generation of a transformation, Vector Loader finds several most relevant examples of tr-files based on the semantic similarity of the task. LLM receives complete examples in the prompt and uses them as templates, adapting the syntax to a specific request. This is a few-shot learning technique in the context of RAG: the model learns from examples during execution, without retraining.

As a result, we achieve high accuracy in generating tr-files for a proprietary language that is not present in the model's training data.

Security and Privacy

Working with corporate documentation and production configurations imposes strict security requirements. All documentation is stored in a private vector database within the company's perimeter, rather than in the cloud. We do not send sensitive data to external APIs. The AI Validation Agent is also trained to find potential leaks of sensitive data in configurations and mark them as a security issue.

Conclusion

We have created an end-to-end solution for automatically generating configurations for streaming data processing, which reduces the time it takes to create a configuration from hours to seconds, eliminates copy-paste errors and typos through automated validation. It automatically applies best practices from documentation via RAG, ensures two-level validation — static and intelligent AI validation, supports inter-agent interaction via the standard A2A protocol for extensibility.

RAG proved to be an ideal solution for generating configurations on proprietary DSL. It allows the model to work with relevant and specific documentation that is not available in the training dataset, reduces the risk of hallucinations by relying on real examples from documentation, and is easily updated — simply by uploading a new version of the documentation to the vector database without retraining the model. It scales to any type of configuration and DSL, making it a universal approach.

The key technologies used in the project are: RAG for contextualizing generation, Platform V Vector DB vector databases for semantic search by meaning, Embeddings for creating high-quality embeddings of Russian-language technical texts, GigaChat as the primary LLM for generation and validation, A2A protocol for inter-agent interaction and extensibility, and microservices architecture for flexibility and scalability.

This solution can be adapted not only for Platform V Synapse, but also for any enterprise systems with extensive documentation and configurations. The architecture is universal — only the content of the vector databases and system prompts change.

Additional materials

• A2A Protocol Specification

• Platform V by SberTech

• Embeddings

• GigaChat

If you have experience implementing RAG or generating code or configurations using LLM, please share in the comments! It's interesting to compare approaches and challenges faced by other teams. Especially interesting are examples with proprietary DSLs, techniques for combating hallucinations, and prompt engineering strategies for code generation.