From Ideas to Code: Testing Attention Theory in Practice

In the first two texts, I discussed personal worldviews, how knowledge is embedded in experience, the role of attention in this process, and broadly touched upon the topic of how attention manifests through user behavior on modern platforms. The main focus then was on formulating ideas and discussing them. In this article, I continue along the same line and move to a practical step - testing these ideas in the form of a working experiment.

Experiment Form

The experiment is implemented as a web application Mindstream:

• application: https://mindstream.app.wiredgeese.com/

• source code: https://github.com/flancer32/mindstream

The application consists of server and browser parts. The server part works the same for all users: it reads publications from tekkix via RSS, generates annotations and brief reviews for them using language models, and based on them, compiles a general feed of publications.

The browser part is responsible for personalization. When entering the page, the user sees a feed of articles with annotations and reviews, as well as markers of personal interest. Upon first use, all interest values are zero, and there are no preliminary preferences.

Interaction with the feed is structured simply: the user reads the title of the publication and a brief annotation, opens and reads the review, and if interested, follows the link to the original publication. These actions are recorded as signals of attention. Based on them, a personal vector of user interests is formed directly in the browser, against which subsequent publications in the feed are evaluated.

Local Computations and Identity

All calculations of personal interest are performed and stored locally, in the user's browser. By default, no data about the user's behavior is sent to the server.

If desired, the user can create an anonymous identity in the form of a UUID. In this case, attention signals begin to be duplicated on the server. Currently, this data is only stored and not used to form collective signals or recommendations. Collective effects are consciously excluded at this stage of the experiment.

Observing the Formation of Interests

As one interacts with the Mindstream feed - reading annotations, opening reviews, and transitioning to original articles - a vector of their own interests is formed in the user's browser. This numerical representation is not interpreted or explained; however, new publications have markers indicating the degree of semantic closeness of their content to already recorded interests.

In this mode, the reader acts as a participant in the experiment and a source of data through their own attention. The use of the application boils down to observing how the personal interest vector changes over time and which tekkix articles begin to correlate with it.

How Interest Markers are Calculated

The calculation of interest markers is based on the vector representation of texts. For each publication, a summary created using a language model is used instead of the full text of the article. The summary represents a retelling of the publication and serves as a compact carrier of its semantics. It is assumed that the meaning of the summary corresponds sufficiently to the meaning of the original text.

For the review, an embedding vector is calculated, describing its position in a multidimensional semantic space. In this space, the distance between vectors is used as a measure of the semantic similarity of texts. Reading a review is considered a signal of attention. In such an event, the embedding vector of the corresponding review is added to the user's personal interest vector. Thus, the interest vector is formed from the same representations as the publications themselves and is located in the same semantic space. For each publication in the feed, the distance between its review's embedding vector and the current user interest vector is calculated. The resulting value is used as a marker of interest and shows the degree of semantic correlation between the content of the publication and the reader's already recorded interests. This value is used as a relative indicator and is not further interpreted.

Identity, Data, and Trust

With this experimental setup, the question of trust boils down to specific aspects: where attention signals are formed, where they are stored, and at what point the user consciously goes beyond the local contour. By default, Mindstream operates completely locally. The personal interest vector is formed and used in the user's browser, and attention signals are not saved or transmitted. In this mode, the experiment remains individual and confined to the user.

Identity is introduced as a separate voluntary step. It represents an anonymous UUID and is not linked to personal data. When it is created, attention signals begin to be duplicated on the server, which in the future allows for the consideration of collective effects and aggregated data. At this current stage, this data is not used for generating recommendations or a general feed. Mindstream is not a service and does not imply guarantees of results. The data is used strictly within the described model and exclusively for researching the behavior of attention signals and vector representations in a practical context. Creating an identity remains a conscious choice of the user.

Agent Development and My Role

The development of Mindstream began on February 1 and took about ten days. During this time, a client-server web application was assembled with access to external data sources (RSS tekkix) and external models for working with texts and embeddings.

The main volume of application code was not written directly by me, but by an agent based on a language model (Codex). Within the framework of the approach used, my role was to form the context for the agent's work, set tasks, evaluate proposed solutions, and adjust the results. Architectural solutions, module structure, and a significant portion of the glue code were generated by the agent within this context. If I had written this application manually, it would have looked different in structure and style. The agent's workflow and decisions made are documented in reports available in the project's repository.

When forming context documents, working on architectural solutions, and clarifying the boundaries of the experiment, ChatGPT was also used as a tool for collaborative thinking and checking formulations. On the server side, my own object container @tefw/di is used, which provides late binding of components at runtime and simplifies testing. This allowed the architecture to remain flexible and not to fix decisions until they became necessary within the framework of the experiment.

The experience of developing Mindstream became the context in which today's discussion on Habr about the possibility of creating full-fledged applications using language models later arose. By the time of the discussion, I already had practical results - a working experiment that was built alone and within a limited timeframe. In the comments to the article "How prompt optimization turned from shamanism into an engineering discipline," I participated in the conversation, relying on this development experience and observations made during the work on Mindstream. In this context, Mindstream can be seen as a documented practical result: the application exists, functions, and solves a specific problem, having been created using exactly the agent approach.

Experiment Limitations

Mindstream is in an experimental stage, and its capabilities are deliberately limited. These limitations set the boundaries for the interpretation of the obtained results.

• The application is not intended to replace tekkix and does not serve as a recommendation system. It does not filter publications or optimize the feed for engagement. Mindstream visualizes potential semantic correlations between user interests and current publications, without interfering with the order and composition of the original feed.

• The model used works with summaries, not full texts of articles. The summary serves as an approximate bearer of the publication's semantics, so interest ratings depend on its quality and do not directly reflect the content of the original text.

• Embedding vectors and the distances between them are a model representation of meaning. They allow for the comparison of texts within the same space but do not provide interpretable explanations and do not reflect the reasons for user interest. Interest markers show relative proximity, not significance or value of the publication.

• Collective attention signals are not utilized in the current version. Even when creating an anonymous identity, data is not aggregated and does not impact the feed. This is done deliberately to avoid mixing personal experimentation with social mechanisms.

These limitations allow considering Mindstream as a laboratory stand for testing individual ideas about attention, meaning, and vector representations in a controlled manner.

Why I am publishing this and what will happen next

This publication is not a report on a completed project. Mindstream was conceived and remains an experiment aimed at testing how ideas about attention and personal meaning manifest in a functioning engineering framework.

It is important for me to understand how interesting this format is to other people as a practical observational tool: for monitoring one's own feed, for attention signals, for seeing which publications begin to correlate with the vector of interests over time. In particular, I am interested in whether users are willing to consciously participate in the experiment and share attention signals for the sake of obtaining aggregated results.

The future development of the experiment depends on this. Collective effects and aggregated statistics are only possible if they are found to be in demand.

If this approach resonates with you, you can try Mindstream on your own and form your own impression. Additional context and other materials of mine are available on the website wiredgeese.com.

Thank you to everyone who was able to read.