Does AI-coding speed us up or just make it more expensive?

How to understand if AI-assisted coding is really effective?

And how do you understand that in cases related to:

1. scalability, maintainability, and security

2. race conditions, distributed system failures, infrastructure interaction, flaky tests, or performance

3. institutional/tribal knowledge

4. cryptography, auth flows, compliance logic, financial systems, and private data

and under certain conditions - deadlines, quality, resources - it was necessary to use AI-coding at all?

Misuse of AI in such areas can easily cost $15,000 per month.

Tokenmaxxing and distorted incentives

At the same time, many organizations have adopted a tokenmaxxing approach, which has led to the following practices:

• “Amazon employees admit to using AI unnecessarily to inflate internal metrics - they complain about pressure to use AI tools”

• “Meta employees used 60.2 trillion AI tokens in 30 days; as part of the same tokenmaxxing initiative, Microsoft also burned a huge number of tokens”

• The message received by Salesforce employees: “use at least $170 per month in tokens, or you will be flagged”

A different look at the problem

At the same time, others are asking more important questions.

Head of Engineering at Shopify - Farhan Thawar:

"I want to understand why they spent, for example, $1,000 per month on Cursor credits. Maybe they are actually building something significant and using an agent-based development model."

AI Code Pulse: measuring AI usage in context

These problems are solved by the JIRA app AI Code Pulse

The app links:

• Token input
• Token output
• Token Cache Create
• Token Cache Read
• Token Cost ($)

with arbitrary nested hierarchies:

• AI Model (Model → Repo → File → Author → Date)
• Author (Author → Repo → File → Date)
• Repo (Repo → File → Author → Date)
• File (File (repo) → Author → Date)
• Task (Task → Repo → File → Author → Date)
• Epic (Epic → Task → Repo → File → Author → Date)
• Date (Date → Repo → File → Author)

This allows analyzing AI usage not as an isolated entity, but in the context of engineering work.

The screenshot shows how Task (Task → Repo → File → Author → Date) is linked to Token Cost ($):

Approaches to analyzing AI cost

For analysis, you can use:

1. correlation of JIRA tasks or commits with AI costs

2. benchmarking similar classes of tasks or epics

3. cost analysis at the repository or file level

4. comparison of AI usage patterns between engineers of different levels

The representation of costs by models looks as follows:

Tokenmaxxing adepts can group data by “Authors” to see the total token costs (or choose the Total tokens metric to see the overall token consumption by authors).

However, the token economy itself does not answer the following engineering question.

Even if tokens are spent efficiently at the level of an individual task - how do you understand whether the implementation takes into account the system architecture and design?

Even in organizations with a strong engineering culture, PRs often become overwhelmed due to their density or large number.

In less mature companies, the situation is even more acute: due to the rapid adoption of AI, engineers start to think less about how their code affects the system as a whole and future maintainability.

Many stop testing properly, excessively trust AI, become less attentive, or are simply forced to work in such a mode because of pressure from managers. A term has even appeared - feature factory.

If SDLC practices are abandoned, which many people advocate for, it is very easy to accumulate technical debt that will have to be paid off later - as has always been the case in engineering.

So what should the criterion be?

Rework rate - code waste/survivability and missed parts of implementation. This metric is one of the few key signals for measuring AI-coding efficiency. It needs to be interpreted in the context of the company, but generally a high rework rate indicates wasted tokens, lost time from reviewers and compute resources, and sometimes lost time from QA, DevSecOps, SRE and other teams.

Even at Meta, where refactoring is encouraged, there are boundaries within which it is permitted.

High rework frequency also often indicates the presence of critical errors, and consequently reputational risks for the company.

A Microsoft research division identified one key point:

High rework frequency is a better predictor of bugs than many other complexity metrics.

Typical signals:

• files are frequently modified

• multiple developers work on the same file

• recent large-scale rework

These signals are strongly correlated with defect density.

Below are two metrics: rework rate and PR cycle time:

How it works

Data Flow:

1. The AI Code Pulse Tracker (npm) installed on a developer's workstation transmits data about AI-coding, but never the original code or text prompts.

2. Deterministic heuristic algorithms in AI Code Pulse link AI-coding usage to commits from GitHub or Bitbucket.

Currently, AI Code Pulse supports:

• Claude Code

• GitHub

• Bitbucket

• Jira

If your toolset is different, write in the comments, and I'll add support soon.

Trial period

Did you know that organizations lose 7 hours per week per team member due to inefficiencies related to AI-coding?