PulseFlow Tecnologia
← Insights
·7 min read·framework-playbook·agent-maturity·governance·metrics
PulseFlow Tecnologia

A framework for assessing AI agent maturity

LangChain's State of AI Agents survey shows a distribution any maturity framework needs to take seriously: 51% of companies already have agents in production, but the top complaint recorded is execution quality - weighted twice as heavily as any other factor. That means "being in production" isn't synonymous with maturity; plenty of companies are in production and still uneasy about the reliability of what they're running.

Why five levels, not two

It's tempting to simplify maturity into "doesn't use" versus "uses" agents. LangChain's data shows that hides the variable that matters most: the difference between tech teams and other sectors isn't whether they adopt agents - it's how many simultaneous control methods each uses. Tech teams use, on average, 51% more simultaneous controls (tracing, offline evaluation, granular permissions, human approval) than other sectors. That control gradient is the basis for the five levels below.

Level 1 - Experimental

Ad hoc use, no formal process. Undocumented use cases, no data or tool policy, no dedicated security, zero observability beyond default logs. Corresponds to the stage before the survey's "51% in production" - still testing whether the use case holds up.

Level 2 - Assisted

The agent acts under constant supervision, every action reviewed by a human before executing. Limited tools, no deep integration with internal systems. Informal metrics, no structured evals.

Level 3 - Integrated

The agent already accesses corporate data and tools consistently, with real integrations (not prototypes). This is where most companies LangChain surveyed sit: clear use cases (research and summarization, personal productivity, customer support), but security and observability still incomplete - "performance quality" stays the top complaint because the evaluation process hasn't matured yet.

Level 4 - Governed

This is where the controls that separate tech teams from the rest in the survey come in: read-only permissions by default, mandatory human approval for critical actions (writes, deletes), systematic offline evaluation (39.8% of companies already do this), and tracing as standard practice, not the exception. Microsoft's thesis - "AI alone won't change your business. The system running it will" - describes exactly this level: the system around the agent, not the isolated agent, is what guarantees reliability.

Level 5 - Controlled autonomy

The agent acts with real autonomy in business processes, but within a formal lifecycle - what Microsoft describes as "source, test, deploy, observe, and improve" - with continuous improvement supervised by humans, not replaced by them. Clear ownership per agent, business metrics (not just technical ones) tied to each one, and scale multiplied across teams, not isolated individuals.

How to use the framework

Assess each of the ten dimensions - use cases, data, tools, integrations, security, observability, metrics, evals, ownership, scale - separately. It's common for a company to sit at level 4 on security and level 2 on evals at the same time. The goal isn't a single "level X" stamp, but a map of where to invest next - usually, the most lagging dimension is the one that most defines the operation's real risk.

Sources