As computing professionals, we’ve grown accustomed to evaluating AI coding tools through familiar metrics: syntactic correctness, benchmark performance and code quality scores. While these measures provide useful baselines, they fail to capture a more transformative capability: the ability to understand development objectives holistically, work persistently toward solutions, and autonomously navigate obstacles without constant human guidance. This genuine agency in AI coding systems represents a fundamental shift from code generation to autonomous development partnership.
When Anthropic announced its Claude 4 models in May 2025, the launch emphasized improved reasoning and coding benchmarks. Rather than rely on benchmark scores alone, I became interested in testing the actual agency of Claude 4 models compared to their predecessors—a quality that matters most in practical development scenarios. I decided to put these models to a real test: building a functional productivity plugin.
This task proved ideal for the testing agency because all necessary context, including API documentation and build instructions, were available in the workspace. This setup allowed me to focus primarily on measurement of agency: each model’s ability to holistically understand the problem, decompose it into manageable tasks, implement solutions, execute code, and resolve errors autonomously.
Testing Agency in Practice
I presented each model, Claude Opus 4, Sonnet 4, and the previous Sonnet 3.7 with an identical task: create an OmniFocus plugin that allows users to send tasks to OpenAI API for analysis, restructuring, and summarization. I deliberately avoided hand-holding, providing only the initial requirements.
Claude Opus 4 demonstrated genuine development partnership. When I encountered a database error, it didn’t just fix the immediate code, it proactively identified the underlying architectural issue: “I see the problem. OmniFocus plugins require using the Preferences API for persistent storage rather than direct database access.” It implemented a complete solution and, without prompting, enhanced the implementation with configuration interfaces, error handling, input validation, and progress indicators. Remarkably, Opus 4 required only two follow-up prompts to reach a fully functional solution.
Claude Sonnet 4 showed collaborative agency, but needed more guidance. When struggling with OpenAI integration, it made an autonomous decision to suggest rule-based default behavior when API calls fail, demonstrating initiative while maintaining focus on delivering a working solution. However, this highlighted a potential drawback of agency: default behaviors can have unexpected consequences, and I prefer explicit error handling. This underscores the importance of developers auditing AI-generated code carefully.
Claude Sonnet 3.7 also functioned as a collaborative tool. While it compiled code without syntactic errors, it required explicit guidance at the development stage. After 10+ interactions focused on fixing errors, we still lacked a fully functional plugin.
The Agency Spectrum
My comparative testing revealed distinct approaches that suggest an “agency spectrum” for AI coding systems, with four categories:
Code Generators: Produce syntactically valid code, but lack persistence and contextual understanding.
Responsive Assistants: Create working code, but require explicit guidance at each development stage.
Collaborative Agents: Balance instruction-following with initiative, working semi-autonomously with periodic guidance.
Development Partners: Internalize objectives and work persistently toward them, proactively identifying and resolving obstacles.
Defining Agency Characteristics
The agency spectrum represents more than performance gradients, it reflects fundamentally different approaches to problem-solving with practical implications for development teams.
Contextual Persistence: Higher-agency systems maintain awareness of project goals across multiple interactions. While code generators lose context between prompts, development partners like Opus 4 remember we’re building “a plugin for task analysis” and make decisions consistent with that objective throughout the development process.
Proactive Problem Identification: True agency involves recognizing problems before they’re explicitly stated. When Opus 4 identified the database access issue, it wasn’t responding to a specific error message; it understood the architectural constraints of the platform we were targeting.
Solution Coherence: Agentic systems produce solutions that work together as unified systems, rather than collections of isolated code snippets. The configuration interface, error handling, and progress indicators that Opus 4 added weren’t requested features; they emerged from understanding what constitutes a complete user experience.
Adaptive Strategy: Higher-agency systems modify their approach based on context. When Sonnet 4 built default summarizing behavior for failed API calls, it demonstrated strategic thinking about project completion versus feature completeness. However, in my test, this adaptive strategy proved unwanted, depicting potential overthinking behavior that requires developer oversight.
Implications for Development Practice
This agency evolution has profound implications for how we collaborate with AI systems:
From Instructions to Development Objectives: With agentic AI, effective collaboration shifts from detailed instructions to communicating higher-level objectives. I found myself giving Opus 4 instructions like “Build a plugin that sends tasks to OpenAI for analysis and summarization,” sufficient direction for a complete solution.
Economic Considerations: While Opus 4 costs more per token ($75 per million output tokens) versus Sonnet 4 ($15 per million output tokens), its autonomy dramatically reduces interaction count. When I needed three interactions with Opus 4 versus more than 10 with Sonnet 3.7, the efficiency gain offset higher per-token costs while saving significant developer time and cognitive load. In my experiment, Sonnet 4 demonstrated better functionality-to-financial-cost efficiency. The economics of model selection will become increasingly important as we account for variables including developer time savings, token costs, and project type variations.
Evolving Development Workflows: As AI systems exhibit genuine agency, they’ll handle implementation planning, error diagnosis, and quality assurance, freeing human developers to focus on architecture, objective definition, solution evaluation, and the human aspects of software development.
Final Thoughts
Claude 4 represents a milestone not because it generates better code, but because it exhibits agency that transforms human-AI development relationships. The frontier has shifted from “can it write correct code?” to “can it understand what we’re trying to build?”
As we move from code generation to development partnership, success will depend not just on selecting the right AI tools, but on understanding how to collaborate effectively with systems that can think strategically about software development.
For the computing community, the question is no longer whether AI will transform development practices, but how quickly we can adapt our workflows, evaluation methods, and collaboration patterns to harness the power of truly agentic systems.
Jenil Shah is a Software Engineering Manager specializing in recommendation systems, personalization, and generative AI applications. He has over a decade of experience working in different organizations focusing on applied machine learning and AI. The views expressed here are his own, and do not represent the views of his employer.
