Blog on David Parry

🔐 Where Security Starts

Wed, 29 Jul 2026 20:49:00 -0500

Today I was asked a simple question:

“Where does security start when it comes to software development?”

My answer is still the same: security starts with the developer.

Not with the annual checkbox. Not with the quarterly training video. Not with the login screen that proves someone completed a security class. Those things may be required in the enterprise, and they can be useful, but they are not where real software security starts.

Real security starts when a developer understands what secure code means.

Secure code is not just code that compiles. It is code that handles input correctly, protects data, uses authentication and authorization properly, manages errors without leaking sensitive information, avoids unsafe memory behavior, uses approved libraries, and follows architecture patterns the enterprise already trusts.

Secure software development is not one activity. It is a system.

It includes secure requirements, threat modeling, framework selection, code review, automated testing, dependency scanning, software composition analysis, CI/CD controls, and production observability. NIST’s Secure Software Development Framework supports this view by recommending secure development practices across the software development life cycle, not after the software is already built.

Security Starts Before the Pull Request

A lot of enterprises require developers to take security training every few months. That is fine, but training alone is not enough.

The better question is this:

Can the developer recognize insecure code before it becomes production code?

That is where code review matters. Reviewing code is not just about formatting, naming, or whether the code follows the team’s style guide. Good review asks deeper questions:

Is this input trusted?
Is this framework being used correctly?
Is this custom implementation replacing something the platform already solves?
Does this change weaken authorization?
Does this expose data?
Does this introduce a dependency that should not be here?
Does this generated code contain a vulnerability that looks harmless at first glance?

Security starts when developers are trained to think this way and when the engineering process reinforces that behavior every day.

Frameworks Are Security Controls

One of the biggest mistakes teams make is treating frameworks as convenience libraries instead of security controls.

A mature framework is not only there to save keystrokes. It gives the team hardened defaults, proven patterns, predictable configuration, tested integrations, and community-reviewed behavior. In enterprise software, using a proven framework correctly is often more secure than building custom code to solve a problem the ecosystem already solved.

This is especially true in Java and Spring-based enterprise systems. Authentication, authorization, input handling, validation, serialization, database access, observability, and configuration should not be invented from scratch by every team.

Custom code is sometimes necessary. Custom security infrastructure should be treated with extreme caution.

A recent VMware Tanzu Spring announcement is a good example of this principle in practice. Tanzu Spring announced SLSA Level 3 compliant Java dependencies for Spring Boot 2.7.x, 3.x, and 4.x. More specifically, this is not a blanket claim that every Spring Boot application is automatically SLSA Level 3 compliant. The announcement is about clean room builds and provenance for the Spring Boot Java dependency tree available through the Spring Enterprise Repository.

That distinction matters.

The value is not just that developers are using Spring Boot. The value is that the enterprise can consume a verified dependency supply chain from a trusted source. According to the announcement, Tanzu Spring customers now have access to secure, clean room builds of the Java dependency tree for Spring Boot 2.7.x, 3.x, and 4.x, including provenance for more than 5,000 verified Java library dependencies from a single trusted source.

That is a security control.

Framework security is no longer only about the code the framework gives you. It is also about the supply chain behind the framework. Modern enterprise applications depend on large transitive dependency trees. If those dependencies come from unknown, unaudited, or unverifiable build paths, the application inherits that risk.

Mature enterprise frameworks and trusted artifact sources reduce the amount of custom security-critical code teams write, and they improve confidence in the dependencies teams consume.

That does not remove the need for code review, dependency scanning, SBOMs, SAST, SCA, or runtime controls. But it does prove the larger point: security starts earlier than the scan. It starts with the engineering choices developers and architects make before the code ever reaches production.

The Buffer Overflow Lesson Still Matters

The classic example is still the buffer overflow.

A buffer overflow is not just an old C or C++ problem people mention in security classes. It is a reminder that vulnerabilities often start as ordinary engineering mistakes. A boundary was not checked. Input was trusted. A library was used incorrectly. A reviewer missed it. A generated code change was accepted without enough scrutiny.

MITRE’s CWE Top 25 continues to include serious software weakness classes such as cross-site scripting, SQL injection, missing authorization, out-of-bounds write, path traversal, use-after-free, out-of-bounds read, command injection, code injection, classic buffer overflow, deserialization of untrusted data, improper input validation, and improper access control.

These are not abstract academic concerns. They are common and dangerous weakness classes that can lead to system compromise, data exposure, privilege escalation, or service disruption.

This is why security cannot belong only to the security team.

Security has to be part of how code is written, reviewed, tested, packaged, and shipped.

Where Snyk-Like Tools Fit

Tools such as Snyk, Dependabot, OpenSSF Scorecard, SBOM tooling, SAST, SCA, container scanners, and infrastructure-as-code scanners play an important role.

But they do not all solve the same problem.

Software composition analysis helps identify vulnerable dependencies. It tells you whether the packages, frameworks, containers, and infrastructure components you are using have known vulnerabilities.

That matters because modern software is assembled as much as it is written. Most enterprise applications depend on open-source packages, transitive dependencies, containers, build tools, plugins, and infrastructure configuration. Risk can enter through code your team wrote, but it can also enter through code your team imported.

That is different from code review.

Dependency scanning asks:

Are we using something vulnerable?

Code review asks:

Did we build something vulnerable?

You need both.

An SBOM can tell you what is inside your software. SCA can tell you whether known components have known CVEs. Code review, static analysis, tests, and architectural review help determine whether your own code introduces risk.

What AI Should Change

This is where AI-assisted code review becomes important.

If AI is used only to generate more code faster, we are going to create security debt faster.

But if AI is used to review code, challenge assumptions, identify risky patterns, check framework usage, explain potential vulnerabilities, and enforce secure engineering standards, then it can improve the software delivery process.

That does not mean AI will eliminate vulnerabilities.

It should mean fewer obvious vulnerabilities make it through review.

The future should not be a world where tools keep finding the same preventable issues after the fact. The goal should be to catch more issues before they become releases, incidents, CVEs, emergency patches, or customer-facing failures.

AI-assisted review should help reduce repeated classes of defects:

Poor input validation
Unsafe deserialization
Broken authorization
Improper error handling
Insecure dependency usage
Custom security code replacing mature framework features
Weak tests around security-sensitive behavior
Risky generated code accepted without review
Configuration mistakes
Architecture decisions that expand the attack surface

This only works if AI is part of a governed engineering system. AI review must be tied to rules, standards, architecture, trusted frameworks, CI gates, and human accountability.

The goal is not to replace security tools. The goal is to move more security knowledge into the daily development workflow.

Better Software Should Mean Fewer Preventable Vulnerabilities

Here is the part I think enterprises should start expecting.

If teams are using better code review, stronger framework defaults, secure dependency sources, SCA, SBOMs, SAST, and AI-assisted review, then the number of preventable vulnerabilities should go down.

Not all vulnerabilities will disappear. That would be an unrealistic claim.

But many classes of avoidable issues should become less common:

The obvious injection flaw.
The missing authorization check.
The unsafe generated code.
The custom security implementation that should have used the framework.
The vulnerable dependency pulled from an untrusted source.
The transitive dependency nobody knew existed.
The risky configuration that should have been caught before merge.
The insecure pattern that keeps showing up across services.

This is where AI-assisted review can add real value. It can help reviewers focus. It can explain why a pattern is risky. It can compare code against secure coding rules. It can help developers learn while they work. It can make security feedback faster, more consistent, and closer to the code.

But the enterprise still needs the full system: secure frameworks, trusted artifact repositories, dependency scanning, SBOMs, automated tests, CI/CD gates, human review, observability, and accountability.

AI should make the system better.

It should not become another excuse to skip the system.

Security Is an Engineering Discipline

Security does not start when the security team scans the finished product. It starts earlier, when engineering makes the right decision before the code exists.

It starts with developers who know how to write secure code and reviewers who know what insecure code looks like. It starts with teams that choose proven frameworks instead of inventing security-critical plumbing, architects who reduce risk through design, and enterprises that consume dependencies from trusted, auditable sources. And it starts when dependency scanning, SBOMs, SAST, SCA, AI-assisted review, and human review work together instead of in silos.

The enterprise does not need more checkbox security.

It needs secure engineering discipline.

And that starts with the developer.

🔍 MCP: Enterprise Trust Through Traceability

Wed, 22 Jul 2026 09:30:00 -0500

The headline everyone pulled out of the Model Context Protocol (MCP) 2026-07-28 Release Candidate is that MCP is going stateless. Fair enough — it’s a genuine architectural improvement. Stateless protocols scale horizontally without sticky sessions, they cut operational complexity, and they make the infrastructure easier to deploy and reason about.

But I don’t think statelessness is the change that will matter most for enterprise adoption. I think traceability is.

When I gave a conference talk on building MCP servers earlier this year, the hallway questions afterward were rarely about what a server could do. They were about what it takes to run one: how the traffic shows up in gateway logs, what a trace looks like when a tool call fails, whether any of it can be audited later. So when the release candidate landed, I read it with those questions in mind.

Two changes stood out to me, and they work as a pair.

The first is SEP-2243 , which makes two headers, Mcp-Method and Mcp-Name, required on the Streamable HTTP transport. The spec is clear about their job: they exist so load balancers, gateways, and rate limiters can route on the operation without inspecting the JSON-RPC body. What interests me is the side effect. Every layer of the stack now gets a piece of standardized, protocol-level metadata — and standardized metadata is exactly the raw material enterprise observability and audit tooling is built on.

The second is SEP-414 , which locks the W3C Trace Context keys — traceparent, tracestate, and baggage — into the spec as reserved keys inside params._meta of every JSON-RPC request. Note where they live: in the message, not in HTTP headers. That’s deliberate. MCP is transport-agnostic and stdio has no headers, and a single Streamable HTTP connection can multiplex many JSON-RPC messages, so trace context has to belong to the individual request rather than the connection. A trace that starts in the host application can now follow a tool call through the client SDK, the MCP server, and whatever the server calls downstream, and land in an OpenTelemetry-compatible backend as one span tree:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "get_weather",
    "arguments": { "location": "New York" },
    "_meta": {
      "traceparent": "00-0af7651916cd43dd8448eb211c80319c-00f067aa0ba902b7-01"
    }
  }
}

Several SDKs were already doing this — the release candidate fixes the key names so traces correlate across implementations instead of by convention. And in the same release, MCP’s own logging capability is deprecated, with OpenTelemetry named as the path for structured observability. Read those together and the direction is unambiguous: the protocol didn’t invent its own tracing story, it adopted the one enterprises already run.

That matters because enterprises are not building greenfield. They already have API gateways, OpenTelemetry, centralized logging, SIEM platforms, operational dashboards, and distributed tracing — and years of trust invested in all of it. A new protocol gets adopted when it slots into that existing machinery, not when it asks to be operated as a special case beside it. Routing headers at the transport layer, W3C Trace Context in the message — small design decisions with a large consequence, because together they mean MCP can be operated alongside everything else instead of babysat beside it.

What Enterprise Traceability Looks Like

The important property is that the new metadata enriches an existing trace without changing how distributed tracing works.

                    End User
                       │
                       ▼
                  API Gateway ────────────────┐
                       │                      │
                       ▼                      │
            Spring Boot / Spring AI           │
                       │                      │
                       ▼                      ▼
                  MCP Client         Logs · Metrics · SIEM
                       │                      ▲
    Mcp-Method         │                      │
    Mcp-Name    ──────►│ ─────────────────────┤
    (HTTP headers)     │                      │
                       │                      │
    traceparent        │                      │
    tracestate  ──────►│                      │
    (in _meta)         │                      │
                       ▼                      │
                  MCP Server ─────────────────┤
                       │                      │
                       ▼                      │
                      Tool ───────────────────┘

  W3C Trace Context propagates through every hop above — over HTTP
  headers on ordinary hops, inside _meta on the MCP hop. The routing
  headers add protocol-specific metadata to that same trace.

The distributed trace flows exactly as it does today. The routing headers give the infrastructure something to key on, and the _meta trace keys keep the MCP hop stitched into the trace you’re already collecting.

The Questions a Platform Team Asks First

Enterprise engineering organizations rarely get stuck because they can’t scale a service. They get stuck because they can’t answer questions after the fact:

Which MCP method executed, on which server, running which tool?
Which distributed trace contains this interaction, and how does it line up with the gateway logs?
Which agent initiated the request, and why did it fail?
Can we reproduce it — and can we audit it six months from now?

These aren’t exotic questions. They’re the ones a platform team asks before it agrees to support new infrastructure in production, and standardized protocol metadata is what makes them answerable. With Mcp-Method and Mcp-Name riding alongside the trace context, requests correlate across services and gateways without custom parsing, dashboards and rate limits key off the operation instead of the payload, and the telemetry looks the same across different MCP implementations. That’s the difference between something a platform team can confidently run in production and something that stays a proof of concept.

Fan-Out Is Where This Gets Expensive

The stakes go up as organizations move toward agentic software. A single user request can fan out across multiple agents, several MCP servers, and dozens of tool invocations before anything comes back. Without standardized metadata, reconstructing what actually happened in that fan-out is slow and often guesswork.

With the routing headers and trace-context keys in place, each MCP interaction correlates cleanly with the telemetry you already collect, and the whole system gets easier to operate and troubleshoot. This is the same argument I keep coming back to: agentic systems only become enterprise systems when they’re as observable, governable, and auditable as every other critical application in the estate. This release moves MCP another step toward that bar.

More Than Stateless

Statelessness earns its headlines. Simpler deployments, better scalability, and lower operational complexity are real wins, and the changelog is worth reading in full. But the larger enterprise story is the standardized metadata that integrates naturally with the observability and tracing ecosystems organizations already trust.

Platform teams want a protocol that fits the infrastructure they already run, and the technologies that win in the enterprise are rarely the ones with the longest feature lists.

The claim I’ll defend is narrower than “MCP is enterprise-ready”: two required routing headers and three reserved trace-context keys quietly gave every gateway, trace, and audit log a common vocabulary for MCP traffic. In my experience, that’s the kind of unglamorous change that decides whether a protocol makes it out of the proof-of-concept stage.

💰 Skills vs MCP: What the Token Bill Actually Measures

Mon, 20 Jul 2026 09:00:00 -0500

Move deterministic business rules out of the model and into code. For the one workload I benchmarked, that single decision cut hosted-model cost by 32% on Claude Opus 4.8 and 76% on GPT-5, ran two to five times faster, and eliminated a silent age-calculation error because an authoritative clock replaced the model’s guess. That is the conclusion. Everything below is how I measured it, why the popular “Skills versus MCP” framing is the wrong dividing line, and where the result stops being defensible.

The question that sent me down this path came after a conference talk I gave about building MCP servers. Another engineer asked me something blunt: Why are you still talking about MCP when Skills can do all of this?

My first reaction was equally blunt. If I worked for a model provider, I might prefer the design that keeps more work inside the model. I work for the organization paying the bill, so I want deterministic business rules to run in ordinary code whenever that is practical.

That is my opinion, not evidence of a vendor conspiracy. Providers have good reasons to make models more capable, and customers are free to choose the architecture. Still, the incentive is worth noticing: when a model does more work, customers usually buy more inference. The two hosted models in this benchmark also price output tokens at five to eight times their input-token rate.

So I built a small benchmark to see how much that choice mattered. The result supported my architectural instinct, but it also exposed a problem with the original framing of this article. “Skills versus MCP” is not the real dividing line. The real one is model-executed rules versus code-executed rules.

What I actually compared

A Skill is a package, not an execution environment. The open Agent Skills specification allows a Skill to contain instructions, reference material, assets, and executable scripts. OpenAI’s description of Skills likewise says a Skill can include code.

MCP is a protocol through which a model-facing client can discover and call tools. An MCP server can contain business logic, but it can just as easily wrap a database, an API, a clock, or a bad nondeterministic service. The protocol itself does not make the result correct.

The benchmark therefore compared these specific implementations:

Instruction-only Skill: the Skill contains the rules and catalog in Markdown. The model performs the calculations.
MCP-backed tool: a Rust server performs the calculations and returns structured data. The model calls it and formats the response.
Prompt-only baseline: the same rules are placed directly in the system prompt and the model performs the calculations.

That is a narrower and more useful comparison: model-executed rules versus code-executed rules.

An executable Skill would make a legitimate fourth configuration, but “the Skill contains code” is not enough information. There are two very different ways an agent can use that code. It can run the script in a real software runtime, or it can read the source and attempt to follow the logic itself. The second option is still model-executed business logic. In fact, it may be worse than giving the model concise rules: the source consumes more context, while the model remains free to miss a branch, mishandle a boundary, or improvise around the implementation.

Running the script is different. Once ordinary software receives the same validated inputs, it can produce the same authoritative output as the Rust server. The remaining uncertainty sits in the orchestration around it. Will the model load the Skill, notice the script, invoke it instead of doing the calculation itself, construct the command and arguments correctly, and present the returned value without “correcting” it? If script execution is merely suggested in prose, the architecture still depends on the model choosing the deterministic path on every request.

If the agent runtime exposes the bundled script as a required, typed capability and reliably routes the request through it, I would expect the token and latency profile to resemble the MCP-backed path. At that point, however, the important win comes from executable code being treated as authoritative, not from the Skill label. If the model is expected to read and mentally execute the bundled source, I would expect more tokens without gaining the guarantee that made the code worth writing.

I have not tested either version of that fourth configuration yet. A useful follow-up would measure them separately: Skill plus enforced script execution and Skill plus source code for the model to interpret. Combining those into one “executable Skill” result would hide the architectural difference, but both versions still hit the core economic question: how much material must the model read, reason about, and generate before the code runs? Loading Skill instructions or source, deciding to invoke a script, constructing the command and arguments, and then consuming the result all use tokens. Executable code makes the calculation authoritative; it does not make the orchestration free. My expectation is that asking the model to interpret source will be the most expensive version, while enforced script execution should be closer to MCP. Whether it is cheaper or more expensive than a compact, typed MCP tool call is something the fourth benchmark must measure rather than assume.

The workload

The customer record contains a birth date, ZIP code, ordered interests, and a maximum budget. The system must calculate age, classify the customer, filter a small activity catalog, apply discounts, rank eligible activities, and return the best matches.

The harness ran all three configurations 15 times on each of three models:

Claude Opus 4.8
GPT-5
qwen3:30b, served locally through Ollama

That produced 135 responses. The harness recorded provider-reported input, output, and total tokens; model and tool calls; end-to-end latency; estimated API cost; and the final text. The Rust server, Skill, prompts, harness, tests, and raw JSON are in the public benchmark repository .

There is one important wrinkle. The customer request did not include the current date. The MCP server read the host’s clock, while the instruction-only configurations had no equivalent clock tool. An earlier draft of this article said the date was part of the scenario. It was not.

That means the cost and latency comparison uses the same user request, but the later age comparison is not a controlled test of code versus model arithmetic. It also tests access to current state. I discuss those results separately rather than pretending the difference does not exist.

Results

Every number below is the mean of 15 successful runs. Prices are standard, non-cached API prices at the time of the test. GPT-5 was $1.25 per million input tokens and $10 per million output tokens . Claude Opus 4.8 was $5 and $25 , respectively.

The “text variation” column is the mean pairwise character-sequence difference among the 15 final answers, calculated with Python’s SequenceMatcher. It measures how different the rendered responses were. It does not measure semantic correctness, business-rule determinism, or auditability.

Claude Opus 4.8

Config	Input tokens	Output tokens	Total tokens	Model calls	Input $	Output $	Total $	Latency (s)	Text variation
Instruction-only Skill	1,210	1,193	2,403	1	$0.00605	$0.02983	$0.03588	12.5	48.5%
MCP-backed tool	2,071	561	2,632	2	$0.01036	$0.01403	$0.02438	9.0	38.8%
Prompt only	1,131	1,231	2,362	1	$0.00566	$0.03078	$0.03643	12.9	61.9%

GPT-5

Config	Input tokens	Output tokens	Total tokens	Model calls	Input $	Output $	Total $	Latency (s)	Text variation
Instruction-only Skill	802	2,089	2,891	1	$0.00100	$0.02089	$0.02189	16.1	76.8%
MCP-backed tool	963	408	1,371	2	$0.00120	$0.00408	$0.00529	4.8	38.7%
Prompt only	783	2,320	3,103	1	$0.00098	$0.02320	$0.02418	20.8	67.8%

qwen3:30b (local via Ollama on an Apple M4 Max with 128 GB RAM)

Config	Input tokens	Output tokens	Total tokens	Model calls	Input $	Output $	Total $	Latency (s)	Text variation
Instruction-only Skill	898	4,192	5,090	1	$0.00	$0.00	$0.00	43.4	76.7%
MCP-backed tool	1,480	1,086	2,565	2	$0.00	$0.00	$0.00	10.8	58.9%
Prompt only	882	5,112	5,994	1	$0.00	$0.00	$0.00	54.4	73.2%

Cost per 1,000 requests

Model	MCP-backed tool	Instruction-only Skill	Prompt only	Tool vs Skill	Tool vs Prompt
Claude Opus 4.8	$24.38	$35.88	$36.43	32% less	33% less
GPT-5	$5.29	$21.89	$24.18	76% less	78% less

What those numbers support

The tool-backed path sent more input because the request included a tool schema and required a second model call. It still cost less because it used far fewer completion tokens. On these models, for this task, the price difference between input and output was large enough to overwhelm the extra round trip.

The same implementation was also faster in this sweep. It reduced mean latency from 12.5 to 9.0 seconds against the instruction-only Skill on Opus, from 16.1 to 4.8 seconds on GPT-5, and from 43.4 to 10.8 seconds on the local model. The local comparison matters because there is no per-token invoice to blame. Generating fewer tokens still consumed less wall-clock time on my hardware.

The provider APIs reported completion-token usage; those counts are what I priced. Depending on the provider and model, that usage can include billed reasoning tokens that never appear in the visible answer. It would be inaccurate to say every calculation was literally printed to the user.

The text-variation result is interesting but weaker. Tool-backed answers were more alike in all three sets, which makes sense because the model received the same structured result each time. A character-level similarity score is sensitive to headings, wording, and answer length, though. I would not use it as evidence that the business logic is deterministic. For that, I would test the function outputs directly.

Most important, this is one synthetic workload with one customer and one catalog. It shows that moving this set of rules into code reduced tokens, latency, and hosted-model cost. It does not prove that an MCP call is always cheaper. A large tool schema, a chatty tool response, network latency, retries, or a tiny calculation could reverse the result.

The clock result: useful, but not a fair arithmetic contest

All configurations returned the same three activities in the same order:

Guided Nature Walk — $20
Mountain Hiking Tour — $45
Jazz Club Evening — $60

The reported age differed:

Config	Claude Opus 4.8	GPT-5	qwen3:30b
MCP-backed tool	36 (15/15)	36 (15/15)	36 (15/15)
Instruction-only Skill	35 (14/15)	36 (15/15)	33 (12/15)
Prompt only	35 (15/15)	36 (15/15)	33 (14/15)

The benchmark ran on July 19, 2026, and the customer was born on May 15, 1990, so 36 was correct. The Rust service read that date from the system clock. Opus often behaved as if the year were 2025, and qwen often behaved as if it were 2023. GPT-5 returned the exact benchmark date even though the user request and harness system prompt did not supply it. The data does not tell me whether that date came from model behavior or provider-side context, so calling it a “guess” would go beyond the evidence.

This result demonstrates a real production lesson: if an answer depends on current state, give the system an authoritative source for that state. It does not demonstrate that MCP is uniquely able to provide one. A bundled Skill script, a local command, a conventional API, or an MCP server could all read a clock.

It also explains why the bad ages did not change the recommendations. Ages 33, 35, and 36 fall into the same age band, generation, and discount tier in this catalog. Near a boundary, the error could matter. A 65-year-old calculated as 62 would miss the senior discount; an 18-year-old calculated as 17 could cross discount and eligibility rules. Those are examples of what the defect could cause, not outcomes observed in this run.

For repeatable testing, the server already has a FixedClock implementation. The benchmark should use it, or pass the same explicit date to all three configurations. I plan to add that controlled case before making broader correctness claims.

Where I draw the boundary

This experiment reminded me of reading a SQL execution plan. Returning the right rows is necessary, but it is not the end of the engineering work. At scale, I also care about the cost of getting those rows, the behavior under failure, and whether I can test the logic without asking a probabilistic model to repeat it.

My rule of thumb is now:

Use Skill instructions for judgment, workflow, and reusable operating procedures.
Use executable code for calculations, eligibility, prices, policy rules, and access to authoritative state.
Use MCP when that executable capability should be a shared, discoverable tool with a stable interface. A bundled Skill script or ordinary service may be simpler when it should not.

That is also what I look for when interviewing engineers who use AI. This is a personal hiring preference, not a universal standard. I care less about whether a candidate can make a model produce working code and more about whether they can explain which parts belong in model reasoning, which parts need a deterministic boundary, and what the choice costs at production volume.

The benchmark changed my wording, not my conclusion. “MCP beats Skills” is too broad. The claim I can defend is this: for this workload, code-executed business rules beat model-executed rules on cost and latency, and an authoritative clock prevented a silent data error.

That is enough to influence an architecture. It is not enough to declare a universal winner.

Reproduce the experiment or challenge it in the skill-vs-mcp repository . The raw 15-run result set used here is included.

☕ Java at Thirty: Still One of the Strongest Bets in the Enterprise

Sat, 18 Jul 2026 11:00:00 -0500

Three decades after its release, Java is still solving the problems that matter most to enterprise software. That staying power is easy to dismiss as inertia. I don’t think it is. I think it’s the predictable result of a platform that optimized for the things large organizations actually care about — and kept investing in them long after the hype cycle moved on.

Let me be careful about the claim I’m making, though.

The honest version of the claim

I’m not going to tell you Java is the optimal language. That’s an opinion dressed up as a fact, and it falls apart the moment you change the context — Java is not my first reach for a data-science notebook, a systems kernel, or a quick shell script.

Here’s the claim I’ll actually defend: Java remains one of the strongest choices for enterprise software where long-term maintainability, stability, and operational reliability are the dominant concerns. That’s a narrower statement, and it’s a stronger one, because it names the conditions under which it’s true.

For most organizations, success is rarely about using the newest language. It’s about delivering reliable systems, controlling operational risk, hiring engineers at scale, and maintaining software for a decade or more. Judge Java against those goals and its longevity stops looking like nostalgia and starts looking like fit.

Backward compatibility is a feature, not a limitation

The thing developers complain about — Java’s conservatism — is exactly what enterprises are buying.

Code written years ago still compiles and runs. Upgrades are usually boring, and boring is the point. When a platform breaks compatibility casually, every upgrade becomes a migration project, and migration projects are where budgets and timelines go to die. Java’s discipline here means a large codebase can move forward without being rewritten, and that compounds over the lifespan of a system.

Predictable releases you can plan around

Since the six-month release cadence and the Long-Term Support (LTS) model, Java gives organizations something they can actually build a roadmap on. LTS releases give you a stable target with years of support; the interim releases let the language keep evolving without forcing everyone onto the treadmill. You choose your pace. For a business planning multi-year investments, predictability like that is worth more than raw novelty.

The platform is still being invested in

Java isn’t coasting. Recent years have brought genuinely significant work to the platform — virtual threads reshaping how the JVM handles concurrency at scale, ongoing improvements to startup time and memory footprint, pattern matching and records modernizing the language itself, and continued GC advances. This is a thirty-year-old platform still receiving serious, well-funded engineering. That matters when you’re betting a decade of software on it.

A mature ecosystem and a deep talent pool

Two of the most underrated enterprise concerns are “what do I build on?” and “who will maintain it?”

On the first, the JVM ecosystem is vast and battle-tested — Spring and Spring Boot alone cover an enormous surface of what enterprises need, with libraries, tooling, observability, and integration patterns refined over many years of production use.

On the second, the Java talent pool is one of the largest in the industry. You can hire for it, you can staff teams at scale, and you can find engineers who’ve operated Java systems in production for years. That’s not a glamorous advantage, but it’s a decisive one when you’re maintaining critical systems over a long horizon.

Operational reliability is where it earns its keep

None of the above would matter if the runtime fell over under load. It doesn’t. The JVM’s observability, profiling, and tuning story is deep, and the operational patterns for running Java in production are extremely well understood. When something goes wrong at 2 a.m., “well understood” is exactly the property you want.

Where Java isn’t the answer

Being honest about the boundaries is what makes the rest credible. If your problem is a small script, a latency-critical systems component, data science and ML research, or a lean startup optimizing for the fastest possible time-to-first-feature, Java is often not the best tool — and you should reach for something that fits. The claim is about a context, not a coronation.

Why this is on my mind

I spend most of my time now building agentic systems on the JVM, and this is precisely why. The interesting, fast-moving part — the reasoning, the model calls, the orchestration — sits on top of a platform I can trust to still be maintainable, staffable, and operable years from now. New capability on a stable foundation is a good trade. It lets me chase what’s genuinely new without gambling the boring, load-bearing parts of the system.

The takeaway

Thirty years in, Java’s relevance isn’t an accident and it isn’t sentiment. It’s the payoff of a platform that consistently optimized for maintainability, stability, backward compatibility, predictable evolution, a deep ecosystem, and a large talent pool — the exact concerns that dominate enterprise software.

That doesn’t make it the best language for everything. It makes it one of the strongest bets you can place when you need systems to run, and keep running, for a very long time.

🤖 From Prompting to Planning: What Embabel Taught Me About Agents

Thu, 16 Jul 2026 09:00:00 -0500

I first ran into Embabel at DevNexus and have been quietly using it ever since, but it wasn’t until a recent class — going deep with Dashaun — that it really hit home. I spent the last stretch working through an Embabel workshop — building a bounded “digital worker” that responds to production incidents — and it reorganized how I think about agents on the JVM. Most of the agent content I read treats the LLM as the brain: you write a clever prompt, hand the model some tools, and hope it strings them together. Embabel pushes the intelligence somewhere far more boring and far more trustworthy: into Java types, into a planner, and into policy that a compiler and a test suite can see. This post is my attempt to write down what I actually learned, how I’d classify Embabel, and why I’m setting aside my own platform, AgentFabric, and starting to use Embabel instead.

The one-sentence reframe

The line from the workshop that stuck with me was: the worker chooses the path, your code defines the world. That is the whole shift. In a normal service you write a method that calls four collaborators in a fixed order. In Embabel you declare the capabilities and the desired outcome, and a planner discovers the order at runtime from the current state. You stop writing the sequence and start describing the world the sequence lives in.

What Embabel actually is

If I had to classify Embabel in one phrase, I’d call it a neuro-symbolic, planning-first agent framework for the JVM. Let me unpack why, because each word is doing work.

It’s symbolic because the core engine is Goal-Oriented Action Planning (GOAP) — the same technique game AI has used for years. GOAP starts from the goal and reasons about which declared actions, given the current state, can reach it. The planner is deterministic. It is not the LLM. This is the part people miss: Embabel does not ask a model “what should I do next?” It computes the plan from types.

It’s neural because an individual action is free to call a model. The LLM lives inside a step, boxed in by the types around it, not sitting above the whole process pulling levers.

It’s planning-first because the mental model is OODA — Observe, Orient, Decide, Act — running as a loop. After every action produces a result (or fails), the planner re-observes the world and reassesses what is now possible. Failure isn’t an exception to swallow; it’s new information that changes the next plan.

And it’s JVM-native because Spring still owns everything. Embabel is a Kotlin framework with clean Java authoring, and your agent is still an ordinary @Component. Constructor injection, interfaces, mocks, tests, observability — none of it changes. Embabel just adds planning metadata on top of methods you’d have written anyway.

Types are preconditions and effects

Here’s the idea that made it click for me. Consider four action signatures from the incident worker:

ServiceObservation observeServices(IncidentRequest request)

RunbookAssessment applyRunbook(
    IncidentRequest request, ServiceObservation observation)

IncidentResponseReport analyzeIncident(
    IncidentRequest request,
    ServiceObservation observation,
    RunbookAssessment assessment)

IncidentWorkflowReport prepareReport(
    IncidentRequest request,
    ServiceObservation observation,
    RunbookAssessment assessment,
    IncidentResponseReport response)

Nobody writes the orchestration. The parameters are the preconditions and the return type is the effect. An action is applicable only when every parameter type already exists on the blackboard; running it deposits its return type, which unlocks the next action. So the plan isn’t authored — it falls out of the data flow:

IncidentRequest
   └─ observeServices ─→ ServiceObservation
        └─ applyRunbook ─→ RunbookAssessment
             └─ analyzeIncident ─→ IncidentResponseReport
                  └─ prepareReport ─→ IncidentWorkflowReport  (GOAL)

At the start only IncidentRequest exists, so only observeServices can fire. Each result makes exactly one more action eligible. I checked this against the workshop’s tiny planner, and the mechanism really is that literal: it filters methods whose parameter types are all present, picks the lowest-cost one, invokes it, and puts the result back on the blackboard. That’s it.

The blackboard is typed working memory

There’s no JSON router and no stringly-typed state machine. State is a set of domain objects — IncidentRequest, ServiceObservation, RunbookAssessment — that the debugger, the compiler, the tests, the logs, and the planner all see identically. When I’ve built agent-ish things in the past, the “state” was usually a bag of strings passed through prompts, and it was untestable by construction. Making state a typed blackboard means the invariants live in Java, not in prose I’m begging a model to respect.

DICE: context is a domain model, not a prompt

The workshop calls this DICE — Domain-Integrated Context Engineering — and it’s the philosophical core. Instead of stuffing everything into a prompt, you encode organizational knowledge as executable, testable Java first, then hand the model only the facts it needs to reason inside those boundaries.

The clearest example is approval. The runbook, in plain Java, decides whether a proposed production change requires human sign-off:

return switch (request.incidentType()) {
    case OUT_OF_MEMORY -> new RunbookAssessment(
        "Heap pressure is consistent with an OutOfMemory failure.",
        evidence,
        "Capture a heap dump, roll back the latest risky change, "
            + "then validate with a canary.",
        true);   // requiresApproval — always
    case HIGH_LATENCY -> new RunbookAssessment(
        "Database timeouts and cache misses indicate dependency saturation.",
        evidence,
        "Check database and cache health, then use an approved "
            + "rollback or scale-out.",
        true);
};

The model never sees a requiresApproval field it could flip. It can’t. The model’s output type doesn’t contain that field, and the goal step copies the deterministic value straight from the runbook. Policy is the compiler’s job; interpretation is the model’s job. That separation is the whole point.

Where Spring AI comes in

Spring AI is the thing doing the actual model call inside an action, and it’s exactly where I’ve spent most of my own time. The pattern is small and lovely:

return chatClient.prompt().user("""
    You are an SRE following a production runbook.

    Incident: %s
    Metrics: %s
    Logs: %s
    Runbook diagnosis: %s
    Approved strategy: %s

    Return concise analysis, diagnosis, and recommendation.
    Keep the recommendation inside the approved strategy.
    """.formatted(/* domain facts */))
    .call()
    .entity(IncidentResponseReport.class);

.entity(IncidentResponseReport.class) is the part I lean on constantly: the model reasons, but Java owns the schema. The reply comes back as a typed object or the action fails — and because it can fail, the plan needs a way to recover.

Failure is just another node in the graph

This was my favorite lesson, and it’s where planning earns its keep. The model call is the preferred path (low cost). If it throws — Ollama cold, provider down, invalid output — the planner doesn’t retry the same prompt and it doesn’t ask the model to improvise. It re-observes the blackboard, notices the request and runbook assessment are still true, and selects a different declared capability with the same effect type:

@Action(description = "Fall back to deterministic runbook output",
        readOnly = true, cost = 10.0)
public IncidentResponseReport fallBackToRunbook(
        IncidentRequest request, RunbookAssessment assessment) {
    return new IncidentResponseReport(
        "The model was unavailable; deterministic policy was retained.",
        assessment.diagnosis(),
        assessment.recommendedAction());
}

Two actions, same return type. Cost expresses preference (1.0 for the model, 10.0 for the fallback) without a hand-written if/else route. The blackboard makes the fallback applicable only after the preferred path fails. Plan repair, expressed as data rather than control flow. And the whole run explains itself afterward: a PlanExecution record lists completed actions, failed actions, and whether the goal was achieved — an audit trail of decisions and outcomes, not a dump of private chain-of-thought.

Autonomy only means anything inside boundaries

The worker is allowed to observe Compose, apply the runbook, ask the model, fall back, and prepare a report. It is not allowed to invent shell commands, restart production, bypass approval, or run forever. Two guardrails enforce the last one: actions are FIRE_ONCE by default (no silent re-running) and the planner has a hard step limit. A read-only worker that stops at human approval is still autonomous — autonomy is goal-directed selection inside a capability boundary, not unrestricted mutation. That framing alone is worth the price of admission.

Why I’m putting AgentFabric down and picking up Embabel

The reason all of this landed so hard is that I’ve spent a long time building AgentFabric — my own durable JVM agent platform on the Spring AI substrate — and Embabel put a name to the philosophy I’d been reaching for by feel. Having seen it done properly, I’m going to stop investing in AgentFabric and start using Embabel instead.

AgentFabric was my attempt to run agents in production, durably, across service boundaries. It stands on:

Spring AI as the model and tool layer — ChatClient, structured-output binding, and MCP tool calling wired through a ToolCallbackProvider.
LangGraph4j for stateful graph topology — nodes that call the model, routers that branch on conditions, and verification loops that re-prompt until output passes a quality gate.
Temporal for durable execution — every graph run is a workflow, so a crashed agent resumes from the last completed node instead of replaying LLM calls.
A2A and MCP as the wire protocols — agent-to-agent messaging and agent-to-tool calls.
PostgreSQL as the source of truth for configuration, checkpoints, and token accounting.

It works. But being honest with myself, most of that is orchestration plumbing I wrote so I could get to the actual agent — and the hardest, most valuable part, the deliberation, is the part I did worst. In AgentFabric I describe the graph — nodes, edges, routers — as declarative topology, which means I’m still hand-authoring the sequence and then maintaining it forever. Embabel’s whole point is that I shouldn’t be doing that at all: declare capabilities as typed actions and a goal, and the planner derives the topology from data flow. The centerpiece of my platform turns out to be a worse version of something a maintained framework already gives me for free.

What convinced me isn’t that Embabel is different — it’s that everything I got right in AgentFabric, I got right by accidentally reinventing Embabel, badly:

My LangGraph4j state channels are a home-grown blackboard — except in Embabel the presence of a type is what makes the next step applicable, so I stop drawing edges entirely.
My verifier loop plus deterministic fallback is hand-wired plan repair — Embabel does it with a second declared action at a higher cost, selected automatically when the model action fails. No re-prompt path to maintain.
My completeAs(Class<T>) calls are just Embabel’s .entity(...) — both are Spring AI underneath, so this is the one place there’s genuinely nothing to migrate; it’s the same call.
My budget guard router is a clumsier cost-based action preference.
My guarded write tools (dry-run unless apply=true) are the same instinct as keeping requiresApproval in Java, never the model — one thing I’ll happily carry over as a habit rather than a codebase.
Even the durability I reached for Temporal to get is largely native: Embabel’s blackboard persists through a pluggable AgentProcessRepository — in-memory by default, but back it with PostgreSQL or MongoDB and a process survives a restart. And human-in-the-loop pause/resume is built in via awaitables: a process returns WAITING with a processId and you resume it later through /continue. I wrote a Temporal integration to get exactly these two things.

There’s a real cost to admitting this — AgentFabric is a lot of my work — but continuing to maintain a bespoke orchestration engine to avoid adopting a better, supported one is just ego with a build file. Embabel gives me the planner; Spring gives me the beans; Spring AI gives me the model calls I already knew; a persistent repository gives me durable state and pause/resume.

So do I still need Temporal? Almost never. The only case that would pull it back in is when I need guarantees Embabel’s step-level persistence doesn’t offer: deterministic replay with exactly-once activities (the JVM dies mid-LLM-call and must resume without re-invoking the model), durable timers (“wait three days, then continue”), or distributed orchestration across a worker fleet with guaranteed delivery and backoff. For a single-process agent that plans, calls a model, and pauses for human approval — which is most of what I actually build — none of that applies. So AgentFabric goes on the shelf, and my next agent starts as an Embabel @Agent.

And with the recent news that Embabel is heading for a 1.0.0 release, whatever hesitation I had about betting on it is gone. I’m all in.

What I’m taking away

The move Embabel names is the move from calling APIs to building workers. A worker navigates a typed domain, invokes real Spring-managed services, plans from preconditions and effects, stops at an explicit goal, repairs a failed path, preserves human approval, and leaves an audit trail. None of that requires trusting a model with the steering wheel. It requires giving the model a small, well-lit room to think in — and letting typed code define everything outside the door. That’s the bet I’m making by setting my own platform down and building on Embabel instead.

If you’re on the JVM and you’ve been building agents by growing ever-larger prompts, I’d genuinely recommend sitting with GOAP for an afternoon. It reframes the problem from “how do I make the model behave” to “what world am I asking it to operate in” — and the second question is one your compiler can help you answer.

📱 My Mobile Applications

Sun, 19 May 2024 08:00:00 -0500

GitHub Projects

David Widgets an Android Library of UI Widgets

Example Android Project showing how to use the widgets from the DavidWidgets Library

App Store Listings

PlayStore

App Store

📝 Why Choose Hugo for Static Content?

Sat, 18 May 2024 16:00:00 -0500

As a developer, my primary focus is on solving unique and challenging problems that have not yet been addressed. Writing my own static content generation system might seem appealing, but it would consume a significant amount of time and effort that could be better spent on more complex tasks. Hugo is a mature, robust, and well-supported static site generator that allows me to focus on what truly matters: innovating and solving new problems.

Hugo provides a highly efficient and flexible framework for creating static content. It is designed to handle a wide range of scenarios with ease, offering features such as Markdown support, custom templates, and shortcodes. These capabilities allow me to quickly and efficiently generate static content without having to reinvent the wheel. By leveraging Hugo, I can avoid the pitfalls of maintaining a custom-built system and instead rely on a proven solution that is constantly being improved by a large community of developers.

My requirements for static content generation are straightforward and do not warrant the development of a custom system. Hugo excels in delivering exactly what I need with minimal configuration and setup. It enables rapid deployment, supports a variety of content types, and integrates seamlessly with modern development workflows. By choosing Hugo, I ensure that my static content generation process is reliable, maintainable, and future-proof, freeing me to tackle the more complex and unique challenges in my projects.

👨‍💻 Codebase Resurrection: Given at JavaMUG

Tue, 10 Oct 2023 19:00:00 -0500

Welcome to this Java conference session, where we delve into the transformative power of AI in refactoring and improving codebases. If your codebase is in dire need of a revamp and you’re grappling with complex, tangled code, this session is tailored for you. We’ll explore how Codium.AI can serve as your secret weapon, helping you tackle subpar code effectively.

Understanding Code Health:

Analyzing Code Health:

We demonstrated how Codium.AI can assess the current state of your codebase, identifying areas that need improvement. This involves evaluating code complexity, detecting code smells, and highlighting technical debt. Visualizing Code Issues: Codium.AI provides detailed reports and visualizations, making it easier to understand the intricacies of your code and prioritize refactoring efforts.

AI-Powered Testing:

Automated Testing:

We showcased Codium.AI’s capability to generate and run AI-driven tests, ensuring comprehensive coverage and identifying potential bugs before they become issues. Confidence in Refactoring: By leveraging AI-powered testing, developers can refactor code with confidence, knowing that the AI will catch any unintended side effects or bugs introduced during the process. Precision Refactoring:

Expert-Recommended Refactorings:

Codium.AI incorporates best practices from industry experts, suggesting precise refactorings to simplify complex code structures and enhance maintainability. Step-by-Step Guidance: The tool provides step-by-step guidance on applying refactorings, ensuring that developers can implement changes efficiently and accurately. Measuring Success:

Evaluating Impact:

We discussed how to use Codium.AI to measure the impact of refactorings, tracking improvements in code quality, performance, and maintainability. Continuous Improvement: The tool supports continuous monitoring and improvement, helping teams maintain high standards of code quality over time.

Interactive Session:

The session was highly interactive, with over 30 attendees actively participating. Attendees asked insightful questions about the practical application of Codium.AI in various scenarios, and we provided detailed answers and demonstrations.

Case Study:

We shared a real-world example from our team, highlighting how Codium.AI worked like a charm in revitalizing a particularly challenging codebase. The tool’s AI-driven insights and recommendations proved invaluable, making the refactoring process efficient and effective.

Conclusion:

By the end of the session, attendees were equipped with the knowledge and tools to harness the power of Codium.AI, transforming their codebases into efficient, robust, and future-proof systems. The positive feedback and high level of engagement reflected the value and potential of integrating AI into coding practices.

💻 Redefining Error Monitoring: Breaking Free from Brittle Logs

Sat, 18 Mar 2023 16:00:00 -0500

In recent years, the landscape of error monitoring has undergone a significant transformation. Traditional log-based approaches, while invaluable for root cause analysis, have shown their brittleness in the face of evolving software applications. However, the good news is that it’s now easier than ever to break free from the shackles of brittle logs and adopt a more reliable error monitoring strategy. This transformation has been made possible by tools like Micrometer, a powerful metrics collection library, and AWS EventBridge, a versatile event bus, each offering distinct approaches to error monitoring.

Micrometer has emerged as a game-changer in the world of error monitoring by providing developers with a simple and consistent way to emit custom metrics and events from their applications. Spring Boot, a popular framework in the Java ecosystem, has seamlessly integrated with Micrometer, making it incredibly easy for developers to harness its capabilities. By leveraging Micrometer, you can emit distinct error events, separate from your regular logs, that are designed to be stable and reliable. These events serve as beacons, allowing your monitoring and alerting systems to pinpoint critical issues, even as your application evolves.

On the other hand, AWS EventBridge offers an alternative approach to error monitoring. It acts as a centralized event bus that seamlessly connects various AWS services and third-party tools. By emitting error events to EventBridge, you can effortlessly integrate with a wide range of monitoring and alerting tools, such as AWS CloudWatch, Datadog, or even custom applications. This means that not only can you reliably capture errors, but you can also set up sophisticated workflows and responses based on these events. The result is a monitoring ecosystem that’s both robust and flexible, adapting to your specific needs and evolving alongside your application.

In conclusion, the days of relying solely on brittle logs for error monitoring are behind us. Thanks to tools like Micrometer and AWS EventBridge, error monitoring has become more accessible and reliable than ever. Whether you choose the integrated simplicity of Micrometer with Spring Boot or the versatility of AWS EventBridge, you can ensure that your application’s critical issues are detected and acted upon, regardless of how your application evolves. It’s a transformation that empowers developers to build more resilient and responsive software systems while gaining deeper insights into application behavior.