No matter how powerful or fast your horse is, imagine riding it without a harness.

Can you?

Without reins, you can’t steer it. You can’t slow it down. You can’t guide it through a narrow path. Power alone doesn’t make it useful. But add a harness, and suddenly that same horse becomes controllable. Directional. Reliable.

That’s exactly what OpenAI experimented with and wrote about in their recent post on Harness Engineering

They’ve spent the last five months running an experiment that feels like a glimpse into the next decade of software engineering: building a product with zero lines of manually written code. Everything - the application logic, tests, CI configuration, observability setup, and even documentation was authored by Codex agents. The result was a million-line codebase built in roughly one-tenth the time it would have taken a traditional human team.

Impressive? Yes.

But the real story isn’t the speed. It’s the harness.

Humans Steer. Agents Execute.

The core philosophy they landed on is simple: humans steer, agents execute

In this world, the engineer’s primary job isn’t typing functions. It’s designing the environment, specifying intent clearly and building the feedback loops that keep those agents on track.

Codex is the horse: powerful, fast, capable of running for hours.

The harness is everything around it: the repository structure, strict architectural boundaries, mechanical enforcement via linters, observability wiring, structured documentation and review loops.

Without that harness, you don’t get a product.

You get chaos at scale.

The Repository as the System of Record

One of the most interesting shifts they described was moving away from “manuals” toward “maps.”

They tried the giant instruction-file approach. It failed. Large instruction blobs rot quickly. They crowd out context. They become stale. When everything is important, nothing is.

Instead, they treated AGENTS.md as a lightweight table of contents, pointing to a structured docs/ directory that acted as the real system of record

That’s a subtle but powerful change.

In many enterprises, critical decisions live in Slack threads, Google Docs, hallway conversations or someone’s memory. In an agent-first system, if knowledge isn’t versioned inside the repository, it effectively doesn’t exist.

That forces clarity. It also forces discipline.

Making the System Legible to the Agent

For an agent to be effective, the codebase has to be legible to it.

OpenAI went further than just structuring documentation. They wired Codex directly into an ephemeral observability stack. Agents could query logs using LogQL and metrics using PromQL. They could inspect traces, reproduce bugs, and validate UI flows automatically

Think about that.

Instead of a developer manually checking logs and stepping through failures, the agent could “see” what was happening and iterate.

That’s not just code generation. That’s operational awareness.

The horse wasn’t just running. It had vision.

Enforcing Taste and Architecture

In my 25+ years in tech, I’ve seen how quickly architectural purity falls apart under delivery pressure. In this experiment, they didn’t treat architecture as aspirational. They made it mandatory. They enforced a rigid layering model:

Dependencies could only flow in approved directions. Custom linters enforced boundaries mechanically. Structural tests ensured drift didn’t creep in.

Instead of arguing over style in code reviews, they encoded “human taste” directly into tooling.

And when the agent produced what they openly called “AI slop,” they didn’t just clean it up manually. They built recurring cleanup processes like background agents that scan the repository, update quality grades and open targeted refactoring pull requests.

It’s essentially garbage collection for a codebase.

Human judgment gets encoded once, then enforced continuously.

That’s the harness tightening itself.

Will This Work for the Rest of Us?

This isn’t magic.

It worked because they invested heavily in scaffolding. They designed the harness first.

Where this model will likely shine:

• Greenfield products

• Teams with strong platform discipline

• Clear domain boundaries

• Organizations willing to encode governance into tooling

Where it will struggle:

• Legacy systems full of tribal knowledge

• Fragmented architectures

• Weak repository hygiene

• Environments where “the real decision” lives outside version control

Most enterprises aren’t ready to just “let the horse run.”

They need to build the harness first.

The Optimization Target Has Shifted

For decades, we optimized for better engineers writing better code.

Now the optimization target is different.

We are optimizing for better repository designers. Better constraint designers. Better feedback-loop architects. As OpenAI put it, the discipline of software engineering is moving out of the code and into the scaffolding

The code becomes a byproduct.

The environment becomes the product.

If we can build a better harness, we can stop fighting the horse and start deciding where we want to go.

And that might be the real evolution here.

References

1. Harness engineering: leveraging Codex in an agent-first world

2. My AI Adoption Journey - Mitchell Hashimoto

3. Harness Engineering - Birgitta Bockeler

-Suren

That is a steep decline. There’s noise about Figma slowing down, competition rising, AI tools generating layouts in seconds.

Fine.

But after years of building enterprise websites, some of them running 10–15 years, across small sites and massive platforms with thousands of pages, I can say this confidently:

Figma is not just a design tool.

It is an alignment engine.

And alignment is the hardest part of enterprise digital work. Actually building the website is the easy part.

The process before code touches production is where the real complexity lives.

• Business sends plans to marketing.

• Marketing shapes positioning and sends direction to design.

• Design iterates.

• Marketing pushes back.

• Accessibility reviews.

• Legal comments.

• Engineering evaluates feasibility.

• Back and forth. Sometimes for weeks.

In large enterprises, I’ve even seen multiple design agencies hired for the same concept. Parallel explorations. Competing directions. The organization chooses the one that best aligns with long-term strategy.

That entire negotiation happens before development begins.

Figma became the shared visual brain for that process. Everyone sees the same thing. Everyone comments in one place. Version history is visible. Decisions are anchored to a visual artifact.

If Figma declines, the replacement cannot simply be:

“AI that generates a homepage.”

That solves the wrong problem.

AI today is excellent at creation. Prompt in, artifact out. It accelerates drafts. It reduces blank-canvas friction. But enterprises are not bottlenecked by blank canvases.

They are bottlenecked by consensus. What replaces Figma must do more than generate.

It must:

• Host structured review cycles.

• Support traceable approvals.

• Bridge business, marketing, design, and engineering.

• Understand design systems at scale.

• Manage change across hundreds or thousands of pages.

We do not yet have an AI-native design collaboration layer that fully does this.

There is no intelligent consensus system that understands enterprise governance while being collaborative and visible. If Figma weakens, the winner won’t be a generative UI tool. It will be the platform that becomes the next shared system of truth for digital experience.

AI will be part of it. It has to be.

But generation is not enough.

The next layer needs to help organizations move from

idea → debate → alignment → approval → implementation with clarity.

Until something does that better than Figma, Figma’s role remains structurally important.

The real disruption won’t be AI replacing designers.

It will be whoever replaces the enterprise design brain.

- Suren

Heroku announced on Feb 6, 2026 that they are transitioning into “sustaining engineering”, a million things were running in my mind, thinking about the engineering teams that have to go through this transition. Wade Wegner (Chief Ecosystem and Growth Officer, DigitalOcean) had some great thoughts but one line stayed with me:

“Migrate is easy to say. Harder to do.”

True. But in most real engineering organizations, it’s not even a migration. It’s a rewrite. And rewrites are where strategy meets friction.

Heroku isn’t going dark tomorrow. That’s not the issue. The real signal is the absence of a forward roadmap. When a platform stops investing, the risk isn’t sudden failure; it’s slow stagnation while the rest of the ecosystem evolves. That’s when teams begin the painful process of reassessing.

The Myth of Portability

On the surface, migration feels simple. Move the app. Move the database. Cut over DNS. But production systems are never just apps. They are layered assumptions: build pipelines, background workers, release phases, network boundaries, secrets management and on-call playbooks.

Code portability is often high. Operational portability rarely is. What looks portable at the code layer becomes deeply entangled at the operational layer. Moving from Heroku isn’t just swapping a “dyno” for a “droplet”; it’s re-establishing a working architecture in a world where your networking topology and data layer must be rebuilt from scratch.

The Triple-Front War

Migrations don’t happen in a vacuum. Engineering teams are not sitting idle waiting for a platform decision. They are currently fighting a three-front war:

1. Maintaining the Legacy: Keeping the current Heroku setup stable so the business doesn’t crash.

2. The Re-Write: Rebuilding pipelines, monitoring and databases on a new platform like DigitalOcean.

3. The AI Mandate: Accelerating AI POCs, standing up inference endpoints and experimenting with agents, the work many orgs are pushing for right now.

Asking already stretched senior engineers to context-switch between modernization and maintenance while shipping AI features isn’t a “lift-and-shift” exercise. It’s controlled reconstruction mid-flight.

Why the Destination Matters

If we are forced into this “controlled reconstruction,” the destination has to offer more than just a place to park old code. It needs an active roadmap.

DigitalOcean’s App Platform is built on the same Cloud Native Buildpack model that made Heroku successful, which helps reduce the “surface area” of the change. But more importantly, they are shipping the things Heroku stopped prioritizing: native Bun support, VPC integration and AI-assisted migration tooling.

The Reality Check

Responsible migration requires something many organizations underestimate: deliberate capacity. Not optimism. Not marketing. Capacity.

It requires an inventory of the architecture and a protection of the engineers doing the work. Sometimes, a migration is a forcing function to modernize everything deferred for years - turning a “migration budget” conversation into a full platform transformation program.

The real question isn’t whether teams can migrate. It’s whether they can transform without destabilizing everything else they’re already responsible for. In 2026, engineering leaders are balancing platform shifts, AI mandates and security hardening all at once. Adding a platform rewrite to that mix demands adult tradeoff conversations.

Migration is easy to say. Rewriting reality is not.

The organizations that navigate this well won’t be the fastest to react. They’ll be the ones that understand the full blast radius before they move.

Social Media Blurb (LinkedIn/X)

“Migrate” is easy to say. Harder to do. Actually, for most teams, it’s not a migration - it’s a complete rewrite. As Heroku enters “maintenance mode,” the real risk is slow stagnation while your team fights a triple-front war: maintaining the old, building the new, and shipping the AI POCs the business is screaming for. Don’t ask for “Migration Heroes” plan for platform reality.

Read my take:

This breakdown of the Heroku sunsetting news provides additional context on what “sustaining engineering” actually means for enterprise customers in 2026.

If this resonated, I’ll be writing more about engineering leadership, AI-assisted development, and building sustainable systems. Subscribe if you want future posts in your inbox.

— Suren

Vidhya Srinivasan’s 2026 third letter reads like an ads update.

It’s not.

It’s an architectural forecast.

If you zoom out, what she’s describing is commerce evolving from page-based interaction to AI-orchestrated systems. Search becomes conversational. Checkout becomes agent-driven. Creative becomes generative. Measurement becomes unified intelligence.

That’s not marketing innovation.

That’s software engineering territory.

Search Is No Longer a List. It’s a Runtime.

AI Mode reframes Search as a live reasoning environment. Users brainstorm, compare, refine, and decide within a flowing context. Ads do not interrupt the experience. They participate in it.

From an engineering perspective, that means:

The system must track conversational state.
Intent must be dynamically re-evaluated.
Sponsored results must pass semantic relevance and policy checks in real time.

We are moving from index-based ranking to orchestration engines.

That is a fundamentally different design problem.

Influence Becomes Programmable

YouTube being positioned as a commerce bridge is really a graph intelligence story. Creator content, audience embeddings, and brand intent all converge in large-scale similarity systems.

But the interesting shift is this: influence is becoming machine-readable.

When AI can understand community affinity at scale, matching brands to creators stops being manual partnership management. It becomes infrastructure.

Commerce pipelines become composable. Influence becomes queryable.

That is a big leap.

Agentic Commerce Is About Protocols, Not Pages

The Universal Commerce Protocol layered on top of Agent Payments Protocol is the most strategic signal in the letter.

Google is building a shared language for AI agents and merchants to transact securely.

That is not checkout optimization.

That is protocol design.

When an AI agent can represent a user, authenticate identity, negotiate offers, and complete payment across retailers without redirect loops, we are looking at a new commerce layer entirely.

Retailers stop being destinations. They become service endpoints.

This introduces real engineering challenges:

Delegated authority
Secure identity verification
Consent validation
Distributed observability

Agentic commerce is distributed systems engineering wrapped in a shopping experience.

Where Enterprises Come In

This is where it becomes especially relevant for large organizations.

Most enterprises today operate on digital marketing platforms such as:

• Adobe

• Salesforce

• OpenText

These platforms power content management, customer data platforms, campaign orchestration, personalization engines, and commerce stacks.

Until now, the integration pattern has largely been:

Experience platform → Campaign execution → Ad network → Reporting loop.

AI-native search and agentic commerce disrupt that flow.

Now enterprises can:

Expose product catalogs and offers as structured, agent-readable endpoints.
Integrate identity systems with commerce protocols.
Feed first-party data into conversational discovery surfaces.
Re-architect measurement stacks to align with AI-driven attribution.

Instead of pushing traffic into a site, they can participate directly in AI-mediated journeys.

That creates entirely new integration surfaces between enterprise platforms and ecosystems like AI Mode.

For software engineers inside large organizations, this means building APIs and data pipelines that are agent-compatible, not just browser-compatible.

It means thinking about product information as structured intelligence, not page content.

It means preparing CDPs and commerce engines to operate in a protocol-first world.

Gemini 3 and the Abstraction Discipline

Gemini 3 improving underlying ad systems without customers needing to change anything is a quiet architectural flex.

It signals strong abstraction layers between model evolution and application interfaces.

Enterprises need to adopt the same discipline.

Models will change. Latency will improve. Capabilities will expand.

If your workflows break every time a model version updates, you are coupling too tightly.

AI-native systems require clean contracts and strong evaluation layers.

The Bigger Shift

What we are witnessing is a reframing of digital commerce as a network of intelligent systems.

Search becomes a reasoning layer.
Creators become structured influence nodes.
Checkout becomes protocol exchange.
Measurement becomes unified intelligence fabric.
Enterprise platforms become integration hubs for AI agents.

This is not just about ads performing better.

It is about enterprises rethinking how their architecture participates in AI-driven ecosystems.

The expansionary moment she described will reward organizations that redesign their systems for interoperability, trust, and semantic intelligence.

The question is no longer whether AI will reshape digital commerce.

The question is whether our software architecture is ready for it.

For a long time, I thought I understood engineering metrics.

I’ve seen velocity charts, delivery dashboards and productivity reports across many teams and systems. They looked precise. They felt reassuring. And yet, they often failed to explain what was actually happening on the ground.

That disconnect is what eventually pushed me toward frameworks like DORA and SPACE. Not because they were trendy, but because they helped replace gut feel with something more honest.

tl;dr (for exec readers)

• Most engineering metrics measure motion, not progress

• DORA shows how work actually flows through the system

• SPACE shows what that system does to people

• Using both together improves delivery without burning teams out

• Metrics should help teams learn, not make them defensive

For years, engineering productivity was judged by proxies.

How busy teams looked.

How fast tickets moved.

How confident leaders sounded in reviews.

From experience, I’ve learned this the hard way: activity is easy to measure, effectiveness is not. And once systems get large, distributed and automated - intuition stops scaling.

That’s where DORA and SPACE changed how I think about metrics.

DORA: When Delivery Stops Being a Personality Debate

DORA focuses on four things:

1. Deployment frequency

2. Lead time for changes

3. Change failure rate

4. Mean time to recovery

On paper, these look technical. In practice, they expose organizational truth.

Scenario (anonymized)

I’ve seen two teams deliver similar features with very different timelines. One shipped weekly. The other took months.

The difference wasn’t talent or effort. It was structure.

More approvals. More handoffs. Testing pushed late. Releases treated as events instead of routines.

DORA didn’t blame anyone. It made friction visible.

Once that happened, the conversation changed from “why is this team slow?” to “why does the system make safe change so hard?”

That’s when automation gets funded and ownership becomes clear.

SPACE: When “High Performance” Starts to Crack

DORA tells you how software moves.

SPACE tells you what it feels like to build it.

SPACE looks at:

1. Satisfaction and well-being

2. Performance

3. Activity

4. Communication and collaboration

5. Efficiency and flow

Scenario (anonymized)

I’ve worked with teams that looked excellent on paper. Strong delivery metrics. Fast releases. Quick recovery.

And yet, people were tired. Senior engineers disengaged. Attrition crept in.

SPACE helped explain why.

Engineers were spending more time coordinating than building. Context switching never stopped. Collaboration increased, but focus disappeared.

Without SPACE, those signals would have been easy to ignore.

Why One Framework Wasn’t Enough for Me

DORA alone can push teams faster than the system can safely support.

SPACE alone can protect teams without fixing delivery problems.

The real value showed up when both were used together.

I’ve seen teams improve deployment frequency and satisfaction at the same time, simply by clarifying ownership, reducing cognitive load and making recovery predictable.

Velocity improved because stress went down.

Metrics Are Mirrors, Not Weapons

One lesson experience teaches quickly: metrics change behavior the moment people believe they’re being judged by them.

Used poorly, metrics make teams defensive.

Used well, they make teams honest.

DORA and SPACE only work when they’re treated as learning tools, not performance sticks.

Closing Thoughts

Engineering excellence today isn’t about heroics.

It’s about systems that don’t require them.

DORA helped me see how software really moves.

SPACE helped me see what that movement costs.

Used together, they shifted my thinking from intuition-driven management to calmer, evidence-based leadership.

That’s where real improvement sticks.

If this resonated, I’ll be writing more about engineering leadership, AI-assisted development, and building sustainable systems. Subscribe if you want future posts in your inbox.

— Suren

Building modern AI-powered applications isn’t just about sending prompts to a model. The real magic happens when you can augment LLMs with external tools and services, like fetching live weather data. Recently, Helidon, one of my favorite java frameworks, introduced MCP servers, giving Java developers a new way to bridge LLMs with real-world APIs through the Model Context Protocol. That means your AI app isn’t limited to what the model knows it can now reach out, call tools, and bring live information into the conversation.

In this blog, we’ll walk through how to wire together LangChain4j, Ollama, and Helidon to create an AI microservice that answers weather-related questions. Along the way, we’ll unpack the moving parts: chat models, MCP clients, tools, and service endpoints.

🔹 High-Level Architecture

At a glance, our app looks like this:

User → Helidon HTTP Endpoint → LangChain4j AI Service
       ↳ Ollama (LLM: Qwen/Llama3/etc.)
       ↳ MCP Client (Weather API tool)

• Helidon → lightweight Java microservice framework

• LangChain4j → Java framework for connecting LLMs + tools

• Ollama → runs local open-source models (Qwen3, Llama3, Mistral, etc.)

• MCP (Model Context Protocol) → standard way to plug external services into AI agents

The Flow:

Ollama ← MCP Client ← MCP Server ← API ← UI

Here’s how the runtime pieces connect:

[User Interface] 
     ↓
 [Helidon API Layer] (WeatherService + WeatherAiChat)
     ↓
 [MCP Client] (DefaultMcpClient + HttpMcpTransport)
     ↓
 [MCP Server] (Helidon MCP server exposing weather tool)
     ↓
 [Ollama LLM] (Llama3/Qwen answering queries with context)

🧩 Separation of Responsibilities

When the user calls /weather, not every layer runs at once.
Each component plays a specific role in the pipeline — and the MCP Client only activates when the LLM decides a tool is needed.

🔹 Component Breakdown

1. MCP Client (mcp-client/)

• Houses WeatherService.java, which wires:

  • An OllamaChatModel (local LLM from Ollama at

  http://localhost:11434).

  • An MCP Client (DefaultMcpClient) that connects to the MCP server at http://localhost:8081/mcp.

• Exposes a simple HTTP GET API so users can send queries like:

GET http://localhost:8080?question=Will it rain in Dallas tomorrow?

• The WeatherAiChat interface defines the AI contract:

@SystemMessage(”You are a helpful assistant that provides weather forecasts.”)
String weather(@UserMessage String question);

2. MCP Server (mcp-server/ and mcp-server-declarative/)

• Two flavors provided:

  • Imperative (mcp-server/) → explicit server wiring via Java code.

  • Declarative (mcp-server-declarative/) → YAML-driven configuration + annotated classes.

• Both expose the MCP endpoint (/mcp) that the client can consume.

• These servers wrap external APIs (like OpenWeather or a mock weather provider) into MCP tools.

3. Ollama (Local LLM)

• Runs the base model (llama3.1, qwen3:1.7b, etc.) locally.

• Provides the “reasoning engine” that interprets the query, decides if it needs external tools, and then integrates the MCP tool results.

4. API Layer (Helidon)

• A thin HTTP service that exposes AI capabilities to the outside world.

• In our example, WeatherService is registered as an HttpService.

• The user calls:

GET /?question=Will+it+rain+in+Dallas+tomorrow

→ The system routes through MCP and Ollama → returns a natural-language answer.

5. User Interface

• Could be as simple as curl, a browser, or a frontend app (React, Angular).

• Doesn’t care about the internal wiring — just talks to the API.

🔹 Project File Structure

weather-application/
 ├── mcp-client/
 │    └── src/main/java/io/helidon/extensions/mcp/weather/server/client/
 │         ├── Main.java
 │         ├── WeatherAiChat.java
 │         ├── WeatherService.java
 │         └── package-info.java
 │    └── resources/application.yaml
 │
 ├── mcp-server-declarative/
 │    └── src/main/java/io/helidon/extensions/mcp/weather/server/declarative/
 │         ├── Main.java
 │         ├── McpServer.java
 │         └── package-info.java
 │    └── resources/application.yaml
 │
 ├── mcp-server/
 │    └── src/main/java/io/helidon/extensions/mcp/weather/server/
 │         ├── Main.java
 │         └── package-info.java
 │    └── resources/application.yaml
 │
 ├── README.md
 └── pom.xml

🔹 Module 1: MCP Client

The MCP Client is the entry point of our application. It’s responsible for:

• Bootstrapping the Helidon WebServer

• Exposing a /weather endpoint to users

• Connecting to Ollama (local LLM)

• Connecting to the MCP Server (via HTTP transport)

• Wiring everything through LangChain4j’s AI service layer

📌 Main.java

class Main {
    private Main() { }

    public static void main(String[] args) {
        LogConfig.configureRuntime();
        Config config = Config.create();

        WebServer.builder()
                .config(config.get(”server”))
                .routing(routing -> routing.register(”/weather”, new WeatherService()))
                .build()
                .start();
    }
}

📌 WeatherAiChat.java

import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;

public interface WeatherAiChat {
    @UserMessage(”You are a weather journalist. {{question}}”)
    String weather(@V(”question”) String question);
}

📌 WeatherService.java

@Service.Singleton
class WeatherService implements HttpService {

    private final WeatherAiChat weather;

    WeatherService() {
        ChatModel model = OllamaChatModel.builder()
                .baseUrl(”http://localhost:11434”)
                .modelName(”llama3.1”)
                .timeout(Duration.ofMinutes(3))
                .build();

        McpTransport transport = new HttpMcpTransport.Builder()
                .timeout(Duration.ofMinutes(10))
                .sseUrl(”http://localhost:8081/mcp”)
                .logRequests(true)
                .logResponses(true)
                .build();

        McpClient mcpClient = new DefaultMcpClient.Builder()
                .transport(transport)
                .build();

        ToolProvider toolProvider = McpToolProvider.builder()
                .mcpClients(List.of(mcpClient))
                .build();

        this.weather = AiServices.builder(WeatherAiChat.class)
                .chatModel(model)
                .toolProvider(toolProvider)
                .build();
    }

    @Override
    public void routing(HttpRules rules) {
        rules.get(this::weatherChat);
    }

    private void weatherChat(ServerRequest request, ServerResponse response) {
        String question = request.query().get(”question”);
        String answer = weather.weather(question);
        response.send(answer);
    }
}

📌 application.yml

server:
  port: 8080
  host: 0.0.0.0

🔹 Module 2: MCP Server (Imperative)

The MCP Server is the “toolbox” side of our app. It exposes the get-weather-alert-from-state tool that queries the National Weather Service API.

public class Main {
    private static final Jsonb JSON = JsonbProvider.provider().create().build();
    private static final WebClient WEBCLIENT = WebClient.builder()
            .baseUri(”https://api.weather.gov”)
            .addHeader(”Accept”, “application/geo+json”)
            .addHeader(”User-Agent”, “WeatherApiClient/1.0 (your@email.com)”)
            .build();

    public static void main(String[] args) {
        Config config = Config.create();
        WebServer.builder()
                .config(config.get(”server”))
                .routing(routing -> routing.addFeature(
                        McpServerConfig.builder()
                                .name(”helidon-mcp-weather-server-imperative”)
                                .addTool(tool -> tool.name(”get-weather-alert-from-state”)
                                        .description(”Get weather alert per US state”)
                                        .schema(createWeatherSchema())
                                        .tool(Main::getWeatherAlertFromState))))
                .build()
                .start();
    }
}

📌 application.yml

server:
  port: 8081
  host: 0.0.0.0

🔹 Module 3: MCP Server (Declarative)

The Declarative MCP Server offers the same tool but with annotations instead of imperative wiring.

@Mcp.Server(”helidon-mcp-weather-server”)
class McpServer {
    @Mcp.Tool(”Get weather alert per US state”)
    List<McpToolContent> getWeatherAlertFromState(String state) {
        // Calls api.weather.gov and returns alerts
    }
}

📌 application.yml

server:
  port: 8081
  host: 0.0.0.0

🔹 Module 4: Demo Walkthrough

First, make sure ollama is running.

ollama run llama3.1

Build the whole weather application from the weather-application directory.

mvn clean package

Then run on of the server application.

java -jar mcp-server/helidon-mcp-weather-server.jar

In another terminal, run the client application.

java -jar mcp-client/helidon-mcp-weather-client.jar

Example query:

curl -G “http://localhost:8080/weather” \
     --data-urlencode “question=Is there a weather alert in state TX?”

Response:

There is a Coastal Flood Advisory and High Rip Current Risk...

🔹 Summary & Key Takeaways

• MCP Client → entry point exposing /weather API, connects Ollama + MCP.

• MCP Server → provides weather tools (imperative or declarative).

• Ollama → runs the local LLM (LLaMA, Qwen, Mistral).

• Demo → queries flow User → API → MCP Client → MCP Server → Weather API → LLM → back to User.

💡 Takeaway: LangChain4j + Ollama + MCP + Helidon = a powerful recipe for AI microservices that are local, extensible, and production-ready.

Java may not be the first language people think of for AI, but that’s exactly why I’m exploring it. Follow along for more experiments, tutorials, and stories from the intersection of AI and Java development.

Enter a fresh paper from NVIDIA Research, dropped recently, arguing that small language models (SLMs)—think models under 10 billion parameters that can run on your laptop—are actually the future of this agentic AI boom. Full disclosure: I'm no NVIDIA insider, but as someone who's been watching out over AI trends closely (remember my post on Capgemini's Resonance AI?), this paper hit me. It's not just tech talk; it's a wake-up call on why we might be overpaying for AI software (agents) we don't fully need. It’s like buying a Ferrari to go go grocery shopping few blocks away. Buckle up, because we're unpacking this short summary in my usual style—analogies, real talk, and a dash of "what ifs".

The paper, titled "Small Language Models are the Future of Agentic AI," is penned by a team of NVIDIA brains like Peter Belcak and Pavlo Molchanov, plus a Georgia Tech collab. It's a position piece, not some dry experiment log—think manifesto for ditching LLMs in favor of SLMs for most agent work. They back it with stats, examples, and even a how-to guide for switching over.

Why care? Agentic AI is already a multi-billion-dollar beast, with surveys showing over half of big IT firms using agents, and markets predicting it'll balloon to $200 billion by 2034. And every tech consulting firm screaming we can build & help!

But the current setup? Agents ping huge LLMs in the cloud for every little task, burning through billions in infrastructure. The authors say that's wasteful—SLMs can handle the grunt work better, cheaper, and greener. NVIDIA's take on SLMs in agentic AI. (Imagine a cool graphic here of a tiny model zipping past a hulking LLM—source: the paper's Figure 1, but I'll describe it later.)

What's the Big Idea? Defining SLMs and the Position First off, they define SLMs simply: models small enough to fit on your phone or laptop, cranking out responses fast for one user without lagging.

Anything bigger? That's an LLM, stuck in data centers. As of 2025, that's mostly under 10B params—tiny compared to GPT-4's rumored hundreds of billions.

Their core stance: SLMs are powerful enough for most agent tasks, more operationally fitting, and way more economical. They call it a "value statement," not a bossy recommendation—basically, if we follow logic and priorities like cost and efficiency, SLMs win. Agents break down big jobs into repetitive subtasks: planning, tool-calling, reasoning in loops. Why waste a generalist LLM on that when a tuned SLM nails it? For chatty, open-ended stuff, mix them in heterogeneous systems—SLMs for the basics, LLMs as the big queries when needed.

They elaborate: LLMs are overkill for agents' narrow, scripted interactions. Agents aren't chit-chatting philosophy; they're parsing code, calling APIs, or summarizing docs. SLMs shine here with lower latency, less power draw, and easier tweaks. Plus, it's sustainable—leaving LLM monoliths could slash AI's carbon footprint. They nod to philosophy too: it's a "Humean moral ought," meaning if we value efficiency and the planet, we should shift.

The Arguments: Why SLMs Rock for Agents

The meat of the paper? A bunch of arguments backing their views. Let's break them down like I did with Capgemini's pillars—step by step, no jargon overload.

Argument 1: SLMs Are Already Beast-Mode Capable.

Forget the old "bigger is better" trope. Scaling laws say more params equal more smarts, but recent SLMs bend that curve. Examples? Microsoft's Phi-3 (7B params) matches 70B models on reasoning and code gen, running 15x faster. NVIDIA's own Nemotron-H (up to 9B) rivals 30B LLMs at a fraction of the compute. Hugging Face's SmolLM2 (1.7B) punches like 14B contemporaries. DeepSeek's distilled models even beat GPT-4o on commonsense. Add tricks like tool augmentation or self-consistency, and a 3B SLM solves math like a 30B one. Bottom line: For agent errands like tool-calling or instruction-following, SLMs deliver without the bloat.

Argument 2: Cheaper Than a Budget Lunch. Inference on a 7B SLM?

10-30x less in energy, FLOPs, and cash than a 175B LLM. Fine-tuning? Hours, not weeks. Edge deployment? Run them offline on your GPU with stuff like NVIDIA's ChatRTX. LLMs waste params on sparse signals; SLMs are denser, more efficient. Modular agents? Stack SLMs like Legos for specialized tasks—cheaper to debug, deploy, and scale.

Argument 3: Flexibility on Steroids.

SLMs cost less to train or tweak, so spin up experts for niche jobs. This democratizes AI—more folks can build agents, fostering diversity and innovation. No more one-size-fits-all.

Argument 4: Agents Barely Scratch LLM Surfaces.

Agents cage LLMs with prompts and tools, using like 10% of their skills. A fine-tuned SLM for that narrow slice? Just as good, plus bonuses.

Argument 5: Alignment Matters Big Time.

Agents chat with code—tool calls need perfect formatting (JSON, not YAML). SLMs, tuned for one style, hallucinate less and fit seamlessly.

Argument 6: Heterogeneity is Baked In.

Agents can call multiple models. Use an LLM for the boss-level planning, SLMs for minions. Or go full SLM with a chatty one for users and specialists for backend.

Argument 7: Free Data Goldmine.

Agent interactions spit out perfect fine-tuning data. Log them (ethically), curate, and boom—better SLMs over time.

These aren't exhaustive, but they paint a picture: SLMs aren't underdogs; they're optimized for the agent game.

But Wait, Counterarguments?

They Tackle Them Head-On: No paper's complete without pushback. They address alternatives like "LLMs always win on language understanding" (scaling laws say so, plus some "semantic hub" theory). But: New architectures close the gap, fine-tuning fixes niches, and agents decompose tasks anyway—no hub needed for simple subtasks.

Another: Centralized LLMs are cheaper at scale. Maybe, but modular inference tech is catching up, and setup costs are dropping.

They admit inertia—LLMs have a head start—but argue advantages will flip it.

Barriers: Why Aren't We There Yet?

Honest talk: Billions sunk into LLM clouds, benchmarks favoring generalists, and SLMs lacking hype. But these are fixable—better scheduling, agent-focused evals, and word-of-mouth on savings.

The Fix: LLM-to-SLM Conversion Recipe

1. Log agent calls (anonymized, secure).

2. Curate data—scrub sensitive stuff, paraphrase if needed.

3. Cluster tasks (e.g., summarization, code gen).

4. Pick SLMs (Phi, Mistral as starters).

5. Fine-tune with LoRA or distillation.

6. Iterate with fresh data.

It's like upgrading your car engine without buying a new ride—painless, iterative.

What’s next?

Build Small Language Models. It's about making AI cheaper and greener to speed up the agent revolution.

My Two Cents: Is This the Future? As someone who's seen AI hype cycles (hello, model collapse fears), this paper feels spot-on. Agents are repetitive workhorses, not philosophers—SLMs fit like gloves. Imagine: Cheaper agents mean more access, from startups to your grandma's recipe bot. But risks? Fine-tuning could amplify biases if data's junky, and heterogeneity might complicate debugging. What's next? Multimodal SLMs for vision agents? On-device privacy wins? Or pushback from LLM giants? I'm betting SLMs surge, especially with energy costs rising. What do you think? Will SLMs dethrone LLMs in agents, or is it hype? Drop a comment—let's keep the AI convo rolling.

So, let’s break it down: how does the Resonance AI Framework work, why does it matter, and what’s the secret sauce behind its promise of “human-AI chemistry”? Buckle up, because we’re diving into the future of enterprise AI.

What’s the Resonance AI Framework, Anyway?

Picture this: AI as a utility, like electricity or Wi-Fi, accessible to everyone, everywhere, anytime. That’s Capgemini’s vision, and the Resonance AI Framework is their roadmap to make it happen. At its core, this framework helps organizations move from AI ambition to tangible results—think faster operations, smarter products, and happier customers. It’s not about slapping AI onto existing processes; it’s about reimagining how businesses operate, innovate, and connect with people. Capgemini calls it a “strategic blueprint” that blends their expertise in AI, data, and human-centric design. The framework revolves around three key dimensions—AI Essentials (ACCESS), AI Readiness (ADAPT), and Human-AI Chemistry (ADOPT)—each tackling a different piece of the AI puzzle. Oh, and it’s backed by a suite of transformation offers and RAISE, a generative AI and AI agents gallery that’s constantly evolving. Sounds like a lot, right? Don’t worry, I’ll unpack it step by step.

The Three Pillars of Resonance AI

Let’s start with the foundation. Capgemini’s framework is like a three-legged stool—each leg is critical, and if one wobbles, the whole thing tips over. Here’s how the three dimensions work together to create what they call “waves of value.”

AI Essentials (ACCESS): The Building Blocks

Think of AI Essentials as the raw materials for your AI transformation. This is about having the right tech and data to make AI work. Capgemini breaks it down into two parts:

• Intelligent-as-a-Service: This includes scalable infrastructure, advanced language models (think LLMs like the ones powering ChatGPT), and software with built-in AI smarts. It’s the engine that drives your AI capabilities.

• Raw Data: Your organization’s data—unique, messy, and often untapped—is the fuel. Whether it’s customer records, supply chain logs, or sensor readings, this data powers the insights that make AI valuable.

Without these essentials, you’re trying to build a house without bricks or mortar. Capgemini’s approach ensures you’ve got the tech and data ready to unlock actionable intelligence, whether you’re automating processes or dreaming up new products.

AI Readiness (ADAPT): Setting the Stage

Here’s where things get practical. AI Readiness is about preparing your organization to use AI effectively and responsibly. It’s not enough to have fancy tech—you need the right enablers to make it stick. This means:

• Workforce Models: Training your team to work alongside AI, not fear it.

• Governance Frameworks: Rules to keep AI ethical, legal, and safe.

• Data Infrastructure: Systems to manage and process your data at scale.

Capgemini also emphasizes guardrails—think of them as bumpers in bowling, keeping your AI from veering into risky territory. This is crucial because, let’s face it, AI gone wrong can be a PR nightmare (or worse). By focusing on readiness, Capgemini ensures your organization can scale AI without tripping over itself.

Human-AI Chemistry (ADOPT): The Secret Sauce

This is my favorite part, and it’s where Capgemini really shines. Human-AI Chemistry is about designing interactions between people and AI that actually work. It’s not just about tech—it’s about trust, collaboration, and culture. Capgemini highlights three elements here:

• Defined Roles: Clarifying who does what—AI handles the heavy lifting, humans bring the judgment.

• Well-Designed Interactions: Workflows that feel natural, not clunky.

• Ethical Alignment: Ensuring AI respects legal and moral standards to build trust over time.

I love the analogy they use: just like team chemistry makes a sports team click, human-AI chemistry determines how deeply AI integrates into your business. Without it, you’ve got a shiny AI tool that nobody trusts or uses. With it, you’ve got hybrid teams that thrive.

A Real-World Example: AI in Action

Still with me? Let’s ground this in a real example. Capgemini’s already rolling out the Resonance AI Framework with clients, and the results are impressive. Take their work with a global pharmaceutical company struggling with its IT service desk—slow resolution times, high costs, and grumpy users. Sound familiar? By bringing in agentic and generative AI, Capgemini transformed the service desk. The results?

• 20% reduction in average handling time.

• Improved first-contact resolution and user satisfaction.

• 80% zero-touch automation (yes, 80%!).

• 40% cut in operational costs.

This isn’t just about saving money—it’s about making IT support faster, smarter, and more user-friendly. And it’s not a one-off. Capgemini’s framework is being used across industries, from manufacturing to financial services, to craft AI roadmaps, hyper-automate processes, and reimagine customer experiences.

The Bigger Picture: Why This Matters

So, why am I excited about this? (maybe it’s partly because I’m on Team Capgemini, but hear me.) The Resonance AI Framework isn’t just another corporate AI pitch—it’s a response to where the world’s headed. With agentic AI set to explode in the next two years, organizations need a way to harness it without losing their soul. Capgemini’s focus on human-AI chemistry is a game-changer. It acknowledges that AI isn’t about replacing people—it’s about amplifying them. Let’s zoom out for a second. AI is becoming a utility, as Capgemini’s Group CEO Aiman Ezzat puts it, “accessible everywhere, anytime, and by anyone.”

But accessibility alone isn’t enough. Without strategy, governance, and trust, AI can become a chaotic mess. The Resonance AI Framework offers a clear path: align your vision with execution, strategy with operations, and innovation with responsibility. And it’s not just talk. Capgemini’s backing this up with serious muscle—over 150,000 team members trained on generative AI, AI Centers of Excellence, and partnerships with heavyweights like AWS, Google Cloud, Microsoft, and Mistral AI. They’ve even got recognition from Forrester’s Wave™ for AI services in Q2 2024.

This is a company walking the walk. And i’m proud to be part of it.

What’s Next for Resonance AI?

The journey’s just beginning, and I’m already curious about where this framework will go. Here are a few possibilities on the horizon:

• Multimodal AI Integration: Imagine AI that seamlessly blends text, images, and audio to solve complex problems—like diagnosing equipment failures from photos and sensor data.

• AI on the Edge: Deploying AI on devices like smartphones or IoT sensors for faster, real-time decisions.

• Explainable AI: Making AI’s decisions transparent so users trust it more (because nobody likes a black box).

Capgemini’s RAISE gallery, with its ever-evolving AI agents, hints at a future where businesses can plug and play AI solutions tailored to their needs. From hyper-automating supply chains to crafting personalized customer experiences, the possibilities are endless. But there’s a catch. As I’ve written before (remember my post on AI model collapse?), scaling AI isn’t without risks. If organizations lean too heavily on synthetic data or lose access to human-generated data, things could go sideways. Capgemini’s emphasis on governance and ethical guardrails is a step in the right direction, but it’ll take community-wide coordination to keep AI sustainable.

Final Thoughts

Capgemini’s Resonance AI Framework is more than a tool—it’s a mindset. It’s about starting at the core of your organization and letting AI radiate outward, creating continuous waves of value. Whether you’re a manufacturing giant, a financial services firm, or a startup with big dreams, this framework offers a way to make AI real, responsible, and human. I’ll admit, I’m a bit biased as a Capgemini insider, but I truly believe this framework could set the standard for enterprise AI. Will it become the global benchmark, as Capgemini hopes? Only time will tell, but one thing’s clear: in a world where AI is the next utility, frameworks like this are what separate the leaders from the laggards. So, what do you think? Are you ready to embrace human-AI chemistry in your organization? Let’s keep the conversation going.

Further Reading

• Capgemini’s Resonance AI Framework Overview: Capgemini Website

• “The Forrester Wave™: AI Services, Q2 2024”: Forrester Website

• My previous post on AI Model Collapse for context on AI’s challenges

The primary theme the last decade was "Humans As Consumers". Digital Transformation was all over us. UI, UX, Websites, Content, Targeted personalized content, Advertisements, Ads based on our search criteria, Analytics, SEO, Websites for desktop, Websites for mobiles, Mobile apps... everything was centered around HUMANS.

AI As Consumer

AI builds' websites, writes content for AI. But the best part is that AI agents/bots do not care how the website/presentation looks; so long as they can read the text to process.

Then generate Summaries (short text. bullets like Google AI Overviews AIO) for Humans to Consume. Yes, we are getting addicted to quick summarized results from AI Chats (like ChatGPT, Google Gemini, Perplexity…). Gone are the days where we scroll through 1000’s of google search results. Who has time and energy to do that?

When was the last time you went beyond like 2nd, 3rd page of google results?

Many things will change

An entire ecosystem of technologies were built around this “Humans As Consumers” was built the last 10-15 years. About 14,106 marketing technology products.

Now if the fundamental consumer is changing, aka AI, i cannot imagine how many of the above will become obsolete, some may change and some may stand up as transformers winners. The next few years will be very interesting and exciting for the core use of several digital products will change and how AI will evolve.

Further reading

• Customer Experience Matrix, David Raab

• LLMs are weird computers, Phillip Carter

tl;dr

This post details how to manipulate nodes in AEM using a Java servlet. Code

The Servlet

A servlet that has some path to be accessible. Remember this (can be) a one-time used code.

The sample I used the path as “/bin/sampleapp/manipulatecontent”. You may use any.

The input to the servlet is the path. So the above can be called as “/bin/sampleapp/manipulatecontent?searchpath=/content/sampleapp”

1: Search query

In this example I am looking for all occurrences of the component “bg” and then adding extra nodes to that component. “SearchPath” would be the input (or if you know the path, you may hard code it).

Map<String, String> queryParameterMap = new HashMap<>();
queryParameterMap.put("path", searchPath);
queryParameterMap.put("type", JcrConstants.NT_UNSTRUCTURED);
queryParameterMap.put("1_property", "sling:resourceType");
queryParameterMap.put("1_property.value", "sampleapp/components/content/bg");

2: Find the component’s node

Find all the paths that have the component.

 List<Hit> hits = searchResult.getHits();
 if (CollectionUtils.isNotEmpty(hits)) {
   for (Hit hit : hits) {
     hitPath = hit.getPath();
     pageResource = resourceResolver.getResource(hitPath);
     nodeToUpdate = resourceResolver.getResource(hitPath).adaptTo(Node.class);
     ...

3: Add the extra nodes

Add the properties you’d like to add to the component’s node.

Note: If you look at the example, I added some extra checks before adding nodes.

nodeToUpdate.setProperty("backgroundSpacing", "theme-3");
session.save();

That’s it!

Note: This is by no means THE solution. This worked for us. I would love to see ore such examples and code more simpler or efficient.

Code snippet

Resources

• Complete servlet at

• Using Groovy scripts - <TBD>

• Using Javascript - <TBD>

And Quantum, ain’t want to be quiet or left ignored. Though a powerful technology, frankly the world still need to see a practical uses (other than movie scripts)!

Some movies have been using this tech to let characters in the movies be in 3D worlds, alternative realities and time travel :-)

The race for Quantum superiority started few years back with IBM, Google started investing heavily on the technology. Recently the new trend started with Google’s Willow quantum chip announced in 9th Dec 2009

The latest development being..

Amazon’s Ocelot

The release of Amazon’s Ocelot (may be named to serve as a way to link the physical animal with the underlying quantum principles of the chip)

last week followed by..

Microsoft’s Majorana 1

on 19th Feb, 2025 by Microsoft’s Majorana 1 (named after Ettore Majorana, an Italian theoretical physicist who, in 1937, theorized the existence of a type of particle known as a Majorana fermion)

and the in Dec 2009

Google’s Willow

release of Willow in 9th Dec 2009

and the race goes on!!!

—

Technology is great, fall in love with it not the hype!

If you are reading this after 2025, between 2023-2025 there was so much noise & announcements & press releases around AI tools, services, products that we could not avoid, but write/talk about it.

When Amazon initially launched Alexa in Nov 6, 2014 along with it’s first-generation Amazon Echo smart speaker, the voice assistant market was young with just a handful of players like Apple’s Siri (launched 2011) and Google’s Now (later became Google Assistant)

As many software products Alexa went through it’s evolution of product improvements and updates, a significant one was the team’s reflection after 5 years of Alexa and I can’t believe they said this in Nov 2014 (AI term was vaguely used then).

Amazon’s mission is to be the earth’s most customer-centric company. With that mission in mind and the Star Trek computer as an inspiration, on November 6, 2014, a small multidisciplinary team launched Amazon Echo, with the aspiration of revolutionizing daily convenience for our customers using artificial intelligence (AI).

Looks like Amazon stuck to its core mission of improving Alexa with Artificial Intelligence; And never a good time than now to release a product with AI Powered / Capabilities. Minus all the marketing trendy words like..

Today, we’re excited to introduce Alexa+, our next-generation assistant powered by generative AI. Alexa+ is more conversational, smarter, personalized—and she helps you get things done. She keeps you entertained, helps you learn, keeps you organized, summarizes complex topics, and can converse about virtually anything. Alexa+ can manage and protect your home, make reservations, and help you track, discover, and enjoy new artists. She can also help you search, find or buy virtually any item online, and make useful suggestions based on your interests. Alexa+ does all this and more—all you have to do is ask.

what excites me is the Product Engineering part of it.

1. Alexa+ got connections: Forget just turning on your lights, Alexa's hooked up with tons of apps and gadgets! Order food with GrubHub, book a table with OpenTable, call an Uber, play your favorite tunes on Spotify—you name it, Alexa+ probably works with it.

2. Your own personal genie: Alexa+ can now juggle multiple tasks like a pro. Need to book a table for dinner AND let your friend know where you'll be? Just ask! Alexa+ will make the reservation and shoot off a text, all in one go.

3. No more BS: LLMs can sometimes be a bit flaky, but Alexa+ is all about keeping it real. It uses some fancy tech and teams up with a bunch of reliable news sources to make sure you get accurate info, whether you're checking the weather or controlling your smart home.

4. Lightning-fast: Nobody likes waiting around, especially in the GenAI era, and Alexa+ gets that. It's been souped up with some serious speed-boosting tech to make sure it responds to your requests in a snap.

5. Alexa gets to know YOU: Alexa+ may know you even better. Worries me some. It remembers your favorite things (like music, books, and food) and uses that info to give you more personalized responses. The more you chat with Alexa+, the more it becomes your perfect AI sidekick.

6. Alexa's got skills (like, real skills): No AI product without Agents these days. Alexa+ has Agent capabilities, essentially teaching Alexa+ to navigate the digital world as a person would.

Developing a product is only half the battle. The real challenge is understanding how it's used and perceived in the real world, and using that feedback to drive continuous improvement.

I wish Amazon Alexa+ team the very best!

—

Technology is great, fall in love with it not the hype!

Okay, let's be real. AI is cool, but it's got this annoying habit of just… making stuff up. We call it "hallucinating," but it's basically like your friend who tells you a wild story they totally invented. Annoying, right? Especially when you're trying to build serious applications.

Well, MongoDB just dropped a bomb on this problem by grabbing Voyage AI. These guys are basically the AI equivalent of fact-checkers on steroids. They've built tech that makes AI find the right information, so it doesn't just spew out random nonsense.

Why Should You Be Pumped?

• Goodbye, AI Gibberish: Ever asked a chatbot a question and got a completely bonkers answer? That's what MongoDB's trying to fix. By bringing Voyage AI's tech into their database, they're making AI way more reliable.

• AI You Can Actually Trust: Imagine building an AI app for your business, and it's actually… accurate. That's the dream, right? With better data retrieval, AI becomes way more trustworthy.

• Faster, Smarter AI: Finding the right data is like giving your AI a brain boost. It means faster, more accurate results, which is a win for everyone.

What's the Secret Sauce?

Voyage AI's tech is all about making AI understand the meaning of data, not just the words. They've got:

• Super-accurate search tools.

• Reranking magic that filters out the junk.

• AI models that are top-notch at understanding all kinds of data.

Basically, MongoDB's building a database that stops AI from being a liar. And that's something we can all get behind.

The “Developer's Dream”

MongoDB Just Made AI Development Way Less Painful (Thanks, Voyage AI!)

Let's talk about building AI apps. Right now, it's a mess. You're juggling all these different tools, trying to make them work together. It's like trying to build a spaceship with spare parts from a lawnmower.

But MongoDB just changed the game. They bought Voyage AI, which means they're bringing super-smart AI search directly into their database.

Why Developers Will Love This:

• No More Patchwork Solutions: Forget about managing separate embedding APIs and vector stores. MongoDB's putting everything you need in one place.

• AI Search That Just Works: Imagine being able to do semantic search, vector retrieval, and ranking without all the headaches. That's what MongoDB's delivering.

• Faster Development Cycles: With everything integrated, you can build AI apps faster and more efficiently.

• Focus on the Fun Stuff: Instead of wrestling with infrastructure, you can focus on building cool features.

In short, MongoDB's making AI development less of a headache and more of a joy. They are enabling developers to easily build AI into their applications, and that is what developers are looking for.

—

Technology is great, fall in love with it not the hype!

As tempting and mouth watering the image looks, ask any chef, it is as such a complex piece of things to do. It’s the perfectly cooked soft pan cakes, temperature, stacking, syrup, the fruit, the presentation.. yes it a complex breakfast with a lot of work going into it.

Now coming to our AI world, what is a Compound AI system? An AI system that does more than juts being a Large Language Model (LLM), but is an LLM in conjunction with other components like integrating with databases, ability to search n the internet etc into a system or a chain that allows it to achieve its State of the Art result.

Some examples of compound AI systems:

• Autonomous vehicles

  • Combining computer vision, sensor data, speech recognition and information retrieval

• Prompt engineering

  • Prompt engineering logic + LLM

• RAG (Retrieval Augmented Generation)

  • LLM + Retrieval systems, Data API’s like Databricks Online Table, Text-To-SQL engines

• and many more like Healthcare diagnostics, Fraud detection systems, Siri, Alexa..

—Concepts

Tasks, intents, pipelines

Compound AI systems are designed to tackle complex tasks by combining multiple AI components. Three fundamental concepts: tasks, intents, and pipelines need to understood.

Tasks

A task is a specific activity or operation that an AI system is designed to perform.

Examples:

• Sentiment analysis

• Language translation

• Image classification

• Text summarization

Characteristics:

• Well-defined input: Tasks typically have a clear and well-defined input, such as text or an image.

• Specific output: Tasks produce a specific output, such as a sentiment score or a translated text.

• Measurable performance: Task performance can be measured using metrics like accuracy, precision, or recall.

Intents

An intent is the underlying purpose or goal that a user wants to achieve when interacting with an AI system.

Examples:

• Booking a flight

• Finding a restaurant

• Getting directions

• Making a purchase

Characteristics:

• User-centric: Intents are focused on the user's needs and goals.

• Context-dependent: Intents can depend on the user's context, such as location or previous interactions.

• Flexible input: Intents can be expressed in various ways, such as through text, voice, or gestures.

Pipelines

A pipeline is a series of tasks or processes that are executed in a specific order to achieve a particular intent.

Examples:

• A language translation pipeline might involve:

  1. Text analysis

  2. Language detection

  3. Machine translation

  4. Post-editing

• A sentiment analysis pipeline might involve:

  1. Text preprocessing

  2. Feature extraction

  3. Model training

  4. Prediction

Characteristics:

• Sequential processing: Pipelines involve a sequence of tasks that are executed one after the other.

• Data flow: Pipelines typically involve the flow of data from one task to the next.

• Configurable: Pipelines can be configured to accommodate different tasks, intents, or AI models.

Putting it all Together

Compound AI systems integrate multiple tasks, intents, and pipelines to create a comprehensive AI solution. By combining these components, developers can build AI systems that:

• Handle complex tasks: Compound AI systems can tackle complex tasks that involve multiple intents, tasks, and pipelines.

• Improve accuracy: By integrating multiple AI components, compound AI systems can improve overall accuracy and performance.

• Enhance user experience: Compound AI systems can provide a more seamless and intuitive user experience by understanding user intents and adapting to their needs.

In summary, tasks, intents, and pipelines are the building blocks of compound AI systems. By understanding these concepts and how they interact, developers can create sophisticated AI solutions that tackle complex challenges and provide real value to users.

—Key Components of a Compound AI System

A Compound AI system typically includes:

1. Language Models: Understand and respond to user inputs in a natural way.

2. Information Retrievers: Fetch relevant data from databases or external sources.

3. Databases: Store and manage data for easy access.

4. External Tools: Use APIs and services to perform specific tasks.

5. Data Converters: Turn data into a format that's easy to search and retrieve.

6. Result Filters: Prioritize and evaluate retrieved results for relevance.

7. Prompt Builders: Create effective inputs for language models.

8. Output Refiners: Ensure generated outputs are high-quality and coherent.

9. Caching Systems: Store frequently accessed responses for faster retrieval.

10. Task Managers: Orchestrate tasks and data flow for optimal performance.

11. Evaluation Tools: Assess system performance and output quality.

12. Monitoring and Feedback: Track performance and gather user feedback for improvement.

13. Model Adapters: Fine-tune pre-trained models for specific tasks or domains.

14. Agent Builders: Create autonomous agents that can perform tasks and make decisions.

These components work together to create a powerful and adaptable Compound AI system.

—Designing Compound AI Systems

In summary here’s a brief summary of a system approach to Application Development Life-cycle

Analysis —> Design —> Development —> Production —> Monitoring

To simplify:

01. Identify Intents

• define possible intent based on some user requirements.

• define intent dependencies

02. Identify Tools

• tools you may need to accomplish a task. Like web search, apis, code execution etc.

03. Build the Chain

• build a workflow based on identified tasks

• architecture you may to solve the problem, like RAG, Text2SQL, etc.

• iterate the process

An example: 3 main intents

1. intent - retrieval / search piece;

2. taking those requests, responses from that search and then building the augmentation of the prompt, grabbing those summaries an building the context;

3. once we have the context, sending to the model then generating answers.

Components

The Compound AI System is a modular, scalable, and flexible architecture designed to integrate multiple AI components to achieve complex tasks. This architecture enables the creation of a robust and adaptable AI system that can be applied to various domains.

1. Data Ingestion Layer

2. AI Component Layer

3. Knowledge Graph

4. Orchestration Layer

5. Decision Support System

6. Continuous Learning and Improvement

—Challenges of Compound AI Systems

Building

• Complexity in integrating multiple components

• Managing multiple models and processing steps

• Adapting to different hardware configurations

• Ensuring robust metrics and logging systems

Optimizing

• Balancing latency, throughput, and resource utilization

• Co-optimizing system components

• Cost optimization while maintaining performance

• Harmonizing interactions among components

Deploying

• Meeting diverse hardware, software, and scalability requirements

• Managing operational complexity with MLOps and DataOps

• Ensuring scalability and elasticity

• Integrating with existing infrastructure

• Addressing security, privacy, and explainability concerns

These challenges highlight the complexity and nuance of building, optimizing, and deploying Compound AI systems.

Some good reads

• https://www.databricks.com/glossary/compound-ai-systems

• The Shift from Models to Compound AI Systems

tl;dr - Vector search and the AI connection

The noise around AI, Generative AI applications and the demand these apps are creating around the need for users to search through huge data models. To put tin simple terms, Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content.

So the more people are using Generative AI.. which is more or less “Searcing through tons of content and getting appropriate results back, and that too fast.” If you have used ChatGPT, Google Bard, Github’s Copilot, all of these give you answers/generate responses based on your questions/requests.

All of this boils down to “search within a content and get results back” aka “search”.

This post details about vector search, how it works (briefly) and some platforms that support this.

What is vector search?

Vector search leverages machine learning (ML) to capture the meaning and context of unstructured data, including text and images, transforming it into a numeric representation. Frequently used for semantic search, vector search finds similar data using approximate nearest neighbor (ANN) algorithms. Compared to traditional keyword search, vector search yields more relevant results and executes faster - Source Elasticsearch.

..wait that’s a lot. Vectors? Semantic search? Nearest neighbor algorithm?

here’s a simpler version - Vector search is a type of search that uses machine learning to find objects that are similar to a given query. It works by converting the query and the objects into vectors, which are then compared to each other. The objects that are most similar to the query are then returned as results.

What is a vector?

In computer science a Vector is a data structure that stores a sequence of elements of same data type. They are similar to arrays but they have some differences:

• Random access the elements in a vector can be accessed by their index making them efficient for operations like searching and sorting.

• Dynamic size while arrays have fixed size, vectors can grow & shrink as needed making them more flexible and efficient.

• Vector operations vectors support a lot of operations that are optimized for their data structure like addition, subtraction, multiplication, and division.

Why search?

A common use case in almost every software system, search enables users to find information within the system or data that the system can access.

But what is the problem? Humans use language which is often ambiguous and fuzzy. There are synonyms (two words mean the same thing), polysems (same word can have multiple meanings). In English for example “accurate” and “precise” can sometimes be synonymous, but “accurate” can also mean many different things like current, good, true, proper. With these ambiguities, developing efficient search is difficult.

How does a vector search engine work?

Traditional search leans more on keywords, similar words, how many time a word repeated, vector search engines use distances in the embedding space to represent similarity.

To address the above, different machine learning techniques such as spelling correction, language processing, category matching, and more are used to structure and make sense of language. These words are converted into vectors (numbers) allowing their meaning to be encoded and processed mathematically. This process of converting is called vectorization.

Usually vectors are used for clustering documents, identifying meaning and intents in queries, ranking results and adding synonyms. There is also something called vector embeddings which are more than vectors + objects like whole documents, audio, video, images files.

Embeddings - aka vectors for business use cases

A major challenge in creating vectors for representing different entities that are meaningful and useful for business use cases. By applying pre-trained deep learning technology models to raw data, you can extract "embeddings" - vectors that map each row of data in a space of their "meanings".

Challenge 1 - Successful extraction of useful vectors (embeddings) from your business data is the key challenge. But after you have done that, the only thing you have to do is search for similar vectors - which is another complex part of the process.

Challenge 2 - Building a fast and scalable vector search engine. Some of the most widely used metrics for calculating the similarity between vectors are L2 distance (Euclidean distance), cosine similarity, and inner product (dot product). These may get too mathematical, and can conveniently avoid understanding the inner working of this. Usually search engines will have these pre-defined/implemented.

To learn more about creating embeddings, this google course will help to understand the fundamentals.

Use cases

Vector search is specialized search and like many software systems, it is not recommended for everything and/or can solve all search problems.

Few use cases:

1. Semantic Search — wait what the heck is this? if you asked above.. please stop there. Read about Semantic Search here.

2. Recommendations - Vector search can be used to recommend products to users based on their past purchases, interests, and browsing history.

3. Question Anwsering -

4. Image search - Vector search can be used to find images that are similar to a given image. This is useful for finding similar products, finding similar artwork, or finding similar landmarks.

5. Natural language search - Vector search can be used to find documents that are similar to a given document. This is useful for finding similar news articles, finding similar research papers, or finding similar blog posts.

A selection of software products with vector search

• Commercial

  • Elasticsearch - Elasticsearch Relevance Engine

  • Apache Solr available from v9.3.0

  • Algolia Neural Search

  • Pinecone

  • Google Cloud Vertex AI Matching Engine

  • Azure Cognitive Search - public preview of Vector search

  • MongoDB Atlas Vector search

• Open Source

  • Qdrant

  • Vespa

Onward

Generative AI is exploding eery single day and as more enterprises & startups jump into the race, more applications are built, the demand for “search” will only grow north. This directly relates to how fast users will be able to view “search results” from the terabytes of data these organizations and LLaMA models are released.

Subscribe now

As a backend java developer, adding some javascript here and there, was a big deal for me, back in 2010’s. Ajax, JQuery simplified this a little, but also made javascript files more complex. Around 2015, we were trying to solve some issues on a big data entry form (I still remember eery details on that), one of the intern on the team showed me how we can solve most of them using Angular (and yes, be open to learn from anyone); It was an eye-opener.. yaaay.. aha.. what-you-can-do-this? moments.

But yes, AngularJS (was end-of-life jan 2022) did change the way I was solving frontend and was very close to how I was developing java based applications. And since, i’ve been working / using / preaching Angular. Was.. has been and is my favorite framework.

This post is about why Angular, above other great/awesome js frameworks.

Why Angular? because..

1.Complete Framework

Angular is a complete framework. As an application framework, Angular include several libraries allowing developers to do solve/implement features from within the library than using other third-party. (Although there are several other third-party libraries that help us to solve certain features better/easier than angular’s libraries)

Core Features/Concepts: Components, Templates, Directives, Dependency Injection

Libraries integrated Angular: Routing & Navigation, Forms, HTTP (my favorite), Testing, Internationalization, Animations, Web Workers & Progressive Web Apps, Server side rendering

2. Great Tooling / Developer Friendly

Another great availability from Angular is the tooling support to build projects better.

• Angular CLI saves a ton of time to build projects.

• Language Services available for all major IDEs helps developers write better code.

• Schematics, Dev tools, and lots of other from community like ngd , codelyzer,

3. Guides, Examples, Help

My list keeps getting better. There is a lot of material we can find on the internet to learn Angular. And the best part, most of it is free. You can find Books, Podcasts, Video tutorials, Blogs, Bootcamps, …

A passionate community of developers keep the content coming!!

4. Awesome Community

This is my favorite part. Angular’s community is great. Lots of ways to connect/talk Gitter, Discord, Twitter

and over the years tons of applications have been built on Angular, and this feedback sent back to Angular teams is making the framework better every day.

5. Open Source

Was.. is and continues to be open-source forever (hopefully Google does not change it’s mind).

Actually I can keep writing.. but trying to keep this short.

Live, and Let Live

I love Angular. I am not preaching “Angular is The best in the market”. There are a lot of other great javascript based frameworks that solve certain problems very well, sometimes, better than Angular. Yes, a heathy competition is always good that pushes us forward.

Being a software developer, i’m alway open to learning new things and keep an eye on what maybe coming. How I can simplify -> a certain piece of code / implementing a feature / deploying & managing applications.

Always Learning!

A recent research paper AI models collapse when trained on recursively generated data published by Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson & Yarin Gal on 24 July, 2024 makes the point.

and they also kind of make sense.

What is a model collapse?

According to the paper, Model collapse is a degenerative process affecting generations of learned generative models, in which the data they generate end up polluting the training set of the next generation. Being trained on polluted data, they then misinterpret reality.

Basics. How AI models are trained

As most of you know there is tons of human-generated data all around us and has been since ages. Books, papers, news paintings, and in the last 2-3 decades most of that physical data is digitized and made available on the web. Then there’s more - like blogs, news, videos, movies, pictures, drawings, social media and the list goes on.

Imagine if there is a way to accumulate all of this data into a software system or a database? That is exactly what has happened and such data is being used to train AI LLMs or Large Language Models.

A few popular data sets used to train LLMs HelpSteer , No Robots , Anthropic HH Golden and several at Hugging Face Datasets

The Problem

These models keep growing bigger and bigger every generation, as model builders keep adding more data. What happens at some point if these builders exhaust / use all of this data?

The LLMs will evaluate themselves and generate their own data - called as, synthetic data [2].

Also LLMs while training raw data, processes the data to generate a refined set. And as new generation of LLMs are trained with more / new datasets, LLMs grow / pile up / add on the previously refined datasets. This is how LLMs data is managed / stored.

So far so good, now comes the bad part.

If in this cycle, some parts are corrupted, that corrupted data just keep piling up on new models - unless someone removes/clears it. And this may lead to an AI Model Collapse.

In the research paper it was demonstrated that training on samples from another generative model can induce a distribution shift, which—over time—causes model collapse. This in turn causes the model to misinterpret the underlying learning task.

The Solution, if there is any?

The “first mover advantage” is evident in training models like Large Language Models (LLMs). However, to sustain learning over time, AI vendors must ensure:

• Preserved access to original data sources

• Continuous availability of new, human-generated data

• Community-Wide Coordination: different parties involved in LLM creation and deployment must share information to resolve provenance questions. Without this coordination, training newer LLM versions may become increasingly difficult due to:

  • Limited access to pre-mass adoption internet-crawled data

  • Insufficient direct access to human-generated data at scale

A key challenge arises in distinguishing between LLM-generated data and other data. This raises questions about the provenance of internet-crawled content. Tracking LLM-generated content at scale is a significant concern.

References

1. Luiza Jarvosky post on X

2. Aparna Ramani, VP of Engineering at Meta, responsible for AI infrastructure, Data Infrastructure and Developer Infrastructure, gives the opening keynote presentation at AI Infra @Scale 2024.

Swagger UI is a great resource to generate and show docs for (java based) APIs. While adding a Swagger UI to a maven based project should be straight forward, adding a few annotations to your class and some dependencies to your pom.xml. It turned out there are a few steps involved for Helidon built projects. This article presents those details.

TL;DR

This article discusses the how-to’s of setting up Swagger UI to a project built with Helidon MP . A working example is on Github.

Why this post?

It took a while for me to setup Swagger UI to my API-only project built with Helidon MP. Wanted to share this article to my fellow developers, to save time.

Getting started

Building from scratch

If you already have an existing project, skip steps 1 and 2

1.Create a new project using Helidon MP’s archetype

mvn -U archetype:generate 
    -DinteractiveMode=false
    -DarchetypeGroupId=io.helidon.archetypes
    -DarchetypeArtifactId=helidon-quickstart-mp
    -DarchetypeVersion=2.3.4
    -DgroupId=io.helidon.examples
    -DartifactId=helidon-quickstart-mp
    -Dpackage=io.helidon.examples.quickstart.mp

2.Build and Run the default project

mvn package
java -jar ./target/helidon-quickstart-mp.jar

3.Add these dependencies to pom.xml

<!-- https://mvnrepository.com/artifact/org.eclipse.microprofile.openapi/microprofile-openapi-api -->
<dependency>
   <groupId>org.eclipse.microprofile.openapi</groupId>
   <artifactId>microprofile-openapi-api</artifactId>
   <version>1.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.helidon.microprofile.openapi/helidon-microprofile-openapi -->
<!-- NOTE: I built the project using Helidon MP 2.3.4, pleaase us the version per your project -->
<dependency>
   <groupId>io.helidon.microprofile.openapi</groupId>
   <artifactId>helidon-microprofile-openapi</artifactId>
   <version>2.3.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.microprofile-ext.openapi-ext/openapi-ui -->
<dependency>
   <groupId>org.microprofile-ext.openapi-ext</groupId>
   <artifactId>openapi-ui</artifactId>
   <version>1.1.5</version>
</dependency>

4. Build and Run the project again

mvn package
java -jar ./target/helidon-quickstart-mp.jar

5. In your browser, use the url “openapi-ui” . By default it would be like http://localhost:9393/your-project/openapi-ui

Screenshot of Swagger-UI (like) when added to a Helidon MP built project - Suren K

That’s it.

Adding docs to your APIs

We need to add the required Annotations for docs to be generated. Example below (in the file)

@Path("/greeting")
@PUT
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
@RequestBody(name = "greeting",
   required = true,
   content = @Content(mediaType = "application/json",
   schema = @Schema(type = SchemaType.STRING, example = "{\"greeting\" : \"Hola\"}")))
@APIResponses({
   @APIResponse(name = "normal", responseCode = "204", description = "Greeting updated"),
   @APIResponse(name = "missing 'greeting'", responseCode = "400",
      description = "JSON did not contain setting for 'greeting'")})
 public Response updateGreeting(JsonObject jsonObject) { ... }

Background

You may have observed, we did not use Swagger-ui dependency but instead used Open-API. We can add MicroProfile OpenAPI annotations to Helidon MP built project, and Helidon will automatically generate the OpenAPI document for you and provide an endpoint where clients can retrieve it.

For more details, you may go here

• Helidon supports OpenAPI and

• Swagger also supports OpenAPI Specifications and

• https://github.com/microprofile-extensions/openapi-ext/tree/main/openapi-ui and

• https://github.com/oracle/helidon/tree/master/examples/openapi

Resources

• Working example on Github - https://github.com/ksurendra/helidon-mp-swagger

• Helidon resources https://github.com/oracle/helidon

• Swagger resources https://swagger.io/docs/