Back to Advanced Topics
Q8Question 8 of 45

What are AI agents and how do they work in a Java backend

Answer: An AI agent is a system where an LLM acts as the reasoning engine that decides which tools to call, interprets the results, and iterates through a plan until the task is complete, enabling autonomous multi-step workflows in a Java backend.

Key Concepts
  • An AI agent is a system where an LLM decides what actions to take, calls tools to execute those actions, observes the results, and iterates until the task is complete.
  • Tools are functions the agent can call: search a database, call an API, run a calculation, or read a file.
  • The agent loop follows a reason-act-observe cycle: the LLM reasons about what to do, calls a tool, observes the result, and decides the next step.
  • State management tracks the agent's progress, intermediate results, and context across multiple tool calls.
  • Planning breaks complex tasks into steps, and the agent executes each step, adapting the plan based on intermediate results.
  • Guardrails limit what the agent can do: maximum iterations, approved tool list, confirmation for destructive actions.

Definition

An AI agent is a system where an LLM acts as the reasoning engine that decides which tools to call, interprets the results, and iterates through a plan until the task is complete, enabling autonomous multi-step workflows in a Java backend.
  • AI agent is a program that uses an LLM to reason, plan, and execute actions through tool calls in an iterative loop.
  • Tool is a function the agent can invoke: querying a database, calling a REST API, performing a calculation, or reading a document.
  • Agent loop is the reason-act-observe cycle: the LLM decides the next action, the tool executes it, and the result is fed back to the LLM.
  • State is the accumulated context (conversation history, intermediate results, variables) the agent carries across iterations.
  • Planning is the agent's ability to decompose a complex task into ordered steps before executing them.

Think of a new loan officer at the National Bank of India. The officer (LLM) receives a customer request: "I want to apply for a home loan." The officer thinks about the steps: check eligibility, verify documents, calculate EMI, check credit score, and prepare the offer letter. For each step, the officer uses a tool: the eligibility calculator, the document verification system, the EMI formula, the credit bureau API. After each step, the officer looks at the result and decides the next action. If the credit score is too low, the officer adjusts the plan. This is an AI agent: reasoning, acting, observing, and adapting.

Understanding the Concepts

How does the agent loop work?

The agent loop is the core pattern. The LLM receives the task and conversation history. It decides which tool to call (or whether to provide a final answer). The tool executes and returns a result. The result is added to the conversation, and the LLM decides again.

JAVA
package com.bank.ai;

import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Map;
import java.util.HashMap;

public class AgentLoopDemo {

    // Tools the agent can call
    static Map<String, Object> callTool(String tool, Map<String, String> params) {
        Map<String, Object> result = new HashMap<>();
        switch (tool) {
            case "check_eligibility" -> {
                String income = params.get("monthly_income");
                boolean eligible = new BigDecimal(income).compareTo(new BigDecimal("50000")) >= 0;
                result.put("eligible", eligible);
                result.put("reason", eligible ? "Income meets minimum" : "Income below 50000");
            }
            case "calculate_emi" -> {
                BigDecimal principal = new BigDecimal(params.get("principal"));
                BigDecimal rate = new BigDecimal("8.5").divide(new BigDecimal("1200"), 10, RoundingMode.HALF_UP);
                int months = Integer.parseInt(params.get("tenure_years")) * 12;
                BigDecimal emi = principal.multiply(rate).multiply(
                    rate.add(BigDecimal.ONE).pow(months)).divide(
                    rate.add(BigDecimal.ONE).pow(months).subtract(BigDecimal.ONE),
                    2, RoundingMode.HALF_UP);
                result.put("emi", emi.toString());
                result.put("total_interest", emi.multiply(BigDecimal.valueOf(months)).subtract(principal).toString());
            }
            case "check_credit_score" -> {
                result.put("score", 750);
                result.put("rating", "GOOD");
            }
        }
        return result;
    }

    public static void main(String[] args) {
        System.out.println("Agent task: Process home loan application for Rajesh");
        System.out.println();

        // Step 1: LLM reasons -> call check_eligibility
        System.out.println("Step 1: [REASON] Check if Rajesh meets income requirement");
        var eligibility = callTool("check_eligibility", Map.of("monthly_income", "75000"));
        System.out.println("  [OBSERVE] " + eligibility);
        System.out.println();

        // Step 2: LLM reasons -> call check_credit_score
        System.out.println("Step 2: [REASON] Eligibility passed, check credit score");
        var credit = callTool("check_credit_score", Map.of("customer_id", "CUST-2001"));
        System.out.println("  [OBSERVE] " + credit);
        System.out.println();

        // Step 3: LLM reasons -> call calculate_emi
        System.out.println("Step 3: [REASON] Credit score good, calculate EMI");
        var emi = callTool("calculate_emi", Map.of("principal", "5000000", "tenure_years", "20"));
        System.out.println("  [OBSERVE] " + emi);
        System.out.println();

        // Step 4: LLM provides final answer
        System.out.println("Step 4: [ANSWER] Rajesh is eligible for a home loan.");
        System.out.println("  Loan amount: 50,00,000 INR");
        System.out.println("  Interest rate: 8.5% p.a.");
        System.out.println("  EMI: " + emi.get("emi") + " INR/month");
        System.out.println("  Credit score: " + credit.get("score") + " (" + credit.get("rating") + ")");
    }
}
TEXT
Agent task: Process home loan application for Rajesh

Step 1: [REASON] Check if Rajesh meets income requirement
  [OBSERVE] {eligible=true, reason=Income meets minimum}

Step 2: [REASON] Eligibility passed, check credit score
  [OBSERVE] {score=750, rating=GOOD}

Step 3: [REASON] Credit score good, calculate EMI
  [OBSERVE] {emi=43391.41, total_interest=5413938.40}

Step 4: [ANSWER] Rajesh is eligible for a home loan.
  Loan amount: 50,00,000 INR
  Interest rate: 8.5% p.a.
  EMI: 43391.41 INR/month
  Credit score: 750 (GOOD)

What tools can a banking AI agent use?

DIAGRAM
| Tool | Purpose | Example |
|------|---------|--------|
| check_balance | Query account balance | SAV2001 balance: 75,000 |
| transfer_funds | Execute a transfer | SAV2001 -> SAV3002: 10,000 |
| calculate_emi | Calculate loan EMI | 50L at 8.5% for 20 years |
| check_credit | Look up credit score | Score: 750 (GOOD) |
| search_policies | RAG search bank policies | Home loan eligibility criteria |
| generate_statement | Create account statement PDF | SAV2001 Jan 2025 statement |

What are guardrails for AI agents?

Guardrails prevent the agent from taking harmful or unauthorized actions.

JAVA
package com.bank.ai;

import java.util.Set;

public class AgentGuardrailsDemo {

    static final int MAX_ITERATIONS = 10;
    static final Set<String> ALLOWED_TOOLS = Set.of(
        "check_balance", "calculate_emi", "search_policies", "check_credit"
    );
    static final Set<String> REQUIRES_CONFIRMATION = Set.of(
        "transfer_funds", "close_account", "update_kyc"
    );

    static boolean canExecute(String tool, int iteration) {
        if (iteration > MAX_ITERATIONS) {
            System.out.println("BLOCKED: Max iterations (" + MAX_ITERATIONS + ") reached");
            return false;
        }
        if (!ALLOWED_TOOLS.contains(tool) && !REQUIRES_CONFIRMATION.contains(tool)) {
            System.out.println("BLOCKED: Tool '" + tool + "' not in approved list");
            return false;
        }
        if (REQUIRES_CONFIRMATION.contains(tool)) {
            System.out.println("CONFIRM: Tool '" + tool + "' requires human approval");
            return false; // In production, pause and ask user
        }
        return true;
    }

    public static void main(String[] args) {
        System.out.println(canExecute("check_balance", 1));    // true
        System.out.println(canExecute("transfer_funds", 2));   // requires confirmation
        System.out.println(canExecute("delete_account", 3));   // blocked
        System.out.println(canExecute("check_balance", 11));   // max iterations
    }
}
TEXT
true
CONFIRM: Tool 'transfer_funds' requires human approval
false
BLOCKED: Tool 'delete_account' not in approved list
false
BLOCKED: Max iterations (10) reached
false

What are common mistakes with AI agents?

Mistake 1: No iteration limit

Wrong: the agent loops forever if it cannot solve the problem.

Correct: set a maximum iteration count and return a graceful failure.

Mistake 2: Allowing destructive actions without confirmation

Wrong: the agent can transfer money, close accounts, or delete data without human approval.

Correct: classify tools as safe (read-only) and dangerous (write). Dangerous tools require human confirmation.

What should a senior engineer know?

- Function calling. Modern LLMs (GPT-4, Claude) support structured function calling where the model returns a JSON object specifying which function to call and with what parameters. This is more reliable than parsing free-text tool calls.

- Multi-agent systems. Complex workflows use multiple specialized agents. A supervisor agent delegates to a data-retrieval agent, a calculation agent, and a formatting agent. Each agent has its own tools and context.

- Memory management. Long-running agents accumulate conversation history that exceeds the context window. Implement summarization: compress older conversation turns into a summary while keeping recent turns in full.

- Observability. Log every agent step: the reasoning, the tool called, the parameters, the result, and the decision. This creates an audit trail for banking compliance and helps debug agent behavior.

- Cost control. Each iteration costs tokens. A 10-step agent loop with GPT-4o might cost $0.10-0.50. Set budgets per request and use cheaper models for simple reasoning steps.

How do you limit agent actions for safety?

An AI agent in a banking context must have strict boundaries. Use allowlists for permitted tools, require human approval for destructive operations, and log every action for audit.

JAVA
package com.bank.ai;

public class AgentSafety {
    public static void main(String[] args) {
        System.out.println("Agent safety boundaries for banking:");
        System.out.println("  1. Allowlist of permitted tools (no direct DB writes)");
        System.out.println("  2. Read-only access by default");
        System.out.println("  3. Human approval for any state-changing operation");
        System.out.println("  4. Maximum iteration count (prevent infinite loops)");
        System.out.println("  5. Token budget per request");
        System.out.println("  6. Audit log of every tool call and result");
    }
}
TEXT
Expected output:
Agent safety boundaries for banking:
  1. Allowlist of permitted tools (no direct DB writes)
  2. Read-only access by default
  3. Human approval for any state-changing operation
  4. Maximum iteration count (prevent infinite loops)
  5. Token budget per request
  6. Audit log of every tool call and result

How do you test an AI agent?

Testing an agent requires deterministic test scenarios with expected tool calls and outcomes. Mock the LLM responses and verify that the agent calls the right tools in the right order.

JAVA
package com.bank.ai;

public class AgentTesting {
    public static void main(String[] args) {
        System.out.println("Agent testing strategy:");
        System.out.println("  1. Unit test: mock LLM, verify tool selection");
        System.out.println("  2. Integration test: real tools, controlled environment");
        System.out.println("  3. Evaluation set: known questions with expected outcomes");
        System.out.println("  4. Safety test: adversarial prompts, boundary violations");
        System.out.println("  5. Cost test: token usage within budget");
    }
}
TEXT
Expected output:
Agent testing strategy:
  1. Unit test: mock LLM, verify tool selection
  2. Integration test: real tools, controlled environment
  3. Evaluation set: known questions with expected outcomes
  4. Safety test: adversarial prompts, boundary violations
  5. Cost test: token usage within budget

What are common interview follow-up questions?

  • What is the difference between an AI agent and a simple chatbot?
  • How do you test an AI agent's behavior?
  • What is the ReAct (Reasoning + Acting) pattern?
  • How do you handle agent errors in production?
  • What frameworks exist for building AI agents in Java?

Visual Overview

DIAGRAM
  Agent Loop: Reason - Act - Observe
  ====================================

  User Task: "Process Rajesh's home loan application"
       │
       ▼
  ┌─────────────────────────────────────┐
  │  LLM: REASON                        │
  │  "I need to check eligibility first"│
  │  Tool: check_eligibility            │
  │  Params: {income: 75000}            │
  └──────────────┬──────────────────────┘
                 │ ACT
                 ▼
  ┌─────────────────────────────────────┐
  │  Tool: check_eligibility            │
  │  Result: {eligible: true}           │
  └──────────────┬──────────────────────┘
                 │ OBSERVE
                 ▼
  ┌─────────────────────────────────────┐
  │  LLM: REASON                        │
  │  "Eligible, now check credit score" │
  │  Tool: check_credit                 │
  └──────────────┬──────────────────────┘
                 │ ACT
                 ▼
  ... (continues until task complete or max iterations)


  Guardrails
  ===========
  ┌──────────────────────┐
  │ Max iterations: 10   │
  │ Allowed tools: [...]  │
  │ Confirm: transfers   │
  │ Block: delete, admin │
  │ Budget: $0.50/request│
  └──────────────────────┘

Key Takeaway

AI agents bring autonomy to backend systems. Instead of hardcoding every step of a workflow, you give the LLM a task and a set of tools, and it figures out the steps. For the National Bank of India, an agent can process a loan application by calling the eligibility checker, the credit bureau API, and the EMI calculator in sequence, adapting its plan based on each result.

The agent loop is the core pattern: reason about what to do next, call a tool, observe the result, and repeat. The LLM is the brain that decides which tool to call and when to stop. The tools are the hands that interact with databases, APIs, and calculators. The state is the memory that tracks what has been done and what remains.

Guardrails are non-negotiable in banking. An agent that can transfer money without human approval is a security risk. Classify tools as safe (read-only queries) and dangerous (writes, transfers, deletions). Dangerous tools require explicit human confirmation. Set iteration limits to prevent infinite loops. Log every step for compliance auditing.

The most important design decision is tool design. Well-defined, focused tools with clear inputs and outputs produce reliable agent behavior. Vague, multi-purpose tools confuse the LLM and lead to errors.

One sentence to remember: an AI agent uses an LLM to reason and plan, calls tools to execute actions, observes results, and iterates until the task is complete, with guardrails to prevent unauthorized actions.

---

Share
Use ← → arrow keys to navigate