NVIDIA NCP-AAI : Agentic AI

You are evaluating your RAG pipeline. You notice that the LLM-as-a-Judge consistently assigns high similarity scores to responses that contain irrelevant information.
What should you investigate as the most likely potential cause with the least development effort?

• A. The size of the knowledge base used to power the RAG pipeline.
• B. The prompt used to instruct the LLM-as-a-Judge to assess the response.
• C. The temperature setting used by the LLM during response generation.
• D. The quality of the synthetic questions used for evaluation.

You're managing an agentic AI responsible for customer support ticket triage. The agent has been consistently accurate in routing tickets to the appropriate departments. However, a team leader has noticed a significant increase in the number of tickets requiring "escalation" - cases where the agent initially misclassified a complex issue as a simple, routine one, leading to delays and frustrated customers.
What would be an appropriate first step in resolving this issue?

• A. Adjusting the agent's reward function to prioritize speed of resolution over accuracy, as a first step in analysis of the problem.
• B. Conducting a "red-teaming" exercise, having human agents deliberately create complex and ambiguous scenarios to analyze the agent's robustness.
• C. Analyzing the agent's decision-making process, focusing on the specific criteria it uses to classify tickets, and identifying potential biases or blind spots.
• D. Increasing the agent's autonomy, granting it more decision-making power during triage to improve its efficiency.

A development team is creating an AI assistant that interacts with employees to help manage schedules and tasks. The team wants to ensure users can easily provide feedback, understand the agent's decisions, and intervene when necessary to maintain control and trust.
Which practice best supports effective human oversight and interaction with the AI agent?

• A. Designing intuitive user interfaces with integrated feedback loops and transparent explanations of agent decisions
• B. Continuously collecting and integrating user feedback throughout the agent's lifecycle to drive ongoing improvements
• C. Incorporating user review stages before finalizing agent decisions to maintain accountability
• D. Enabling flexible user interactions beyond predefined commands to accommodate diverse needs

Integrate NeMo Guardrails, configure NIM microservices for optimized inference, use TensorRT-LLM for deployment, and profile the system using Triton Inference Server with multi-modal support.
Which of the following strategies aligns with best practices for operationalizing and scaling such Agentic systems?

• A. Deploy all agents on a single high-performance GPU node to reduce latency, and use cron jobs for periodic health checks and updates.
• B. Use Docker containers orchestrated by Kubernetes, implement MLOps pipelines for CI/CD, monitor agent health with Prometheus/Grafana.
• C. Deploy agents on bare-metal servers to maximize performance and avoid container overhead, using manual scripts for orchestration and monitoring.
• D. Run agents as independent serverless functions to minimize infrastructure management, relying primarily on cloud provider auto-scaling and logging tools.

Implement Memory Systems for Contextual Awareness
An enterprise AI system needs to maintain contextual information over multiple interactions with users.
Which memory implementation approach would be MOST effective for managing both immediate context and long-term historical interactions within an agentic workflow?

• A. Rely predominantly on the context window of the base LLM model to store all historical interactions with minimal external memory supplementation.
• B. Store all user interactions in a simple key-value database which will by default provide organization and retrieval strategy for historical context management.
• C. Implement a hybrid memory system with short-term memory for immediate context and a vector database for long-term memory with semantic retrieval capabilities.
• D. Use a static prompt template with fixed context for all interactions, thereby providing memory information in that form across conversation sessions.