> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnao.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Playbook

> Step-by-step guide to start context engineering for your analytics agent

## Getting Started with Context Engineering

This playbook provides a systematic approach to building and maintaining effective context for your analytics agent. Follow these steps in order to ensure a solid foundation and scalable growth.

## First POC on small, reliable context

**Step 1: Add Your Data Context**

Start with a **restricted perimeter** of your data warehouse:

* **Maximum 20 tables** to begin with
* Focus on **clean, gold, or mart layer** tables (avoid raw staging tables)
* Choose tables that represent core business domains

<Tip>
  Starting small helps you validate your approach before scaling. You can always add more tables later.
</Tip>

**Step 2: Add Your Documentation Repository**

Include your documentation sources in context:

* **dbt documentation** (schema.yml, docs blocks)
* **Semantic layer** definitions
* Any other relevant documentation repositories

This helps the agent understand business logic, relationships, and data lineage.

**Step 3: Add Company and Domain Rules**

Create rules that provide context on:

* **Your company** - business context, terminology, conventions
* **Different domains** covered by your 20 tables - e.g., sales, marketing, finance, operations

These high-level rules set the foundation for domain-specific understanding.

**Step 4: Add Sub-Rules for Each Sub-Domain**

For each sub-domain covered, create detailed sub-rules that include:

* **Business definitions** - what key terms mean in your organization
* **Metrics definitions** - how metrics are calculated and used
* **List of tables** - which tables belong to this domain
* **Relevant docs yaml** - specific documentation for this domain

This modular approach makes your context easier to maintain and scale.

## Measure, test and iterate

**Step 5: Create a Set of 20 Key Questions**

Develop a **test suite** of 20 key questions that represent:

* Common user queries
* Critical business questions
* Edge cases
* Different complexity levels

These questions will serve as your quality benchmark throughout the process.

**Step 6: Test and Iterate**

Test the chat on your 20 questions:

* Run all questions through the agent
* Verify answers are correct and complete
* Identify gaps in context or understanding
* **Iterate on context** - add missing information, clarify ambiguities, refine rules

Repeat until all 20 questions are answered correctly.

<Card title="Evaluation Guide" icon="flask" href="/nao-agent/context-engineering/evaluation">
  Learn how to build comprehensive test suites and integrate testing into your workflow
</Card>

**Step 7: Roll Out to Users**

Once your test suite passes:

* **Roll out** to a small group of users initially
* **Track usage** - monitor what questions users are asking
* **Monitor real-life performance** using logs of questions and feedback
* Collect user feedback to identify improvement areas

**Step 8: Version Control and Quality Assurance**

Maintain context quality over time:

* **Version your context** using git repositories
* **Run `nao test` frequently** (e.g., weekly or after major changes)
* Ensure context quality doesn't drift as you make updates
* Set up automated tests in CI/CD pipelines
* Track test results over time to monitor performance trends

## Close the feedback loop

**Step 9: Let Recommendations Surface the Next Fix**

Your 20 test questions catch regressions, but they can't predict every question real users will ask. Once the agent is live, turn production usage into your next backlog:

* Enable [Recommendations](/nao-agent/context-engineering/recommendations) so nao audits its own usage on a schedule and ranks where your context is missing, wrong, or unclear.
* Work the impact-ordered list: tool errors, downvotes, and recurring questions each point at a specific file to fix.
* Open the fix as a pull request straight from a recommendation when a GitHub repo is connected, then re-run `nao test` to confirm it holds.

This is the online half of evaluation: `nao test` guards the benchmark you wrote, Recommendations finds the gaps you didn't. Feeding production friction back into context (and into new test cases) is what keeps the agent improving instead of drifting.

## Scale

**Step 10: Scale Gradually**

As adoption grows:

* **Extend the number of datasets** available in the agent
* **Make documentation and rules modular** to support scalability
* Add new domains incrementally, following the same process
* Maintain the same quality standards as you expand

<Card title="Context Principles" icon="book" href="/nao-agent/context-engineering/principles">
  Review the core principles that guide effective context engineering.
</Card>