> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnao.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Recommendations

> Close the feedback loop - nao audits its own usage to recommend context fixes (online evals)

Context engineering is a feedback loop: ship context, watch how the agent performs on real questions, fix what breaks, repeat. nao gives you both halves of that loop.

| | Offline evals | Online evals |
| - | - | - |
| **Tool** | [`nao test`](https://docs.getnao.io/nao-agent/context-engineering/evaluation) | Recommendations |
| **When** | Before you ship, in CI | Continuously, on production usage |
| **Signal** | A fixed suite of questions you wrote | What real users actually asked and where the agent struggled |
| **Answers** | "Did I regress against my benchmark?" | "Where is my context missing, wrong, or unclear?" |

`nao test` catches regressions against a benchmark you control. Recommendations catch the gaps you didn't think to test, by mining what users do once the agent is live. Together they close the loop between shipping context and improving it.

[nao Context Recommendations Demo](https://www.youtube.com/embed/ybdNfvGXPjQ?modestbranding=1\&rel=0\&iv_load_policy=3\&showinfo=0)

## How it works

On a schedule you choose (daily, weekly, or monthly), or on demand with **Run now**, nao runs an analysis agent that audits your own project context against real usage. The agent reads your usage data only through a read-only, project-scoped sandbox (auth and PII columns are excluded, and it can only see the current project). It produces a ranked list of context improvements, each pointing at the file to change and what is missing.

## What it scans

The agent mines nao's own usage to find friction, then reads your context files to locate where each fix belongs.

**Usage signal** (read-only SQL over nao's usage views):

* **Tool errors** - queries or tools that fail repeatedly
* **Negative feedback** - messages users downvoted
* **Regenerations and corrections** - answers users had to redo or correct
* **Coverage gaps** - questions the agent couldn't answer, or recurring analysis patterns it handles inconsistently

**Context files** it audits to place the fix: `RULES.md`, `semantics/*.md`, `databases/**`, `docs/`, and synced repositories under `repos/<name>/`.

**Read cost** - nao also tracks what the `read` tool spends on each context file and flags three patterns, so the audit can propose a cheaper shape rather than only a more correct one:

| Flag | What it means | What the agent tends to suggest |
| - | - | - |
| `truncated_on_read` | The file is longer than the `read` tool's 32,000-character limit, so its tail is never seen | Split it into several readable files |
| `frequent_and_expensive` | The agent reads it often and it costs a lot of tokens every time | Split it, move a recurring process into a [skill](https://docs.getnao.io/nao-agent/context-engineering/skills.md), or cut the parts nobody uses |
| `rare_but_outlier` | Rarely read, but very expensive when it is | Split it, or summarize it |

## What it outputs

Review recommendations as an admin under **Settings** -> **Recommendations**:

* **Impact-ordered cards** - the highest-friction issues first.
* **A category** - `Tool errors`, `Hallucinations`, `Semantic missing`, `Context bloat`, `Skills`, or `Other`.
* **A concrete target** - each recommendation names the file to edit and what is missing, wrong, or unclear.
* **Provenance** - the chat the friction came from (with the user and the chat title, so you can jump to it), and the model that produced the recommendation.
* **Lifecycle actions** - **acknowledge**, **snooze**, or **dismiss** each one as you work through them.
* **A drafted patch** - when nao can write the fix itself, the card says how many files it drafted. **Show diff** opens a side panel with the proposed edit rendered file by file, with added and removed line counts and collapsed unchanged context you can expand.
* **Manual fix guidance** - when the fix belongs in an auto-generated file, the card shows **How to fix** instead, with written guidance and a ready-to-use prompt.

### Filtering and sorting

The toolbar above the list narrows a long backlog:

* **Filter** by category, by the user whose chat triggered the recommendation, or by whether it came with explicit feedback. Each filter shows its count.
* **Sort** by date, by number of triggering chats, or by pull request state, ascending or descending.

## Running an audit

The Recommendations page shows the state of the latest run.

* **Run now** starts an audit immediately, outside the schedule. The button shows **Running...** while an audit is in progress.
* If a run fails, the error is shown on the page rather than failing silently.

## What you can configure

From the Recommendations settings page you control:

* **Analysis model** - which LLM provider and model run the audit.
* **Run frequency** - daily, weekly, or monthly, at a fixed time of day. You can also trigger a run any time with **Run now**.
* **Custom system prompt instructions** - extra guidance appended to the built-in audit instructions on every run. For example: *"Spot recurring analysis patterns and propose new skills to structure them"* or *"Scan the conversations of the last 3 weeks"*.
* **GitHub repository** - the `owner/name` repo where fixes are opened as pull requests (see below). Project files are not synced from this repo; it is only the target for PRs.
* **YOLO mode** - when on, nao opens pull requests automatically after each run, without human review, and marks those recommendations as applied. Leave it off to review each recommendation before opening a PR.

The audit is enabled by default on your deployment. To turn it off, set `BETA_CONTEXT_RECOMMENDATIONS_ENABLED=false`.

## Working from an MCP client

Admins and context admins can also handle recommendations from an external AI client connected to the [nao MCP](https://docs.getnao.io/nao-agent/connectors/mcp.md) server. The `list_recommendations`, `get_recommendation`, and `set_recommendation_status` tools list recommendations by impact, return the full fix (root cause, triggering chats, proposed edits, and a ready-to-run `agentPrompt`), and mark a recommendation as applied, dismissed, or open again. Pair them with `get_chat` to read the chats that triggered a recommendation. See [Available tools](https://docs.getnao.io/nao-agent/connectors/mcp.md#available-tools).

## Ways to use it

* **Catch tools that fail regularly** - a metric the agent keeps querying wrong usually means a definition is missing or ambiguous in `semantics/`.
* **Act on new user feedback** - downvotes and corrections surface as recommendations instead of sitting unread in the feedback log.
* **Encode recurring analysis patterns** - when many users ask the same kind of question, the agent suggests documenting it once in your context so every answer is consistent.

## Turn a recommendation into a pull request

When a GitHub or GitLab repository is connected, you can open the fix as a pull request (GitHub) or merge request (GitLab) directly from a recommendation - nao writes the edit to the right context file and opens it for your review.

1. **Connect GitHub or GitLab to your nao account.** nao uses the same OAuth app as [GitHub SSO](https://docs.getnao.io/nao-agent/chat/admin/setup.md#configure-github-sso) or [GitLab SSO](https://docs.getnao.io/nao-agent/chat/admin/setup.md#configure-gitlab-sso) for repository access. Connect it once at the account level.
2. **Choose the target repository** on the Recommendations settings page, in `owner/name` format. nao suggests the repositories already declared in your `nao_config.yaml`.
3. **Open the PR or MR.** Review the drafted edit with **Show diff**, then click **Create PR**. nao proposes the concrete edit (for human-written files like `RULES.md` or `semantics/**`, or for upstream files under `repos/<name>/` when that repo maps to a GitHub or GitLab URL) and opens a pull request or merge request. You review and merge it like any other change, keeping your context under version control.

Once the pull or merge request exists, the card replaces **Create PR** with **View PR** and shows its current state.

For fixes that belong in generated files (`databases/**`, which `nao sync` rewrites) or in unconnected sources, nao instead hands you clear guidance to apply yourself rather than opening a PR.

### Batching several fixes into one pull request

Opening one PR per recommendation gets noisy once you have a backlog. Select several cards instead and ship them together.

Turn on selection mode, tick the recommendations you want, and an action bar appears at the bottom of the page:

* **Open 1 PR** commits each selected fix separately and opens a **single** pull request containing all of them, so your reviewer sees one change with one commit per recommendation. Only recommendations with a drafted patch and no existing PR are eligible - the bar shows how many of your selection qualify.
* **Copy prompts** copies the fix prompts for everything selected, so you can paste them into your own coding agent instead.
* **Select all** / **Deselect all** work on the currently filtered list.

Up to 20 recommendations can be actioned at once.

> [!NOTE]
>
> When the nao project lives in a subdirectory of the connected repository (a monorepo, for example), nao writes the fixes at the right path inside it rather than at the repository root.

To skip the manual step entirely, turn on **YOLO mode** (see [What you can configure](#what-you-can-configure)) and nao opens these PRs automatically after each run for you to review and merge.

## Next steps

> [card] [Evaluation](https://docs.getnao.io/nao-agent/context-engineering/evaluation.md)
>
> Build the offline test suite that complements recommendations

> [card] [Playbook](https://docs.getnao.io/nao-agent/context-engineering/playbook.md)
>
> See where recommendations fit in the end-to-end workflow


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.