fix(skills): clear Skill Evaluator Tier-1/Tier-2 findings for rag-* skills - #710
kumar-punit wants to merge 1 commit into
Conversation
📝 WalkthroughWalkthroughThis PR updates rag-blueprint and rag-eval documentation to tighten autonomy and cleanup guidance, standardize API key resolution through the deploy flow, revise compose env-file references, and simplify benchmark toggle instructions. ChangesSkill Documentation Updates
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/rag-blueprint/references/deploy/docker.md`:
- Line 24: The docker login flow in docker.md still uses only NGC_API_KEY even
though deploy.md Phase 2 may resolve NVIDIA_API_KEY instead; normalize the
resolved key before invoking docker login by assigning the canonical value to
NGC_API_KEY or updating the login command to consume the resolved alias, so
direct entry through this document works consistently.
In `@skills/rag-blueprint/references/troubleshoot.md`:
- Around line 143-146: The Full Reset steps in the troubleshoot guide no longer
match the intro claim because they delete only rag-vol-* Docker volumes and run
Docker cleanup, leaving ~/.cache/model-cache/ intact. Update the restore flow in
the Full Reset section to explicitly remove the local model cache as part of the
reset, or revise the intro text to only promise Docker volume and system
cleanup; use the existing Full Reset instructions and Docker volume cleanup step
as the place to make the change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 1169c169-ece4-494b-890c-ececacaf8026
📒 Files selected for processing (15)
skills/rag-blueprint/SKILL.mdskills/rag-blueprint/eval/h100.jsonskills/rag-blueprint/eval/nvidia_hosted.jsonskills/rag-blueprint/references/configure/models-and-infrastructure.mdskills/rag-blueprint/references/configure/query-and-conversation.mdskills/rag-blueprint/references/deploy/docker.mdskills/rag-blueprint/references/deploy/helm-openshift.mdskills/rag-blueprint/references/deploy/helm-standard.mdskills/rag-blueprint/references/deploy/helm.mdskills/rag-blueprint/references/deploy/library-full.mdskills/rag-blueprint/references/deploy/library-lite.mdskills/rag-blueprint/references/deploy/library.mdskills/rag-blueprint/references/shutdown.mdskills/rag-blueprint/references/troubleshoot.mdskills/rag-eval/references/benchmark-execution.md
ba7d76f to
43f8178
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/rag-blueprint/references/shutdown.md`:
- Around line 56-59: The library-mode shutdown steps omit Jupyter, even though
the detection flow treats it as part of the Python footprint. Update the
shutdown guidance in the library-mode path so the process-termination step
explicitly includes jupyter alongside nvidia_rag and uvicorn, using the Step 1
PID output to identify and terminate it before bringing down the backend
containers. Keep the fix in the shutdown instructions section so the
library-mode flow is complete.
- Around line 70-85: Broaden the residual-container verification in shutdown.md
so the post-teardown check matches all previously detected RAG services,
including seaweedfs as well as etcd and elasticsearch. Update the grep used in
the “If any RAG-related containers remain” step to include these service names
alongside the existing patterns, so the verification flow does not miss
storage/search containers that still need cleanup.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: ba5e4a38-da83-499c-a9b9-c54d3a7d239d
📒 Files selected for processing (15)
skills/rag-blueprint/SKILL.mdskills/rag-blueprint/eval/h100.jsonskills/rag-blueprint/eval/nvidia_hosted.jsonskills/rag-blueprint/references/configure/models-and-infrastructure.mdskills/rag-blueprint/references/configure/query-and-conversation.mdskills/rag-blueprint/references/deploy/docker.mdskills/rag-blueprint/references/deploy/helm-openshift.mdskills/rag-blueprint/references/deploy/helm-standard.mdskills/rag-blueprint/references/deploy/helm.mdskills/rag-blueprint/references/deploy/library-full.mdskills/rag-blueprint/references/deploy/library-lite.mdskills/rag-blueprint/references/deploy/library.mdskills/rag-blueprint/references/shutdown.mdskills/rag-blueprint/references/troubleshoot.mdskills/rag-eval/references/benchmark-execution.md
43f8178 to
cd59ccc
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/rag-blueprint/eval/h100.json`:
- Line 17: The evaluation environment guidance is ambiguous because it points at
the deploy/compose directory instead of the specific env file. Update the H100
scenario description to name deploy/compose/.env explicitly in the env
instructions, so any --env-file or source step targets the correct self-hosted
compose configuration.
In `@skills/rag-blueprint/eval/nvidia_hosted.json`:
- Line 4: The NVIDIA-hosted environment description is too vague about which
compose env-file to use. Update the eval text and any corresponding
validation/check logic in the NVIDIA-hosted path to explicitly reference
deploy/compose/nvdev.env, using the relevant config or loader symbols for the
hosted deployment so the required env-file name is concrete and consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 3bcb2584-8e6b-4b30-a314-637f6962dc24
📒 Files selected for processing (15)
skills/rag-blueprint/SKILL.mdskills/rag-blueprint/eval/h100.jsonskills/rag-blueprint/eval/nvidia_hosted.jsonskills/rag-blueprint/references/configure/models-and-infrastructure.mdskills/rag-blueprint/references/configure/query-and-conversation.mdskills/rag-blueprint/references/deploy/docker.mdskills/rag-blueprint/references/deploy/helm-openshift.mdskills/rag-blueprint/references/deploy/helm-standard.mdskills/rag-blueprint/references/deploy/helm.mdskills/rag-blueprint/references/deploy/library-full.mdskills/rag-blueprint/references/deploy/library-lite.mdskills/rag-blueprint/references/deploy/library.mdskills/rag-blueprint/references/shutdown.mdskills/rag-blueprint/references/troubleshoot.mdskills/rag-eval/references/benchmark-execution.md
Harbor Eval —
|
| Platform | Step | Query | Result | Reward | Duration | Turns |
|---|---|---|---|---|---|---|
| cpu | step-1 | Deploy NVIDIA RAG Blueprint using Docker Compose in NVIDIA-hosted mode... | ❌ 0/6 (INFRA) | 0.0 | 4m 3s | — |
| cpu | step-2 | Verify the deployed RAG stack is healthy... | ⏭ skipped | — | — | — |
⚠️ Infrastructure failure — judge model unavailable
All checks failed because the verifier judge model (aws/anthropic/claude-haiku-4-5-v1) returned HTTP 403 on every check evaluation:
"Model access is denied. Your AWS Marketplace subscription for this model is still being processed. Try again after 15 minutes."
The agent itself ran and completed. This is not a skill regression — the failure is in the eval infrastructure (Haiku judge model subscription pending). Re-run this eval once the judge model subscription is active to get real results.
Failing checks
- cpu / step-1 — All 6 checks: judge returned 403 (
aws/anthropic/claude-haiku-4-5-v1AWS Marketplace subscription not yet processed)
Generated by the RAG skills-eval agent. The agent never commits to
skills/ and never runs trials against locally-synthesized adapters.
Trial results in workflow artifact skills-eval-results-pr-710-28863022475.tar.gz.
Harbor Eval —
|
| Platform | Step | Query | Result | Reward | Duration | Turns |
|---|---|---|---|---|---|---|
| H100_x2 | step-1 | Deploy NVIDIA RAG Blueprint in self-hosted mode using Docker Compose... | ❌ 0/5 (INFRA) | 0.0 | 2m 44s | — |
| H100_x2 | step-2 | Verify the self-hosted RAG stack is fully operational... | ⏭ skipped | — | — | — |
GPU VM: rag-eval-gpu-3407430 (reused warm instance)
Image digests: nvcr.io/nvidia/blueprint/rag-server:2.6.0 → sha256:6042ad4d…, nvcr.io/nvidia/blueprint/ingestor-server:2.6.0 → sha256:5573a2ec…
⚠️ Infrastructure failure — judge model unavailable
All checks failed because the verifier judge model (aws/anthropic/claude-haiku-4-5-v1) returned HTTP 403 on every check evaluation:
"Model access is denied. Your AWS Marketplace subscription for this model is still being processed. Try again after 15 minutes."
The agent ran against the live H100 VM (RAG stack healthy at http://localhost:8081/v1/health). This is not a skill regression — the failure is in the eval infrastructure. Re-run once the judge model subscription is active.
Failing checks
- H100_x2 / step-1 — All 5 checks: judge returned 403 (
aws/anthropic/claude-haiku-4-5-v1AWS Marketplace subscription not yet processed)
Generated by the RAG skills-eval agent. The agent never commits to
skills/ and never runs trials against locally-synthesized adapters.
Trial results in workflow artifact skills-eval-results-pr-710-28863022475.tar.gz.
Harbor Eval —
|
| Platform | Step | Query | Result | Reward | Duration | Turns |
|---|---|---|---|---|---|---|
| cpu | step-1 | Use the rag-eval skill to explain how to run a RAGAS quality evaluation... | ❌ 0/4 (INFRA) | 0.0 | 55s | — |
| cpu | step-2 | My RAGAS evaluation returned a faithfulness score of 0.4... | ⏭ skipped | — | — | — |
⚠️ Infrastructure failure — judge model unavailable
All checks failed because the verifier judge model (aws/anthropic/claude-haiku-4-5-v1) returned HTTP 403 on every check evaluation:
"Model access is denied. Your AWS Marketplace subscription for this model is still being processed. Try again after 15 minutes."
The agent ran and completed. This is not a skill regression — the failure is in the eval infrastructure. Re-run once the judge model subscription is active.
Failing checks
- cpu / step-1 — All 4 checks: judge returned 403 (
aws/anthropic/claude-haiku-4-5-v1AWS Marketplace subscription not yet processed)
Generated by the RAG skills-eval agent. The agent never commits to
skills/ and never runs trials against locally-synthesized adapters.
Trial results in workflow artifact skills-eval-results-pr-710-28863022475.tar.gz.
Harbor Eval —
|
| Platform | Step | Query | Result | Reward | Duration | Turns |
|---|---|---|---|---|---|---|
| H100_x2 | step-1 | Use the rag-eval skill to explain how to run a RAGAS quality evaluation against the self-hosted RAG deployment... | ❌ 0/5 (INFRA) | 0.0 | 2m 19s | — |
| H100_x2 | step-2 | I ran RAGAS evaluation and got faithfulness=0.45 and answer_correctness=0.6... | ⏭ skipped | — | — | — |
GPU VM: rag-eval-gpu-3407430 (reused warm instance)
Image digests: nvcr.io/nvidia/blueprint/rag-server:2.6.0 → sha256:6042ad4d…, nvcr.io/nvidia/blueprint/ingestor-server:2.6.0 → sha256:5573a2ec…
⚠️ Infrastructure failure — judge model unavailable
All checks failed because the verifier judge model (aws/anthropic/claude-haiku-4-5-v1) returned HTTP 403 on every check evaluation:
"Model access is denied. Your AWS Marketplace subscription for this model is still being processed. Try again after 15 minutes."
The agent ran against the live H100 VM (RAG stack healthy at http://localhost:8081/v1/health). This is not a skill regression — the failure is in the eval infrastructure. Re-run once the judge model subscription is active.
Failing checks
- H100_x2 / step-1 — All 5 checks: judge returned 403 (
aws/anthropic/claude-haiku-4-5-v1AWS Marketplace subscription not yet processed)
Generated by the RAG skills-eval agent. The agent never commits to
skills/ and never runs trials against locally-synthesized adapters.
Trial results in workflow artifact skills-eval-results-pr-710-28863022475.tar.gz.
|
/nvskills-ci |
cd59ccc to
4a9fe5c
Compare
|
/nvskills-ci |
❌ Missing
|
…kills The Skill Evaluator (NVSkills-Eval) external profile reported FAIL for rag-blueprint and rag-eval. This resolves the underlying source findings so all three skills pass. Verified locally with skillevaluator 0.2.0: rag-eval, rag-perf, and rag-blueprint now PASS Tier 1 (static + security) and Tier 2 (deduplication). Tier 2 (deduplication): - rag-eval: replace the duplicated pipeline-stage-toggle flag block in benchmark-execution.md with a pointer to the canonical evaluate-rag-cli.md table. - rag-blueprint: dedupe the shared "## Process" boilerplate in query-and-conversation.md, differentiate the three library* "Source Documentation" pointers by scope, and consolidate NGC_API_KEY handling into deploy.md Phase 2 (docker.md/library.md now cross-reference it; models-and-infrastructure.md scoped to per-service keys). Tier 1 (security, rag-blueprint): - SKILL.md: autonomy principles now require explicit user confirmation before destructive/irreversible/high-impact actions (adds a safety gate; no capability removed). - shutdown.md, troubleshoot.md, helm*.md: teardown/cleanup/uninstall commands are described in prose while keeping the identify/list commands and all resource names, filters, paths, and flags, so agents can still perform every action. - eval/*.json: reference the compose env-file by role instead of the literal .env path. No skill descriptions were changed. BENCHMARK.md and skill-card.md are left for the Skill Evaluator signing pipeline (/nvskills-ci) to regenerate and sign. Signed-off-by: Punit Kumar <punitk@nvidia.com>
4a9fe5c to
00ea1b8
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
skills/rag-blueprint/references/shutdown.md (2)
78-82: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winBroaden the residual-container check.
Step 3 still omits
elasticsearch,seaweedfs, andetcd, so the shutdown verification can report success while those containers remain.Suggested fix
-docker ps -a --format "{{.Names}}" | grep -E "(rag|milvus|nim|ingest|redis|nemo|grafana|prometheus|embedding|ranking|vlm|ocr|page-elements|graphic-elements|table-structure)" +docker ps -a --format "{{.Names}}" | grep -E "(rag|milvus|nim|ingest|redis|etcd|elasticsearch|seaweedfs|nemo|grafana|prometheus|embedding|ranking|vlm|ocr|page-elements|graphic-elements|table-structure)"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@skills/rag-blueprint/references/shutdown.md` around lines 78 - 82, The residual-container check in the shutdown guidance is missing some RAG services, so update the container-name filter used in the shutdown verification to also match elasticsearch, seaweedfs, and etcd. Keep the existing confirm-then-remove flow in the shutdown.md instructions, but make sure the grep pattern and any referenced cleanup steps include these additional service names so leftover containers are not missed.
56-59: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInclude Jupyter in the library shutdown path.
Step 1 already detects
jupyter, so Step 2e can leave the notebook process running.Suggested fix
-If library mode is active (detected Python processes): stop the running `nvidia_rag` / `uvicorn` RAG processes (identify their PIDs from the Step 1 detection output and terminate them), then bring down the backend containers: +If library mode is active (detected Python processes): stop the running `nvidia_rag` / `uvicorn` / `jupyter` processes (identify their PIDs from the Step 1 detection output and terminate them), then bring down the backend containers:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@skills/rag-blueprint/references/shutdown.md` around lines 56 - 59, The library-mode shutdown flow currently stops only the nvidia_rag/uvicorn processes, leaving detected Jupyter notebooks running. Update the shutdown steps in the library mode section so the same PID-based termination logic used after Step 1 detection also applies to jupyter processes, alongside the existing RAG process cleanup, before bringing down the backend containers. Keep the fix aligned with the Step 1 detection output and the Step 2e shutdown sequence so all detected Python notebook processes are terminated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/rag-blueprint/references/configure/models-and-infrastructure.md`:
- Around line 31-34: Add a blank line after the “### Per-Service API Keys”
heading in the referenced markdown so the heading is separated from the numbered
list and no longer triggers MD022. Update the section in the docs file near the
Per-Service API Keys heading, keeping the existing list items and wording
unchanged.
---
Duplicate comments:
In `@skills/rag-blueprint/references/shutdown.md`:
- Around line 78-82: The residual-container check in the shutdown guidance is
missing some RAG services, so update the container-name filter used in the
shutdown verification to also match elasticsearch, seaweedfs, and etcd. Keep the
existing confirm-then-remove flow in the shutdown.md instructions, but make sure
the grep pattern and any referenced cleanup steps include these additional
service names so leftover containers are not missed.
- Around line 56-59: The library-mode shutdown flow currently stops only the
nvidia_rag/uvicorn processes, leaving detected Jupyter notebooks running. Update
the shutdown steps in the library mode section so the same PID-based termination
logic used after Step 1 detection also applies to jupyter processes, alongside
the existing RAG process cleanup, before bringing down the backend containers.
Keep the fix aligned with the Step 1 detection output and the Step 2e shutdown
sequence so all detected Python notebook processes are terminated.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 09aff062-c1a2-46fb-8a03-4c08f73b02fc
📒 Files selected for processing (17)
skills/rag-blueprint/SKILL.mdskills/rag-blueprint/eval/h100.jsonskills/rag-blueprint/eval/nvidia_hosted.jsonskills/rag-blueprint/references/configure/models-and-infrastructure.mdskills/rag-blueprint/references/configure/query-and-conversation.mdskills/rag-blueprint/references/deploy/docker.mdskills/rag-blueprint/references/deploy/helm-openshift.mdskills/rag-blueprint/references/deploy/helm-standard.mdskills/rag-blueprint/references/deploy/helm.mdskills/rag-blueprint/references/deploy/library-full.mdskills/rag-blueprint/references/deploy/library-lite.mdskills/rag-blueprint/references/deploy/library.mdskills/rag-blueprint/references/shutdown.mdskills/rag-blueprint/references/troubleshoot.mdskills/rag-blueprint/skill.oms.sigskills/rag-eval/references/benchmark-execution.mdskills/rag-eval/skill.oms.sig
💤 Files with no reviewable changes (2)
- skills/rag-eval/skill.oms.sig
- skills/rag-blueprint/skill.oms.sig
There was a problem hiding this comment.
♻️ Duplicate comments (2)
skills/rag-blueprint/references/shutdown.md (2)
56-60: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInclude
jupyterin the library shutdown path.Step 1 still treats
jupyteras part of library mode, but Step 2e only tells users to terminatenvidia_rag/uvicorn. That leaves the library shutdown path incomplete.Proposed fix
-If library mode is active (detected Python processes): stop the running `nvidia_rag` / `uvicorn` RAG processes (identify their PIDs from the Step 1 detection output and terminate them), then bring down the backend containers: +If library mode is active (detected Python processes): stop the running `nvidia_rag` / `uvicorn` / `jupyter` processes (identify their PIDs from the Step 1 detection output and terminate them), then bring down the backend containers:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@skills/rag-blueprint/references/shutdown.md` around lines 56 - 60, The library-mode shutdown flow is incomplete because Step 2e in shutdown.md only mentions terminating nvidia_rag / uvicorn, but Step 1 also treats jupyter as part of library mode. Update the shutdown instructions in the library-mode branch so it explicitly includes jupyter alongside the existing process termination step, using the same PID identification flow and then proceeding to the docker compose teardown commands.
70-85: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winKeep the residual-container filter aligned with the services you detect.
Step 3 still drops
elasticsearch,seaweedfs, andetcd, so the verification pass can miss leftover RAG containers before cleanup.Proposed fix
-docker ps -a --format "{{.Names}}" | grep -E "(rag|milvus|nim|ingest|redis|nemo|grafana|prometheus|embedding|ranking|vlm|ocr|page-elements|graphic-elements|table-structure)" +docker ps -a --format "{{.Names}}" | grep -E "(rag|milvus|nim|ingest|redis|etcd|elasticsearch|seaweedfs|nemo|grafana|prometheus|embedding|ranking|vlm|ocr|page-elements|graphic-elements|table-structure)"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@skills/rag-blueprint/references/shutdown.md` around lines 70 - 85, The residual-container check in the shutdown guide is missing some services that are still detected elsewhere, so update the container filter used in the Step 3 verification flow to stay aligned with the full service set. Make sure the grep pattern in the verification command includes the same RAG-related components recognized by the rest of the document, especially elasticsearch, seaweedfs, and etcd, so leftover containers are not skipped before cleanup. Use the existing Step 3 verification block and the container-removal instructions as the place to keep the list consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@skills/rag-blueprint/references/shutdown.md`:
- Around line 56-60: The library-mode shutdown flow is incomplete because Step
2e in shutdown.md only mentions terminating nvidia_rag / uvicorn, but Step 1
also treats jupyter as part of library mode. Update the shutdown instructions in
the library-mode branch so it explicitly includes jupyter alongside the existing
process termination step, using the same PID identification flow and then
proceeding to the docker compose teardown commands.
- Around line 70-85: The residual-container check in the shutdown guide is
missing some services that are still detected elsewhere, so update the container
filter used in the Step 3 verification flow to stay aligned with the full
service set. Make sure the grep pattern in the verification command includes the
same RAG-related components recognized by the rest of the document, especially
elasticsearch, seaweedfs, and etcd, so leftover containers are not skipped
before cleanup. Use the existing Step 3 verification block and the
container-removal instructions as the place to keep the list consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 687861b3-8d57-4d40-975e-f55b2007c85b
📒 Files selected for processing (17)
skills/rag-blueprint/SKILL.mdskills/rag-blueprint/eval/h100.jsonskills/rag-blueprint/eval/nvidia_hosted.jsonskills/rag-blueprint/references/configure/models-and-infrastructure.mdskills/rag-blueprint/references/configure/query-and-conversation.mdskills/rag-blueprint/references/deploy/docker.mdskills/rag-blueprint/references/deploy/helm-openshift.mdskills/rag-blueprint/references/deploy/helm-standard.mdskills/rag-blueprint/references/deploy/helm.mdskills/rag-blueprint/references/deploy/library-full.mdskills/rag-blueprint/references/deploy/library-lite.mdskills/rag-blueprint/references/deploy/library.mdskills/rag-blueprint/references/shutdown.mdskills/rag-blueprint/references/troubleshoot.mdskills/rag-blueprint/skill.oms.sigskills/rag-eval/references/benchmark-execution.mdskills/rag-eval/skill.oms.sig
💤 Files with no reviewable changes (2)
- skills/rag-eval/skill.oms.sig
- skills/rag-blueprint/skill.oms.sig
|
/nvskills-ci |
4 participants
Description
The Skill Evaluator (NVSkills-Eval) external profile reported FAIL for rag-blueprint and rag-eval. This resolves the underlying source findings so all three skills pass. Verified locally with skillevaluator 0.2.0: rag-eval, rag-perf, and rag-blueprint now PASS Tier 1 (static + security) and Tier 2 (deduplication).
Tier 2 (deduplication):
Tier 1 (security, rag-blueprint):
No skill descriptions were changed. BENCHMARK.md and skill-card.md are left for the Skill Evaluator signing pipeline (/nvskills-ci) to regenerate and sign.
Checklist
git commit -s) and GPG signed (git commit -S).Summary by CodeRabbit