NVIDIA NeMo Curator 1.3.0

Changelog Details

• ci: Package version bump to 1.2.0 by @thomasdhc :: PR: #1784
• Add SlurmRayClient for multi-node Ray cluster management on SLURM by @abhinavg4 :: PR: #1712
• ci: Fix gpu coverage and add gpu test check by @thomasdhc :: PR: #1790
• Build: drop rc0 pre-release tag and add dynamic git versioning by @thomasdhc :: PR: #1788
• ci: Gpu test matrix ci_worklod check by @thomasdhc :: PR: #1793
• ci: Fix build-test-publish-wheel by maintaing previous workflow by @thomasdhc :: PR: #1799
• fix: give nemotron-cc stages distinct names by @nightcityblade :: PR: #1763
• fix: preserve utf-8 characters in JSONL output by @nightcityblade :: PR: #1762
• Add CPU requirement to Llama Nemotron tutorial by @sarahyurick :: PR: #1773
• build(deps-dev): bump pytest from 9.0.2 to 9.0.3 by @dependabot[bot] :: PR: #1805
• build(deps): bump lxml-html-clean from 0.4.3 to 0.4.4 by @dependabot[bot] :: PR: #1564
• build(deps): bump pillow from 12.1.1 to 12.2.0 by @dependabot[bot] :: PR: #1803
• Add GLiNER instruction by @sarahyurick :: PR: #1772
• Add 60 second timeout to ArXiv check by @sarahyurick :: PR: #1776
• ci: add sync-skills workflow by @ko3n1g :: PR: #1806
• [ci] fix: retry apt-get installs to handle mirror sync failures by @ko3n1g :: PR: #1814
• refactor: extract Ray Serve backend boundary - inference (1/5) by @praateekmahajan :: PR: #1813
• [benchmarking] Adds gpustat data for processes running on each GPU by @rlratzel :: PR: #1823
• [benchmarking] Increases timeout for math_preprocess_llm_cleanup benchmark by @rlratzel :: PR: #1819
• [benchmarking] Updates timeouts based on analysis of prior CI runs by @rlratzel :: PR: #1831
• feat!: Typed Serve Config + Dyamo config stub - inference (2/5) by @praateekmahajan :: PR: #1820
• fix(image): addressing nvbug6075086 (when --images-per-tar exceed --batch-size) by @huvunvidia :: PR: #1828
• docs: add nemo-curator-docs Claude Code skill by @lbliii :: PR: #1825
• Add qwen3 by @oyilmaz-nvidia :: PR: #1827
• Update NDD (nemo data-designer) version by @huvunvidia :: PR: #1755
• Enhance VideoReader initialization with path validation by @suiyoubi :: PR: #1845
• More CVE fixes by @ayushdg :: PR: #1843
• [Tutorial] Merge multiple file prefixes generated by MegatronTokenizerWriter by @asolergi-nv :: PR: #1427
• docs: add OTEL_SDK_DISABLED workaround for NVBUG 6078008 SIGSEGV by @shubhamNvidia :: PR: #1842
• Improve math tutorial README documentation by @lbliii :: PR: #1766
• docs: cross-link Nemotron-CC recipe as a production NeMo Curator example by @arhamm1 :: PR: #1767
• Fix waveform tensor leak in audio pipeline and redesign AudioDataFilterStage for all VAD/Speaker combinations by @shubhamNvidia :: PR: #1765
• ci: add base_sha to codecov/codecov-action upload step by @ko3n1g :: PR: #1864
• Fix Sortformer tutorial issues and add InferenceSortformerStage benchmark by @melllinia :: PR: #1764
• feat: Placement Group based subprocess manager - inference (3/5) by @praateekmahajan :: PR: #1833
• feat: (experimental) Dynamo aggregated backend - inference (4/5) by @praateekmahajan :: PR: #1834
• [benchmarking] Adds support for fine-grained config overrides and global ray config by @rlratzel :: PR: #1840
• Tutorials: Add interleaved getting-started tutorial by @VibhuJawa :: PR: #1774
• Add ReadSpeech audio curation benchmark by @shubhamNvidia :: PR: #1851
• Readspeech tutorial: README, tests, and lower audio data filter GPU defaults by @shubhamNvidia :: PR: #1841
• feat(tutorials/readspeech): add interactive Jupyter notebook tutorial by @shubhamNvidia :: PR: #1870
• feat: (experimental) Dynamo Disaggregated serving + validators + router polish - inference (5/5) by @praateekmahajan :: PR: #1861
• Add generic audio processing pipeline for ASR and TTS data preparation by @sushmitha-deva-09 :: PR: #1679
• 26.04 staging by @lbliii :: PR: #1655
• Regenerate uv.lock from prev version from main by @ayushdg :: PR: #1875
• Bump python requirements to 3.11 min and 3.13 max by @ayushdg :: PR: #1855
• Add Nemotron Omni Model for Video caption pipeline by @suiyoubi :: PR: #1883
• fix: Support aarch64 torch install by @thomasdhc :: PR: #1891
• benchmark: add ndd_dynamo_dp4 entry; cache weights to drop ray-serve startup time by @praateekmahajan :: PR: #1896
• Fix checkpoint crash when ManifestWriterStage returns FileGroupTask by @melllinia :: PR: #1884
• [benchmarking] Adds paths config for specifying Docker container mounts by @rlratzel :: PR: #1879
• Bump Ray 2.55.1 / CUDA 12.9.1 / torch 2.10 / vllm 0.18; HAProxy ingress by @praateekmahajan :: PR: #1895
• docs: move install into Get Started, refresh Setup & Deployment index by @lbliii :: PR: #1878
• docs(26.04): close release-note gaps and add missing feature pages by @lbliii :: PR: #1893
• Lazy import for text module by @oyilmaz-nvidia :: PR: #1888
• Lazy init for openai client by @oyilmaz-nvidia :: PR: #1890
• [benchmarking] Adds --use-host-curator-benchmarking and --skip-curator-image-build* options by @rlratzel :: PR: #1910
• ci: Update container registry by @thomasdhc :: PR: #1934
• Add translation by @rkalaniNV :: PR: #1844
• ci: Update runner name by @thomasdhc :: PR: #1958
• Add fit_data_fraction and cache_path to KMeansStage by @sarahyurick :: PR: #1927
• build(deps): bump notebook from 7.5.1 to 7.5.6 by @dependabot[bot] :: PR: #1944
• build(deps-dev): bump gitpython from 3.1.45 to 3.1.50 by @dependabot[bot] :: PR: #1962
• docs: fix broken links — redirects, GitHub source links, drift by @lbliii :: PR: #1938
• docs: add rolling main version, remove translation from v26.04 by @lbliii :: PR: #1969
• docs(26.04): port SME video encoder docs (VP9 + BYO H.264) by @lbliii :: PR: #1926
• Don't show uv sync progress when installing extras to clean up CI logs by @ayushdg :: PR: #1971
• build(deps): bump lxml from 6.0.2 to 6.1.0 by @dependabot[bot] :: PR: #1848
• build(deps): bump mistune from 3.1.4 to 3.2.1 by @dependabot[bot] :: PR: #1939
• build(deps): bump jupyter-server from 2.17.0 to 2.18.0 by @dependabot[bot] :: PR: #1928
• build(deps): bump mako from 1.3.10 to 1.3.12 by @dependabot[bot] :: PR: #1941
• build(deps): bump python-multipart from 0.0.22 to 0.0.27 by @dependabot[bot] :: PR: #1942
• chore: Update dynamo to 1.1.0 by @praateekmahajan :: PR: #1957
• fix(filters): escape - in regex_url character class (#1601) by @SAY-5 :: PR: #1880
• build(deps): bump urllib3 from 2.6.3 to 2.7.0 by @dependabot[bot] :: PR: #1970
• build(deps): batch Dependabot lockfile updates by @ayushdg :: PR: #1974
• Add Skill to Nemo Curator for "Getting Started" by @chris-alexiuk :: PR: #1746
• Package translation prompt yamls by @rkalaniNV :: PR: #1972
• ci: Major refactor of release-workflows by @ko3n1g :: PR: #1911
• Nemotron-CLIMB data curation recipe by @sarahyurick :: PR: #1727
• docs(fern): switch to NVIDIA global theme as source of truth by @lbliii :: PR: #2000
• build(deps): bump idna from 3.11 to 3.15 by @dependabot[bot] :: PR: #2001
• Add filtration and PnCWithLLM stages for audio tagging pipeline by @sushmitha-deva-09 :: PR: #1863
• ci: validate release branch-rules by @ko3n1g :: PR: #2009
• fix(docs): correct version label to v1.2.0 by @lbliii :: PR: #1983
• [benchmarking] Migrate test-paths.yaml to paths section, drop duplicates by @rlratzel :: PR: #2026
• [benchmarking] Add configurable GPU memory warning threshold by @rlratzel :: PR: #1966
• [benchmarking] Add ping_users_on_failure toggle, disable in nightly config by @rlratzel :: PR: #2039
• Add ray data option for video benchmarks by @oyilmaz-nvidia :: PR: #2002
• ci: Remove sync skills workflow by @chtruong814 :: PR: #2041
• ci: disable uv cache for cpu tests by @thomasdhc :: PR: #2043
• Add long-form audio cutting pipeline for ALM pretraining by @yqwangustc :: PR: #1898
• test: add image reader sample data by @nightcityblade :: PR: #2032
• fix(utils): include port 65535 in get_free_port() scan range by @miclaldogan :: PR: #2024
• docs: rewrite README and expand CONTRIBUTING per PM review and OSS audit by @lbliii :: PR: #2030
• docs: show how to resume Common Crawl downloads by @omribz156 :: PR: #2029
• docs: fix broken links surfaced in 2026-05-14 scan by @lbliii :: PR: #1991
• ci: bump _release_library.yml to v1.4.3 by @ko3n1g :: PR: #2046
• Expose translation control knobs by @rkalaniNV :: PR: #2038
• docs: retire docs/ Sphinx tree, replace with pointer to fern/ by @lbliii :: PR: #1961
• fix: gate nemo_text_processing to x86_64 to unblock aarch64 build by @thomasdhc :: PR: #2049
• build(deps): bump aiohttp from 3.13.4 to 3.14.0 by @dependabot[bot] :: PR: #2047
• [benchmarking] Multiple fixes to stabilize the nightly benchmark suite by @rlratzel :: PR: #2035
• Unify task identity: collapse _uuid into deterministic task_id by @abhinavg4 :: PR: #2036
• build(deps): bump starlette from 0.50.0 to 1.0.1 by @dependabot[bot] :: PR: #2048
• Strip user-set task_id from tutorials & getting-started script by @abhinavg4 :: PR: #2058
• fix(audio): download FLEURS via hf_hub_download to avoid HTTP 429 by @mohammadaaftabv :: PR: #2057
• Refactor empty/sentinel tasks: EmptyTask class + SentinelTask base by @abhinavg4 :: PR: #2062
• feat(benchmarking): config-driven CI timeouts (default/cleanup/min) by @thomasdhc :: PR: #2068
• fix(audio): lazy-load audio stages and fix tutorial notebook paths by @mohammadaaftabv :: PR: #1835
• Fix EmptyTask references in benchmark and tutorials by @rlratzel :: PR: #2070
• Nemotron OCR SDG Pipeline by @suiyoubi :: PR: #1899
• caption quality evaluation by @weijiac0619 :: PR: #1980
• Minor updates to CLIMB tutorial by @sarahyurick :: PR: #2072
• docs: add Related Topics section to Quality Assessment & Filtering by @chenopis :: PR: #2071
• fix: exclude eval/ from check-manifest sdist validation by @thomasdhc :: PR: #2081
• Update Ray Data Backend to use compute strategy + Allow min/max/initial workers to be specified by @praateekmahajan :: PR: #2077
• fix: extract arxiv archives outside download dir by @nightcityblade :: PR: #20...

NVIDIA NeMo Curator 1.2.0

NeMo Curator 26.04 (v1.2.0)

⚠️ Python 3.10 support ends in 26.06. This is the last release to support Python 3.10 — upgrade environments to 3.11+ before the next release.

Highlights

• vLLM & Sentence Transformers embeddings — new VLLMEmbeddingModelStage and SentenceTransformerEmbeddingModelStage; EmbeddingCreatorStage gains use_sentence_transformer and cache_dir.
• Inference Server (Ray Serve) — new InferenceServer / InferenceModelConfig to serve OpenAI-compatible LLMs inside a Ray cluster; new inference_server and sdg_cuda12 extras.
• RayDataExecutor promoted out of experimental → nemo_curator.backends.ray_data.
• Semantic dedup defaults to vLLM with google/embeddinggemma-300m (was SentenceTransformers + all-MiniLM-L6-v2).
• Per-stage runtime environments — declare runtime_env on a ProcessingStage for isolated Python deps per stage.
• Cosmos-Xenna 0.2.0 — simplified Resources API (gpu_memory_gb or gpus; nvdecs/nvencs/entire_gpu removed); Ray ≥ 2.54.
• Multi-node Ray on SLURM — drop-in SlurmRayClient + reference tutorials (container & bare-metal).
• NeMo Data Designer integration — new DataDesignerStage + NDD-backed Nemotron-CC stages.
• Megatron tokenization writer — produce Megatron-LM .bin/.idx directly from a Curator pipeline.
• Audio overhaul — AudioBatch → AudioTask; new VAD / Band / SIGMOS / UTMOS / Speaker-Separation stages, AudioDataFilterStage composite, streaming Sortformer diarization, ALM pipeline, DNS Challenge ReadSpeech tutorial.
• Video — Nemotron Nano 12B V2 VLM captioning (bf16/fp8/nvfp4); fused DocumentIterateExtractStage for 3-stage acquisition pipelines.
• Interleaved IO — InterleavedParquetReader + InterleavedWebdatasetWriter close the WDS ⇄ Parquet round-trip; four new filters (blur, QR-code, CLIP-score, image/text ratio).
• PDF pipeline — four-stage Nemotron-Parse Xenna pipeline (pypdfium2 dep added).
• CommonCrawl S3 transport — opt-in via use_s3=True / CC_USE_S3.
• Workflow results API — all dedup workflows now return WorkflowRunResult with structured per-stage metadata.
• Multi-user metrics isolation — per-UID metrics dirs, PID-based tracking, auto Ray dashboards.

Security

• nemo-toolkit RCE (CVE-2025-33245, CVE-2025-33253) — bumped to >=2.7.2.
• xgrammar DoS (CVE-2026-25048) — override to >=0.1.32.
• jackson-core DoS (GHSA-72hv-8253-57qq) — Ray's bundled ray_dist.jar removed from the container image.

Breaking Changes

• Minimum Ray 2.54 (was 2.50).
• TextSemanticDeduplicationWorkflow: default backend is now vLLM; default model is google/embeddinggemma-300m; removed embedding_model_inference_batch_size, embedding_pooling, embedding_padding_side, embedding_max_seq_length — use embedding_vllm_init_kwargs.
• Resources: removed nvdecs, nvencs, entire_gpu — use gpus or gpu_memory_gb.
• AudioBatch removed → use AudioTask (single dict, not list[dict]).
• RayDataExecutor moved: nemo_curator.backends.experimental.ray_data → nemo_curator.backends.ray_data.
• DocumentExtractStage removed; DocumentIterateStage replaced by DocumentIterateExtractStage. Data acquisition is now 3 stages (URL gen → download → iterate-extract).
• Dedup workflow run() returns: ExactDeduplicationWorkflow, FuzzyDeduplicationWorkflow (was None), SemanticDeduplicationWorkflow, TextSemanticDeduplicationWorkflow (was dict), TextDuplicatesRemovalWorkflow (was list[FileGroupTask] | None) — all now return WorkflowRunResult.

Deprecations

• Python 3.10 — last supported in 26.04; removed in 26.06.

Full Release Notes

docs.nvidia.com/nemo/curator/v26.04/about/release-notes

NVIDIA NeMo Curator 1.1.0

New Features

• Stage and Pipeline Benchmarking: Benchmarking for all modalities (text, image, video, audio)
• YAML Configuration: Declarative pipeline configuration with pre-built configs for code filtering, deduplication, heuristic filtering, and FastText
• Pipeline Performance and Metric Logging: Automatic tracking of processing time, throughput, and resource usage; detailed logs and error reporting for failed stages

Improvements

• Video: Removed InternVideo2; vLLM 0.15.1, FFmpeg 8.0.1
• Audio: Enhanced ASR/WER docs, robust manifest handling
• Image: Optimized batch sizes (batch_size=100, num_threads=16), memory guidance
• Text: Better memory management for large-scale semantic deduplication
• Deduplication: Cloud storage (S3, GCS, Azure) for ParquetReader/Writer, non-blocking ID generation, empty batch handling

Dependency Updates

• Transformers 4.55.2, vLLM 0.15.1, FFmpeg 8.0.1
• Security patches: aiohttp, urllib3, python-multipart, setuptools

Bug Fixes

• FastText numpy>2 compatibility, NeMo doc links, ID generator blocking, vLLM video API, Gliner/SDG tutorials, semantic dedup test reliability

Infrastructure

• Secrets detection, Dependabot, enhanced install tests, AWS runner support, Docker/uv optimization, Cursor rules

Breaking Changes

• InternVideo2 Removed: Use Cosmos-Embed1 for video embeddings

Documentation

• Heuristic filter guide, distributed classifier memory guidance, installation troubleshooting, memory management, AWS credentials

NVIDIA NeMo Curator 1.0.0

This major release represents a fundamental architecture shift from Dask to Ray, expanding NeMo Curator to support multimodal data curation with new video and audio capabilities. This refactor enables unified backend processing, better heterogeneous computing support, and enhanced autoscaling for dynamic workloads.

Installation Updates

• New Docker container: Updated Docker infrastructure with CUDA 12.8.1 and Ubuntu 24.04 base; obtainable through the NGC Catalog (nvcr.io/nvidia/nemo-curator:25.09)

• Docker file to build own image: Simplified Dockerfile structure for custom container builds with FFmpeg support

• UV source installations: Integrated UV package manager (v0.8.22) for faster dependency management

• PyPI improvements: Enhanced PyPI installation with modular extras for targeted functionality:

  Extra	Installation Command	Description
  All Modalities	nemo-curator[all]	Complete installation with all modalities and GPU support
  Text Curation	nemo-curator[text_cuda12]	GPU-accelerated text processing with RAPIDS
  Image Curation	nemo-curator[image_cuda12]	Image processing with NVIDIA DALI
  Audio Curation	nemo-curator[audio_cuda12]	Speech recognition with NeMo ASR models
  Video Curation	nemo-curator[video_cuda12]	Video processing with GPU acceleration
  Basic GPU	nemo-curator[cuda12]	CUDA utilities without modality-specific dependencies

  All GPU installations require the NVIDIA PyPI index:

  uv pip install --extra-index-url https://pypi.nvidia.com nemo-curator[EXTRA]

New Modalities

Video

NeMo Curator now supports comprehensive video data curation with distributed processing capabilities:

• Video splitting: Fixed-stride and scene-change detection (TransNetV2) for clip extraction
• Semantic deduplication: K-means clustering and pairwise similarity for near-duplicate clip removal
• Content filtering: Motion-based filtering and aesthetic filtering for quality improvement
• Embedding generation: InternVideo2 and Cosmos-Embed1 models for clip-level embeddings
• Ray-based distributed architecture: Scalable video processing with autoscaling support

Audio

New audio curation capabilities for speech data processing:

• ASR inference: Automatic speech recognition using NeMo Framework pretrained models
• Quality assessment: Word Error Rate (WER) and Character Error Rate (CER) calculation
• Speech metrics: Duration analysis and speech rate metrics (words/characters per second)
• Text integration: Seamless integration with text curation workflows via AudioToDocumentStage
• Manifest support: JSONL manifest format for audio file management

Modality Refactors

Text

• Ray backend migration: Complete transition from Dask to Ray for distributed text processing
• Improved model-based classifier throughput: Better overlapping of compute between tokenization and inference through length-based sequence sorting for optimal GPU memory utilization
• Task-centric architecture: New Task-based processing model for finer-grained control
• Pipeline redesign: Updated ProcessingStage and Pipeline architecture with resource specification

Image

• Pipeline-based architecture: Transitioned from legacy ImageTextPairDataset to modern stage-based processing with ImageReaderStage, ImageEmbeddingStage, and filter stages
• DALI-based image loading: New ImageReaderStage uses NVIDIA DALI for high-performance WebDataset tar shard processing with GPU/CPU fallback
• Modular processing stages: Separate stages for embedding generation, aesthetic filtering, and NSFW filtering
• Task-based data flow: Images processed as ImageBatch tasks containing ImageObject instances with metadata, embeddings, and classification scores

Learn more about image curation.

Deduplication Improvements

Enhanced deduplication capabilities across all modalities with improved performance and flexibility:

• Exact and Fuzzy deduplication: Updated rapidsmpf-based shuffle backend for more efficient GPU-to-GPU data transfer and better spilling capabilities
• Semantic deduplication: Support for deduplicating text, image, and video datasets using unified embedding-based workflows
• New ranking strategies: Added RankingStrategy which allows you to rank elements within cluster centers to decide which point to prioritize during duplicate removal, supporting metadata-based ranking to prioritize specific datasets or inputs

Core Refactors

The architecture refactor introduces a layered system with unified interfaces and multiple execution backends:

User Layer: Pipeline → ProcessingStage X→Y → ProcessingStage Y→Z → ProcessingStage Z→W
           ↓
Orchestration Layer: BaseExecutor Interface
           ↓
Backend Layer: XennaExecutor (Production Ready) | RayActorPoolExecutor (Experimental) | RayDataExecutor (Experimental)
           ↓
Adaptation Layer: Xenna Adapter | Ray Actor Adapter | Ray Data Adapter
           ↓
Execution Layer: Cosmos-Xenna (Streaming/Batch) | Ray Actor Pool (Load Balancing) | Ray Data API (Dataset Processing)

Pipelines

• New Pipeline API: Ray-based pipeline execution with BaseExecutor interface
• Multiple backends: Support for Xenna, Ray Actor Pool, and Ray Data execution backends
• Resource specification: Configurable CPU and GPU memory requirements per stage
• Stage composition: Improved stage validation and execution orchestration

Stages

• ProcessingStage redesign: Generic ProcessingStage[X, Y] base class with type safety
• Resource requirements: Built-in resource specification for CPU and GPU memory
• Backend adapters: Stage adaptation layer for different Ray orchestration systems
• Input/output validation: Enhanced type checking and data validation

Tutorials

• Text tutorials: Updated all text curation tutorials to use new Ray-based API
• Image tutorials: Migrated image processing tutorials to unified backend
• Audio tutorials: New audio curation tutorials
• Video tutorials: New video processing tutorials

For all tutorial content, refer to the tutorials directory in the NeMo Curator GitHub repository.

Known Limitations

(Pending Refactor in Future Release)

Generation

• Synthetic data generation: Synthetic text generation features are being refactored for Ray compatibility
• Hard negative mining: Retrieval-based data generation workflows under development

PII

• PII processing: Personal Identifiable Information removal tools are being updated for Ray backend
• Privacy workflows: Enhanced privacy-preserving data curation capabilities in development

Blending & Shuffling

• Data blending: Multi-source dataset blending functionality being refactored
• Dataset shuffling: Large-scale data shuffling operations under development

Docs Refactor

• Local preview capability: Improved documentation build system with local preview support
• **...

NVIDIA NeMo Curator 0.9.0

Major Features and Enhancements

• New How-to Data Recipes (Tutorials)
  • Multimodal DAPT Curation w/ PDF Extraction
  • Llama Nemotron Data Curation
  • LLM NIM - PII Redaction
• Performance and Code Optimizations
  • Simplified Clustering Logic: Significantly improved semantic deduplication clustering performance
  • Removed convoluted backend switching logic that caused performance issues
  • Eliminated expensive length assertions that could cause timeouts on large datasets
  • Improved GPU utilization during KMeans clustering operations
  • Tested on 37M embedding dataset (80GB) across 7 GPUs with substantial performance gains

Bug Fixes

• FastText Download URL Fix
  • Corrected the fasttext model download URL in nemotron-cc tutorial
  • Changed from dl.fbaipublicfiles.com/fastText/ to dl.fbaipublicfiles.com/fasttext/
  • Ensures reliable model downloads for language identification
• NeMo Retriever Tutorial Bug Fix
  • Fixed lambda function bug in RetrieverEvalSetGenerator
  • Corrected score assignment from df["question"].apply(lambda: 1) to df["score"] = 1
• API Usage Updates
  • Updated examples and tutorials to use correct DocumentDataset API
  • Replaced deprecated write_to_disk(result, output_dir, output_type="parquet") with result.to_parquet(output_dir)
  • Updated exact deduplication workflows: deduplicator.remove() now returns DocumentDataset directly

NVIDIA NeMo Curator 0.8.0

• Llama Based PII Redaction
• Trafilatura Text Extractor
• Chinese & Japanese Stopwords for Text Extractors
• Writing gzip compressed jsonl datasets
• Training dataset curation for retriever customization using hard-negative mining
• Implemented a memory efficient pairwise similarity in Semantic Deduplication

NVIDIA NeMo Curator 0.8.0rc3.dev0

Prerelease: NVIDIA NeMo Curator 0.8.0rc3.dev0 (2025-04-15)

NVIDIA NeMo Curator 0.8.0rc2.dev0

Prerelease: NVIDIA NeMo Curator 0.8.0rc2.dev0 (2025-04-07)

NVIDIA NeMo Curator 0.7.1

• Fix Transformers + Cuda Context bug
• Fix rate limit in SDG Retriever Eval Tutorial

NVIDIA NeMo Curator 0.7.0

• Python 3.12 Support
• Curator on Blackwell
• Nemotron-CC Dataset Recipe
• Performant S3 for Fuzzy Deduplication