This section provides API reference documentation for NeMo Curator’s core classes and interfaces.
Core Classes
Pipeline
The main orchestrator for executing sequences of processing stages.
ProcessingStage
Base class for all data processing stages in NeMo Curator.
CompositeStage
High-level stages that decompose into multiple execution stages.
Task Types
DocumentBatch
Task type for text document processing.
ImageBatch
Task type for image processing.
VideoTask
Task type for video processing.
AudioTask
Task type for audio processing.
Executors
XennaExecutor
Production executor using Cosmos-Xenna for distributed execution.
Experimental Executors
Ray-based experimental executors.
Configuration
Source Code
For complete implementation details, see the NeMo Curator source code on GitHub.