Overview

Relevant source files

• docs/additional-functionality/rapids-udfs.md
• docs/archive.md
• docs/configs.md
• docs/dev/testing.md
• docs/download.md
• docs/img/AWS-EMR/spark-history-server-UI.png
• docs/img/Databricks/sparkconfig.png
• docs/supported_ops.md
• integration_tests/README.md
• integration_tests/src/main/python/data_gen.py
• integration_tests/src/main/python/hash_aggregate_test.py
• integration_tests/src/main/python/window_function_test.py
• jenkins/printJarVersion.sh
• sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
• sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala
• tests/src/test/scala/com/nvidia/spark/rapids/WindowFunctionSuite.scala

The RAPIDS Accelerator for Apache Spark is a plugin that transparently intercepts Apache Spark SQL queries and executes them on NVIDIA GPUs using the cuDF library. The plugin integrates into Spark's Catalyst optimizer through the org.apache.spark.sql.SparkSessionExtensions mechanism, replacing CPU-based physical plan operators with GPU-accelerated equivalents when possible.

This page provides a high-level introduction to the accelerator's architecture, capabilities, and integration points. For detailed information about specific subsystems, see:

• Plugin integration mechanism: Plugin Architecture
• Configuration parameters and control: Configuration System
• Core transformation components: Key System Components
• Detailed operation support: Core Architecture

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala1-100 docs/download.md1-112

Purpose and Scope

The accelerator transforms Spark SQL operations to run on GPUs by:

1. Intercepting physical plans after Spark's optimization phase via the columnarRules extension point
2. Validating type compatibility through the TypeSig and TypeChecks systems
3. Replacing operators when both type signatures and configuration permit GPU execution
4. Managing CPU/GPU transitions through operators like HostColumnarToGpu and GpuColumnarToRowExec
5. Executing on GPU using the cuDF library for columnar data processing

The plugin does not modify Spark's logical plan or optimizer. It operates exclusively on physical plans after standard Spark optimizations complete.

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala461-750 docs/supported_ops.md1-70

High-Level Architecture

Figure 1: Plugin Integration and Transformation Flow

The diagram shows how user queries flow through Spark's standard pipeline, then get intercepted by the RAPIDS plugin via SQLPlugin.columnar(). The GpuOverrides class orchestrates transformation by wrapping plan nodes in RapidsMeta, validating them with TypeChecks, and converting compatible nodes to GPU operators that execute via cuDF.

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala461-567 sql-plugin/src/main/scala/com/nvidia/spark/rapids/TypeChecks.scala1-60

Core Transformation Process

Figure 2: Plan Transformation Components

The transformation uses a three-phase approach:

1. Wrapping Phase: GpuOverrides.wrapAndTagPlan() traverses the Spark plan and wraps each node in a RapidsMeta subclass (SparkPlanMeta, BaseExprMeta, ScanMeta, etc.) based on registered ReplacementRule instances
2. Tagging Phase: Each RapidsMeta calls tagForGpu() to determine GPU compatibility using TypeChecks validation and RapidsConf settings
3. Conversion Phase: convertIfNeeded() generates GPU operators for tagged nodes or inserts CPU/GPU transition operators where needed

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala461-523 sql-plugin/src/main/scala/com/nvidia/spark/rapids/TypeChecks.scala120-260

Key Components

Component	Class/Package	Purpose
Plugin Entry	com.nvidia.spark.SQLPlugin	Implements SparkPlugin interface to register columnar rules
Plan Replacement	GpuOverrides	Orchestrates plan transformation with 854.08 importance score
Metadata Wrappers	RapidsMeta hierarchy	Wraps Spark nodes for GPU compatibility analysis
Type Validation	TypeSig, TypeChecks	Validates data type support for operations
Configuration	RapidsConf	Manages 200+ configuration parameters
GPU Operators	GpuExec subclasses	GPU implementations of Spark physical operators
Transitions	HostColumnarToGpu, GpuColumnarToRowExec	Handle CPU↔GPU data movement
Execution Library	ai.rapids.cudf	NVIDIA cuDF library for GPU DataFrame operations

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala78-318 sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala124-167

Replacement Rule System

The plugin defines GPU implementations through ReplacementRule instances registered in GpuOverrides:

Figure 3: Replacement Rule Type Hierarchy

Each ReplacementRule contains:

• doWrap: Function to create the appropriate RapidsMeta wrapper
• desc: Human-readable description of the operation
• checks: Optional TypeChecks defining supported type signatures
• tag: ClassTag identifying the Spark class to replace
• confKey: Configuration key like spark.rapids.sql.expression.Add

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala78-318 sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala794-861

Supported Operations Overview

The accelerator supports most common SQL operations across multiple categories:

Category	Examples	Importance Score	Supported Types
Type Casting	GpuCast	241.62	Most primitives, decimals, strings (see limitations)
Math Operations	GpuAdd, GpuSubtract, GpuMultiply, GpuUnaryMath	233.37	Numeric types, with ANSI mode support
String Operations	GpuUpper, GpuSubstring, GpuRegExpReplace	320.40	String operations with regex transpilation
Aggregations	GpuSum, GpuAvg, GpuCount, GpuMin, GpuMax	Varies	Numeric types, with overflow detection
Joins	GpuBroadcastHashJoinExec, GpuShuffledHashJoinExec	190.89	All join types, AST condition support
File I/O	GpuParquetScan, GpuOrcScan, GpuCsvScan	245.71	Parquet, ORC, CSV, Avro formats
Collections	GpuArrayTransform, GpuMapKeys, GpuGetStructField	213.67	Arrays, maps, structs with lambdas

Compatibility Notes:

• Timestamp operations require UTC timezone (spark.sql.session.timeZone=UTC)
• Decimal precision limited to 128-bit (precision ≤ 38)
• Floating-point aggregations have configuration guards due to non-associative behavior
• Some regex patterns not supported; transpiled to cuDF-compatible format when possible

Sources: docs/supported_ops.md1-110 tools/generated_files/supportedExprs.csv1-100 tools/generated_files/operatorsScore.csv1-50

Configuration Control

The RapidsConf class manages configuration through a builder pattern:

spark.rapids.sql.enabled=true                    # Master enable/disable
spark.rapids.sql.explain=NOT_ON_GPU              # Explain why operations didn't GPU-accelerate
spark.rapids.sql.expression.Add=true             # Per-operation control
spark.rapids.sql.castFloatToIntegral.enabled=true
spark.rapids.memory.gpu.allocFraction=0.9

Common configuration patterns:

• Operation toggles: spark.rapids.sql.{expression|exec|input}.ClassName
• Type safety: spark.rapids.sql.castFloatToIntegral.enabled for potentially unsafe casts
• Memory management: spark.rapids.memory.gpu.* for GPU memory allocation
• Compatibility modes: spark.rapids.sql.incompatibleOps.enabled for operations with behavior differences

See Configuration System for detailed parameter documentation.

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala322-656 docs/configs.md1-57

Type System

The TypeSig class represents sets of supported data types:

TypeSig.BOOLEAN + TypeSig.integral + TypeSig.fp        // Primitives
TypeSig.DECIMAL_128                                     // Decimal(precision ≤ 38)
TypeSig.ARRAY.nested(TypeSig.STRING + TypeSig.INT)    // Array<String|Int>
TypeSig.STRUCT.nested()                                // Struct with any nested types

Type checking occurs in tagForGpu() via TypeChecks objects:

• ExprChecks: Validates expression parameter types
• ExecChecks: Validates executor input/output types
• PartChecks: Validates partitioning key types
• ScanChecks: Validates file format types

When types don't match, willNotWorkOnGpu() marks the operation for CPU fallback.

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/TypeChecks.scala95-260 docs/supported_ops.md73-104

Execution Flow Example

For a simple query like SELECT sum(amount) FROM sales WHERE region = 'WEST':

1. Spark produces a physical plan: HashAggregateExec → FilterExec → FileSourceScanExec
2. SQLPlugin.columnar() calls GpuOverrides.apply()
3. wrapAndTagPlan() creates:
   • HashAggregateExecMeta wrapping the aggregate
   • FilterExecMeta wrapping the filter
   • FileSourceScanExecMeta wrapping the scan
4. Each meta calls tagForGpu():
   • Validates data types against TypeChecks
   • Checks RapidsConf settings
   • Marks incompatible nodes
5. convertIfNeeded() generates:
   • GpuHashAggregateExec for aggregation
   • GpuFilterExec for filtering
   • GpuBatchScanExec for file reading
6. GpuTransitionOverrides inserts any needed HostColumnarToGpu transitions
7. Execution proceeds through GpuExec operators calling cuDF APIs

Sources: sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala528-750

Testing Infrastructure

The accelerator includes comprehensive testing:

Test Type	Location	Purpose
Integration Tests	integration_tests/ (importance 224.38)	Python pytest comparing CPU vs GPU results
Unit Tests	tests/src/test/scala/	Scala tests for individual components
Data Generation	data_gen.py	Deterministic test data with seeds
Assertion Utilities	asserts.py	CPU/GPU result comparison with tolerance
Compatibility Docs	docs/supported_ops.md	Known behavioral differences

Integration tests use markers:

• @approximate_float: Allows floating-point tolerance
• @allow_non_gpu: Permits partial CPU fallback
• @ignore_order: Handles non-deterministic result ordering

Sources: integration_tests/src/main/python/hash_aggregate_test.py1-50 integration_tests/src/main/python/data_gen.py1-100 integration_tests/README.md1-50

Build and Distribution

The plugin uses a multi-version build system supporting Spark 3.2.x through 4.0.x:

1. Shim System: Version-specific code in src/main/spark{330,340,350}/ directories
2. Maven Profiles: Build with -Dbuildver=330 or similar for target Spark version
3. Parallel World JAR: Single JAR containing all Spark version implementations
4. Binary Deduplication: Common classes extracted to spark-shared/ package

Distribution artifacts:

• rapids-4-spark_2.12-VERSION.jar (Scala 2.12)
• rapids-4-spark_2.13-VERSION.jar (Scala 2.13)
• Available from Maven Central: com.nvidia:rapids-4-spark_2.12:VERSION

Sources: docs/download.md1-112 jenkins/printJarVersion.sh1-38

Getting Started

To use the accelerator:

1. Install Requirements:

   • NVIDIA GPU (Volta, Turing, Ampere, Ada Lovelace, Hopper, or Blackwell)
   • CUDA 12.x driver (R525+)
   • Apache Spark 3.2+ or 4.0+

2. Add JAR to Spark:

   spark-submit --jars rapids-4-spark_2.12-VERSION.jar \
                --conf spark.plugins=com.nvidia.spark.SQLPlugin \
                --conf spark.rapids.sql.enabled=true
   

3. Configure for Environment:

   • Set spark.rapids.memory.gpu.allocFraction based on GPU memory
   • Enable operations: spark.rapids.sql.incompatibleOps.enabled=true if needed
   • Review spark.rapids.sql.explain=ALL output for optimization opportunities

4. Monitor Execution:

   • Check Spark UI for GPU stages
   • Use explain() to see physical plans
   • Set spark.rapids.sql.explain=NOT_ON_GPU to identify CPU fallbacks

For detailed configuration, see Configuration System. For operation support details, see Supported Operations Matrix.

Sources: docs/download.md1-112 docs/configs.md1-57