Neo4j Connector for Unity Catalog

Enables Databricks and Neo4j customers to maintain governance over their Neo4j graph data from Unity Catalog and federated queries across both data platforms.

Key Benefits

• Unified analytical plane - one consistent interface for analysts across graph and relational data.
• No more loading bulk data into the graph - keep fault data, weather data, and time-series in the lakehouse; query it alongside graph data seamlessly.
• SQL-only access to graph data - Analysts query everything with standard SQL; Cypher translation happens automatically.
• Federated joins across graph + lakehouse - Unity Catalog distributes queries to both sources and joins the results.
• Materialized views as a graph cache - Schedule refreshes to avoid hitting the graph directly on every query.
• Genie natural language queries - Materialized graph views plug into Databricks Genie for natural language access across all data.

What Is It?

A single shaded (fat) JAR that bundles the Neo4j JDBC driver, the SQL-to-Cypher translator, and the Spark subquery cleaner for use with Databricks Unity Catalog federated queries.

This single JAR gives users one artifact to upload to a UC Volume and one path to reference in their connection configuration.

For examples of how to set up and use this connector with Databricks Unity Catalog, see neo4j-uc-connector-demos.

Databricks Compatibility

This connector loads through a Unity Catalog connection of TYPE JDBC, the "bring your own driver" path. That path rolled out by compute type:

Compute	Requirement
Standard / dedicated clusters	Databricks Runtime 17.3 LTS or above
Serverless	Supported
SQL warehouses	Pro or serverless, Databricks SQL 2025.35 or above, plus the "Enable networking for isolated workloads in Serverless SQL Warehouses" preview

The feature as a whole reached Public Preview at Databricks Runtime 18.1 and Databricks SQL 2025.40. On earlier supported runtimes, enable the Custom JDBC on UC Compute and remote_query preview flags in the workspace.

What the JDBC path gives you and what it does not:

• Governance is connection-level only. Foreign catalogs are not supported with JDBC connections, so there is no browsable three-level namespace and no per-table grants.
• Authentication is Static Credential (username and password in a Databricks secret) or OAuth Machine-to-Machine, which is in Beta. Unity Catalog credentials and service credentials are not supported.
• Filters push down to Neo4j by default for both the Spark Data Source API and remote_query.
• The bundled translator covers WHERE, ORDER BY, LIMIT/OFFSET, DISTINCT, INNER/NATURAL JOIN, GROUP BY, HAVING, and the aggregates COUNT, COUNT DISTINCT, SUM, AVG, MIN, MAX, percentileCont, percentileDisc, and standard deviation. Outer joins, window functions, set operations, and relationship-property aggregation are not supported; use the /*+ NEO4J FORCE_CYPHER */ hint for those.

Prerequisites

• Java 17+

Build

./mvnw clean verify

The shaded JAR is produced at:

target/neo4j-unity-catalog-connector-0.0.0-SNAPSHOT.jar

Sync Neo4j JDBC

Use the uv script to sync neo4j-jdbc.version in pom.xml to the latest released org.neo4j:neo4j-jdbc-bom version from Maven Central and run the build:

uv run scripts/sync_neo4j_jdbc.py

To check what would be used without changing files:

uv run scripts/sync_neo4j_jdbc.py --dry-run

The script prompts before updating a locally modified pom.xml. After a successful build, it can also commit the pom.xml change with a message like Sync Neo4j JDBC to 6.12.2 and push the commit.

Run Tests

Tests verify that the bundled translators are discoverable via SPI, the Spark subquery cleaner handles Databricks/Spark query patterns, and the JDBC driver class is loadable.

./mvnw test

Release

The release GitHub Actions workflow publishes a GitHub Release with the built JAR when you push a tag. The version in the JAR filename is derived from the tag name automatically.

Create an annotated tag with a release message and push it:

git tag -a 1.1.0 -m "Release 1.1.0: description of changes"
git push origin 1.1.0

You can also build a specific version locally:

./mvnw clean verify -Drevision=1.1.0

What's Inside

The shaded JAR bundles:

Dependency	Purpose
neo4j-jdbc	Core JDBC driver for Neo4j
neo4j-jdbc-translator-impl	SQL-to-Cypher translation engine
neo4j-jdbc-translator-sparkcleaner	Cleans Spark subquery wrapping (SPARK_GEN_SUBQ_0 WHERE 1=0)

All transitive dependencies (Jackson, Netty, jOOQ, Bolt protocol, Cypher DSL, Reactive Streams) are relocated under org.neo4j.jdbc.internal.shaded.* to avoid classpath conflicts with the Databricks runtime.

Design

Problem

Connecting Neo4j to Databricks Unity Catalog previously required users to download and upload two separate JARs to a Unity Catalog Volume:

1. neo4j-jdbc-full-bundle-6.x.x.jar: the main JDBC driver with SQL-to-Cypher translation
2. neo4j-jdbc-translator-sparkcleaner-6.x.x.jar: handles Spark's subquery wrapping (SPARK_GEN_SUBQ_0 WHERE 1=0)

Both had to be referenced individually in the java_dependencies array when creating a UC JDBC connection. This meant two manual downloads from Maven Central, two uploads to a Volume, two paths to manage, and two version numbers to keep in sync. If a user forgot the sparkcleaner JAR or used mismatched versions, the connection silently broke with confusing errors.

Solution

This project uses maven-shade-plugin to merge all three dependencies (neo4j-jdbc, neo4j-jdbc-translator-impl, neo4j-jdbc-translator-sparkcleaner) into a single self-contained JAR. Users upload one file to a UC Volume and reference one path, with the driver, SQL translator, and Spark cleaner versioned together.

SPI Service Registration

Java's Service Provider Interface (SPI) is a plugin mechanism built into the JDK. A library declares an interface (in this case, TranslatorFactory) and other JARs provide implementations of that interface. At runtime, java.util.ServiceLoader discovers these implementations automatically by reading text files under META-INF/services/ inside the JAR. Each file is named after the interface and lists the fully qualified class names of the implementations.

The Neo4j JDBC driver uses SPI to discover SQL translators. When the driver starts, it calls ServiceLoader.load(TranslatorFactory.class) which scans the classpath for META-INF/services/org.neo4j.jdbc.translator.spi.TranslatorFactory files. Any translator factory listed in those files gets loaded and used in the translation pipeline.

This project bundles two translator JARs that each provide their own SPI registration:

• neo4j-jdbc-translator-impl registers SqlToCypherTranslatorFactory: converts SQL to Cypher
• neo4j-jdbc-translator-sparkcleaner registers SparkSubqueryCleaningTranslatorFactory: strips Spark's SPARK_GEN_SUBQ_0 WHERE 1=0 wrapping before translation

When the maven-shade-plugin merges these JARs into one, it uses ServicesResourceTransformer to concatenate the separate SPI files into a single merged file. Without this transformer, one file would overwrite the other and only one translator would be discovered at runtime.

If a custom DatabricksTranslator is needed in the future to handle Databricks-specific SQL patterns beyond what the Spark cleaner covers, it can be added by implementing TranslatorFactory and registering it via the same SPI mechanism.

User-Agent Identification

The project includes a META-INF/neo4j-jdbc-user-agent.txt file containing:

neo4j-unity-catalog-connector/${project.version}

This string is sent by the Neo4j JDBC driver to the Neo4j server with every connection. The ${project.version} placeholder is substituted by Maven at build time (via <filtering>true</filtering> in the pom.xml). This lets Neo4j (especially Aura) distinguish connections coming from the Databricks UC connector vs the plain JDBC driver, which supports support workflows, usage analytics, and debugging.

Package Relocation

All bundled dependencies are relocated under org.neo4j.jdbc.internal.shaded.* to avoid classpath conflicts with whatever JARs are already on the Databricks SafeSpark sandbox classpath.