BigQuery JDBC Driver

Modern JDBC driver for Google BigQuery, optimized for development tools and database IDEs. Built from scratch for Java 21+ with JDBC 4.3 compliance and fast, high-quality metadata support.

Features

🎯 Database IDE Optimized

• Fast, comprehensive metadata operations
• Parallel dataset loading with intelligent caching
• Fixes critical JetBrains driver issues (DBE-18711, DBE-12954, DBE-22088, DBE-12749, DBE-19753)
• 30x faster schema introspection for large projects

✨ Modern Java 21+

• Records, sealed classes, pattern matching
• Virtual thread support
• CompletableFuture-based async operations

🔐 Comprehensive Authentication

• Application Default Credentials (ADC)
• Service Account (JSON key)
• User OAuth 2.0
• Workforce Identity Federation
• Workload Identity Federation

📊 BigQuery Sessions

• Temporary tables (CREATE TEMP TABLE)
• Multi-statement SQL scripts
• Transaction support (BEGIN, COMMIT, ROLLBACK)

⚡ Performance

• BigQuery Storage Read API for large result sets
• Configurable result pagination
• Connection pooling compatible
• Query timeout enforcement with automatic cancellation

🎯 Complete Type Support

• All BigQuery primitive types
• Temporal types (TIMESTAMP, DATE, TIME, DATETIME)
• Numeric types (NUMERIC, BIGNUMERIC)
• Complex types (ARRAY, STRUCT, JSON, GEOGRAPHY)

IntelliJ IDEA Integration

🚀 Optimized for Database IDEs and Development Tools

This driver addresses critical limitations in existing BigQuery JDBC drivers for IntelliJ IDEA and other database IDEs, with a focus on fast metadata operations and schema introspection:

✅ Reliable Schema Introspection - Complete DatabaseMetaData implementation (fixes DBE-18711, DBE-12954)

✅ High Performance with Large Projects - Parallel loading + caching for 90+ datasets (fixes DBE-22088)

• JetBrains driver: Hangs or takes 90+ seconds
• tbc-bq-jdbc: 2-3 seconds (30x faster)

✅ Safe STRUCT/ARRAY Handling - JSON representation prevents crashes (fixes DBE-12749)

✅ Robust Authentication - Automatic token refresh for long sessions (fixes DBE-19753)

📖 Complete IntelliJ Setup Guide →

Quick Start for IntelliJ

1. Download Driver JAR

   wget https://repo1.maven.org/maven2/vc/tbc/tbc-bq-jdbc/1.0.61/tbc-bq-jdbc-1.0.61.jar

2. Add Driver in IntelliJ

   • Go to Settings → Database → Drivers
   • Click + to add new driver
   • Name: BigQuery (tbc-bq-jdbc)
   • Driver Files: Select downloaded JAR
   • Class: vc.tbc.bq.jdbc.BQDriver

3. Connect to BigQuery

   jdbc:bigquery:my-project/my_dataset?authType=ADC
   

4. For Large Projects (50+ datasets):

   jdbc:bigquery:my-project?authType=ADC&metadataCacheEnabled=true&metadataCacheTtl=600
   

See IntelliJ Integration Guide for:

• Complete installation instructions
• Performance tuning for large projects
• Comparison with JetBrains driver
• Troubleshooting guide

Quick Start

Installation

Maven

<dependency>
    <groupId>vc.tbc</groupId>
    <artifactId>tbc-bq-jdbc</artifactId>
    <version>1.0.61</version>
</dependency>

Gradle

dependencies {
    implementation 'vc.tbc:tbc-bq-jdbc:1.0.61'
}

Standalone (Fat JAR)

# Download shaded JAR with all dependencies included
wget https://repo1.maven.org/maven2/vc/tbc/tbc-bq-jdbc/1.0.61/tbc-bq-jdbc-1.0.61.jar

Basic Usage

import java.sql.*;

public class Example {
    public static void main(String[] args) throws SQLException {
        // Connect using Application Default Credentials
        String url = "jdbc:bigquery:my-project/my_dataset?authType=ADC";

        try (Connection conn = DriverManager.getConnection(url);
             Statement stmt = conn.createStatement();
             ResultSet rs = stmt.executeQuery("SELECT name, age FROM users LIMIT 10")) {

            while (rs.next()) {
                String name = rs.getString("name");
                int age = rs.getInt("age");
                System.out.printf("%s is %d years old%n", name, age);
            }
        }
    }
}

Prepared Statements

String url = "jdbc:bigquery:my-project/my_dataset?authType=ADC";

try (Connection conn = DriverManager.getConnection(url);
     PreparedStatement pstmt = conn.prepareStatement(
         "SELECT * FROM users WHERE age > ? AND active = ?")) {

    pstmt.setInt(1, 18);
    pstmt.setBoolean(2, true);

    try (ResultSet rs = pstmt.executeQuery()) {
        while (rs.next()) {
            System.out.println(rs.getString("name"));
        }
    }
}

Using Sessions for Transactions

String url = "jdbc:bigquery:my-project/my_dataset?" +
             "authType=ADC&enableSessions=true";

try (Connection conn = DriverManager.getConnection(url)) {
    conn.setAutoCommit(false); // Begin transaction

    try (Statement stmt = conn.createStatement()) {
        // Create temp table
        stmt.execute("CREATE TEMP TABLE temp_data AS SELECT 1 as id");

        // Use temp table
        ResultSet rs = stmt.executeQuery("SELECT * FROM temp_data");

        conn.commit(); // Commit transaction
    } catch (SQLException e) {
        conn.rollback(); // Rollback on error
        throw e;
    }
}

Documentation

📚 Complete Guides:

• Quick Start - Get started in 5 minutes
• Authentication Guide - All authentication methods with examples
• Connection Properties - Complete configuration reference
• Type Mapping - BigQuery ↔ JDBC type conversions
• Compatibility Matrix - JDBC features and limitations
• Integration Tests - Running integration tests

URL Format

Traditional Format

jdbc:bigquery:[project]/[dataset]?property1=value1&property2=value2

Examples:

// Application Default Credentials
"jdbc:bigquery:my-project/my_dataset?authType=ADC"

// Service Account
"jdbc:bigquery:my-project/my_dataset?authType=SERVICE_ACCOUNT&credentials=/path/to/key.json"

// With sessions and location
"jdbc:bigquery:my-project/my_dataset?authType=ADC&enableSessions=true&location=EU"

// With timeout and page size
"jdbc:bigquery:my-project/my_dataset?authType=ADC&timeout=600&pageSize=50000"

Simba BigQuery Driver Compatibility

tbc-bq-jdbc supports Simba BigQuery JDBC driver URL format for easy migration from Simba-based applications. Use the same connection strings without modification:

jdbc:bigquery://[Host]:[Port];ProjectId=[Project];OAuthType=[AuthValue];[Property1]=[Value1];...

Simba Format Examples:

// Application Default Credentials (OAuthType=3)
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=my-project;OAuthType=3"

// Service Account (OAuthType=0)
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=my-project;DefaultDataset=my_dataset;OAuthType=0;OAuthPvtKeyPath=/path/to/key.json"

// User OAuth (OAuthType=1)
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=my-project;OAuthType=1;OAuthClientId=id;OAuthClientSecret=secret;OAuthRefreshToken=token"

// With additional properties
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=my-project;DefaultDataset=my_dataset;OAuthType=3;Timeout=120;Location=EU"

OAuthType Values:

OAuthType	Authentication Method	Notes
0	Service Account	Requires OAuthPvtKeyPath
1	User OAuth	Requires OAuthClientId, OAuthClientSecret, OAuthRefreshToken
3	Application Default	Recommended for most use cases
4	External Account	Requires credentialConfigFile via Properties

Property Mapping:

Simba properties are automatically mapped to tbc-bq-jdbc equivalents:

Simba Property	tbc-bq-jdbc Property
ProjectId	projectId
DefaultDataset	datasetId
OAuthPvtKeyPath	credentials
Timeout	timeout
MaxResults	maxResults
Location	location

See Connection Properties for complete property mapping and all available options.

Connection Pooling

Works with all major connection pools:

HikariCP (Recommended)

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;

HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:bigquery:my-project/my_dataset?authType=ADC");
config.setMaximumPoolSize(10);
config.setMinimumIdle(2);
config.setConnectionTimeout(30000);

HikariDataSource dataSource = new HikariDataSource(config);

// Use the pool
try (Connection conn = dataSource.getConnection()) {
    // Execute queries...
}

Authentication

Application Default Credentials (ADC)

Best for: Local development, Google Cloud environments

# Set up ADC
gcloud auth application-default login

String url = "jdbc:bigquery:my-project/my_dataset?authType=ADC";
Connection conn = DriverManager.getConnection(url);

Service Account

Best for: Production, automation

String url = "jdbc:bigquery:my-project/my_dataset?" +
             "authType=SERVICE_ACCOUNT&" +
             "credentials=/path/to/service-account-key.json";
Connection conn = DriverManager.getConnection(url);

See Authentication Guide for all methods.

Building from Source

Requirements

• Java 21 or later
• Maven 3.9+
• Docker (for integration tests)

Build Commands

# Build slim JAR
./mvnw clean install

# Build shaded JAR (includes all dependencies)
./mvnw clean package

# Run unit tests
./mvnw test

# Run integration tests (requires Docker)
./mvnw verify -Pintegration-tests

Build Artifacts

After building:

• Slim JAR: target/tbc-bq-jdbc-1.0.61.jar (60K)
• Shaded JAR: target/tbc-bq-jdbc-1.0.61.jar (51M)
• Sources JAR: target/tbc-bq-jdbc-1.0.2-SNAPSHOT-sources.jar (41K)
• Javadoc JAR: target/tbc-bq-jdbc-1.0.2-SNAPSHOT-javadoc.jar (267K)

Testing

Unit Tests

91 unit tests covering:

• Driver registration and URL parsing
• Connection property validation
• Authentication configuration
• Type mapping
• Exception handling
• JDBC 4.3 methods

./mvnw test

Integration Tests

113 integration tests covering:

• Connection lifecycle
• Query execution
• Prepared statements
• Metadata operations
• Type conversions
• ResultSet operations

./mvnw verify -Pintegration-tests

See Integration Tests Guide for details.

Benchmarks

JMH benchmarks for performance testing:

# Set BigQuery connection URL
export BENCHMARK_JDBC_URL="jdbc:bigquery:my-project/my_dataset?authType=ADC"

# Run benchmarks
./mvnw clean package
java -jar target/benchmarks.jar

JDBC Compliance

JDBC Version: 4.3

Compliance Level: Partial (due to BigQuery limitations)

✅ Supported Features

• Connection lifecycle (open, close, isValid)
• Statement, PreparedStatement execution
• ResultSet forward iteration (TYPE_FORWARD_ONLY)
• ResultSetMetaData, DatabaseMetaData
• JDBC 4.3 methods (beginRequest, endRequest, enquoteLiteral, etc.)
• Sessions and transactions (with enableSessions=true)
• All BigQuery data types
• Query timeout and cancellation

❌ Unsupported Features

• Traditional transactions (without sessions)
• Scrollable or updatable ResultSets
• Batch operations
• CallableStatement
• Stored procedures (limited routine support)
• Full Array/Struct JDBC support (work in progress)

See Compatibility Matrix for complete details.

Performance

Query Latency

Query Type	Typical Latency
Small (SELECT 1)	200-500ms
Medium (< 100MB)	2-10s
Large (> 100MB)	10s - minutes

Optimization Tips

• Use pageSize property for large results
• Enable Storage API for queries > 10MB
• Use connection pooling
• Cache frequently executed queries
• Set appropriate timeouts

See Connection Properties - Performance Tuning for detailed optimization strategies.

Known Limitations

BigQuery Architecture

• No transactions outside of sessions (use enableSessions=true)
• No indexes (BigQuery auto-optimizes)
• No primary/foreign keys (data warehouse, not OLTP)
• No row-level locking

JDBC Limitations

• Forward-only ResultSets (no scrollable)
• Read-only ResultSets (no updatable)
• No batch operations (use BigQuery array syntax)
• Limited Array/Struct support

See Compatibility Matrix for complete list.

Roadmap

Version 1.0 (Current) - IDE Integration Focus

• ✅ Fast, comprehensive DatabaseMetaData implementation
• ✅ Parallel dataset loading with intelligent caching
• ✅ Core JDBC 4.3 implementation
• ✅ All authentication methods
• ✅ Session support
• ✅ Complete type mapping
• ✅ Extensive testing (199 total tests)
• ✅ Comprehensive documentation

Future Versions

• Full Array/Struct JDBC support
• Complete Storage API Arrow deserialization
• Routine (UDF) metadata
• Enhanced DatabaseMetaData
• Additional authentication methods

Contributing

Contributions welcome! Please see CONTRIBUTING.md for guidelines.

Development Setup

1. Clone the repository
2. Install Java 21+
3. Run ./mvnw clean install (Maven Wrapper included, no need to install Maven)

Running Tests

# Unit tests
./mvnw test

# Integration tests (requires Docker)
./mvnw verify -Pintegration-tests

# Format code
./mvnw spotless:apply

License & Disclaimer

Apache License 2.0 - see LICENSE file for details.

Use at your own risk. This software is provided "as is" without warranties of any kind. See LICENSE for complete disclaimer.

Support

• 📖 Documentation
• 🐛 Issue Tracker
• 💬 Discussions

Acknowledgments

• Architecture inspired by looker-open-source/bqjdbc
• Built for Java 21+ with modern features
• Uses Google Cloud BigQuery Client Library

Project Status

Status: ✅ Version 1.0 Release - Optimized for Database IDEs

This initial release focuses on providing fast, high-quality database metadata for development tools like JetBrains IDEs, DataGrip, and other database clients. Comprehensive JDBC 4.3 implementation with extensive testing (91 unit tests, 108 integration tests).