playbooks

home / skills / jeremylongshore / claude-code-plugins-plus-skills / bigquery-scheduled-query

bigquery-scheduled-query skill

/skills/14-gcp-skills/bigquery-scheduled-query

This skill guides and generates production-ready configurations for bigquery scheduled queries, following best practices and validating results.

npx playbooks add skill jeremylongshore/claude-code-plugins-plus-skills --skill bigquery-scheduled-query

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
2.2 KB
---
name: "bigquery-scheduled-query"
description: |
  Manage bigquery scheduled query operations. Auto-activating skill for GCP Skills.
  Triggers on: bigquery scheduled query, bigquery scheduled query
  Part of the GCP Skills skill category. Use when working with bigquery scheduled query functionality. Trigger with phrases like "bigquery scheduled query", "bigquery query", "bigquery".
allowed-tools: "Read, Write, Edit, Bash(gcloud:*)"
version: 1.0.0
license: MIT
author: "Jeremy Longshore <[email protected]>"
---

# Bigquery Scheduled Query

## Overview

This skill provides automated assistance for bigquery scheduled query tasks within the GCP Skills domain.

## When to Use

This skill activates automatically when you:
- Mention "bigquery scheduled query" in your request
- Ask about bigquery scheduled query patterns or best practices
- Need help with google cloud platform skills covering compute, storage, bigquery, vertex ai, and gcp-specific services.

## Instructions

1. Provides step-by-step guidance for bigquery scheduled query
2. Follows industry best practices and patterns
3. Generates production-ready code and configurations
4. Validates outputs against common standards

## Examples

**Example: Basic Usage**
Request: "Help me with bigquery scheduled query"
Result: Provides step-by-step guidance and generates appropriate configurations


## Prerequisites

- Relevant development environment configured
- Access to necessary tools and services
- Basic understanding of gcp skills concepts


## Output

- Generated configurations and code
- Best practice recommendations
- Validation results


## Error Handling

| Error | Cause | Solution |
|-------|-------|----------|
| Configuration invalid | Missing required fields | Check documentation for required parameters |
| Tool not found | Dependency not installed | Install required tools per prerequisites |
| Permission denied | Insufficient access | Verify credentials and permissions |


## Resources

- Official documentation for related tools
- Best practices guides
- Community examples and tutorials

## Related Skills

Part of the **GCP Skills** skill category.
Tags: gcp, bigquery, vertex-ai, cloud-run, firebase

Overview

This skill automates management of BigQuery scheduled queries within GCP workflows. It guides creation, update, validation, and troubleshooting of scheduled queries and produces production-ready configurations and code snippets. Use it to enforce best practices, permissions, and efficient resource usage.

How this skill works

The skill inspects request context for BigQuery scheduled query intents and then generates step-by-step instructions, IAM recommendations, deployment snippets, and validation checks. It can produce cron schedules, SQL templating patterns, service account IAM bindings, and Terraform or gcloud commands. It flags common configuration issues and suggests fixes for permissions, dependencies, and scheduling conflicts.

When to use it

  • Create or update BigQuery scheduled queries
  • Automate daily/weekly ingestion, transformation, or partition maintenance tasks
  • Validate scheduled query configuration and permissions before deployment
  • Generate Terraform or gcloud commands for scheduled queries
  • Troubleshoot scheduling, quota, or permission errors

Best practices

  • Use a dedicated service account with least-privilege BigQuery roles and grant additional roles only when needed
  • Prefer partitioned tables and incremental SQL patterns to reduce processing costs
  • Use parameterized SQL or scheduled parameters for date-driven incremental loads
  • Set clear retry and notification policies for failed runs (Pub/Sub, Cloud Logging, or alerts)
  • Validate SQL and dry-run queries to estimate cost before scheduling
  • Version scheduled query definitions in infrastructure-as-code (Terraform/gcloud) for reproducibility

Example use cases

  • Create a nightly ETL scheduled query that loads incremental data into a partitioned table with a service account
  • Migrate existing cron jobs to BigQuery scheduled queries with Terraform and automated IAM setup
  • Generate monitoring and alerting guidance for failed scheduled query runs using Cloud Logging and alerts
  • Validate and fix permission-denied errors when a scheduled query cannot access a destination dataset
  • Produce a templated scheduled query that accepts run-time parameters for date ranges

FAQ

What permissions are required to run a scheduled query?

The scheduled query needs a service account with roles/bigquery.jobUser and roles/bigquery.dataEditor (or dataset-level permissions) on the destination. Grant least privilege and test with a dry run.

How do I avoid high cost from scheduled queries?

Use partitioned tables, incremental SQL filtering (based on _PARTITIONTIME or timestamp columns), dry-run cost estimates, and schedule lower-frequency runs when possible. Monitor bytes processed per run.