Public Data Processing Templates

SWFSC CoastWatch maintains two public, sanitized GitHub templates for groups that also process data on GCP. They are optional starting points: an ERDDAP deployment does not need either template. Each repository contains example files and placeholders that must be adapted to the new project.

Template	Choose it when	Repository
Cloud Run Job	A container should run a batch workflow on demand or on a schedule.	cloud-run-template
Processing VM	A workflow runs directly on a VM, with scripts, notebooks, and a persistent workspace.	vm-dataproc-template

Cloud Run Job template

The Cloud Run template includes a Dockerfile, entrypoint.sh, deploy_job.sh, and example config/, scripts/, src/, and templates/ directories. Its deployment script contains project, region, image, service account, and secret placeholders. Review the entire script and replace those values before running it; it can create cloud resources and deploy a job.

Start with the repository’s README, then use Use this template on GitHub to create a repository for your workflow. Add your processing code, configure its GCP identity and storage permissions, and review the deployment script against your organization’s infrastructure ownership.

Processing VM template

The processing VM template provides example config/, notebooks/, scripts/, src/, data/, docs/, logs/, templates/, and tests/ directories. Its data/ directory contains an example file; it is not a preconfigured link to a persistent disk. Choose the storage layout and mount paths appropriate for your project.

First establish the VM, disk, service account, and required storage access using Optional Processing VM Setup. Then follow the template’s README and use Use this template to create your workflow repository. The template organizes code and documentation; it does not provision a VM or install a scientific environment.

Create a repository from a template

Open the Cloud Run template or processing VM template on GitHub. Select Use this template → Create a new repository, choose the repository owner and name, and create it. Clone your new repository, then replace the example configuration and files with your workflow’s code and settings. Review any deployment script before running it against your GCP project.