Jobs Module#
The Jobs module unlocks end-to-end orchestration capabilities and enables teams to implement an SDLC (Software Development Lifecycle) workflow for their data projects. When enabled, a Jobs tab appears in the Datameer navigation where users can configure the module, view job history, and create, update, delete and run jobs.
Key Concepts#
Service Account#
A Service Account represents a Snowflake identity used by the Jobs module to execute work in Snowflake on your behalf. Administrators configure service accounts with the appropriate Snowflake credentials and permissions.
See Service Accounts for configuration details.
Environment#
An Environment combines a Snowflake Role (via a service account identity) with a Snowflake Warehouse. It defines the access and compute resources used when executing jobs, and can include a notifications channel and sane defaults for target locations. Jobs are defined to use a single Environment.
See Managing Job Environments for configuration details.
Job Type#
The Jobs module supports several types of job workloads:
| Job Type | Description |
|---|---|
| Load / Unload | Load data from or unload data to a Snowflake stage |
| Stored Procedure | Execute a stored procedure in Snowflake |
| Model Job | Create or update a dataset represented as a Model |
Job Definition#
A Job Definition combines an Environment, a workload definition (based on Job Type), and scheduling or trigger configuration. Common configuration across all job types includes:
- Environment: which environment the job runs in
- Triggers:
- Schedule: run the job on a recurring schedule
- Event Driven: run the job when another job completes or a deployment runs
Job Run#
A Job Run is the historical record of a single execution of a Job Definition. Job runs can be reviewed in the Jobs Dashboard and History views.
Model#
A Model represents a data model that can be materialized via a Model Job into an arbitrary Snowflake target location. It defines the data sources, transformation logic and maintains a version history of the SQL. Models are the successor to the legacy Deployments feature.
See Models for full documentation.
Managing Jobs#
For detailed how-to documentation on configuring and running jobs, see: