Skip to content

Deploying General Information

After transforming your data, you can deploy your dataset to your Snowflake instance.

General Information#

Definition of Terms#

Datameer View

A view is basically a named definition of a transformed dataset in Datameer.

Example: Your source dataset is called 'Car_List' and you apply a filter to the asset. The output is the Datameer View 'FILTER'. The Datameer view is not deployed to your Snowflake instance. The new asset can be deleted from here, deployed to Snowflake or one can open the asset details page.

Snowflake Views and Tables

In Datameer you can deploy both a Snowflake view or table.

  • Views in Snowflake are a point-in-time of the view in Datameer and defined as 'basically a named definition of a query' in Snowflake. Views enable you to consume just a portion of the data in a table. The performance is slower than tables and the results are not stored for the future use.
  • A view that was created in Snowflake is a 'CREATE VIEW', not materialized and not a table.

  • Example: Filtering data from a table results in a Datameer view and then can be deployed to a Snowflake view.

  • Tables in Snowflake are static artifacts which are stored in a columnar format, e.g. a fix table with historical data.

  • This allows faster access, but requires storage space and active maintenance.
  • A table that was created in Snowflake is a 'CREATE TABLE' and materialized.

  • Example: Filtering data from a table results in a Datameer view and then can be deployed to a Snowflake view.

Find more information about the Snowflake definition of terms here and here.

Deploying Requirements#

  • The location chosen for the view or table must be an existing database and schema. The available schemas are configured on the 'Snowflake Settings' page (admin only). You are not allowed to create a new schema at this time.
  • Deployed views and tables appear in both Snowflake and Datameer immediately. These assets can be shared across other Projects as well.
  • If a referring dataset becomes out of sync with the deployed view/ table in Snowflake due to changes in the Project, the asset is highlighted and must be re-deployed. Note that using an asset from Project A as the source from Project B requires active maintenance when Project B is not scheduled.
  • Overwriting an existing view/ table is not allowed. To do so, you have to re-deploy. Otherwise the view/ table must be renamed.
  • After deploying a view or table, you can add this node again as a source to the same Project or a new Project.

Deploying Permissions#

When deploying data to Snowflake only those schemas are displayed for which the required permissions are granted to create new tables or views. The following permissions are required:

  • to deploy as a Snowflake view, grant 'CREATE_VIEW'
  • to deploy as a Snowflake table, grant 'CREATE_TABLE' and 'CREATE_PROCEDURE'

Note that all of the above can be executed independently when having the 'OWNERSHIP' privilege of the artifact. In case the required schema is not part of the deployment list, the permissions must be set in your Snowflake account. Find more general permission setup information here.

Detecting Non-Synchronized Views/ Tables#

You can simply detect when a view or table that has been deployed, is no longer in synchronization with the one from which the deployed view or table is based on. Non-synchronized views and tables are highlighted with a red border. The visual warning will persist until the view/ table is re-deployed or until the changes are reverted.

You can investigate the following information for the view/ table:

  • last deployment date
  • last change date to any of the views to generate that view/ table