Skip to content

Exporting to Cloud Storage

Exporting to Azure Data Lake Storage Gen2

Exporting to Azure Data Lake Storage Gen2#

INFO

Find here all information and how-to export to Azure Data Lake Storage Gen2.

Prerequisites#

Having Azure Data Lake Storage Gen2 Configured as a Connection#

INFO

Make sure you have established a connection to your Azure Data Lake Storage Gen2 instance.

Configuring Export Jobs to the Azure Data Lake Storage Gen2#

To export a worksheet through a Azure Data Lake Storage Gen2 connector:

  1. Click the "+" button and choose "Export Job" or right-click in the "File Browser" and select "Create New" → "Export Job". The 'New Export Job' tab appears in the menu bar.
  2. Click "Select" to select the Workbook from the File Browser. The dialog "Select Workbook" opens.
  3. Select the Workbook and confirm with "Select". The name of the Workbook appears in the Workbook field.
  4. Confirm with "Next".

  5. Select a worksheet from the drop-down. The 'Connection' tab opens.

    INFO: Only kept sheets are available for export. If the drop-down is missing the sheet you want to export, modify the Workbook settings to keep the sheet.

  6. Click on "Select Connection" to select the Azure Data Lake Storage connection. The dialog 'Select Connection' opens.

    INFO: If you haven't create the connection yet, you can do it here by clicking on "New Connection".

  7. Select the relevant connection and confirm with "Next". The selected connection is displayed under 'Connection'.

  8. Select the file type from the drop-down and confirm with "Next". The 'Data Details' tab opens.

    INFO: The selected file type must be congruent to the file type of the Workbook.

  9. Fill in the file path in the GCS.

    INFO: If the data is being exported to an existing file path, the previous data is written over by the current export.

    INFO: Activate the 'Replace data' checkbox optionally.

  10. If needed, fill in the advanced settings within 'Maximum file size(MB) and 'File name consecutive numbering(Digits)'.

  11. If needed, select a compressor method and confirm with "Next". The 'Mapping' tab opens.

  12. Have a look a the preview and decide which columns will be included and whether empty data is being accepted.

    INFO: All checkboxes are activated by default.

  13. Decide how to handle invalid data and confirm with "Next". The 'Schedule' tab opens.

  14. Select the schedule method for when the export job shall be run and confirm with "Next". The 'Save' tab opens.

  15. If needed, add a description.

    INFO: Deactivating the checkbox 'Start export immediately after save will prevent the export starting immediately after saving.

  16. If needed, add your email address in section 'Notification' to receive possible error notifications and confirm with "Save". The dialog 'Save Export Job' opens.

    INFO: Spectrum can only send email notifications, if an email server has been configured.

  17. Select the place to save the export job to and name the export job, confirm with "Save". The export job is shown in the File Browser.

Exporting to Google Cloud Storage

Exporting to Google Cloud Storage#

INFO

Find here all information and how-to export to Google Cloud Storage.

Prerequisites#

Having Google Cloud Storage Configured as a Connection#

INFO

Make sure you have established a connection to your Google Cloud Storage instance.

Configuring Export Jobs to the Google Cloud Storage#

To export a worksheet through a Google Cloud Storage connector:

  1. Click the "+" button and choose "Export Job" or right-click in the "File Browser" and select "Create New" → "Export Job". The 'New Export Job' tab appears in the menu bar.
  2. Click "Select" to select the Workbook from the File Browser. The dialog "Select Workbook" opens.
  3. Select the Workbook and confirm with "Select". The name of the Workbook appears in the Workbook field.
  4. Confirm with "Next".

  5. Select a worksheet from the drop-down. The 'Connection' tab opens.

    INFO: Only kept sheets are available for export. If the drop-down is missing the sheet you want to export, modify the Workbook settings to keep the sheet.

  6. Select the GCS connector by clicking on "Select Connection". The dialog 'Select Connection' opens.

    INFO: If there's a preselected connection, you'll find it listed under 'connection'.

    INFO: If you haven't create the connection to GCS yet, you can do it here by clicking on "New Connection".

  7. Select the relevant GCS connection and confirm with "Next". The selected connection is displayed under 'Connection'.

  8. Select the file type from the drop-down and confirm with "Next". The 'Data Details' tab opens.

    INFO: The selected file type must be congruent to the file type of the Workbook.

  9. Fill in the file path in the GCS.

    INFO: If the data is being exported to an existing file path, the previous data is written over by the current export.

    INFO: Activate the 'Replace data' checkbox optionally.

    Example:

    • '/exports/%year%%month%%day%T%hour%:%min%/12354-%id%.csv' → This file is being written to a new directly each time the export job runs. A new file is created each time and the old file is not deleted.
    • '/exports/12354/%year%%month%%day%T%hour%%min%-%id%.csv' → This file is being written to the same directly each time the export job runs. The incoming export job overwrites the previous export job.

  10. If needed, fill in the advanced settings within 'Maximum file size(MB) and 'File name consecutive numbering(Digits)'.

  11. Adjust the prefilled input within the section 'CSV', if needed and confirm with "Next". The 'Mapping' tab opens.

  12. Have a look a the preview and decide which columns will be included and whether empty data is being accepted.

    INFO: All checkboxes are activated by default.

  13. Decide how to handle invalid data and confirm with "Next". The 'Schedule' tab opens.

  14. Select the schedule method for when the export job shall be run and confirm with "Next". The 'Save' tab opens.

  15. If needed, add a description.

    INFO: Deactivating the checkbox 'Start export immediately after save will prevent the export starting immediately after saving.

  16. If needed, add your email address in section 'Notification' to receive possible error notifications and confirm with "Save". The dialog 'Save Export Job' opens.

    INFO: Spectrum can only send email notifications, if an email server has been configured.

  17. Select the place to save the export job to and name the export job, confirm with "Save". The export job is shown in the File Browser.

Exporting to Snowflake

Exporting to Snowflake#

INFO

All records are exported as Avro files to a temporary (and configurable) location on S3. The Avro files are then added to Snowflake using a single Snowflake COPY INTO <table> command allowing for high performance on the Snowflake side. (See https://docs.snowflake.net/manuals/sql-reference/sql/copy-into-table.html). The temporary files and the Snowflake STAGE needed for the import to S3 are deleted once the export job has completed.

Prerequisites#

A Snowflake connection with Spectrum must be created before exporting data.

Configuring Export Jobs with an Snowflake Connection#

To export a worksheet using the Snowflake connector:

  1. Select the "+" at the top left of the "File Browser" and select "Export Job" or right-click on the Workbook to export from and select "Add to New Export Job".
  2. Select the Workbook to export from in the browser.
  3. Select the worksheet to export from the drop down menu.
  4. Select the Snowflake connector.

  5. The Snowflake database name is that set in the connector. If not previously set from the connector, enter the Snowflake Warehouse name. To export to an existing table on Snowflake, select the radio button "Use existing". Choose a schema and table name from the drop-down lists. To export to a new table or replace an existing table on Snowflake, select the radio button "Create new or replace existing table". Choose a schema from the drop-down list and enter a new table name. Select a record replacement policy.

  6. Replace records - All records from the existing table are deleted and the new records are written to the table. No table properties are modified.

  7. Append records - The records append (added in addition) to the existing table.

  8. Preview how the data to is being mapped to the Snowflake table and select how to handle invalid data.

  9. Select when to run the export job. This can be done manually by a user, run each time the Workbook in Spectrum is calculated, or at specifically scheduled times.

    INFO: Custom properties can be added in the advanced section as well as logging specifications.

  10. Enter a description and click "Save".