Skip to main content

Snowflake API

Check your docs version

This version of the Anyscale docs is deprecated. Go to the latest version for up to date information.

info

If you want to access this feature, contact the Anyscale team.

SnowflakeDatasource​

SnowflakeDatasource(**connection_parameters: str)

A Datasource that reads and writes from Snowflake data sets.

Parameters​

  • **connection_parameters: Keyword arguments to pass to snowflake.connector.connect. To view supported parameters, read the Snowflake Python Connector API.

Examples​

from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)

ray.data.read_datasource​

ray.data.read_datasource(
    datasource: SnowflakeDatasource,
    *,
    sql: str
) -> Dataset

Read data from a Snowflake table into a Ray Dataset.

Parameters​

  • datasource: A SnowflakeDatasource.
  • sql: The SQL query you want to execute.

Returns​

A Ray Dataset that contains the query result set.

Examples​

import ray
from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)
ds = ray.data.read_datasource(
    datasource,
    sql="SELECT * FROM SNOWFLAKE_SAMPLE_DATA.TPCDS_SF100TCL.CUSTOMER"
)

Dataset.write_datasource​

Dataset.write_datasource(
    datasource: SnowflakeDatasource,
    *,
    table: str
) -> None

Write data in a Ray Dataset to a Snowflake table.

Parameters​

  • datasource: A SnowflakeDatasource.
  • table: The table you want to write to.

Examples​

import ray
from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)
dataset = ray.data.from_items([
    {"title": "Monty Python and the Holy Grail", "year": 1975, "score": 8.2},
    {"title": "And Now for Something Completely Different", "year": 1971, "score": 7.5},
])
dataset.write_datasource(datasource, table="MY_DATABASE.MY_SCHEMA.MOVIES")