Skip to main content

Snowflake API

info

If you want to access this feature, contact the Anyscale team.

SnowflakeDatasource​

SnowflakeDatasource(**connection_parameters: str)

A Datasource that reads and writes from Snowflake data sets.

Parameters​

  • **connection_parameters: Keyword arguments to pass to snowflake.connector.connect. To view supported parameters, read the Snowflake Python Connector API.

Examples​

from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)

ray.data.read_datasource​

ray.data.read_datasource(
    datasource: SnowflakeDatasource,
    *,
    sql: str
) -> Dataset

Read data from a Snowflake table into a Ray Dataset.

Parameters​

  • datasource: A SnowflakeDatasource.
  • sql: The SQL query you want to execute.

Returns​

A Ray Dataset that contains the query result set.

Examples​

import ray
from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)
ds = ray.data.read_datasource(
    datasource,
    sql="SELECT * FROM SNOWFLAKE_SAMPLE_DATA.TPCDS_SF100TCL.CUSTOMER"
)

Dataset.write_datasource​

Dataset.write_datasource(
    datasource: SnowflakeDatasource,
    *,
    table: str
) -> None

Write data in a Ray Dataset to a Snowflake table.

Parameters​

  • datasource: A SnowflakeDatasource.
  • table: The table you want to write to.

Examples​

import ray
from ray_extensions.data import SnowflakeDatasource

datasource = SnowflakeDatasource(
    user=...,
    account="ABCDEFG-ABC12345",
    password=...,
)
dataset = ray.data.from_items([
    {"title": "Monty Python and the Holy Grail", "year": 1975, "score": 8.2},
    {"title": "And Now for Something Completely Different", "year": 1971, "score": 7.5},
])
dataset.write_datasource(datasource, table="MY_DATABASE.MY_SCHEMA.MOVIES")