The Snowflake Resource Monitor can be used to control warehouse credit usage, avoiding unexpected behaviors. When executing queries on the Snowflake platform, virtual warehouses are exploited to retrieve answers and, by doing so, Snowflake credits are consumed. Credits are used to pay for the processing time/power used by each virtual warehouse. If desired, the warehouse can be suspended when it reaches a credit limit.
Limits can be set for specific intervals or date ranges and when reached, they can trigger various actions such as notifications or warehouse shut down.
Resource Monitor Properties
Credit Quota
Specifies the number of credits allocated to the monitor and can be any (reasonable, if you don’t want to spend billions) number. If a specified interval is given, it resets back to 0 at the end of it.
Monitor Level
This setting specifies if the Resource Monitor should monitor the credit usage of the entire Account or a specific set of virtual warehouses assigned to it. The Resource Monitor is dormant if this property is not set, so basically this option is required.
Schedule
If this option is not set, the Resource Monitor starts monitoring credit usage right away and credits reset back to 0 at the beginning of the standard Snowflake billing cycle, which is the calendar month.
Scheduling can further be tuned to achieve specific objectives by setting the following properties:
- Frequency the interval at which the used credits reset according to the start date. Possible values are:
- Daily
- Weekly
- Monthly
- Yearly
- Never (used credits never reset)
- Start date and time when the monitoring starts. Supported values are:
- Immediately
- Timestamp
- End date and time when Snowflake suspends the warehouses associated with the Resource Monitor, regardless of whether the credits exceeded a threshold. Needs to be a timestamp.
Actions
Actions (or triggers) are defined using the percentage of the credit quota assigned to the Resource Monitor and are executed upon reaching the specified threshold. Supported actions are:
- Notify and Suspend send a notification to all account administrators and suspend all assigned warehouses (after all statements being executed by the warehouses have completed). Basically it warns you that credits are ending
- Notify and Suspend Immediately send a notification to all account administrators and suspend all assigned warehouses immediately, regardless of any statements being executed. That’s the red button: just shut everything down because you don’t want to take out a mortgage to pay Snowflake
- Notify send an alert notification to all account administrators
Resource Monitors and Warehouses
A Resource Monitor can be set at the account level or at multiple warehouses to control credit usage. However, a warehouse can only be assigned to a single Resource Monitor below the account level.
Based on the example in the image above, we can say that:
- The credit quota for the account is 5000 and it’s controlled by Resource Monitor 1. When this quota is reached all the actions defined for the RM are enforced to all five warehouses
- Warehouse 3 can consume a maximum of 1000 credits and it is assigned to Resource Monitor 2
- Warehouse 4 and 5 can consume a maximum combined total of 2500 credits and are assigned to Resource Monitor 3
It is notable that the actual credits consumed by warehouses 3, 4 and 5 may be less than the account quota if it is reached first, because RM 1 also monitors warehouse 1 and 2, so if the maximum combined total of all warehouses is 5000, warehouses are shut down.
Warehouse Suspension and Resumption
Used credits for a Snowflake Resource Monitor reflects the sum of credits consumed by all assigned warehouses within the specified interval, along with the cloud services used to support those warehouses during the same interval.
If a monitor reaches his credit threshold and one of Suspend or Suspend Immediately actions were set for it, any warehouse assigned to the monitor is suspended. Each suspended warehouse can’t be resumed until some conditions happen:
- the next interval, if any, starts, as dictated by the start date of the monitor
- the credit quota for the monitor is increased
- the credit threshold for the suspend action is increased
- the warehouses are no longer assigned to the monitor
- the monitor is dropped
Overview on Costs
Now that we covered our back with Resource Monitors, let’s take a quick dive into understanding costs, in order to be able to understand why the account or warehouses have been suspended, even if few operations were performed.
The total cost of using Snowflake is the aggregate of the cost of using data transfer, storage and compute resources. Snowflake’s innovative cloud architecture separates the cost of accomplishing any task into one of these usage types.
Compute Resources Cost
Using compute resources within Snowflake consumes credits. The billed cost of using compute resources is calculated by multiplying the number of consumed credits by the price of a credit. Credits have different costs, depending on the type of warehouse you choose. To learn more about this click here.
There are three types of compute resources that consume credits within Snowflake:
- Virtual Warehouse Compute: Virtual warehouses are user-managed compute resources that consume credits when performing operations. Because Snowflake utilizes per-second billing, warehouses are billed only for the credits they actually consume when they are actively working. Note that a 60-second minimum period each time the warehouse starts is applied.
- Serverless Compute: There are Snowflake features such as Search Optimization and Snowpipe that use Snowflake-managed compute resources rather than virtual warehouses. To minimize cost, these serverless compute resources are automatically resized and scaled up or down by Snowflake as required for each workload.
- Cloud Services Compute: The cloud services layer of the Snowflake architecture consumes credits as it performs behind-the-scenes tasks. Usage of the cloud services layer is charged only if the daily consumption of cloud services resources exceeds 10% of the daily warehouse usage.
Storage Cost
The monthly cost for storing data in Snowflake is based on a flat rate per terabyte (TB). It depends mainly on the type of your account and the region your account is hosted. Storage is calculated monthly based on the average number of on-disk bytes stored each day in your Snowflake account.
Data Transfer Cost
Snowflake does not charge data ingress fees to bring data into your account, but does charge for data egress.
Snowflake charges a per-byte fee when you transfer data from a Snowflake account into a different region on the same cloud platform or into a completely different cloud platform. This per-byte fee for data egress depends on the region where your Snowflake account is hosted.
To understand also how to contain costs, check this other article by Can Hobek that can be found here.
References
Snowflake Documentation: https://docs.snowflake.com/
