One of the ways we can see that a query might be handling too much data for a particular warehouse is by looking at spilling. This article doesn’t apply to adaptive warehouses as I haven’t had the chance to experiment…
There are some really cool details about Snowflake’s clustering. Working with these details can help reduce the cost of clustering. In the last article, I dug into how aligning a clustering key with data arrival patterns can help you reduce…
Choosing a clustering key isn’t just about which columns or functions to include — it’s about how your data arrives. In most real-world systems, data lands in approximate or exact date order. That arrival pattern interacts with your clustering key…
Queries in Snowflake try to use the entire standard warehouse. So what happens when we’re running dozens of queries on the same warehouse at the same time, and each of those queries is attempting to use all of the resources…
The variety and number of ways that people test performance on Snowflake incorrectly continues to amaze me. I don’t think that testing the performance of a query in isolation is difficult, but it requires a very scientific approach. We need…
The age of AI may be here, but there are still some things an LLM does better than others. There have been different levels of automation or autonomics added to the systems I work with my whole career. With LLMs,…
When using Snowflake, compute and storage are fully separated. This means choosing different sizes (and types) of compute to use against the same data for different purposes. Even at the same time. It used to be that size was the…
Benchmarking is a topic I’ve seen discussed for my entire career working with databases. We used to watch the TPC-C results published, when I was working with primarily OLTP systems, and was amazed to see software I worked with every…
Hints have been a thing in database engines for years. I’ve always seen them as an absolute last resort, but have used them on other platforms. In general, Snowflake has not had customer configured hints. But like any optimizer, sometimes…
My day job involves working with customers on performance problems. I have worked with dozens of customers and probably hundreds of queries just this year. I’ve been working in this role for more than two years now. What has all…
At work, I read a lot of customer-written SQL. This is a major part of my day job, and there are some underrated things that I regularly recommend. The ANY_VALUE() function is one of these things. The ANY_VALUE() Function The…
In a recent series of articles, I walked through the query optimizer. I’ve also covered using EXPLAIN to look at queries. One of the primary disadvantages of EXPLAIN is that it does not execute the query, and is therefore missing…
