# Snowflake Configuration for Optimal Search Performance ## Overview Panther has determined how to configure Snowflake to yield optimal search performance. These configuration settings include [warehouse size](#warehouse-size), [query acceleration](#query-acceleration), and [search optimization](#search-optimization). These guidelines are used in determining configurations for [Panther-managed Snowflake](https://docs.panther.com/~/changes/Cd1BxbxeaFl8dlynhNpt/system-configuration/panther-deployment-types/legacy-configurations/snowflake-setup#panther-managed-snowflake) instances, and can serve as a reference for [customer-configured Snowflake](https://docs.panther.com/~/changes/Cd1BxbxeaFl8dlynhNpt/system-configuration/panther-deployment-types/legacy-configurations/snowflake-setup#customer-configured-snowflake) instances. ## Warehouse size [Warehouse size](https://docs.snowflake.com/en/user-guide/warehouses-overview) determines the amount of compute resources used when performing an operation in Snowflake (e.g., searching Snowflake tables). ### **Search performance and cost expectations** In general, the larger a warehouse is, the faster a query runs. An increase in warehouse size means an across-the-board increase in query speed. However, it also means an across-the-board increase in compute spend. For example, a `SMALL` warehouse will typically run queries twice as fast as an `X-SMALL` warehouse, but costs twice as much to run. Below is a high-level warehouse size recommendation based on your ingest volume in Panther. These ingest volume/warehouse size pairings will all yield similar search performances.
Ingest volume in TB/month (uncompressed)Recommended warehouse size
< 8X-SMALL
8-15SMALL
16-31MEDIUM
32-63LARGE
64-127X-LARGE
128-2552X-LARGE
256-5123X-LARGE
Learn more about warehouse size on [Snowflake's Overview of Warehouses documentation](https://docs.snowflake.com/en/user-guide/warehouses-overview). ## Query acceleration {% hint style="warning" %} Query acceleration is only available within [Snowflake's Enterprise Edition](https://docs.snowflake.com/en/user-guide/intro-editions#enterprise-edition). {% endhint %} [Query acceleration](https://docs.snowflake.com/en/user-guide/query-acceleration-service) is a Snowflake service that uses idle Snowflake warehouses to speed up slow-running queries. ### The scale factor Query acceleration is configured with a scale factor, which is a cost control mechanism that sets an upper limit on the amount of compute resources that can be borrowed for query acceleration. Panther determines the scale factor for [Panther-managed Snowflake instances](https://docs.panther.com/~/changes/Cd1BxbxeaFl8dlynhNpt/system-configuration/panther-deployment-types/legacy-configurations/snowflake-setup#panther-managed-snowflake). For example, a `SMALL` warehouse with a scale factor of `10` means that up to `10` more `SMALL` warehouses may be allocated for a particular query. Learn more about the scale factor in [Snowflake's Adjusting the Scale Factor documentation](https://docs.snowflake.com/en/user-guide/query-acceleration-service#adjusting-the-scale-factor). ### **Search performance and cost expectations** * Query acceleration does not kick in until a running query is deemed "slow." Generally, queries running for longer than one minute are candidates to be sped up. * The cost of running a query with acceleration is roughly the same as if the query were run without acceleration, because the cost of the additional warehouses is balanced by the reduced compute time. The theoretical maximum cost, however, is determined by the [scale factor](#setting-the-scale-factor)—so a warehouse with a scale factor of `10` *could,* at worst, cost ten times more than the bare warehouse cost. Learn more about query acceleration on [Snowflake's Using the Query Acceleration Service documentation](https://docs.snowflake.com/en/user-guide/query-acceleration-service). ## Search optimization {% hint style="warning" %} Search optimization is only available within [Snowflake's Enterprise Edition](https://docs.snowflake.com/en/user-guide/intro-editions#enterprise-edition). This feature is only available to Panther Enterprise customers. {% endhint %} [Search optimization](https://docs.snowflake.com/en/user-guide/search-optimization-service) is a Snowflake service that indexes ingested data to dramatically improve speed when performing “needle in a haystack” searches (i.e., one in a million-type events). ### **Search performance and cost expectations** * With search optimization enabled, queries can execute 10-100x faster. * The more unique a value is, the greater impact search optimization has. * For example, if you are searching for `isHuman = True` and `50%` of events are `True`, search optimization will not improve performance at all; however, if only `.0001%` of the events are `True`, search optimization will have a significant impact. * The cost of search optimization can vary based on the following: * Both during search optimization initialization and thereafter, as data is ingested, the cost is influenced by the number and size of the tables and fields that are indexed. The more tables and fields, and the larger tables and fields, the more expensive. * When search optimization is initialized, all existing data must be indexed, meaning there is an upfront cost proportional to the amount of historical data being indexed. * Following initialization, the cost is calculated at data ingest based on the compute required to generate the search optimization indexes. * When an index is used to speed up a search, there is no extra cost. This means searches may become cheaper, since they run more quickly. Learn more about search optimization on [Snowflake's Using the Search Optimization Service documentation](https://docs.snowflake.com/en/user-guide/search-optimization-service).