The power of LLMs on your data, more than two orders of magnitude faster and cheaper

Google Cloud has introduced proxy models to accelerate and reduce the cost of LLM-powered AI functions in databases. These ultra-lightweight models are tailored to specific queries and can be trained on-the-fly or ahead of time. They replace the majority of LLM calls during query execution, resulting in significant performance and cost improvements. This optimization is already implemented in BigQuery and AlloyDB under the optimized mode feature. Proxy models can deliver accurate answers in many cases, but may require LLMs in certain scenarios.