With the Snowflake Summit a little over a month behind us, our team has had a chance to preview and work with some of the new and exciting features that were announced during the event. With more compute power per credit, deeper usage insights, and a host of other innovative new features and updates, Snowflake is truly becoming the most powerful Cloud Data Platform available. The move from just a traditional Data Warehouse into a one-stop-shop for all data, applications, and analytics will allow companies to store and process data quicker, reduce costs even further, and most importantly, gain actionable insights that will impact an organization’s bottom line. While the applications of these advancements are wide-ranging, there are a few specific aspects that AllCloud is excited about and that we think will have a direct impact on our current and future data and analytics work.
The releases of Python & Snowpark
Python UDFs (user defined functions) and Python UDTFs (User defined table functions) were an area of focus and will allow customers and our internal engineers to now leverage Snowpark Python workloads directly on Snowflake. We can write custom functions in Python and now call those as though they were built-in functions. Since the majority of our engineers are already Python experts, we’ll now be able to simplify and automate these processes across the board. Imagine you’re trying to set up a pipeline for a Healthcare client’s HL7 files – after some simple setup you can create and call a UDTF with SQL to retrieve and process results from the files directly in the Snowflake GUI!
Unistore & On-Prem External Table Access
These are both exciting changes as they enable customers to both access their transactional data hosted in on-prem systems with new external table functionality – or use Snowflake as an OLTP system to work with transactional workloads. With our clients, we often deal with source system and on-prem system ingestion, but for use cases where data may not be moved from on-prem systems or situations where clients keep their analytical and transactional datasets separate, they can now manage both datasets in a single location. The release of ‘hybrid tables’ powered by Unistore will provide high speed single-row operations and allow customers to build transactional applications directly on Snowflake. We can now use external or hybrid tables to process this data all with the power and flexibility of the Snowflake Data Platform. This is a major release and shows the true flexibility of the platform. Clients no longer need to worry about setting up another system for transactional data workloads; they can leverage Snowflake and act on the data immediately, build better experiences, and streamline their costs and analytics on that data. This consolidation will eliminate complex data integration needs and unify governance by keeping everything on one platform. Personally, I’m most excited about this release and can’t wait to implement this for some of our clients.
The Streamlit integration is a really exciting acquisition for Snowflake, essentially allowing users to build front end applications with minimal code for Snowflake data backends. As an open source Python project, Streamlit allows us to customize ‘building block’ capabilities to fit every use case. Streamlit treats widgets as variables, and every interaction re-runs the script from top to bottom and deploys the app directly from private Git repos updating on every commit. We see this as a huge advantage in development for situations where a custom front-end may be needed but where budget may not necessarily allow for other vendors to develop the required application. We recently had a prospect this year who needed a custom front end application with a data store backend to enable cattle herd management (one of the more interesting data use cases that has come across my desk this year). At the time this custom development was to be outsourced, but with Streamlit, we could have coded the entire application with Python and saved thousands of dollars for the customer – a huge win!
The release of account level replication
While this is somewhat of a minor release, this update excites me because it allows us to expand on strategic data ops scenarios. For example, for our clients with multiple accounts and environments, we can now replicate all objects on an account level down to the QA or DEV accounts with one line of code. Previously database objects and what they contained had to be replicated and refreshed to other accounts individually, but with account level replication we save time and major headaches for the clients with multiple accounts.
Native Machine Learning models within Snowflake
By including native machine learning models within Snowflake, functions like time-series forecasting can be called directly on a dataset within Snowflake (just as you would call a regular function). SQL Machine Learning, now in private preview, adds a powerful set of algorithms for us to use, and these algorithms can be added to everyday BI users’ workflows with minimal training – and will enhance decision quality and speed. This release also includes a memory upgrade. With Larger Memory Warehouses, users can execute more memory-intensive actions like model training on large data sets using open source Python libraries. Imagine some of the more advanced BI Analytics users who now have the ability to explore data science models inside their everyday work. There is no doubt this will enable and inspire them to get deeper insights into their data.
Why it Matters
While we’re certainly excited about the technical aspects of all that we heard and learned at the 2022 Snowflake Summit, the real opportunity (and excitement) is in how we can take these tools and use them to better help our clients solve problems, enhance their businesses, and use data as a tool to move forward.