Snowflake and NVIDIA Powerhouse Partnership Ignites AI Revolution

The recently concluded Snowflake Summit 2023 witnessed the announcement of a new partnership between Snowflake and NVIDIA. The CEO of Snowflake, Frank Slootman, and Nvidia’s CEO Jensen Huang disclosed that the partnership would bring Nvidia’s accelerated GPU computing and AI framework to the Snowflake data cloud. This collaboration promises to revolutionize how enterprises interact with their proprietary data, unlock insights and do business.

In this blog post, we break down the details of this product announcement, highlighting the moving parts, and we discuss the implications of this product on the average Snowflake user while pointing out some relevant use cases.

If you’re excited about what’s in store for your company’s Snowflake data, then stick around and enjoy a great read!

Envision the Possibilities of Data With AI

The rise of generative artificial intelligence and Large Language Models (LLMs) has brought a new era of interacting with data, whereby we can query and interact with data in our own natural language, making data manipulation and analysis possible for users of all kinds.

Data plays a vital role in building generative AI models. These models process the data and perform various complex operations, intelligently adapting their outputs and results based on the data they receive. In simpler terms, the data is essential fuel that powers these AI applications, allowing them to generate responses and outcomes that align with user prompts.

Picture generative AI models intuitively generating outputs based on your organization’s proprietary data. These outputs can then drive business-specific applications and services such as virtual assistant chatbots, recommendation algorithms, and fraud detection and risk assessment systems.

What if I told you that you would be able to develop, deploy and manage customized applications powered by Generative AI models that utilize your company’s proprietary data without ever leaving the platform that secures and governs this data?

If it sounds too good to be true, keep reading.

Generative Data Analytics: Supercharging the Data Cloud

Together, NVIDIA and Snowflake will create an AI factory that helps enterprises turn their own valuable data into custom generative AI models to power groundbreaking new applications — right from the cloud platform that they use to run their businesses. – Jensen Huang, CEO of NVIDIA .

The partnership sees NVIDIA’s NeMo framework and their GPU-accelerated computing power coming to Snowflake Data Cloud, enabling users to create custom AI models for their data all from the comfort of their data warehouses.

Bringing the compute engine to Snowflake ensures that data governance and security are not tampered with and that latency is kept at a minimum – data gravity in its true implementation.

This game-changing technology will revolutionize how businesses engage with their data and uncover authentic insights. Businesses can create multiple Large Language Models and augment them with their data, allowing them to interact with the data through these models, just like having a conversation with a person. The models can then be used to power data applications.

These full-stack data applications can also be deployed and run within Snowflake, all while utilizing NVIDIA’s cutting-edge AI algorithms and compute engines – more details below.

Truly this collaboration offers limitless possibilities, promising to revolutionize how enterprise and big data fuel growth across all sectors that rely on data-driven operations.

A Little More Technical Detail

To enable the seamless deployment and operation of both Large Language Models (LLMs) and full-stack applications within a secure and well-governed data environment, Snowflake presents the Snowpark Container Services (currently in private preview).

Containers allow developers to properly package applications along with any libraries and dependencies to ensure they are portable and can run in multiple environments.

Snowpark Container Services allows users to securely deploy and run sophisticated generative AI models and full-stack applications in Snowflake, leveraging the power of containers and Snowflake’s governed data boundary.

Some notable features of  Snowpark Container Services are:

1. Containerized Workloads

Snowpark Container Services allows developers to deploy, manage, and scale containerized workloads within Snowflake’s secure infrastructure, including jobs, services, and service functions.

2. Support for Any Programming Language

Developers can build containers with code written in any programming language, such as C/C++, Node.js, Python, R, and React, expanding the range of AI/ML and app workloads that can be brought to Snowflake using Snowpark.

3. Configurable Hardware Options

Containers can be executed with configurable hardware options, including support for GPUs, enabling accelerated execution of machine learning libraries, computationally intensive generative AI models, and optimized logic.

4. Simplified Development and Management

SnowparkServices eliminates the complexities of managing compute and clusters for containers, allowing developers to focus more on business problems rather than managing infrastructure.

5. Secure and Governed Data Processing

Containers run within Snowflake, eliminating the need to move governed data outside of Snowflake, ensuring data security, and minimizing security risks.

6. Snowflake Native App Integration

Snowpark Container Services can be part of Snowflake Native Apps, enabling the secure installation and running of sophisticated third-party software and apps within a customer’s Snowflake account, protecting proprietary IP.

7. Partner Integrations

Snowpark Container Services integrates with partner solutions such as Alteryx, Astronomer, Dataiku, NVIDIA, SAS, and others, providing enhanced capabilities and services for processing and working with governed data in Snowflake.

8. Three Execution Options

Snowpark Container Services supports three execution options for containerized workloads: jobs (time-bounded processes), service functions (repeatedly triggered processes), and services (long-living processes with secured ingress endpoints).

9. Future Expansion

Snowpark Container Services opens the door for further app development possibilities, including bringing container images via the Snowflake Marketplace and expanding the capabilities and offerings available to developers.

Datameer: The Transformative Choice

While Snowpark is an intuitive library that is ideal for implementing complex data pipelines that can include custom ai models and full-stack applications, it implements these complexities using code. In fact, any operation that will be performed on data using Snowpark must be done with one programming language or the other.

This can prove tasking for data teams that are not programming-centric. It can also be time-consuming when carrying out data transformations because it all has to be done with code.

An essential alternative for the transformation of large proprietary datasets would be a low/no-code software with a proven track record for saving time during the transformation process, allowing you to allocate more resources to focus on bolstering your data pipelines with custom AI models and data applications.

And such a solution is DATAMEER!

Indeed, if you have non-programmers that work with your data, or you would like to speed up the analytics workflow for technical teams and enable self-service for business users without having to move your data out of Snowflake, then Datameer is the tool for you.

Choosing Datameer also lets you utilize other beneficial features for your data, such as:

1. Data Fusion and Context Creation

Seamlessly merge extensive captured data with master and other datasets to generate context-rich, meaningful data for analysis.

2. Advanced Data Enrichment

Enhance analytics datasets comprehensively by incorporating a diverse range of graphical formulas and functions, adding depth and richness to your analysis.

3. Collaborative Knowledge Sharing

Easily create extensive documentation and facilitate collaboration by allowing users to contribute their own attributes, comments, tags, and more, enabling searchable knowledge sharing across the entire analytics community.

4. Data Governance Empowerment

Leverage these catalog-like documentation features to crowdsource your data governance processes, promoting data democratization and fostering data literacy among users.

5. Enhanced Compliance and Governance

Establish complete audit trails to track data transformations and usage within the community, bolstering governance and compliance procedures.

With all these features, you will be able to reduce to streamline your data transformations and still utilize the power of Snowpark Container Services for your more complex AI and data application needs.

Give Datameer a try today, and you will be glad you did!