Informatica Unveils New Product Innovations and Enterprise-Grade Generative AI Blueprint for Databricks

Informatica, a market leader in business cloud data management, has announced an extended relationship with Databricks, which will include additional product features and solutions. This cooperation brings together the full functionality of Informatica’s AI-powered Intelligent Data Management Cloud (IDMC) platform with the Databricks Data Intelligence Platform. The goal is to make it easier to implement enterprise-grade Generative AI (GenAI) systems on a large scale, supported by high-quality, trustworthy data and metadata.

#Informatica and @databricks are expanding their partnership with new product capabilities and enabling customers to deploy enterprise grade GenAI applications at scale with trusted data.

Learn more about the collaborative innovations: https://t.co/uk0T40O4Yt pic.twitter.com/uZMLSfZGcJ

— Informatica (@Informatica) June 10, 2024

Amit Walia, Chief Executive Officer at Informatica, expressed enthusiasm about the expanded collaboration: “We are excited to expand our innovation and collaboration with Databricks with complete support for Unity Catalog and our enhanced comprehensive GenAI solution blueprint. Together, we can deliver an AI-powered foundation for data management and data intelligence in the cloud to accelerate our joint customers into the era of GenAI with truly transformative applications.”

Key Innovations in the Expanded Partnership

The partnership introduces four significant capabilities:

GenAI Solution Blueprint for Databricks DBRX

Informatica’s GenAI blueprint for Databricks provides a roadmap for customers to develop Retrieval Augmented Generation (RAG)-based GenAI applications using Databricks DBRX, an open, general-purpose large language model (LLM). The blueprint allows for the creation of GenAI solutions leveraging Informatica’s IDMC no-code/low-code interface, metadata-driven intelligence, and Databricks Vector Database for embedding data. This integration ensures access to trusted and actionable data insights at scale.

Key Benefits:

• Generative AI application inference and summaries are grounded on corporate data to provide high-fidelity, trustworthy replies.
• Prompts are being enriched with full corporate data and metadata to provide contextualised replies based on industry and organisational semantics.
• Integrating data quality indicators to prioritise data set selection, ensuring that GenAI applications are built on reliable data.
• RAG chain and agent-based GenAI systems may be orchestrated and defined without using any code, allowing for quick creation, maintenance, and growth.
• Comprehensive data governance promotes openness and trust by enforcing data access and usage regulations for enterprise-compliant apps.

Native Databricks SQL ELT

• This feature allows customers to conduct in-database transformations with complete pushdown capabilities on Databricks SQL, taking advantage of native Databricks services like the AI-powered Assistant and serverless computation. Customers may create ETL pipelines to get AI-optimized performance and scalable data transformations and connections.

Cloud Data Integration-Free Service (CDI-Free) on Databricks Partner Connect

• Informatica’s CDI-Free solution is now available through Databricks Partner Connect, allowing users free access to Informatica’s cloud data ingestion and transformation capabilities. This service provides ELT processing for up to 20 million rows of data or 10 ELT compute hours per month for free.

Full IDMC Support via Unity Catalog

• The Informatica IDMC platform is now fully validated with Databricks Unity Catalog, a unified governance layer for data and AI within the Databricks platform. This integration enhances Informatica’s capability to manage data lineage and governance on the IDMC Platform.

Adam Conway, SVP of Products at Databricks, underscored the importance of access to high-quality data in the GenAI era: “In the era of GenAI, access to trusted, high-quality data is becoming more essential than ever, especially for RAG implementations. The addition of Informatica’s low-code/no-code CDI-Free solution to Databricks Partner Connect, plus the integration of IDMC with Unity Catalog and new analytical data metadata exploration capabilities for RAG-based GenAI, accelerates access to high-quality data for customers building AI solutions and applications on the Databricks Data Intelligence Platform.”

Databricks enabled KPMG to construct a scalable and robust lake house architecture, supporting data transformation, analytics, and AI workloads to meet the firm’s rising AI requirements, while also lowering data integration complexity and costs. Databricks Assistant’s conversational interface allows KPMG developers to query data using natural language. It then produces SQL or Python queries based on context from code cells, libraries, tags, popular databases, and Unity Catalogue schemas.

Dennis Tally, Managing Director at KPMG, also highlighted the value of this partnership: “At KPMG, we are thrilled about the new product offerings and innovations from Informatica and their inclusion in Databricks Partner Connect. This partnership enables us to leverage cutting-edge technologies from both market leaders to drive innovation and deliver tangible business value not only to our internal IT but also to the solutions offered to our clients. With Informatica’s leading AI-powered data management solutions integrated seamlessly into the Databricks ecosystem, we can accelerate our data-driven initiatives, build trusted AI models, and unlock actionable insights at scale. We look forward to further collaboration with Informatica and Databricks to drive digital transformation and help our clients thrive in today’s data-driven world.”