Data Vault 101

Data Vault Architecture Introduction The data vault is typically described as a hybrid of traditional star schema and third-normal-form (3NF) warehouse architectures. The data vault architecture addresses several of shortcomings of 3NF like rigidity and easily breaks with change or hard to add additional data sources. Unlike a star schema design, a data vault architecture normalizes andContinue reading “Data Vault 101”

Azure Synapse vs Snowflake – Which Data Platform is best for you?

Every day we generate 2.5 quintillion bytes. Almost all organizations are placing a high emphasis on building out their technology infrastructure to handle the ever-increasing challenges with data volume, velocity and variety. Now more than ever, companies need a way to collect and consolidate data into a single platform to derive insights quickly and efficiently.Continue reading “Azure Synapse vs Snowflake – Which Data Platform is best for you?”

Configure Single Sign On for Snowflake with Azure AD

In this article, I will show you how to integrate Snowflake with Azure Active Directory (Azure AD). When you integrate Snowflake with Azure AD, you can: Control in Azure AD who has access to Snowflake. Enable your users to be automatically signed-in to Snowflake with their Azure AD accounts. Manage your accounts in one centralContinue reading “Configure Single Sign On for Snowflake with Azure AD”

Azure Storage and Snowflake DW Integration

Snowflake is one of the most popular cloud data warehouses today . Snowflake has a number of features that make it an amazing cloud data warehouse to work with for data engineers. Snowflake leverages several new Azure features, including limitless storage accounts, accelerated networking, and storage soft delete. The goal of Snowflake is to provideContinue reading “Azure Storage and Snowflake DW Integration”

Using Azure Synapse Notebooks to Connect to Relational Databases

Apache Spark is a parallel processing framework that supports in-memory processing to boost the performance of big-data analytic applications. Apache Spark in Azure Synapse Analytics is one of Microsoft’s implementations of Apache Spark in the cloud. Azure Synapse makes it easy to create and configure a serverless Apache Spark pool in Azure. Spark pools inContinue reading “Using Azure Synapse Notebooks to Connect to Relational Databases”

Working with Azure Storage and Azure Functions

Azure Function – Blob Trigger Azure Function is a cloud service available on-demand that provides all the continually updated infrastructure and resources needed to run your application. The Blob storage trigger starts a function when a new or updated blob is detected. The blob contents are provided as input to the function. You focus on the pieces ofContinue reading “Working with Azure Storage and Azure Functions”

Working with Snowflake DW using SnowSQL

Snowflake’s Data Cloud is powered by an advanced data platform provided as Software-as-a-Service (SaaS). Snowflake enables data storage, processing, and analytic solutions that are faster, easier to use, and far more flexible than traditional offerings. The Snowflake data platform is not built on any existing database technology or “big data” software platforms such as Hadoop.Continue reading “Working with Snowflake DW using SnowSQL”

Link Azure ML workspace and Azure Databricks workspace

Azure Machine Learning gives us a workbench to manage the end-to-end Machine Learning lifecycle that can be used by coding & non-coding data scientists Databricks gives us a scalable compute environment: if we want to run a big data machine learning job, it should run on Databricks In this article, we will look at how Databricks canContinue reading “Link Azure ML workspace and Azure Databricks workspace”

Text Analytics with Azure Cognitive Services

When it comes to extracting valuable insights from unstructured data, I always fall back on Azure cognitive services. This gives a very strong set of build in AI capabilities to easily analyze text and classify unstructured data seamlessly. The Text Analytics API is a cloud-based service that provides Natural Language Processing (NLP) features for textContinue reading “Text Analytics with Azure Cognitive Services”