Using Python to work with Azure Storage

In this blog I will show you a very useful package in python that has come in very handy for me over the past few months while interacting with Azure blob storage for programmatically uploading, downloading and listing blobs from a container in an Azure storage account. For this I am using the azure.storage.blob package.

On windows use the command prompt to fire a pip install command to install the azure.storage.blob package in your python environment.

pip install azure.storage.blob

For this to work make sure you the below version or higher. To find the version run the code below in a python notebook

import os, uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, version
try:
print("Azure Blob storage v" + version + " - Python Azure blob sample")
except Exception as ex:
print('Exception:')
print(ex)

Now make sure you setup your environment variables with the connection string your Azure storage account. You can also use Azure key vaults for authentication instead of windows environment variables. I prefer Azure key vault and if you have not done that before I will be creating a blog post showing how you can use key vaults for authentication in python.

connection string example

DefaultEndpointsProtocol=https;AccountName=yourazurestorageaccountname;AccountKey=youraccountkey;EndpointSuffix=core.windows.net

fire the command below in command prompt to set the environment variable

setx AZURE_STORAGE_CONNECTION_STRING "DefaultEndpointsProtocol=https;AccountName=yourazurestorageaccountname;AccountKey=youraccountkey;EndpointSuffix=core.windows.net"

back in your python notebook you should be able to connect to your storage account using the code below

#Create the BlobServiceClient object which will be used to create a container client
connect_str = os.getenv('AZURE_STORAGE_CONNECTION_STRING')
blob_service_client = BlobServiceClient.from_connection_string(connect_str)
container_name = "yourcontainername"
#Create a blob client using the local file name as the name for the blob
local_path = "c\temp"
local_file_name = "quickstart" + str(uuid.uuid4()) + ".txt"
blob_client = blob_service_client.get_blob_client(container=container_name, blob=local_file_name)

use code below to list all containers

containers = blob_service_client.list_containers()
for c in containers:
print(c.name)

upload local blob

local_path = "./"
local_file_name = "your local file name"
upload_file_path = os.path.join(local_path, local_file_name)
#Create a blob client using the local file name as the name for the blob
blob_client = blob_service_client.get_blob_client(container=container_name, blob=local_file_name)
#Instantiate a ContainerClient
container_client = blob_service_client.get_container_client("containername")
with open(upload_file_path, "rb") as data:
blob_client.upload_blob(data)

List blobs in a container

blob_list = container_client.list_blobs()
print("\n----------List of blobs in the container ----------")
for blob in blob_list:
print("----> " + blob.name)

Download blob

local_path = "./"
download_file_path = os.path.join(local_path, str.replace(local_file_name ,'.txt', 'DOWNLOAD.txt'))
print("\nDownloading blob to \n\t" + download_file_path)
with open(download_file_path, "wb") as download_file:
download_file.write(blob_client.download_blob().readall())