How To SetUp Linux Nvidia T4 GPU VM on Azure Part1
Recently, I needed to provision on T4 GPU on Azure, I had searched various methods and online resource, but I was not able to find the correct way to enable GPU on the Azure platform. The create Virtual Machine interface releases images and linux distros that either come with CUDA 10.1 when we are currently at CUDA 12.2 and it just never works… until now
For AI engineers, the AIOps cycle runs to the point where all experimentation is done and it’s time to go live on production.
At this point, setting up GPUs involves a series of crucial steps to ensure a smooth and successful deloyment. Many of such service that enables this is Azure, Google Cloud Platform, AWS Web Service and others. This comprehensive guide provides detailed instructions along with the necessary commands for setting up a GPU enable Linux VM on Azure.
GPUs significantly elevate computational capabilities for compute intensive tasks such as AI/ML, scientific simulations, and more. Proper setup is essential to ensure both optimal performance and reliability.
Setting up the infrastructure on Azure
first things first, ensure you have an account with microsoft and setup an account on https://portal.azure.com
When successfully signed, you should have the image below:
Select Virtual machines
Next you will need to click the Create dropdown and select the first option.
You will need to select your resourceGroup and provide the VM name
Here now, we will need to select a special image, specifically for Nvidia
If you do not have the Nvidia GPU-Optimized VMI-v24.03.4-x64 Gen2 image from your dropdown. select See all images and search for it in the Marketplace.
Next, you will need to choose a Size, the codename for Nvidia T4 GPU instance is Standard_NC4as_T4_* (* implying there can be different version per vCPU config)
If you do not have the NC4as size in your dropdown you need to click See all sizes and search for it.
Most times it will be unavailable, so you need to check different regions one after the other until you find it.
When you do, you may likely need to request for it using Request quota
Ensure to set your GPU keys and username here
Adding the Extension
very important, you need to add the Nvidia Driver extension to make the GPU preinstall one.
Click Select an extension to install and search for Nvidia GPU Driver Extension
select it and then, Review and Create the VM instance.
By following these steps, along with the provided snapshots, you can confidently set up a linux GPUs in your Azure environment.
Hope that was useful! Do subscribe to get a notification whenever a new article of mine comes out.
Thanks reading!🚀
