by Uddave Jajoo, CTA & Indianapolis CUGC Leader

In 2021, I was asked to migrate the vGPU enabled VMs from one platform to another as our customer was migrating from VMWare Horizon to Citrix XenDesktops. I did not have enough knowledge about NVIDIA and how the GPU processing work in highly intensive graphics processing applications.

I took help from community members and support teams (both Citrix and NVIDIA) and deployed the solution to provision NVIDIA vGPU-enabled Citrix VDIs on vSphere. This blog post was created 2 years ago but was not published that time. However, I am happy to publish it now to the community so that others could also benefit from it.

(See also: Configuring NVIDIA vGPU VMs in Azure with Citrix DaaS.)

Below are the overall configuration steps covering deployment of all the major infrastructure and other components.

1. NVIDIA vGPU License Server Deployment
2. Pre-Requisites Open JDK installation
3. NVIDIA License Server Installation
4. Access License Server management Interface
5. Creating license server on NVIDIA portal
6. Creating VDI Master Image (NVIDIA Driver Installation and vGPU profile)
7. VDA installation and Machine Catalog creation
8. Configuring Virtual Machine to enable vGPU profiles

NVIDIA vGPU License Server Deployment: Open JDK JRE Prerequisite

1. Install OpenJDK JRE on the server as prerequisite component before installing vGPU License server component
2. Download java-1.8.0-openjdk-1.8.0.252-2.b09.ojdkbuild.windows.x86_64.msi

3. Run the MSI installer and proceed with the installation wizard
4. Click Next
5. Click Next and Install
6. Monitor the installation and click Finish

7. Configure system environment variable on the server pointing to jdk location.
   • Edit the System Environment Variables.
   • Add the location of the bin folder of the JDK installation to the PATH variable in System Variables.
   • The following is a typical value for the PATH variable: C:\WINDOWS\system32;C:\WINDOWS;”C:\Program Files\Java\jdk-11\bin”

8. Set JAVA_HOME: 
   • Under System Variables, click New. 
   • Enter the variable name as JAVA_HOME. 
   • Enter the variable value as the installation path of the JDK (without the bin sub-folder). 
   • Click OK. 

9. Configure the PATH environment to enable to run JAVA from a command prompt
   • Select the System variables “Path”
   • Click Edit
   • Click on New
   • Type “%JAVA_HOME%\bin” 
   • Click OK.
   • And OK again to apply changes.

10. Verification:  
    • Change to the Java directory 
    • Type java.exe -version 

NVIDIA Legacy License Server Installation

Legacy License server is set to EOL by July 2023, hence NVIDIA offers two different methods for provisioning license server. (DLS) On Premise and Cloud(CLS). In this blog, I am going to cover how to setup a CLS based license server. Steps are very simple and described properly in the NVIDIA documentation as well.

License System User Guide – NVIDIA Docs – Converting Legacy NVIDIA vGPU Software License Servers to NLS License Servers

1. Right click the setup file and select run as administrator to proceed with configuration of license server

2. Open administrative command prompt and navigate to the setup folder and launch setup.exe
3. When prompted select allow the file to launch, select Allow this file and click OK.
4. Wait for the setup file to launch and navigate through the wizard.
5. Click Next in the Introduction Tab
6. Click Next for EULA
7. Click Next for Accepting the Apache License

8. Select the location as default and proceed further with installation, click Next.
9. In the Choose Firewall Options dialog box, select the ports to be opened in the firewall.
10. To enable remote clients to access licenses from the server and prevent remote access to the management interface, use the default setting, which sets ports as follows:
    • ‣ Port 7070 is open to enable remote clients to access licenses from the server.
    • ‣ Port 8080 is closed to ensure that the management interface is available only through a web browser running locally on the license server host.
11. Click Install and proceed further with the installation process.

12. Wait for the License Server installation script to execute
13. Click Done once installation is complete.

Access License Server Management Interface

1. In a web browser, visit the home page for license server management interface HTTP: http://localhost:8080/licserver
2. If administrative security is enabled for the license server, log in to the license server.
   • In the license server management interface, select Login.
   • In the Login page that is displayed, enter your user name and password for logging in to the license server and click Authorize.

The default credentials for the license server administrator account are as follows:
‣ admin
Admin@123

3. Record the license server’s MAC address, by launching the license server management console and navigating to configuration
   MAC Address – 00-XX-XX-XX-XX-XX-00 Note to change the MAC address binding on vecneter level
4. Add Environment variable on the license server as below, this is needed to ensure the nvidialsadmin utility commands run perfectly fine.
   Variable Name:- FLEXNETLS_BASEURL
   Variable Value:- http://localhost:7070/api/1.0/instances/~
5. Reboot the server
6. Enable Administrative security for license server
   Perform this task in a command shell on the license server host.
   Set the property security.enabled to true.
   nvidialsadmin -config -set security.enabled=true Updated the property:security.enabled with value:true successfully

7. Authenticating with license server
   To provide your password in the command, specify the credentials as follows:
   
   nvidialsadmin -authorize username password
   
   In any subsequent nvidialsadmin command, users must provide the required credentials for authenticating with the license server
8. Run status check on the license server
   nvidialsadmin -authorize admin Admin@123 -status

Creating a License server on the NVIDIA Portal

1. Login to NVIDIA Dashboard link to register the license server on portal
   https://nvid.nvidia.com/dashboard/#/dashboard
2. Login with the Organization user credentials and create License Server

3. On the NVIDIA Licensing Portal dashboard, click CREATE LICENSE SERVER.
4. If you are adding a license server to an organization or virtual group for which a license server has already been created, click CREATE SERVER.
5. Enter the details as below:
   ServerName: TestNVIDIA
   Description: License Server for NVIDIA vGPU
   MAC Address: 00-XX-XX-XX-XX-XX-00
   Select product – Virtual PC 2.0-10 Licenses
   Click Add Click create license server

6. Verify the License server details along with the available license displayed in portal

7. Download the License File

8. Login to License server and upload the license file to the server.
9. Navigate to Licensed Feature Usage
   Verify the details for the License added to the console.

Creating VDI Master Image (NVIDIA Driver Installation and vGPU profile)

For vGPU card installation on vSphere or hypervisor please refer to the links: https://blogs.vmware.com/apps/2018/09/using-gpus-with-virtual-machines-on-vsphere-part-3-installing-the-nvidia-grid-technology.html

https://virtualizationreview.com/articles/2015/05/29/how-to-install-an-nvidia-vgpu-in-esxi-hosts.aspx

1. Create a new VM and install NVIDIA and VDA agent.
2. In our case virtual machine was cloned from existing WIN10 Master image.
3. Uninstall VDA from the image.
4. Shutdown the machine
5. Navigate to VMware vcenter, and locate the virtual machine.
6. Click on Edit settings and add Shared PCI Device, Click Add and select the vGPU profile and click Reserve all Memory
7. The New Shared PCI device with vGPU profile will show as configured
8. Select Ok to complete the configuration

9. Power on the virtual machine and monitor through remote console.
10. Login as administrator
11. Open Device Manager, it will show Microsoft Basic Display Adapter with exclamation mark, which is normal
    
    Caution:
    NVIDIA vGPU vSphere VIB version and NVIDIA driver for Windows version need to match
    
12. Locate the media for NVIDIA driver and copy to local C Drive
13. Right click on the NVIDIA driver setup file and run as administrator
14. Select Yes to allow the UAC
15. Leave the extraction path as default and click OK.

16. In the installation wizard, select Agree and Continue and proceed further
17. Installation options select, Custom (advanced), it would let you select components to install as needed.
18. Click Next, in the next window, it will display the list of components which will be install on the image.

19. Select the check box to Perform a Clean Installation and select Next.

20. Wait for the installation to complete and monitor the progress.

21. Select restart now on successful installation of NVIDIA Driver.
    
    After restarting, the mouse cursor may not track properly using VNC or vSphere console. If so, use Remote Desktop.

22. RDP to the virtual machine and verify the device manager configuration.
23. Upon login to machine you may receive a prompt that NVIDIA license is not present. Ignore the alert for now.
24. In order to validate the successful installation of the graphics drivers as well as the vGPU device, open Windows Device Manager and expand the Display Adapter section.
25. It will show NVIDIA GRID M60-8Q adapter.’
26. New Device NVDIA GRID V100D-8Q, it will show up as this in the display adapters

27. Shutdown the virtual machine and remove the Shared PCI device added on the master image.
28. Right click machine, edit settings
29. Click on the x mark against the Shared PCI device
30. It will show as device will be removed, click OK.
31. The recent Task Pane will display the status of completion once the device will be removed. A XenDesktop machine catalog can be safely configured now.

VDA installation and Machine Catalog creation

1. RDP to the virtual machine, copy the media to C:\Support
2. Run the AutoSelect.exe file
3. It will detect the VDA to be installed in Single Session OS by default.
4. Select the option for VDA in single session OS and click Next.

5. In the next screen select Create a Master MCS image, click Next.
6. VDA will be selected by default, click Next.

7. In the component screen, select the option, Citrix Supportability Tools, Citrix User profile manager and Citrix User profile manager_WMI Plugin. Click Next.

8. In the controller’s Tab, select configure later and click Next.
9. In the Features Tab, select the options- Optimize Performance, use windows Remote Assistance, Enable Real Time Audio Transport for audio and MCS IO, click Next.

10. In the Firewall Tab, leave the default and click Next.

11. In the Summary Tab, review the selected configuration and proceed with Install. Click Install.

12. Uncheck the box and click Next.
13. Click Finish and Restart the machine, it will reboot.
14. Login to the machine again to resume the VDA installation process.

15. RDP to the machine and run Citrix Optimizer against the respective windows OS version template.
16. Shut down the master image and take snapshot for the virtual machine.
17. Login to Citrix studio and create machine catalog using the snapshot taken in above step.
18. Navigate to the Machine Catalog Tab and select create Machine catalog, in the wizard select Next.
19. In the next screen select Single Session OS and click Next.
20. Select the options as highlighted and click Next.
21. Check the option for Machines that are Power Managed
    Check Citrix MCS and select the resources corresponding to Cluster (ESXi Grid Server, where vGPU Cards and driver was configured) and click Next.

22. Select the options as highlighted and click Next.
23. In the Master Image, select the Snapshot created for the master image.
    Ex – NVIDATESTVM_SNAP
24. Select the minimum functional level for this catalog as 2206 or newer.
25. In the Network cards tab, keep the VLAN selected and change if needed.

26. In the Virtual Machines page, select the number of virtual machine and allocate the memory:
    2 Machines with 32 GB RAM

27. Create the Active directory machine accounts for the Desktops
28. Name the Machine Catalog with Description:
    TEST-MC-NVIDA-vGPU-Pool
29. Monitor the Catalog creation process in studio and vcenter.

Configuring Virtual Machine to enable vGPU profiles

1. Assign vGPU profiles as per the requirement
2. Assign B series profile to the provisioned desktops for allocating Virtual PC

3. Assign a series profile to the provisioned desktops for allocating virtual Apps.

Reference:
License System User Guide – NVIDIA Docs – Converting Legacy NVIDIA vGPU Software License Servers to NLS License Servers

https://blogs.vmware.com/apps/2018/09/using-gpus-with-virtual-machines-on-vsphere-part-3-installing-the-nvidia-grid-technology.html

https://virtualizationreview.com/articles/2015/05/29/how-to-install-an-nvidia-vgpu-in-esxi-hosts.aspx

License System User Guide – NVIDIA Docs

Legacy License server is set to EOL by July 2023

See More Recent Posts:

• Testing the New Teams in Citrix VDI
• Citrix WEM Database Migration
• On-Prem New and Cool
• Recap: CUGC Great Plains XL 2023
• How to Migrate GPOs from Microsoft to Citrix WEM