cugcblogs

Azure IoT and Citrix NetScaler

by Ingmar Verheij, Citrix, CTP

Earlier I wrote an article on Citrix Blogs on how a NetScaler can help scale and secure an IoT data stream. For organisations who host their IoT service, this is key for their success. After all, you don’t want your solution to get compromised! 

But what if you want to use Azure IoT instead of building your own IoT service? That makes perfect sense to me; it’s a mature IoT stack which not only features the IoT Hub but also brings machine learning and other capabilities to create a complete solution. Another great capability of Azure IoT is the Azure IoT Edge (v2 in tech preview at the time of writing), which enables organisations to connect devices at the Edge instead of the Cloud. And–this is the best part in my humble opinion–you can process data at the edge before you forward it to the Cloud (if ever).

Azure IoT is capable of communicating via a number of protocols, including HTSSP, AMQP and SMQTT. What’s interesting is that the Azure IoT has implemented a subset of the MQTT stack; it supports Device-to-Cloud (D2C) and Cloud-to-Device (C2D) but not Device-to-Device (D2D). Meaning; a device can only subscribe to messages published by the IoT Hub, not from other devices as you would with a “normal” MQTT broker. But what’s more interesting for this article, they only support the secure implementation of MQTT. A good decision if you ask me, you shouldn’t communicate anything over a non-secure connection to a Cloud service.

If we introduce the Azure IoT Edge component–which will probably end up somewhere in a datacenter or (factory building)–does this still apply? I think there are numerous devices which only talk the non-secure version of MQTT. Though you can resolve this with a custom module, I doubt you should (for such a simple reason).  

As mentioned before, I feel the IoT Edge component is essential if your devices should process data at the edge, before reaching the Cloud. This could be due to performance reasons (eg. shutdown a valve when an anomaly is detected in the sensor readings) but also due to compliancy reasons (eg. personally identifiable information [PII]). Using modules in the IoT Edge runtime, this can be handled.

Now what if we bring the Azure IoT Edge to a production site, how do we handle resiliency? There’s little information known about the scalability of the Azure IoT Edge, but what’s clear is that one equals none. Meaning; we need at least two to provide a high available set. So, we need a Load Balancer capable of handling (S)MQTT.

Just like the IoT Hub the IoT Edge only accepts SMQTT, which requires a TLS between the Device and the Edge. This requires quite some compute power–not to mention the administrative efforts–for which a CPU isn’t optimized. A Reverse Proxy capable of offloading the SSL handshake does improve scalability.

Lastly, some (proprietary) devices aren’t able to communicate the secure version of MQTT. If the devices are in a controlled network segment (yes, you should really segment all your IoT devices) this might be acceptable. A Reverse Proxy capable of translating MQTT into SMQTT would be a great solution.

What you see here is an example configuration where all IoT Devices communicate using the same connection string (single point of access); a Citrix NetScaler. The NetScaler has two endpoints: MQTT and SMQTT (with a CA signed certificate). The NetScaler in its turn communicates with the Azure IoT Edges (plural) over SMQTT. In my example, they’ve got the default self-signed certificates on them, this reduces the administrative overhead. 

Three obvious benefits of a Citrix NetScaler with Azure IoT:

  • Create a resilient pool of Azure IoT Edge servers;
  • Offload SSL transactions;
  • Enable non-secure MQTT.

And I can probably sum a whole lot more, but I won’t. What I will do is share with you how to you can set this up yourselves.

Requirements

Before we get to the dirty details, we need to ensure we have the tools at hand to test our setup. To test the publish and subscribe actions, I’ll be using the client tools provided by mosquitto on my Ubuntu 16.04 server.

We’ll also need some information about your Azure IoT environment. What we need is:

  • [IoTHub] – The name of your Azure IoT Hub (this makes up [IoTHub].azure-devices.net). You can find this in the Overview section of your IoT Hub in Azure Portal;
  • [DeviceID] – The ID of a device we’ll be testing with. Create a new device in the IoT Devices section of your IoT Hub and give it an ID (I named mine MosquittoDevice). You can create the Device Explorer for this too (see next line);
  • [SharedAccessSignature] – The SharedAccessSignature (SAS) of the device (!) you just created. This isn’t available through the Azure Portal, you’ll need the Device Explorer to create a SAS for the device. TIP: Set the TTL longer than 0;
  • Example: SharedAccessSignature sr=.azure-devices.net%2Fdevices%2F&sig=xxxxxx&se=xxxxxx
  • [AzureEdgeFQDN1] – The FQDN to your first Azure Edge server;
  • [AzureEdgeFQDN2] – The FQDN to your second Azure Edge server;
  • [NetScalerFQDN] – The FQDN where your devices will connect to your NetScaler;
  • [NetScalerVIP] – The Virtual IP on the NetScaler where your devices connect to;
  • [NetScalerCertKeyName] – The name of your SSL server.

You also need the mqtt.lua file (see docs.citrix.com) in /var/download/extensions on your NetScaler (I’m using FileZilla on my Mac).

Setup

With all the requirements in place we can start with configuring the Citrix NetScaler. Open up the CLI and issue the following commands; 

import ns extension local:mqtt.lua mqtt_code
add user protocol MQTT -transport TCP -extension mqtt_code
add user protocol MQTT_SSL -transport SSL -extension mqtt_code
add server AzureIOTEdge1 [AzureEdgeFQDN1]
add server AzureIOTEdge2 [AzureEdgeFQDN2]
add service lb_service_azureiot1 AzureIOTEdge1 USER_SSL_TCP 8883
add service lb_service_azureiot2 AzureIOTEdge2 USER_SSL_TCP 8883
add lb vs lb_vserver_azureiot USER_SSL_TCP
bind lb vs lb_vserver_azureiot lb_service_azureiot1
bind lb vs lb_vserver_azureiot lb_service_azureiot2
bind ssl vserver lb_vserver_azureiot -certkeyName [NetScalerCertKeyName]
set ssl vserver lb_vserver_azureiot -ssl3 DISABLED
add user vs u_vserver_azureiot MQTT [NetScalerVIP] 1883 -defaultlb lb_vserver_azureiot
add user vs u_vserver_azureiot_ssl MQTT_SSL [NetScalerVIP] 8883 -defaultlb lb_vserver_azureiot
bind ssl vserver u_vserver_azureiot_ssl -certkeyName [NetScalerCertKeyName]
set ssl vserver u_vserver_azureiot_ssl -ssl3 DISABLED

Let’s take a look at the steps

import ns extension local:mqtt.lua mqtt_code
add user protocol MQTT -transport TCP -extension mqtt_code
add user protocol MQTT_SSL -transport SSL -extension mqtt_code

This creates a protocol extension with the name mqtt_code in your NetScaler, as described here.

add server AzureIOTEdge1 [AzureEdgeFQDN1]
add server AzureIOTEdge1 [AzureEdgeFQDN2] 
add service lb_service_azureiot1 AzureIOTEdge1 USER_SSL_TCP 8883
add service lb_service_azureiot2 AzureIOTEdge2 USER_SSL_TCP 8883

We then add server objects pointing to the Azure IoT Edge servers and two service objects, referring to the Azure IoT Servers, on port 8883 (the port the Azure IoT Edge is listening SMQTT).

add lb vs lb_vserver_azureiot USER_SSL_TCP
bind lb vs lb_vserver_azureiot lb_service_azureiot1
bind lb vs lb_vserver_azureiot lb_service_azureiot2
bind ssl vserver lb_vserver_azureiot -certkeyName [NetScalerCertKeyName]
set ssl vserver lb_vserver_azureiot -ssl3 DISABLED

Next, we create a load balancing virtual server and add bind both Azure IoT Servers. We add the SSL certificate and disable SSLv3.

add user vs u_vserver_azureiot MQTT [NetScalerVIP] 1883 -defaultlb lb_vserver_azureiot

We then create a user virtual server for the non-secure MQTT, which listens on the specified Virtual IP and port 1883.

add user vs u_vserver_azureiot_ssl MQTT_SSL [NetScalerVIP] 8883 -defaultlb lb_vserver_azureiot
bind ssl vserver u_vserver_azureiot_ssl -certkeyName [NetScalerCertKeyName]
set ssl vserver u_vserver_azureiot_ssl -ssl3 DISABLED

Last, we create another user virtual server for the secure MQTT, which listens on the specified Virtual IP and port 8883.

Don’t forget to save your config!

Testing

Now that everything is set up, we can start testing by subscribing and publishing messages to the Azure IoT hub, via the Azure IoT Hub. We skip the NetScaler portion for now, we need to make sure this part works first.

I’ve been testing on the same host as where my Azure IoT Edge is installed, so the self-signed (CA) certificates are on the same machine. Before we can use them, we need to create symbolic links to them using c_rehash.

Issue the following command

c_rehash /var/lib/azure-iot-edge/certs/edge-chain-ca/cert

We can now test publishing a message using the mosquitto client:

mosquitto_pub -d -h

[AzureEdgeFQDN1]

 -i

[DeviceID]

 -u "

[AzureEdgeFQDN1]

.azure-devices.net/

[DeviceID]

 /DeviceClientType=azure-iot-device%2F1.2.2&api-version=2017-06-30" -P "

[SharedAccessSignature]

" -t "devices/

[DeviceID]

/messages/devicebound/#" -m "Hello World!" -t "devices/

[DeviceID]

/messages/events/readpipe/" -p 8883 -V mqttv311 --capath /var/lib/azure-iot-edge/certs/edge-chain-ca/cert 

What you should see is: 

Client MosquittoDevice sending CONNECT
Client MosquittoDevice received CONNACK
Client MosquittoDevice sending PUBLISH (d0, q0, r0, m1, 'devices/MosquittoDevice/messages/events/readpipe/', ... (12 bytes))
Client MosquittoDevice sending DISCONNECT

If you want to test subscribing to a message:

mosquitto_sub -d -h

[AzureEdgeFQDN1]

 -i

[DeviceID]

 -u "

[AzureEdgeFQDN1]

.azure-devices.net/

[DeviceID]

 /DeviceClientType=azure-iot-device%2F1.2.2&api-version=2017-06-30" -P "

[SharedAccessSignature]

" -t "devices/

[DeviceID]

/messages/devicebound/#" -p 8883 -V mqttv311 --capath /var/lib/azure-iot-edge/certs/edge-chain-ca/cert

You should see:

Client MosquittoDevice sending CONNECT
Client MosquittoDevice received CONNACK
Client MosquittoDevice sending SUBSCRIBE (Mid: 1, Topic: devices/MosquittoDevice/messages/devicebound/#, QoS: 0)
Client MosquittoDevice received SUBACK
Subscribed (mid: 1): 0

In the Device Explorer, you can go to the Message To Device tab and select your device. Type “Hello World!” at Message and press Send. You should see the message come in.

.

NetScaler

Now, let start testing with the NetScaler! Since we use a CA signed certificate we can use the regular CA certificates on the device (in /etc/ssl/certs).

We’ll issue the same commands as before BUT we change the FQDN from the Azure IoT Hub to the NetScaler and, as mentioned, we change the certificate path:

Publish

mosquitto_pub -d -h

[NetScalerFQDN]

 -i

[DeviceID]

 -u "

[AzureEdgeFQDN1]

.azure-devices.net/

[DeviceID]

 /DeviceClientType=azure-iot-device%2F1.2.2&api-version=2017-06-30" -P "

[SharedAccessSignature]

" -t "devices/

[DeviceID]

/messages/devicebound/#" -m "Hello World!" -t "devices/

[DeviceID]

/messages/events/readpipe/" -p 8883 -V mqttv311 --capath /etc/ssl/certs

Subscribe

mosquitto_sub -d -h

[NetScalerFQDN]

 -i

[DeviceID]

 -u "

[AzureEdgeFQDN1]

.azure-devices.net/

[DeviceID]

 /DeviceClientType=azure-iot-device%2F1.2.2&api-version=2017-06-30" -P "

[SharedAccessSignature]

" -t "devices/

[DeviceID]

/messages/devicebound/#" -p 8883 -V mqttv311 --capath /etc/ssl/certs

And that’s it!

Cheers,

Ingmar

Leave a Reply