by Braxton Koppelman

Introduction

I recently completed a project with a client to build out a brand new, on-premises Exchange 2016 environment that required load balancing with Citrix ADC. As with most enterprise Exchange deployments, a reverse proxy and load balancing solution should be implemented for redundancy and to secure those services. This customer was previously using the Exchange SMTP relay receive connectors to explicitly allow certain hosts to send mail via SMTP relay but wanted to transition to a load balanced configuration.

The Problem

Configuring SMTP load balancing is fairly straightforward on the appliance as we just need to define the Exchange servers in a Service Group and create a TCP load balancing virtual server on TCP port 25 for SMTP. The issue is that with the Citrix ADC in place, the Source IP of all packets is now the Subnet IP (SNIP) of the Citrix ADC. From a security perspective, we don’t want to just configure the SMTP Relay connector in Exchange to allow the SNIP as all clients would be able to use the SMTP relay feature of Exchange. With that being the case, how do we accomplish filtering of this traffic to only allow specific devices to send SMTP relay now that the source IP is that of the Citrix ADC SNIP? We assessed a couple of options and were presented with the following requirements from the customer:

1. The solution should provide the same granular SMTP relay filtering provided by Exchange.
2. No architectural or networking changes to the Citrix ADC.
3. New IP’s and subnets could be easily added to the relay by their Level 1 support staff.

Reviewing Potential Options

When looking at passing the client IP address to a backend server with the Citrix ADC there are a couple options, but we have to also consider the Layer 7 protocol used here which is SMTP. The Citrix ADC has a feature that allows the client’s IP address to be forwarded via an HTTP header but since we’re load balancing SMTP and not HTTP, this is not an option. With that said, we had some options we could use which had some drawbacks:

1. Configure Use Source IP (USIP) Mode on the Citrix ADC Appliance which will forward packets to the backend Exchange server with the Source IP of the client. This process is documented within Citrix Docs here but there are also a couple of drawbacks with this scenario.
   1. By enabling this mode, the client-to-server reuse ratio is compromised because the appliance cannot reuse the connections for other clients.
   2. Tweaking of the timeout values is required based on the service being load balanced.
2. Configure the Direct Server Return feature in conjunction with USIP mode on the Citrix ADC which routes the return traffic directly back to the client instead of through the Citrix ADC appliance. This process is actually documented in a great article here (credit to Dan Schlimme on the CUGC blog) but this approach wouldn’t be able to meet the customers requirements based on the following:
   1. A new relay server would need to be deployed, maintained, monitored, etc.
   2. Networking changes are required on the Relay Server to route traffic properly
   3. This configuration requires USIP mode, so the drawbacks noted in the previous option apply here as well.

Again, these options work just fine but there are some additional architectural and performance considerations along with the fact that USIP mode is not the most efficient method of passing the client IP to the backend servers. With this being the case, I normally try to avoid USIP mode if possible. There has to be a better way!

The Solution

Another feature of the Citrix ADC that can be leveraged to allow for filtering of traffic is the Responder feature. This allows us to create an action like a TCP Reset or HTTP Redirect to be applied when certain criteria are met within a policy expression. This alone could be used to meet this requirement if we’re only allowing a small number of IP addresses. We can filter for a specific IP address in the following expression:

add responder policy Filter_SMTP “!CLIENT.IP.SRC.EQ(192.168.1.100)” RESET

If you need to add more IP’s, you could simply create an OR statement in the expression:

add responder policy Filter_SMTP “!(CLIENT.IP.SRC.EQ(192.168.1.100) || CLIENT.IP.SRC.EQ(192.168.1.100))” RESET

The first policy will drop any traffic sent to the SMTP load balancing virtual server unless it’s sourced from 192.168.1.100 and the second will allow both 192.168.1.100 and 192.168.1.101. This works just fine, but what if we need to add 100 or even 1,000 SMTP relay hosts or what if we want to add a whole subnet? This specific approach isn’t very scalable as it would be very cumbersome to maintain. This specific customer we worked with had over 1200+ relay hosts defined in Exchange, so this is where using a simple Responder policy falls short.

Enter the Citrix ADC pattern sets and policy expressions. First, with the pattern set, we can define all of our IP Addresses and Subnets that we need to allow for SMTP relay. Second, we then build a policy expression which allows us to take multiple Citrix ADC expressions (referencing our pattern set in this case) and aggregate them into a single expression that can be referenced within the Responder policy.

Now that we’ve determined how we can meet the requirements, let’s take a look at the configuration. First, the load balancing virtual server will need to be created and this guide assumes that this is already in place.

Pattern Set Configuration:

First, we need to create a pattern set:

add policy patset Allow_SMTP_IPs

Next, we need to define the IP addresses and subnets within the pattern set. One thing to note is that we’ll be doing this with a specific syntax: 0.0.0.0/32. We need to add the mask bits in there since we need to use the policy expression to look for certain subnet masks. This is what gives us the ability to define both individual IP Address AND subnets within the same pattern set. The commands would be as follows:

bind policy patset Allow_SMTP_IPs “10.1.1.10/32” -index 1 -charset ASCII

bind policy patset Allow_SMTP_IPs “192.168.160.0/22” -index 2 -charset ASCII

bind policy patset Allow_SMTP_IPs “10.1.10.0/24” -index 3 -charset ASCII

bind policy patset Allow_SMTP_IPs “10.10.0.0/16” -index 4 -charset ASCII

There are a couple of things to note here:

1. Make sure that the index values are incremented properly when running this via CLI.
2. The charset should be ASCII

Now that we have that defined, we can reference it in a policy expression.

Policy Expression:

Now we need to define a policy expression to look through our pattern set and allow traffic based on the IP Address of subnet that’s sourcing SMTP traffic as shown below.

add policy expression SMTP_Whitelist “!((CLIENT.IP.SRC + \”/32\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(31) + \”/31\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(30) + \”/30\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(29) + \”/29\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(28) + \”/28\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(27) + \”/27\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(26) + \”/26\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(25) + \”/25\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(24) + \”/24\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(23) + \”/23\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(22) + \”/22\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(21) + \”/21\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(20) + \”/20\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(19) + \”/19\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(18) + \”/18\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(17) + \”/17\”).EQUALS_ANY(\”Allow_SMTP_IPs\”) || (CLIENT.IP.SRC.SUBNET(16) + \”/16\”).EQUALS_ANY(\”Allow_SMTP_IPs\”))”

This policy expression looks at the subnet mask bits from /32 all the way down to /16. This means that we can define an IP Address or subnets with masks spanning from /32 (IP Address) all the way down to a /16 subnet. This should cover most environments but if not, simply add another OR statement with the mask bits you require. Please also note that in each statement, the ‘Allow_SMTP_Whitelist’ Pattern Set we created earlier is referenced.

Responder Policy:

The final configuration that’s required on the Citrix ADC is to create a Responder policy and bind it to the SMTP virtual server. The following commands will need to be run.

add responder policy POL_RES_Allow_SMTP SMTP_Whitelist RESET

bind lb vserver LB_SMTP_25 -policyName POL_RES_Allow_SMTP -priority 100 -gotoPriorityExpression END -type REQUEST

As you can see within the Responder policy, we only need to reference the policy expression we created earlier which is ‘SMTP_Whitelist’. Once this is completed, we add the SNIP to the SMTP relay connector in Exchange so that only traffic sourced from the NetScaler will be allowed.

Summary

After all of the configuration is in place, we can see the Citrix ADC sending a TCP RESET packet back to the client during a packet capture if their IP is not defined in the pattern set. Over time, there will be a need to add more IP Addresses and/or subnets to be allowed for SMTP relay. In order to maintain this, we simply need to add additional entries into the Pattern Set and they’ll be immediately allowed. This is also a simple task and with a little bit of documentation, Level 1 support resources have the ability to make these changes as needed. 

Hopefully this was helpful and this configuration can be applied to Responder policies for other use cases outside of SMTP. If you have questions or would like some further clarification, feel free to reach out to me via the following:

Twitter: https://twitter.com/bkoppelman86
LinkedIn: https://www.linkedin.com/in/braxtonk86/

Acknowledgements
I would like to thank my colleague Alex Bouzykanov for assisting with this configuration and content. I would also like to thank Dan Schlimme for his post on the CUGC Blogs detailing the DSR method of configuring SMTP Load Balancing on the Citrix ADC.