by Owen Reynolds, CTA

Watch Now: “Best Practices for Sustainable Citrix Implementation” with Owen Reynolds.

I recently returned as a consultant for a client that I was working for in 2020, and it’s inspired this piece. The reason being, I realized I’d left some items in place that should have been removed and had confused the local staff.

This is a common scenario; where you are the trusted expert for an implementation as an SME, but don’t fully hand off to other people on your team so they can continue to support your implementation when you’re gone (or on vacation).

The BUS TEST

Around 2014, I was working for a large financial company that was going full ITIL for their change management process. I was already familiar with change records, but the full ITIL compliance introduced a lot of extra steps that I found cumbersome. At one point, I got annoyed by having my change records rejected by a change managment person, so we decided to talk on the phone rather than bounce emails back and forth. He told me why my change records weren’t being approved: they didn’t pass the “BUS TEST.” He said: 

“Owen, if you were hit by a BUS before the implementation start date, and were laid up in the hospital, would anyone else on your team be able to complete the change on your behalf?” 

I had to answer “NO.” The instructions I included were written by me, for me, and were certainly not usable by anyone else. I went back, and re-wrote the instructions from an outsider’s perspective, and my change was approved. To this day, I follow the “BUS TEST” model for writing change MGMT records as well as documentation for clients / co-workers / blog posts. 

With that MORBID intro out of the way, here are some common items of consideration for a successful & sustainable Citrix implementation:

The dirty dozen (and then some)

• Scheduled tasks that run under a non-service account: Don’t put in your OWN DOMAIN ID! I had to do this once, but I set a reminder for a future date to re-visit and set to a proper shared group service account.
  
• Scheduled tasks that don’t contain any info on what they do via the description field, or link to external doc. When I used to work for a big financial institution years back, I’d host the accompanying docs on SharePoint, and link them in the description field.
  
  
• PowerShell code: Other than the mighty Guy Leech, there are few of us who are formally trained in programming, but, we are writing code whenever we open Powershell ISE or MS Visual Studio Code to create PowerShell code.
  
  I’m NOT an expert on PowerShell code, I’d consider myself “intermediate.” One thing I’m good at, is documenting! Document your code inside the script, or in greater detail via the README on github. Don’t assume everyone understands your code. People will run into an error, not report it (especially if they don’t know you) and never try to use it again. This happened to me with a health check script I wrote last year. V1 required more exit statements at the start for restricted environments, and I only caught it when someone working for the same company as me reported it. The Internet is like space: no one can hear you scream. People won’t report issues and will move on to the next thing they find by search engine search. Generally speaking, you should assume that any code you write for a client, is code that you “own” and ultimately have some level of responsibility should it stop working as expected for the clients who continue to use it.
• Related to the above point ; scripts that don’t include a synopsis / header to explain their function, or, for more complex scripts, no accompanying documentation in wiki/SharePoint/OneNote/etc. Create a README on your git hub or at least include what the script does in contents of the script
  
• If you’ve got the time, it’s a good idea to leave the client with a basic means to health check the Citrix / hosting resources you setup for them: Shameless plug, I’ve got two scripts that work for this exact purpose: One is fully documented on my personal blog here, and another for health checking vCenter ESXi environments is on my git hub here . Both of these scripts require very little config to run, and give you a quick overview of your environment that you can once, or as a scheduled task 
• GPO’s and OU’s that were used for testing called “TEST,” “TST” that don’t contain any active items. If they ain’t used, delete ’em!
  
• A GPO for each setting/settings group: This comes down to preference, I suppose, but for me, I like to have large sets of common EUC-related settings in just two GPOs: One for computer, one for user-side settings. Having lots of GPOs can be confusing if they aren’t labeled, and can adversely impact logon times as each GPO must be processed at user logon / computer start-up
  
• Not commenting your GPOs/GPP entries: With Citrix, there are obscure registry keys that affect user experience that aren’t well documented, CTX hooks for instance! Always use the comments field in your GPO/GPP settings to link to the blog/KB where you got it, and explain why you are using it. If it’s only supposed to be in place temporarily , make a note that it’s to be removed at date “MM/DD/YYYY”
  
• Use of un-supported settings/registry keys/etc. that were in place to resolve an issue, that were later resolved by a patch from the vendor. I’ll admit, this one isn’t easy to track. I’m open to ideas on how you’ve re-visited old registry keys/GPO settings for clients you’ve set up.
  
• For work-arounds you provide to fix symptomatic issues, keep in mind, “quick and dirty” can get dirtier as time goes on. Ever forgotten a banana peel behind your trash can? I had a job once years ago, where PRPs (problem resolution procedures) that had beginning / end date! When the end date came up, you had to evaluate if the work-around was still valid, and justify renewal of the PRP. This was a great process, as you really shouldn’t be using a work-around for more than a few months, right!?
  
• Citrix products that introduce extra infra components / agents: WEM is a popular component that provides an alternative to the normal method of applying user settings to VDA sessions: GPO/GPPs. However, if you aren’t using the performance optimization features of the WEM agent, the use of WEM to control your user experience such as resetting desktop icons/mapped drives/printers introduces two caveats: 
  
  1: On-site staff who are already familiar with GPOs/GPPs will need to learn the new WEM console mgmt interface.
  
  2: The WEM agent is another binary to update in your golden image / platform layer. There is no LTSR for WEM, and each release introduces a risk of a new “bug” that can be user experience impacting.
  
  3: If you’re using WEM on-prem, you’ll have to maintain an SQL instance as well as the windows server instance where your WEM instance is installed. If/when either of these components suffers an outage, the WEM agents will go into “cache” mode, and you won’t be able to apply new settings to your WEM agents.
  
  For my clients that have engaged me in 2020/2021, and already have GPOs in place to control the base required user experience, I’m recommending they only use WEM via Citrix Cloud for it’s “killer feature,” which is its CPU/memory management.
  
• App-layering: The inspiration for this presentation came from a project I was leading in 2019 to migrate a client on a Win 7 VDI with PvD to Win 10. The original plan was to deploy app-layering + user layers as “like for like” replacement for the depreciated PvD feature. The POC using Citrix App layering was successful, but process docs we had written for the client were so complex / long, I knew the client would be challenged to continue to support the implementation once I was gone. Instead, we went with a solution that had ALL the key apps installed on one golden image, and used Microsoft FSLogix app masking to selectively hide/un-hide apps based on AD group membership. The project was a complete success. We were able to complete on-time, despite staff shortages due to covid budget restraints. 
   
• PVS vs MCS: I started using PVS in 2016, and MCS in 2019. I’m familiar enough with both products, and am happy to work with either, but the administrative overhead of PVS is significantly higher with PVS. If your client’s local staff aren’t dedicated to Citrix, which is often the case, PVS is not going to be the right choice for them. As well, MCS provides on-prem clients a path to Citrix Cloud, PVS as of the time of this writing does not
  
• Documentation: Ask anyone who’s worked with me in the past 20 years, I’m BIG on documentation. Part of it is my own personal preference ;  I actually use Microsoft OneNote to plan smart phone purchases , apartment moves, dog names, IT cert study, etc. So, when it comes to client work, I’m going to probably give you MORE than you need 🙂
  
  I prefer to create “living documents” as I work in Microsoft OneNote and then transfer to TEAMS WIKI once the docs are finalized for client hand-off.
  
  I’ve created two scenarios, one where you have lots of time towards the end of a project for documentation, the other where you’ve got very little time
  
  Short amount of time
  -AD Group > Delivery group mapping
  -Golden image virtual machine names > AD machine account mapping
  -Inventory of on-prem Citrix components and service account names
  -Storefront internal / external URLs
  
  Greater amounts of time
  -All of the above items
  -MCS / PVS / App Layering image update process
  -Full documentation on scheduled tasks
  -Full documentation on user migration scripts
  -Architecture diagrams / connection flows
  -Problem resolution procedures for the helpdesk 
  

The above list is culled from my work as a Citrix consultant here in Montreal, Quebec, Canada and from my past experience working for large financial companies. Your own experience will vary.

For me, the key to a successful implementation isn’t just about checking off the all the project requirements, but ensuring that the platform you deliver will continue to work a day after you’re gone, and a year later!

Here are some good litmus tests you can use to gauge staff acceptance during the course of the project implementation:

• If you’ve already written process docs, have the local staff read/reviewed/understood what’s written? Are they capable of completing basic changes when you’re not gone? Sick days, holidays, etc.
  
• How many emails do you get asking questions you had previously answered?  
  
• Have you ever been called on a day off for assistance on your implementation? 

If any of the above come up with any regularity, you might have created a monster that only YOU can tame. The client will call you back, your sales folks might be happy as you’ve increased billable hours, but your professional rep (and your company brand) may take a hit

Watch the corresponding webinar here!