Other Citrix performance monitoring – Quick guide August 17, 2017 By: Silvio Rahle 9 min read This is a quick guide on Citrix performance monitoring. Check out powerful Citrix performance monitoring tool - Syskit Monitor. Table of contents Why and how to monitor Citrix performance?Common Citrix performance issues and how to prevent themTips on improving overall Citrix performanceWhich Citrix performance metrics and services to monitor?Citrix performance monitoring tool - Syskit Monitor Citrix Server is a virtualization solution that enables central hosting of applications and resources. There are multiple components in a Citrix virtualization infrastructure, but globally it is available in two main editions – XenApp and XenDesktop. Citrix XenApp – Enables seamless application delivery to all platforms by isolating applications from the underlying OS and streaming them to the target device where they are executed. Basically, the application user sends keystrokes and mouse clicks to the server and receives screenshot updates. There is no need for installation or configuration on client servers. Citrix XenDesktop – Enables publishing of complete virtual desktops from a hypervisor to remote client devices. That way every user gets his own instance of the desktop without sharing any resources with other users. To put it simple, Citrix Servers deliver applications to Windows, Mac, and Linux desktops without any installation or application compatibility problems. Why and how to monitor Citrix performance? Well, first of all – poor Citrix performance can have a major impact on your business. How? Symptoms of poor Citrix performance are sudden session disconnections, slow logins, slow application startups, screen freezes, and general usage lag. That greatly affects user productivity, and even worse, symptoms can affect all users within a Citrix XenApp farm. The problem with identifying poor Citrix performance causes lies in the different Citrix layers that are mutually dependable and therefore a problem on any layer can disrupt the functionality of another layer making the entire Citrix environment unstable. If a user is experiencing any of the aforementioned symptoms, it is most likely that the performance of the entire Citrix service is bad, and not only that of a specific XenApp/XenDesktop server. Performance issues can be related to the virtualization platform, network, storage, Active Directory along with many other parts of your environment. For that reason, you should track metrics in your environment such as user logon times, network bandwidth, used applications, system resources, various Citrix performance counters, and crucial Citrix services. Common Citrix performance issues and how to prevent them We can group the common Citrix performance issue causes by their symptoms: Slow logons / logon delays Cause: Poor performance of External File Server Description: Roaming user profiles allow users with a computer joined to a Windows Server domain to log on to any computer on the same network and access their documents. If an external file server is holding roaming user profiles it can cause significant problems while accessing Group Policies and Profile Data, therefore making the logon slow. An even bigger problem is that all users who have their roaming profiles on that server are bound to experience the same problems. Solution: Detect the events with “Srv” source in the System Event Log with Error code “2022” and troubleshoot servers. Edit the server’s Windows registry value for Max/Min Free Connections as instructed here. Cause: Master FSMO as Domain Controller/Global Catalogue Description: Global Catalogue servers can cause significant logon delays as group memberships are referenced and permissions are established from the Active Directory. The first Domain Controller in the domain is always automatically configured as a Global Catalogue, and therefore it shouldn’t act as a Master FSMO. In case of a multi-rolled Domain Controller, the LDAP queries can’t be quickly accessed and that can cause problems. Solution: Deploy dedicated Domain Controllers for Global Catalogue and Master FSMO roles. Cause: Bad Profile Management Description: Profile management is very important for optimizing user logon times. More importantly, slow logons will affect the global Citrix performance due to the excessive load that logging on places on your system. Solution: Use pre-configured (mandatory) profiles for all users with all the unnecessary stuff removed until the size is reduced to around 256K. Also, make sure the profiles are cached locally on Citrix servers and deleted once you log off. Slow application performance Cause: Antivirus Configuration Description: Antivirus software can disrupt optimal application performance with unnecessary scanning. All such software should be installed and configured correctly for specific Citrix Server in order to ensure that there is no performance overhead. Solution: Temporarily disable all anti-virus components such as on-access scanners and the application buffer overflow protections to detect the problem source. After that, configure the anti-virus software to: Scan on write events only Scan local drives only Exclude the Page File from being scanned Exclude the Program FilesCitrix folder from being scanned since it contains the local host cache and Resource Manager local database Exclude the user‘s XenApp Plugin bitmap cache and the XenApp Plugin folders Cause: Session Latency Description: Problems with session latency can be traced back to network issues such as bad network topology, port mismatches, inadequate Maximum transmission unit size, or network saturation due to the bad Quality of service rules. Solution: Track session latency using the WMI performance counters for ICA Session that are installed when Citrix is installed on a Windows Server and resolve detected problems internally. Cause: Speed Screen Configuration Description: Speed Screen Configuration is often overlooked in the Citrix Server settings and it can lead to slow response when entering text into published applications. Solution: Configure Speed Screen and replicate those settings across the server farm. Global poor performance Cause: Shared Resources Description: The problem with shared resources is that they are shared, and you are most likely using it for SharePoint, Exchange or something else, but the bottom line is, it can’t guarantee optimal Citrix performance. You always need to be in control over your resources, and they have to be dedicated to specific tasks. Solution: Utilize local storage for desktop provision to ensure the optimal Citrix performance. You can keep the critical core services such as controllers on the SAN shared since they generally do not affect the performance. Cause: Not Using Session Sharing Description: Using Session Sharing means that every user will only use one session for all published applications without the need to open multiple sessions. That way a user will always launch applications on their current session. This means there is no need for a user to be logged on to multiple servers and to go through the whole logon process at every application start. It will reduce server load and improve the overall performance for all users. Solution: Publish all applications with the identical sessions settings (color depth, sound settings, etc.). Cause: File Server Description: File Server configuration and deployment is directly connected to the Citrix servers in your environment. Monitoring the File Server performance counters such as disk queue lengths is important because bad values can significantly affect Citrix performance. Solution: Make sure that File Servers used for Citrix Profiles are in the same subnet as the Citrix Session servers. That way they don’t need to traverse the gateway. When using DFS make sure that the DFS server subnets are defined in Active Directory Sites and Services. Tips on improving overall Citrix performance Smart capacity planning Citrix Server capacity planning is in fact very straightforward and basically a prerequisite to successfully running the Citrix environment. Let’s run some numbers that will guarantee optimal Citrix performance (these numbers can differ from environment to environment): XenApp 6GB RAM per Session Server 400MB RAM for each session No more than 14 users per CPU Core with hyperthreading 25 Disk IOPS per session 100 IOPS during logon 100 IOPS during logoff XenDesktop 3GB RAM per Session Desktop No more than 4 users per CPU Core with hyperthreading 25 Disk IOPS per session 200 IOPS during logon 200 IOPS during logoff Adapter advanced settings On your Session HostsVirtual Delivery Agents you should configure the settings as follows to receive the maximum performance from them: IPv4 Checksum Offload = Disabled IPv4 TSO Offload = Disabled Large Send Offload V2 (IPv4) = Disabled Large Send Offload V2 (IPv6) = Disabled Offload IP Options = Disabled Offload tagged traffic = Disabled Receive Side Scaling = Disabled TCP Checksum Offload (IPv4) = Disabled TCP Checksum Offload (IPv6) = Disabled UDP Checksum Offload (IPv4) = Disabled UDP Checksum Offload (IPv6) = Disabled Network Provider Order On your Session HostsVirtual Delivery Agents ensure that the Citrix Client Network is first in the Provider Order. You can access it by navigating to Control Panel -> Network Connections -> press “Alt” -> Advanced -> Advanced Settings. This will require a reboot. Which Citrix performance metrics and services to monitor? Citrix advises the following services to be closely monitored. Depending on your Citrix environment, you will not necessary have all services below running, but in case a crucial service for your environment stops, you will have a major problem. Service Name Executable Description Citrix AD Identity Service Citrix.ADIdentity.exe Manages Active Directory Computer Accounts. Citrix Analytics Citrix.Analytics.exe Collects analytical data on Citrix products. Citrix Broker Service BrokerService.exe Brokers connections from endpoint devices to desktops and applications. Citrix Configuration Logging Service Citrix.ConfigurationLogging.exe Logs Administrator activity and configuration changes in a XenDesktop deployment. Citrix Configuration Service Citrix.Configuration.exe Stores configuration information on the Citrix Services. Citrix Delegated Administration Service Citrix.DelegatedAdmin.exe Manages configuration of delegated administration permissions. Citrix Diagnostic Facility COM Server CdfSvc.exe Manages and controls Citrix diagnostic trace sessions. Citrix Environment Test Service Citrix.EnvTest.exe Manages tests for evaluating the state of a XenDesktop Site. Citrix Host Service Citrix.Host.exe Manages hosts and hypervisor connections. Citrix Machine Creation Service Citrix.MachineCreation.exe Creates new virtual machines. Citrix Monitor Service Citrix.Monitor.exe This service monitors the FlexCast system. Citrix Storefront Service Citrix.Storefront.exe Manages deployment of Storefront. Citrix advises the following performance counters to be closely monitored. Most thresholds need to be configured according to your environment, however the following counters should be monitored 24/7. Object Counter Explanation Threshold Monitor On Processor %Processor TimeAll instances The primary indicator of processor activity, it displays the average percentage of busy time Alert on 80% sustained for 1 minute or longer All XML brokers, Most Data Collectors, Web Interface / IIS server, Server hosting Data Store Memory Available MBytes The amount of physical memory available on the computer Alert if less than 15% of RAM sustained for 1 minute All XML brokers, Most Data Collectors, Web Interface / IIS server, Server hosting Data Store Logical Disk Free Megabytes The unallocated space on the disk drive in megabytes This threshold should be determined based on the size of the drives and duties of the server. Server hosting Data Store, All Presentation Servers, Web Interface / IIS server Citrix MetaFrame Presentation Server Application Resolution Time (ms) Measures the time required to resolve the Least-Loaded Server during an application launch A baseline needs to be established to account for increases during peak logon times. All XML brokers Citrix MetaFrame Presentation Server Data Store Connection Failure The number of minutes that the Presentation Server has been disconnected from the data store Threshold should consider events such as reboots and scheduled maintenance All Presentation servers Citrix MetaFrame Presentation Server Number of busy XML threads The number of XML requests that are currently being processed. The maximum at any time is 16. Alert on a count of 16 sustained for 1 minute or longer All XML brokers Citrix MetaFrame Presentation Server WorkItem Queue Ready Count The number of work items that are ready and waiting to be processed by IMA Alert if this counter is sustained above 0 for a minute All XML brokers, Most Data Collectors Citrix MetaFrame Presentation Server Resolution WorkItem Queue Ready Count The number of work items (related to application launches) waiting to be processed by IMA Alert if this counter is sustained above 0 for 1 minute All XML brokers, Most Data Collectors Citrix MetaFrame Presentation Server Zone Elections Won Amount of times this server has won an election Alert if this counter increases by 2 in 1 hour Most Data Collectors ASP.NET Zone Elections Triggered Amount of times server has called for an election. It is incremented if a new Zone Data Collector was elected or not. Alert if this counter increases by 2 in 1 hour All Presentation servers ASP.NET Requests Queued The number of requests waiting to be processed by ASP A baseline would need to be established in your environment to calculate accurate thresholds (accounting for peak logon times) Web Interface / IIS server ASP.NET Requests Rejected The number of requests rejected because the request queue was full Alert if this counter increases by 1 Web Interface / IIS server Citrix performance monitoring tool – Syskit Monitor To summarize, Citrix environments are very complex and one issue can quickly become a global problem for all users. This is why you need a reliable tool to make the process of Citrix monitoring successful. Meet Syskit Monitor – our powerful Citrix monitoring tool! Syskit Monitor is an agentless tool that allows you to monitor Citrix servers by using Windows API so you don’t need to install a third-party software on your Citrix servers. All your Citrix farms are monitored from a single console, and you can easily separate many servers into logical server groups if you have e.g. UAT, DEV, or production farms. With Syskit Monitor, you can monitor all Citrix performance counters and services, in real time and historically. If any Citrix service stops running, Syskit Monitor will restart it and inform you what happened. Also, if any Citrix performance counter crosses its threshold, Syskit Monitor will alert you immediately. To make performance monitoring ever easier, we provide you with monitoring templates for different aspects of Citrix environment, helping you track your Citrix servers health even if you are not sure what to monitor. Other than performance, Syskit Monitor enables you to track user activities by monitoring logon activity and time spent in different session states, as well as tracking the application usage across all servers. You can also document your entire Citrix environment and keep track of Citrix licensing. If you have any questions be sure to contact us. Get deeper insight into server performance and user activity It only takes 3 minutes to install and you have 30 days to see why thousands of system admins have chosen Syskit Monitor to monitor servers, users, and applications! Try for free Related Posts Other SharePoint 2013 search topologies explained In SharePoint 2013, there are numerous components in Search Topology and due to… April 17, 2014 6 min read Other Upgrading from Classic to Modern SharePoint Who says once you move into the cloud that upgrades and migrations can’t be fun… May 15, 2019 10 min read Other How to boost your server security with Cryptolocker prevention In this blog, we go over the best practices on how to secure your domain and co… March 29, 2017 10 min read