Citrix performance monitoring – Quick guide

Citrix Server is a virtualization solution that enables central hosting of applications and resources. There are multiple components in a Citrix virtualization infrastructure, but globally it is available in two main editions – XenApp and XenDesktop.

• Citrix XenApp – Enables seamless application delivery to all platforms by isolating applications from the underlying OS and streaming them to the target device where they are executed. Basically, the application user sends keystrokes and mouse clicks to the server and receives screenshot updates. There is no need for installation or configuration on client servers.
• Citrix XenDesktop – Enables publishing of complete virtual desktops from a hypervisor to remote client devices. That way every user gets his own instance of the desktop without sharing any resources with other users.

To put it simple, Citrix Servers deliver applications to Windows, Mac, and Linux desktops without any installation or application compatibility problems.

Why and how to monitor Citrix performance?

Well, first of all – poor Citrix performance can have a major impact on your business. How? Symptoms of poor Citrix performance are sudden session disconnections, slow logins, slow application startups, screen freezes, and general usage lag. That greatly affects user productivity, and even worse, symptoms can affect all users within a Citrix XenApp farm.

The problem with identifying poor Citrix performance causes lies in the different Citrix layers that are mutually dependable and therefore a problem on any layer can disrupt the functionality of another layer making the entire Citrix environment unstable.

If a user is experiencing any of the aforementioned symptoms, it is most likely that the performance of the entire Citrix service is bad, and not only that of a specific XenApp/XenDesktop server. Performance issues can be related to the virtualization platform, network, storage, Active Directory along with many other parts of your environment.

For that reason, you should track metrics in your environment such as user logon times, network bandwidth, used applications, system resources, various Citrix performance counters, and crucial Citrix services.

Common Citrix performance issues and how to prevent them

We can group the common Citrix performance issue causes by their symptoms:

Slow logons / logon delays

Cause:

Poor performance of External File Server

Description:

Roaming user profiles allow users with a computer joined to a Windows Server domain to log on to any computer on the same network and access their documents. If an external file server is holding roaming user profiles it can cause significant problems while accessing Group Policies and Profile Data, therefore making the logon slow. An even bigger problem is that all users who have their roaming profiles on that server are bound to experience the same problems.

Solution:

Detect the events with “Srv” source in the System Event Log with Error code “2022” and troubleshoot servers. Edit the server’s Windows registry value for Max/Min Free Connections as instructed here.

Cause:

Master FSMO as Domain Controller/Global Catalogue

Description:

Global Catalogue servers can cause significant logon delays as group memberships are referenced and permissions are established from the Active Directory. The first Domain Controller in the domain is always automatically configured as a Global Catalogue, and therefore it shouldn’t act as a Master FSMO. In case of a multi-rolled Domain Controller, the LDAP queries can’t be quickly accessed and that can cause problems.

Solution:

Deploy dedicated Domain Controllers for Global Catalogue and Master FSMO roles.

Cause:

Bad Profile Management

Description:

Profile management is very important for optimizing user logon times. More importantly, slow logons will affect the global Citrix performance due to the excessive load that logging on places on your system.

Solution:

Use pre-configured (mandatory) profiles for all users with all the unnecessary stuff removed until the size is reduced to around 256K. Also, make sure the profiles are cached locally on Citrix servers and deleted once you log off.

Slow application performance

Cause:

Antivirus Configuration

Description:

Antivirus software can disrupt optimal application performance with unnecessary scanning. All such software should be installed and configured correctly for specific Citrix Server in order to ensure that there is no performance overhead.

Solution:

Temporarily disable all anti-virus components such as on-access scanners and the application buffer overflow protections to detect the problem source. After that, configure the anti-virus software to:

• Scan on write events only
• Scan local drives only
• Exclude the Page File from being scanned
• Exclude the Program FilesCitrix folder from being scanned since it contains the local host cache and Resource Manager local database
• Exclude the user‘s XenApp Plugin bitmap cache and the XenApp Plugin folders

Cause:

Session Latency

Description:

Problems with session latency can be traced back to network issues such as bad network topology, port mismatches, inadequate Maximum transmission unit size, or network saturation due to the bad Quality of service rules.

Solution:

Track session latency using the WMI performance counters for ICA Session that are installed when Citrix is installed on a Windows Server and resolve detected problems internally.

Cause:

Speed Screen Configuration

Description:

Speed Screen Configuration is often overlooked in the Citrix Server settings and it can lead to slow response when entering text into published applications.

Solution:

Configure Speed Screen and replicate those settings across the server farm.

Global poor performance

Cause:

Shared Resources

Description:

The problem with shared resources is that they are shared, and you are most likely using it for SharePoint, Exchange or something else, but the bottom line is, it can’t guarantee optimal Citrix performance. You always need to be in control over your resources, and they have to be dedicated to specific tasks.

Solution:

Utilize local storage for desktop provision to ensure the optimal Citrix performance. You can keep the critical core services such as controllers on the SAN shared since they generally do not affect the performance.

Cause:

Not Using Session Sharing

Description:

Using Session Sharing means that every user will only use one session for all published applications without the need to open multiple sessions. That way a user will always launch applications on their current session. This means there is no need for a user to be logged on to multiple servers and to go through the whole logon process at every application start. It will reduce server load and improve the overall performance for all users.

Solution:

Publish all applications with the identical sessions settings (color depth, sound settings, etc.).

Cause:

File Server

Description:

File Server configuration and deployment is directly connected to the Citrix servers in your environment. Monitoring the File Server performance counters such as disk queue lengths is important because bad values can significantly affect Citrix performance.

Solution:

Make sure that File Servers used for Citrix Profiles are in the same subnet as the Citrix Session servers. That way they don’t need to traverse the gateway. When using DFS make sure that the DFS server subnets are defined in Active Directory Sites and Services.

Tips on improving overall Citrix performance

Smart capacity planning

Citrix Server capacity planning is in fact very straightforward and basically a prerequisite to successfully running the Citrix environment. Let’s run some numbers that will guarantee optimal Citrix performance (these numbers can differ from environment to environment):

XenApp

• 6GB RAM per Session Server
• 400MB RAM for each session
• No more than 14 users per CPU Core with hyperthreading
• 25 Disk IOPS per session
• 100 IOPS during logon
• 100 IOPS during logoff

 XenDesktop

• 3GB RAM per Session Desktop
• No more than 4 users per CPU Core with hyperthreading
• 25 Disk IOPS per session
• 200 IOPS during logon
• 200 IOPS during logoff

Adapter advanced settings

On your Session HostsVirtual Delivery Agents you should configure the settings as follows to receive the maximum performance from them:

• IPv4 Checksum Offload = Disabled
• IPv4 TSO Offload = Disabled
• Large Send Offload V2 (IPv4) = Disabled
• Large Send Offload V2 (IPv6) = Disabled
• Offload IP Options = Disabled
• Offload tagged traffic = Disabled
• Receive Side Scaling = Disabled
• TCP Checksum Offload (IPv4) = Disabled
• TCP Checksum Offload (IPv6) = Disabled
• UDP Checksum Offload (IPv4) = Disabled
• UDP Checksum Offload (IPv6) = Disabled

Network Provider Order

On your Session HostsVirtual Delivery Agents ensure that the Citrix Client Network is first in the Provider Order. You can access it by navigating to Control Panel -> Network Connections -> press “Alt” -> Advanced -> Advanced Settings. This will require a reboot.

Which Citrix performance metrics and services to monitor?

Citrix advises the following services to be closely monitored. Depending on your Citrix environment, you will not necessary have all services below running, but in case a crucial service for your environment stops, you will have a major problem.

Service Name	Executable	Description
Citrix AD Identity Service	Citrix.ADIdentity.exe	Manages Active Directory Computer Accounts.
Citrix Analytics	Citrix.Analytics.exe	Collects analytical data on Citrix products.
Citrix Broker Service	BrokerService.exe	Brokers connections from endpoint devices to desktops and applications.
Citrix Configuration Logging Service	Citrix.ConfigurationLogging.exe	Logs Administrator activity and configuration changes in a XenDesktop deployment.
Citrix Configuration Service	Citrix.Configuration.exe	Stores configuration information on the Citrix Services.
Citrix Delegated Administration Service	Citrix.DelegatedAdmin.exe	Manages configuration of delegated administration permissions.
Citrix Diagnostic Facility COM Server	CdfSvc.exe	Manages and controls Citrix diagnostic trace sessions.
Citrix Environment Test Service	Citrix.EnvTest.exe	Manages tests for evaluating the state of a XenDesktop Site.
Citrix Host Service	Citrix.Host.exe	Manages hosts and hypervisor connections.
Citrix Machine Creation Service	Citrix.MachineCreation.exe	Creates new virtual machines.
Citrix Monitor Service	Citrix.Monitor.exe	This service monitors the FlexCast system.
Citrix Storefront Service	Citrix.Storefront.exe	Manages deployment of Storefront.

Citrix advises the following performance counters to be closely monitored. Most thresholds need to be configured according to your environment, however the following counters should be monitored 24/7.

Object	Counter	Explanation	Threshold	Monitor On
Processor	%Processor TimeAll instances	The primary indicator of processor activity, it displays the average percentage of busy time	Alert on 80% sustained for 1 minute or longer	All XML brokers, Most Data Collectors, Web Interface / IIS server, Server hosting Data Store
Memory	Available MBytes	The amount of physical memory available on the computer	Alert if less than 15% of RAM sustained for 1 minute	All XML brokers, Most Data Collectors, Web Interface / IIS server, Server hosting Data Store
Logical Disk	Free Megabytes	The unallocated space on the disk drive in megabytes	This threshold should be determined based on the size of the drives and duties of the server.	Server hosting Data Store, All Presentation Servers, Web Interface / IIS server
Citrix MetaFrame Presentation Server	Application Resolution Time (ms)	Measures the time required to resolve the Least-Loaded Server during an application launch	A baseline needs to be established to account for increases during peak logon times.	All XML brokers
Citrix MetaFrame Presentation Server	Data Store Connection Failure	The number of minutes that the Presentation Server has been disconnected from the data store	Threshold should consider events such as reboots and scheduled maintenance	All Presentation servers
Citrix MetaFrame Presentation Server	Number of busy XML threads	The number of XML requests that are currently being processed. The maximum at any time is 16.	Alert on a count of 16 sustained for 1 minute or longer	All XML brokers
Citrix MetaFrame Presentation Server	WorkItem Queue Ready Count	The number of work items that are ready and waiting to be processed by IMA	Alert if this counter is sustained above 0 for a minute	All XML brokers, Most Data Collectors
Citrix MetaFrame Presentation Server	Resolution WorkItem Queue Ready Count	The number of work items (related to application launches) waiting to be processed by IMA	Alert if this counter is sustained above 0 for 1 minute	All XML brokers, Most Data Collectors
Citrix MetaFrame Presentation Server	Zone Elections Won	Amount of times this server has won an election	Alert if this counter increases by 2 in 1 hour	Most Data Collectors
ASP.NET	Zone Elections Triggered	Amount of times server has called for an election. It is incremented if a new Zone Data Collector was elected or not.	Alert if this counter increases by 2 in 1 hour	All Presentation servers
ASP.NET	Requests Queued	The number of requests waiting to be processed by ASP	A baseline would need to be established in your environment to calculate accurate thresholds (accounting for peak logon times)	Web Interface / IIS server
ASP.NET	Requests Rejected	The number of requests rejected because the request queue was full	Alert if this counter increases by 1	Web Interface / IIS server

 

Citrix performance monitoring tool – Syskit Monitor

To summarize, Citrix environments are very complex and one issue can quickly become a global problem for all users. This is why you need a reliable tool to make the process of Citrix monitoring successful. Meet Syskit Monitor – our powerful Citrix monitoring tool!

Syskit Monitor is an agentless tool that allows you to monitor Citrix servers by using Windows API so you don’t need to install a third-party software on your Citrix servers. All your Citrix farms are monitored from a single console, and you can easily separate many servers into logical server groups if you have e.g. UAT, DEV, or production farms.

With Syskit Monitor, you can monitor all Citrix performance counters and services, in real time and historically. If any Citrix service stops running, Syskit Monitor will restart it and inform you what happened. Also, if any Citrix performance counter crosses its threshold, Syskit Monitor will alert you immediately. To make performance monitoring ever easier, we provide you with monitoring templates for different aspects of Citrix environment, helping you track your Citrix servers health even if you are not sure what to monitor.

Other than performance, Syskit Monitor enables you to track user activities by monitoring logon activity and time spent in different session states, as well as tracking the application usage across all servers. You can also document your entire Citrix environment and keep track of Citrix licensing.

If you have any questions be sure to contact us.