Tuesday 26 June 2012

Tuning PCoIP


I've been working with a few customers and partners recently on View deployments where we needed to tune PCoIP to particular network scenarios or to cope with limited bandwidth and try and squeeze the best performance out of the available resources. Normally I would start by tuning PCoIP parameters manually using the PCoIP tuning guide but I thought I would document the most common settings and what has worked well for many installations.


The bit to look at in the guide is from page 28 onwards with regard to tuning for the locations or sites with constrained bandwidth: http://www.vmware.com/files/pdf/view/VMware-View-5-PCoIP-Network-Optimization-Guide.pdf


One of the first changes I would have traditionally made was to reduce the frame rate to 15 (you can go considerably lower but test in increments to make sure you are still getting a good user experience) but there are lots more options and considerations that I will cover below.


Back to Basics: Tweak the User Interface Visual Effects

Before we get into tuning the protocol it’s worth starting with the desktop image and ensuring that we are not causing extra screen changes and thus additional traffic.

  • Set Visual Effect to Best Performance
  • Disable Desktop Wallpaper
  • Disable Screen Saver or set it to None
  • Revert back to the classic Start menu
  • Disable Themes (if possible)
  • Disable additional fading
  • System icon and text changes
  • Disable any unnecessary Windows services - Help and Support, Windows Audio (if you don't need sound), Wireless, Remote Registry (be careful, though: some applications need this service, so make sure you properly test), Error Reporting and any other service that is not needed

A good place to start on this is the Optimization Guide for Windows 7. This will help you tune down the visual effects and also disable unnecessary services that would otherwise consume resources. There are even a couple of attached scripts that you can use to make all the changes for you.




Using a GPO

There are a few ways that you can change the PCoIP parameters, but to be clear we are going to make these changes on the virtual desktop(s).
  1. Use a Group Policy (GPO)
    • For local isolated changes to a specific VM, copy this file to c:\windows\inf within the windows VM and implement using your group policy editor.
    • Larger deployments can apply the pcoip.adm to an Organizational Unit (OU), or apply the settings to the template prior to deployment
  1. Apply the setting directly to the registry.

Usually I would recommend using a GPO applied to an OU. The GPO files are located on the View Connection Server in C:\Program Files\VMware\VMware View\Server\Extras\GroupPolicyFiles. Copy them over to somewhere you can get to from your Group Policy Admin tool and add the ones you need into the group policy you are applying to your desktops.





Edit the Group Policy you want to apply these to and expand the Computer Configuration until you see the Administrative Templates. Right click on it and add in the templates you copied over.




If you want to make the changes directly to the registry of the virtual desktop (the master VM may make more sense), you can do so at the following location in the View desktop:
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Teradici\PCoIP\pcoip_admin_defaults\



PCoIP Tuneable Parameters

Build-to-Lossless - pcoip.enable_build_to_lossless

If you want to maximize bandwidth savings vs. lossless image quality, this gives you the option to disable PCoIP build to lossless in favour of build to perceptual lossless. 

When BTL is disabled, PCoIP rapidly builds the client image to a high quality, but lossy image. By default, if the image remains constant, PCoIP would continue to refine the image in the background until it reaches a fully lossless state. Stopping the build process when the image reaches the "perceptually lossless" stage can deliver significant bandwidth savings -- for typical office workflows, we are seeing around a 30% bandwidth reduction.

Be careful when disabling Build-to-Lossless if your users require high image quality or there is a need to provide lossless imaging (e.g. medical, graphic design).



PCoIP Network Parameters

Maximum PCoIP Session Bandwidth (Kbps) - pcoip.max_link_rate

Specifies the maximum bandwidth, in kilobits per second, in a PCoIP session. The bandwidth includes all imaging, audio, virtual channel, USB, and control PCoIP traffic.

When this setting is disabled or not configured on an endpoint, the endpoint imposes no bandwidth constraints. When this setting is configured, the setting is used as the endpoint's maximum bandwidth constraint in kilobits per second.

  • Set the maximum bandwidth on a user to prevent them from impeding other users productivity.
  • Be careful not to set too low since the ability to peak is key to desktop performance


  • Setting applies to the soft host only.
  • Value is in Kbps
  • Ranges between 100 and 1,000,000; and must be set in increments of 100
  • Default value is 900,000.
  • Setting to 0 = no bandwidth constraints

PCoIP Session Bandwidth Floor (Kbps) - pcoip.device_bandwidth_floor

This setting determines the lower bound PCoIP will throttle down to when bandwidth is required but there is congestion detected on the network. PCoIP will still concede bandwidth below this value when it is not needed. The default value is 0, which means that no minimum bandwidth is reserved.


PCoIP session MTU (bytes) - pcoip.mtu_size

Typically, you do not have to change the MTU size. Only change this value if are seeing packet fragmentation due to VPN or other encapsulation.  This setting applies to the server and client. If the two endpoints have different MTU size settings, the lowest size is used.

If this setting is disabled or not configured, the client uses the default value of 1300 bytes in the negotiation with the server.


PCoIP Image Quality Levels

Minimum Image Quality (30-100) - pcoip.minimum_image_quality

This determines the lower bounds of image quality “compression” when network congestion triggers increased build-to-lossless.
The default value here is 50 (from a range of 30 to 100) and in most deployments this value has worked well.
Use this to balance image quality and frame rate for limited-bandwidth scenarios. A lower value allows higher frame-rates, but with a potentially lower quality display. A higher value provides higher image quality, but with potentially lower frame rates when network bandwidth is constrained. When network bandwidth is not constrained, PCoIP maintains maximum quality regardless of this value. Find a good medium that works for you.

Maximum Initial Image Quality (30-100) - pcoip.maximum_initial_image_quality

A lower bound on the image quality that PCoIP tries to deliver “immediately” when screen updates occur. The higher this setting the more “pixel perfect” initial screen updates will be at the cost of higher bandwidth peaks.

The range is 30 to 100 and the default value is 90. In most deployments, values have varied between 70 and 90. This setting behaves as follows:
·         A higher initial image quality means that larger bandwidth bursts will be used when refreshing or updating a larger end-user screen change
·         A lower initial image quality means that less bandwidth bursts will be used when refreshing or updating a larger end-user screen change:


Maximum Frame Rate (1-120) - pcoip.maximum_frame_rate

This setting determines the maximum frequency of client screen updates. Lower values will reduce bandwidth when there are high rates of motion that need to be rendered.
·         A higher value can use more bandwidth but provides less jitter, which allows smoother transitions in changing images such as video.
·         A lower value uses less bandwidth but results in more jitter.

The default value is 30, but you could safely drop this to 15 unless you need an HD movie experience. This setting deals with the frequency of frames, or how many frames per second at which your end-user screen refreshes. Of course, the higher the rate the better the experience; the lower the rate the less data you send across the wire. So, be sure to test:

Audio

PCoIP Audio Policy (1/0) - pcoip.enable_audio

Determines whether audio is enabled in PCoIP sessions. Unless audio is explicitly needed, this setting should be disabled to save significant bandwidth and improve the user experience:

PCoIP Session Audio Bandwidth Limit (Kbps) - pcoip.audio_bandwidth_limit

This setting will limit the maximum bandwidth that audio traffic can consume. PCoIP can still dynamically adjust this setting down based upon current network conditions. Setting this value below 50Kbps may result in no audio being transmitted at all.


PCoIP Encryption Algorithm (1/0)

pcoip.enable_salsa20_256_round12
pcoip.enable_aes128
Some have had better performance setting this to Salsa256 than to AES128, but be sure to run your own tests and validate:


Client-side Cache - pcoip.image_cache_size_mb

This allows you to configure the PCoIP client image cache size (default of 250MB). The client uses image caching to store portions of the display that were previously transmitted and reduce the amount of data that is retransmitted (min 50 MB to a max of 300 MB.)


Network Best Practice


  • PCoIP is a real-time protocol so ensure that it gets proper QoS/CoS classification
    • Classify PCoIP traffic as real-time interactive, typically just below VoIP
    • Insure that QoS/CoS mappings are preserved across WAN links
  • Utilize the View Security Server for remote access as it is the most efficient remote access solution.
  • If you must use VPN, avoid SSL-based solutions. Use IPSEC, L2TP/IPSEC, GRE, DTLS VPN solution that support UDP tunnelling.
  • Insure that PCoIP is bypassed on all WAN acceleration devices.
  • Insure that PCoIP is bypassed or trusted on any IDS/IPS devices in the network path and in endpoint protection software.
  • Prefer fixed bandwidth WAN circuits over “burstable” circuits. Make sure you understand your use case well and perform accurate measurements to allow for proper circuit sizing.
  • If you must use “burstable” circuits insure that the CIR is high enough to cover all existing high priority traffic and the total average traffic for all PCoIP sessions.
    • PCoIP may see high packet loss when it consumes burst bandwidth.
      • Carriers tag burst packets as “out of contract” and low priority.
      • May artificially limit the total bandwidth PCoIP “sees” across the circuit.
  • Utilize WRED for congestion avoidance:
    • Avoid tail-drop.
    • Do not configure WRED on the physical interface as it will override all other QoS policies.
  •  Avoid use cases where round-trip latency is greater than 300ms.
  • Do not utilize per-packet load balancing as this will cause out of order packet delivery leading to PCoIP perceived packet loss.
  • Insure that affinity or session “stickiness” is enabled.


PCoIP Tuning Guidelines

  • Disable Build-to-lossless
    • First and easiest way to shave 10-15% bandwidth.
    • Only enable when there is a defined requirement for pixel perfect accuracy (Medical, CAD/CAM, Graphic Design)
  •  Configure the maximum session bandwidth
    • For low bandwidth links set the limit at or slightly below (10%) the maximum link rate.
    • Even on the LAN it may make sense to apply a limit.
  •  Configure the session floor when:
    • PCoIP is experiencing packet loss but the network link has plenty of headroom
      • May not always improve user experience
    • Packet loss is seen on WiFi or 3/4G networks
    • Be careful to avoid unintentional oversaturation
  • Configure the maximum frame rate
    • In almost all cases the maximum frame rate can be reduced to 18-20fps with little noticeable impact.
    • Settings below 15fps may be noticeable in use cases which require rich media
    • Task workers without media requirements can often utilize settings as low as 6-8fps without significant visual impact
    • Examine the PCoIP Server log files and WMI Image stats to determine average frame rate for desired use case:
  • Configure the maximum initial image quality
    • When on a WAN link with constrained bandwidth reduce this setting to 60-70%
    • For use cases that use large amounts of multimedia/video – large impact
    • Setting this value too low may result in noticeably “fuzzy” or “blurry” images
  • Configure the minimum image quality:
    • This value must be below the maximum initial image quality setting
    • The default value of 50% is acceptable for most cases
  •  Configure the audio bandwidth limit:
    • For use cases that utilize significant amounts of audio - legal/medical transcription for example – reducing audio bandwidth may increase user density
    • Audio bandwidth limit is a target, not a literal value
    • Vary the audio bandwidth limit between 450Kbps – 50Kbps until the desired mix of bandwidth savings and audio intelligibility is achieved
  •  Configure the Client-side cache size:
    • When using thin client devices with limited RAM using a larger cache size than the device can support may lead to dropped sessions
    • Reduce the cache size until connections are unaffected, typically 50-100MB

Summary


  • Always start with the basics before resorting to PCoIP tuning
    • Majority of PCoIP issues are external to the protocol
    • Optimize VDI Base image
    • Insure proper implementation of network configuration QoS/CoS, UDP tunneling through VPN, etc.
    • Proper network sizing for desired use case.
  • Utilize the information in the PCoIP logs and real-time WMI counters to determine where the trouble spots are:
  • Determine proper settings to adjust:
    • Vary one item at a time, make as few changes as possible
    • Test, test, and re-test against the intended use case. Utilize a repeatable set of user actions and/or a scripted set of actions to validate the impact of changes


Monday 25 June 2012

PCoIP Ports and Traffic



[EDIT] I've updated this for View 5.1 which made one change in that the Security Servers and Connection Servers instances negotiate IPSec and respond on UDP 500. (I also made a typo on the USB port).

I’ve been asked by a few people recently what ports need to be opened between different components in a View environment using PCoIP.
All of this information is avaliable in the View Architecture Planning Guide but I have found it useful to boil this down to a couple of simple diagrams.
Internal LAN connections.



External Clients connecting through a PCoIP gateway.


Speeding Up Provisioning Linked Clones in View 5.1


I’m not going to go into detail on why changing the way that View provisions and creates linked clones with Composer is a good idea. If you are running on recent hardware with a well designed solution that provides good disk I/O you can benefit from increasing the number of operations Composer will do concurrently.
Wayne over on IT Blood Pressure covered this for View 5.0 in a Feb blog.
There is information on this is the Best Practices Whitepaper and instruction  on how to change the setting in View 5.0 and previous versions in KB article 1013760
What’s new is that changing this has become easier with View 5.1 as the setting for pae-SVICreationRampFactor can now be made through the View Administrator GUI
Under View Configuration and Servers, Edit your vCenter Server Settings. If you press Edit on the vCenter Server Settings box that appears you will get the following dialog. Change the last setting to increase the provisioning operations concurrency and speed up certain operations.


Change this in increments, test the effect and only do this if your hardware (and Composer) configuration can cope.