VNC Server crashes during single sign-on if different GSSAPI libraries are available on the host computer

Connecting to VNC Server for UNIX with Single sign-on produces an error:

"The connection closed unexpectedly"

VNC Server is in Service Mode is killed and then restarts; in User Mode and Virtual Mode the root helper process is killed and no further authentication attempts succeed.

The problem may be clearly diagnosed by assertion failures in libgssapi_krb5. One of the following errors will be printed to the terminal window before VNC Server is killed:

threads.c:321: krb5int_key_register: Assertion `keynum >= 0 && keynum < K5_KEY_MAX' failed.
threads.c:351: krb5int_key_register: Assertion `destructors_set[keynum] == 0' failed.

There can be problems when two different versions of the Kerberos libraries installed on the machine, for example:

1. In /opt/pbis/lib64 (from the PowerBroker pbis-open package)
2. In /usr/lib64 (from the distribution's krb5 package)

In this case, the linker (ld.so) is loading /opt/pbis/lib64/libgssapi_krb5.so correctly, but loads helper libraries from the wrong directory, so /usr/lib64/libkrb5support.so is used rather than the correct /opt/pbis/lib64/libkrb5support.so. The incompatible libraries cause the application to crash.

If you intend to use libraries that are installed in non-standard locations (eg /opt), the dynamic linker must have been configured to look in those locations (see the manual page for ldconfig). For example, on Linux, run echo /opt/pbis/lib > /etc/ld.so.conf.d/pbis.conf; ldconfig to instruct the linker where to find the PowerBroker single-sign on libraries installed in /opt/pbis/lib.

Alternatively, set the LD_LIBRARY_PATH environment variable to point to the libraries before starting VNC Server. Note however that using environment variables will not affect setuid-root binaries such as vncserver-x11 or Xvnc, so this will not work for standard installations of VNC Server.