You see a gray screen if the desktop environment of the remote computer requires capabilities that are not available to the X server built-in to RealVNC Server in Virtual Mode (Xvnc).

This is the case for the default desktop environments used by Ubuntu 13.04+ and CentOS/RHEL 7+.

Enable SystemXorg

From RealVNC Server 6.2.0 onwards we recommend configuring RealVNC Server in Virtual Mode to work with the system Xorg server, rather than the outdated version built in to Xvnc. More desktop environments (such as Gnome 3), applications and extensions will be compatible out-of-the-box. See how to set this up.

Custom xstartup files

If the desktop environment you've installed is not the default (for example, you've installed LXDE on a regular Ubuntu system), you will need to modify the xstartup script used by RealVNC Server to start your chosen desktop environment. Each desktop session should have a corresponding .desktop file in /usr/share/xsessions, which contains the command used to start the environment.

You can use the following command to list the startup commands for all the available sessions installed on the system:

grep Exec= /usr/share/xsessions/*.desktop

Next, use a text editor to replace the existing xstartup script with the following template:

#!/bin/sh
DESKTOP_SESSION=name-of-desktop-file
export DESKTOP_SESSION
exec-command-from-desktop-file
#dbus-launch --exit-with-session exec-command-from-desktop-file # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

This should be saved to ~/.vnc/xstartup, or /etc/vnc/xstartup.custom for all users on the system, and made executable using chmod +x.

Below are some template xstartup files for common desktop environments.

Xfce

#!/bin/sh
DESKTOP_SESSION=xfce
export DESKTOP_SESSION
startxfce4
#dbus-launch --exit-with-session startxfce4 # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

LXDE

#!/bin/sh
DESKTOP_SESSION=LXDE
export DESKTOP_SESSION
startlxde
#dbus-launch --exit-with-session startlxde # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

KDE Plasma Standard

#!/bin/sh
DESKTOP_SESSION=1-kde-plasma-standard
export DESKTOP_SESSION
startkde
#dbus-launch --exit-with-session startkde # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

GNOME

#!/bin/sh
DESKTOP_SESSION=gnome
export DESKTOP_SESSION
gnome-session
#dbus-launch --exit-with-session gnome-session # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

GNOME Classic

#!/bin/sh
DESKTOP_SESSION=gnome-classic
export DESKTOP_SESSION
env GNOME_SHELL_SESSION_MODE=classic
gnome-session --session gnome-classic
#dbus-launch --exit-with-session gnome-session --session gnome-classic # alternative to above that may provide better compatibility for some applications
vncserver-virtual -kill $DISPLAY

Known issues

There is a known issue with the RENDER extension in current versions of Xvnc that causes problems with Xfce, resulting in the desktop failing to start. This can be resolved by either enabling System Xorg, or disabling the RENDER extension; to do this, create /etc/vnc/config.custom (if it does not already exist) and add the command:

-extension RENDER