You may have a requirement to use Wayland on your Raspberry Pi device, but unfortunately, Wayland is not currently supported by RealVNC Server.

However, you can connect to the built-in WayVNC in order to establish a connection to a Raspberry Pi device running Wayland using RealVNC Viewer by changing the default configuration of WayVNC.

For devices using a WayVNC version prior to 0.7.0

If you wish to use RealVNC Viewer to connect to WayVNC < 0.7.0 you will need to ensure that authentication is disabled, as WayVNC uses the VeNCrypt (TLS) encryption method which is not compatible with RealVNC Viewer.

To disable authentication, you will need to set enable_auth to false. You can do this by setting the ’enable_auth’ line within ‘$HOME/.config/wayvnc/config’ to ‘false’ (enable_auth=false) or by commenting out the ‘enable_auth’ line (#enable_auth=…).

For devices using WayVNC 0.7.0 or later

If you are using WayVNC 0.7.0 or later, RSA-AES authentication has been added which is compatible with RealVNC Viewer, but requires enabling first.

To enable this you will first need to generate an appropriate certificate by running:

ssh-keygen -m pem -f $HOME/.config/wayvnc/rsa_key.pem -t rsa -N ""

And then adding either:

rsa_private_key_file=rsa_key.pem
use_relative_paths=true

or

rsa_private_key_file=$HOME/.config/wayvnc/rsa_key.pem

After making these changes, WayVNC will need to be restarted before initiating a new connection from RealVNC Viewer.

What if I don’t want to disable authentication and do not have a specific requirement for using Wayland?

If you do not wish to disable authentication and do not have a requirement to use Wayland, you can instead opt disable Wayland which will allow for you to use RealVNC Server.

Details on doing this can be found here: How do I disable Wayland to use RealVNC Connect?