how to switch to xfce4 session manager?

How do I change my session manager to xfce4?  It is available on the server, but there seems to be no way to select it.  Running xfce4-session results in

xfce4-session: Another session manager is already running

Killing the current session manager logs me off.  I tried to put "xfce4-session &" in ~/.vnc/xstartup, but that seems to be ignored.

Thanks.