Install VNC software
sudo apt-get install vnc4server
Start VNC server
vncserver :1 -geometry 1024x768 -depth 16 -pixelformat rgb565
Now you will be able to connect to ubuntu VPS with VNC.
On ubuntu, it won't start "vncconfig" on booting. You need to run "vncconfig" to share clip board between vnc.
vncconfig
You can start gui application from this window. It use less CPU. If you need gnome desktop, edit file
vi /root/.vnc/xstartup
Find
x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & x-window-manager &
Replace it with following:
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & gnome-session &