Run Shiny app on a Ubuntu server on the Amazon Cloud

This guide is more for self reference than anything else. 
Since I struggled for two days trying to find all the correct setting to complete this task, gathering information from several websites, I decided to write a little guide on this blog so that if I want to do it again in the future and I do not remember anything (this happens a lot!!) at least I have something to resuscitate my memory.

I found most of the information and code I used from these websites:
http://tylerhunt.co/2014/03/amazon-web-services-rstudio/
http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/
http://www.rstudio.com/products/shiny/download-server/


Preface
First I would like to point out that this guide assumes you (I am talking to myself of the future) remember how to open an instance in the Amazon Cloud. It is not that difficult, you go to this page:
http://aws.amazon.com/ec2/

you log in (if you remember the credentials) and you should see the "Amazon Web Services" page, here you can select EC2 and launch an instance. Remember to select the correct server from the menu on the top right corner since the last time you run all the instances from Oregon, and you live in freaking Switzerland!!


Guide
NOTE
Instead of installing cygwin and cover step 1 and 2 we can just first cover step 4 and connect to the ubuntu server using WinSCP, then start putty from WinSCP and it will be already connected.


1) Install Cygwin
This software is needed to communicate with the Ubuntu server.
It is important to follow the instructions on this page (http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/) to install the software correctly.
In particular during the installation process a "select packages" windows appears where we need to select openssh and click on "skip", until there is a cross on the column bin.

When Cygwin is installed we need to click with the right button on the icon and select "run as administrator", then open it.
Now we can run the following line to install ssh:

ssh-host-config

During the process several questions will be asked, the following answers apply:
- Should privilege separation be used? YES
- New local account sshd? YES
- Run ssh as a service? YES
- Enter a value for daemon:  ntsec
- Do you want to use a different name? NO
- Create a new privilege account user? YES  -> then insert a password


After the installation we need to insert the following line to start the sshd service:

net start sshd

Then this line to configure the service:

ssh-user-config

Again it will ask a series of questions. There is a difference between the new version and what is written on the website.
Now it asks only about an SSH2 RSA identity file to be created, the answer is YES.
Then it asks other two questions regarding DSA files and another thing, the answers here are two NO.



2) Connect to the Amazon Server
Open Cygwin.
Go to the folder where the .pem file is saved, using the following line:

cd D:/<folder>/<folder>



NOTE:
Cygwin does not like folder names with spaces!


Now we need to be sure that the .pem key will not be publicly available using the following line

chmod 400 <NAME>.pem

and then we can connect to the ubuntu server using the following line:

ssh -i <NAME>.pem ubuntu@<PUBLIC IP>

These information are provided in Amazon if we click on "Connect" once the instance has properly been launched.
Once we are in we can installing R and Shiny.


3) Install R and Shiny
The first thing to do is set up the root user with the following line:

sudo passwd root

The system will ask to input a password.
Then we can log in using the following line:

su

Now we are logged in as root users.

Now we need to update everything with the following:

apt-get update

At this point we can install R with the following line:

apt-get install r-base


NOTE:
It may be that during the installation process an older version of R is installed and this may create problems with some packages.
To solve this problem we need to modify the file sources.list located in /etc/apt/sources.list
We can do this by using WinSCP, but first we need to be sure that we have access to the folder.
We should run the following two lines:

cd /etc/

chmod 777 apt

This gives us access to modify the files in the folder apt via WinSCP (see point 4).
This line of code gives indiscriminate access to the folder, so it is not super secure.

Now we can connect and add the following line at the end:

deb http://cran.stat.ucla.edu/bin/linux/ubuntu trusty/

Then we need to first remove the old version of R using:

apt-get remove r-base

or

apt-get remove r-base-dev

Then we need to run once again both the update and the installation calls.



We can check the amount of disk space left on the server using the following command:

df -h


Then we can start R just by typing R in the console.
At this point we need to install all the packages we would need to run shiny using standard R code:

install.packages("raster")


Now we can exit from R with q() and install shiny suing the following line in ubuntu:

sudo su - \
-c "R -e \"install.packages('shiny', repos='http://cran.rstudio.com/')\""

Now we need to install gdebi with the following lines (check here for any update:http://www.rstudio.com/products/shiny/download-server/):

apt-get install gdebi-core
wget http://download3.rstudio.org/ubuntu-12.04/x86_64/shiny-server-1.3.0.403-amd64.deb
gdebi shiny-server-1.3.0.403-amd64.deb



4) Transfer file between windows and Ubuntu
We can use WinSCP (http://winscp.net/eng/index.php) for this task.

First of all we need to import the .pem file that is need for the authentication.
From the "New Site" window we can go to "advanced", then click on "SSH -> Authentication".
From the "Private key file" we can browse and open the .pem file. We need to transform it into a ppk file but we can do that using the default settings.
We can just click on "Save private key" to save the ppk file and then import it again on the same field, and click OK.

Now the Host name is the name of the instance, for example:
ec2-xx-xx-xxx-xxx.eu-west-1.compute.amazonaws.com

The user name is ubuntu and the password is left blank. The protocol is SFTP and the port is 22.

The shiny server is located in the folder /srv/shiny-server

We need to give WinSCP access to this folder using again the command

cd srv
chmod 777 shiny-server





5) Transfer shiny app files on the server
In WinSCP open the folder /srv/shiny-server and create a new folder with the name of your shiny app.
Then transfer the files from your PC to the folder.
Remember to change the address of the files or working directory in the R script with the new links.


6) Allow the port 3838 to access the web
To do this we need to change the rule in the "security groups" menu.
This menu is visible in the main window where all your instances are shown. However, do not access the menu from the panel of the left, that area may be a bit confusing.
Instead select the instance in which you need to add the rule, look at the window at the bottom of the page (the one that shows the name of the instance) and click on the name in light blue near "security group".
This will open the security group menu specific for the instance you selected.
Here we need to add a rule by clicking on "add rule", select custom TCP, write the port number 3838 in the correct area and the select "anywhere" in the IP section.

Now if you go to the following page you should see the app:
<PUBLIC IP>:3838/<FOLDER>

It is sometimes necessary to open port 22 as well, using the same procedure to keep connecting to the server with cygwin or putty.


7) Stop, start and restart Shiny server

sudo start shiny-server

sudo stop shiny-server

sudo restart shiny-server


8) Installing rgdal
This package require some tweaks before its installations. On this site I found what I needed: http://askubuntu.com/questions/206593/how-to-install-rgdal-on-ubuntu-12-10

Basically from the Ubuntu console just run these three lines of code:
sudo apt-get install aptitude
sudo aptitude install libgdal-dev
sudo aptitude install libproj-dev

Then go back to R and install rgdal normally (with install.packages)



9) Installing rCharts
Before installing rCharts we need to run the line above to install rgdal (I also run the first two lines suggested in the comments, but I do not know if it helped). I also run the following lines from here: http://stackoverflow.com/questions/16363144/install-rcharts-package-on-r-2-15-2

sudo apt-get install libcurl4-openssl-dev
sudo apt-get install openjdk-6-jdk
export LD_LIBRARY_PATH=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server
R CMD javareconf 

but I do not know if they helped or not. 

If we not do that "devtools" will not install and therefore we will not be able to install rCharts from github:

require(devtools)
install_github('rCharts', 'ramnathv')

To run rCharts from the ubuntu server we need to make sure that the links to the javascript library are not referring to


NOTE:
You may need to add packages in R after the installation. For doing that you always need to remember to access ubuntu as root user, so first thing to do is write the code:

su

and insert the password. Now you can start R and install the packages. Otherwise the lib folder where the packages are installed would not be accessible.




#UPDATE from Mike Rutter
To make the install easier, add the following PPAs:

sudo apt-add-repository ppa:marutter/rrutter
sudo apt-add-repository ppa:marutter/c2d4u

The first is the same as the CRAN repository, but you don't need to edit "sources.list". The second has over 2,500 R packages ready to install. For example:

sudo apt-get install r-cran-raster r-cran-rgdal r-cran-shiny

will install the R packages mentioned in the the post. No need to install the "dev" packages either, as that will be taken care of by apt. And the will be updated via the regular Ubuntu update process.

NOTE
Without updating the file in /etc/apt we can just run the first two lines suggested by Mike, then remove r-base and re-install it to have the updated version.