Showing posts with label grid. Show all posts
Showing posts with label grid. Show all posts

Sunday, 10 May 2015

Exchange data between R and the Google Maps API using Shiny

A couple of years ago I wrote a post about using Shiny to exchange data between the Google Maps API and R: http://r-video-tutorial.blogspot.ch/2013/07/interfacing-r-and-google-maps.html

Back then as far as I remember Shiny did not allow a direct exchange of data between javascript, therefore I had to improvise and extract data indirectly using an external table. In other words, that work was not really good!!

The new versions of Shiny however features a function to send data directly from javascript to R:
Shiny.onInputChange

This function can be used to communicate any data from the Google Maps API to R. Starting from this I thought about creating an example where I use the Google Maps API to draw a rectangle on the map, send the coordinates of the rectangle to R, create a grid of random point inside it and then plot them as markers on the map. This was I can exchange data back and forth from the two platforms.

For this experiment we do not need  an ui.R file, but a custom html page. Thus we need to create a folder named "www" in the shiny-server folder and add an index.html file.
Let's look at the HTML and javascript code for this page:

 <!DOCTYPE html>  
 <html>  
 <head>  
 <title>TEST</title>  
   
 <!--METADATA-->    
 <meta name="author" content="Fabio Veronesi">  
 <meta name="copyright" content="©Fabio Veronesi">  
 <meta http-equiv="Content-Language" content="en-gb">  
 <meta charset="utf-8"/>  
   
 <style type="text/css">  
   
 html { height: 100% }  
 body { height: 100%; margin: 0; padding: 0 }  
 #map-canvas { height: 100%; width:100% }  
   
 </style>  
   
      
   
 <script type="text/javascript"  
    src="https://maps.googleapis.com/maps/api/js?&sensor=false&language=en">  
   </script>  
        
 <script type="text/javascript" src="http://google-maps-utility-library-v3.googlecode.com/svn/tags/markerclusterer/1.0/src/markerclusterer.js"></script>  
   
 <script src="https://maps.googleapis.com/maps/api/js?v=3.exp&signed_in=true&libraries=drawing"></script>  
   
   
   
        
 <script type="text/javascript">  
      //We need to create the variables map and cluster before the function  
      var cluster = null;  
      var map = null;  
        
      //This function takes the variable test, which is the json we will create with R and creates markers from it  
      function Cities_Markers() {  
           if (cluster) {  
                cluster.clearMarkers();  
                }  
           var Gmarkers = [];  
           var infowindow = new google.maps.InfoWindow({ maxWidth: 500,maxHeight:500 });  
   
           for (var i = 0; i < test.length; i++) {   
                var lat = test[i][2]  
                var lng = test[i][1]  
                var marker = new google.maps.Marker({  
                     position: new google.maps.LatLng(lat, lng),  
                     title: 'test',  
                     map: map  
                });  
         
           google.maps.event.addListener(marker, 'click', (function(marker, i) {  
                return function() {  
                     infowindow.setContent('test');  
                     infowindow.open(map, marker);  
                }  
                })(marker, i));  
           Gmarkers.push(marker);  
           };  
           cluster = new MarkerClusterer(map,Gmarkers);  
           $("div#field_name").text("Showing Cities");  
      };  
   
   
      //Initialize the map  
      function initialize() {  
           var mapOptions = {  
           center: new google.maps.LatLng(54.12, -2.20),  
           zoom: 5  
      };  
   
      map = new google.maps.Map(document.getElementById('map-canvas'),mapOptions);  
        
   
      //This is the Drawing manager of the Google Maps API. This is the standard code you can find here:https://developers.google.com/maps/documentation/javascript/drawinglayer  
  var drawingManager = new google.maps.drawing.DrawingManager({  
   drawingMode: google.maps.drawing.OverlayType.MARKER,  
   drawingControl: true,  
   drawingControlOptions: {  
    position: google.maps.ControlPosition.TOP_CENTER,  
    drawingModes: [  
     google.maps.drawing.OverlayType.RECTANGLE  
    ]  
   },  
   
   rectangleOptions: {   
    fillOpacity: 0,  
    strokeWeight: 1,  
    clickable: true,  
    editable: false,  
    zIndex: 1  
   }  
        
  });  
   
//This function listen to the drawing manager and after you draw the rectangle it extract the coordinates of the NE and SW corners
  google.maps.event.addListener(drawingManager, 'rectanglecomplete', function(rectangle) {  
   var ne = rectangle.getBounds().getNorthEast();  
      var sw = rectangle.getBounds().getSouthWest();  
   
      //The following code is used to import the coordinates of the NE and SW corners of the rectangle into R  
      Shiny.onInputChange("NE1", ne.lat());  
      Shiny.onInputChange("NE2", ne.lng());  
      Shiny.onInputChange("SW1", sw.lat());  
      Shiny.onInputChange("SW2", sw.lng());  
        
 });  
   
   
  drawingManager.setMap(map);  
    
 }  
   
   
 google.maps.event.addDomListener(window, 'load', initialize);  
</script>  
        
   
        
   
 <script type="application/shiny-singletons"></script>  
 <script type="application/html-dependencies">json2[2014.02.04];jquery[1.11.0];shiny[0.11.1];bootstrap[3.3.1]</script>  
 <script src="shared/json2-min.js"></script>  
 <script src="shared/jquery.min.js"></script>  
 <link href="shared/shiny.css" rel="stylesheet" />  
 <script src="shared/shiny.min.js"></script>  
 <meta name="viewport" content="width=device-width, initial-scale=1" />  
 <link href="shared/bootstrap/css/bootstrap.min.css" rel="stylesheet" />  
 <script src="shared/bootstrap/js/bootstrap.min.js"></script>  
 <script src="shared/bootstrap/shim/html5shiv.min.js"></script>  
 <script src="shared/bootstrap/shim/respond.min.js"></script>  
   
 </head>  
   
   
 <body>  
   
  <div id="json" class="shiny-html-output"></div>  
  <div id="map-canvas"></div>  
    
 </body>  
 </html>  

As you know an HTML page has two main elements: head and body.
In the head we put all the style of the page, the metadata and the javascript code. In the body we put the elements that would be visible to the user.

After some basic metadata (written in orange), such as Title, Author and Copyright, we find a style section (in yellow) with the style of the Google Maps API. This is standard code that you can find here, where they explain how to create a simple page with google maps: Getting Started

Below we have some script calls (in blue) where we import some elements we would need to run the rest of the code. We have here the scripts to run the Google Maps API itself, plus the script to run the drawing manager, which is used to draw a rectangle onto the map, and the js script to create the clusters from the markers, otherwise we would have too many overlapping icons.

Afterward we can write the core script of the Google Maps API; here I highlighted the start and the end of the script in red and all the comments in pink so that you can work out the subdivision I made.

First of all we need to declare two variables, map and cluster as null. This is because these two variables are used in the subsequent function and if we do not declare them the function will not work. Then we can define a function, which I call Cities_Marker() because I have taken the code directly from Audioramio. This function takes a json, stored into a variable called test, loops through it and creates a mark for each pair of coordinates in the json. Then it cluster the markers.

Afterward there is the code to initialize the map and the drawing manager. The code for the drawing manager can be found here: Drawing Manager

The crucial part of the whole section is the Listener function. This code, as soon as you draw a rectangle on the map, extracts the coordinates of the NE and SW corners and store them in two variables. Then we can use the function Shiny.onInputChange to transfer these variable from javascript to R.

The final step to allow the communication back to javascript from R is create a div element in the body of the page (in blue) of the class "shiny-html-output" with the ID "json". The ID is the part that allow Shiny to identify this element.

Now we can look at the server.R script:

 # server.R  
 library(sp)  
 library(rjson)  
    
 shinyServer(function(input, output, session) {  
    
 output$json <- reactive({  
 if(length(input$NE1)>0){  
   
 #From the Google Maps API we have 4 inputs with the coordinates of the NE and SW corners  
 #using these coordinates we can create a polygon  
 pol <- Polygon(coords=matrix(c(input$NE2,input$NE1,input$NE2,input$SW1,input$SW2,input$SW1,input$SW2,input$NE1),ncol=2,byrow=T))  
 polygon <- SpatialPolygons(list(Polygons(list(pol),ID=1)))  
   
 #Then we can use the polygon to create 100 points randomly  
 grid <- spsample(polygon,n=100,type="random")  
   
 #In order to use the function toJSON we first need to create a list  
 lis <- list()  
 for(i in 1:100){  
 lis[[i]] <- list(i,grid$x[i],grid$y[i])  
 }  
   
 #This code creates the variable test directly in javascript for export the grid in the Google Maps API  
 #I have taken this part from:http://stackoverflow.com/questions/26719334/passing-json-data-to-a-javascript-object-with-shiny  
    paste('<script>test=',   
       RJSONIO::toJSON(lis),  
       ';Cities_Markers();', # print 1 data line to console  
       '</script>')  
      }  
  })  
 })  

For this script we need two packages: sp and rjson.
The first is needed to create the polygon and the grid, the second to create the json that we need to export to the webpage.

Shiny communicates with the page using the IDs of the elements in the HTML body. In this case we created a div called "json", and in Shiny we use output$json to send code to this element.
Within the reactive function I first inserted an if sentence to avoid the script to start if no polygon has been drawn yet. As soon as the user draws a polygon onto the map, the four coordinates are transmitted to R and used to create a polygon (in blue). Then we can create a random grid within the polygon area with the function spSample (in orange).

Subsequently we need to create a list with the coordinates of the points, because the function toJSON takes a list as main argument.
The crucial part of the R script is the one written in red. Here we basically take the list of coordinates, we transform it into a json file and we embed it into the div element as HTML code.
This part was taken from this post: http://stackoverflow.com/questions/26719334/passing-json-data-to-a-javascript-object-with-shiny

This allow R to transmit its results to the Google Maps API as a variable named test, which contains a json file. As you can see from the code, right after the json file we run the function Cities_Markers(), which takes the variable test and creates markers on the map.


Conclusion
This way we have demonstrated how to exchanged data back and forth between R and the Google Maps API using Shiny.


Wednesday, 2 April 2014

Merge .ASC grids with R

A couple of years ago I found online a script to merge several .asc grids into a single file in R.
I do not remember where I found it but if you have the same problem, the script is the following:

 setwd("c:/temp")  
 library(rgdal)  
 library(raster) 

 
 # make a list of file names, perhaps like this:  
 f <-list.files(pattern = ".asc")  


 # turn these into a list of RasterLayer objects  
 r <- lapply(f, raster) 

 
 # as you have the arguments as a list call 'merge' with 'do.call'  
 x <- do.call("merge",r) 

 
 #Write Ascii Grid  
 writeRaster(x,"DTM_10K_combine.asc")  

It is a simple and yet very affective script.
To use this script you need to put all the .asc grids into the working directory, the script will take all the file with extension .asc in the folder, turn them into raster layers and then merge them together and save the combined file.

NOTE:
if there are other file with ".asc" in their name, the function
list.files(pattern = ".asc") 

will consider them and it may create errors later on. For example, if you are using ArcGIS to visualize your file, it will create pyramids files that will have the same exact name of the ASCII grid and another extension.
I need to delete these file and keep only the original .asc for this script and the following to work properly.



A problem I found with this script is that if the raster grids are not properly aligned it will not work.
The function merge from the raster package has a work around for this eventuality; using the option tolerance, it is possible to merge two grids that are not aligned.
This morning, for example, I had to merge several ASCII grids in order to form the DTM shown below:


The standard script will not work in this case, so I created a loop to use the tolerance option.
This is the whole script to use in with non-aligned grids:

 setwd("c:/temp")  
 library(rgdal)  
 library(raster)  
   
 # make a list of file names, perhaps like this:  
 f <-list.files(pattern = ".asc")  
   
 # turn these into a list of RasterLayer objects  
 r <- lapply(f, raster)  
   
   
 ##Approach to follow when the asc files are not aligned  
 for(i in 2:length(r)){  
 x<-merge(x=r[[1]],y=r[[i]],tolerance=5000,overlap=T)  
 r[[1]]<-x  
 }  
   
 #Write Ascii Grid  
 writeRaster(r[[1]],"DTM_10K_combine.asc")  

The loop merge the first ASCII grid with all the other iteratively, re-saving the first grid with the newly created one. This way I was able to create the DTM in the image above.