Thursday, Sept. 21, 2017

Introduction

10:00 – 10:30: Welcome
10:30 – 12:00: Big data concepts and technologies – Overview (presentation slides)

Presenter: Arne de Wall – 52°North, Marius Appel – ifgi, Thomas Paschke – Esri Germany

This presentation provides a general introduction and an overview of today’s big data concepts and technologies. After a short discussion of the big data landscape, concepts such as stream and batch processing and common big data architectures are presented and specific challenges for geographic data are discussed. The presentation will emphasize widely used technologies including Hadoop, Spark, and array databases and give an outlook on their application on geographic data and how cloud-based solutions make these technologies accessible to users.

Lunch Break

13:30 – 15:00: Technologies & Interfaces

13:30 – 14:00: ArcGIS Real-Time & Big Data Solutions (presentation slides)
14:00 – 14:30: R / Python and Big Data, openEO (presentation slides)
14:30 – 15:00: Scalable Earth observation analytics with SciDB (presentation slides)

Presenter: Marius Appel – ifgi

This presentation will introduce how the data management and analytical system SciDB and its data representation as multidimensional arrays can be used to perform complex analytics on Earth observation datasets. The first part will discuss SciDB’s basic concepts such as chunking and sparse storage before some extensions that facilitate the conversionfrom file-based scenes to multidimensional arrays with the geographic data abstraction library (GDAL) will be presented. A case study for land use change analysis based on Landsat 7 imagery will then demonstrate how user-defined functions can be scaled by streaming. The presentation will close with a discussion of the limitations and an outlook for which applications SciDB might provide a reasonable platform.

15:00 – 15:30: Flash Talks

Big Data from Citizen Science. Use Cases from senseBox and openSenseMap
How Spatio-Temporal Modeling Can Support Casual Carpooling

Presenter: Wi Wang, ecov

Analyzing Mobility Data and Predicting Human Movement

Presenter: Dominik Bucher, ETH Zürich

Detecting Waterbodies in Sentinal 1 Satellite Images to Support Decision Making in Flood Emergency (presentation slides)

Presenter: Maurin Radtke & Diego Morales, ifgi

Coffee Break

16:00 – 18:00: Application areas

16:00 – 16:30: Big Spatial Data in Agriculture (presentation slides)

Presenter:  T. Steckel, H. Warkentin – CLAAS E-Systems

Spatial data are an important source of information in agriculture and thus not new. Up to now, most Farm Management-Systems incorporate some capabilities for spatial and temporal analytics. Increased availability of sensor data and improved wireless communication allow for deeper analysis of machines and processes. Well known IT-Systems encounter their limits. This presentation provides some insight into new environments with particular focus on GIS-aspects.

16:30 – 17:00: A Query Language for Handling Big Observation Data Sets in the Sensor Web (presentation slides)
17:00 – 17:30: ArcGIS Big Data Analytics use cases (presentation slides)

PresenterThomas Paschke – Esri Germany

This presentation will highlight some use cases where Esri’s Real-Time & Big Data solutions were used to solve Big Data analysis related problems.

17:30 – 18:00: Processing and analysis of Earth Observation data (presentation slides)

PresenterCarsten Brockmann – Brockmann Consult

Satellite remote sensing is producing large amount of Earth Observation (EO) raw data. Only the Sentinels of the European Space Agency are generating more than 6 TB/day new data in 2017, increasing over the coming years. A special adaptation of Hadoop for massive parallel processing of EO data has been implemented in the Calvalus processing system. Calvalus is part of several operational processing systems including the Copernicus Data and Exploitation Platform – Deutschland (CODE-DE). Closely linked with Calvalus is SNAP, a Java based toolbox for processing of EO data. Together they allow in-memory chaining of complex processing graphs and exploiting the map-reduce programming model. The production of Essential Climate Variables is chosen as an example to demonstrate the application of Calvalus on full mission data, the analysis with the python based CATE toolbox, and the combination with other earth system data in the Earth System Datacube.

Dinner (19:30)

Friday, Sept. 22, 2017

9:00 – 13:00: Parallel tutorials & hands-on sessions

ArcGIS & Real Time processing

Presenter:  Thomas Paschke – Esri Germany

In the morning user’s will learn how to set up and configure the current ArcGIS Real-Time & Big Data technology stack and gain familiarity with different aspects of the products. They will learn, how to ingest and analyze data feeds in real-time, store large amounts of this observation data and perform big data analysis on them.

R-ArcGIS-Bridge (presentation slides)

Presenter:  Edzer Pebesma – ifgi, Melanie Brandmeier – Esri Germany, Benedikt Graeler – 52°North

ArcGIS offers many spatial analysis tools out of the box. By combining ArcGIS with the full capabilities of the open source programming language R, users are able to work with a rich toolset for geostatistical computing. The R-ArcGIS-bridge allows to integrate R into the toolbox to perform statistical analysis on spatial data. We will showcase the entire workflow from installing the R-ArcGIS-bridge, through the design of R scripts and their integration into a tool box and the execution of these tools. Participants will have the opportunity to follow the showcases in a hands-on manner.

System Requirements: For the practical work with the technology in this hands-on session, please bring your own laptop and make sure that you have

  • administrator rights to install software,
  • access to ArcGIS Pro (Free Trial),
  • a local installation of R including the packages sp, gstat, raster, shiny, rgdal, spatstat, maptools, mclust, and
  • a local installation of RStudio Desktop
Processing Earth observation time series with SciDB and R (presentation slides)

13:00: Closing session

14:00 – 17:00: Hackathon & open space for exploring methods & technologies

We prepare some challenges, data and the infrastructure. You can use the sessions to get some hands-on experience, or try to explore the technologies with your own data.

ArcGIS Real-Time & Big Data analysis

Presenter:  Thomas Paschke – Esri Germany

The experience gained in the morning should provide the foundation and inspire for the Hackathon in the afternoon, where Real-Time and Big Data analysis use cases, based on the user’s own data (bring your own data) or provided sample data, can be explored.

R-ArcGIS-Bridge

Presenter:  Edzer Pebesma – ifgi, Melanie Brandmeier – Esri Germany, Benedikt Graeler – 52°North

We will provide the opportunity to develop and run your R code in ArcGIS along challenges that we propose. Participants are also invited to bring their own data sets and analysis scripts that they wish in integrate into ArcGIS. We will provide step-wise solutions and guidance throughout the afternoon.

System Requirements: For the practical work with the technology in this hands-on session, please bring your own laptop and make sure that you have

  • administrator rights to install software,
  • access to ArcGIS Pro,
  • a local installation of R including the packages sp, gstat, raster, shiny, rgdal, spatstat, maptools, mclust, and
  • a local installation of RStudio Desktop
Processing Earth observation time series with SciDB and R

Presenter:  Marius Appel – ifgi

Following up on the hands-on session in the morning, participants will work on a few simple study cases that use SciDB and R to process a Sentinel 2 dataset.

Drinks and open end