Developer Workshop: Building the Modern Research Data Portal

New high-speed networks make it possible, in principle, to transfer and share research data at tremendous speeds and scales–but have also proved challenging to use in practice. Two new technologies now allow us to translate this potential into reality: Science DMZ architectures provide frictionless end-to-end network paths; and Globus APIs allow programmers to create powerful research data portals that leverage these paths for data distribution, staging, synchronization, and other useful purposes.

In this workshop, we use real-world examples to show how these new technologies can be applied to realize immediately useful capabilities. Examples include the National Center for Atmospheric Research’s Research Data Archive, which provides for high-speed delivery of research data to thousands of geoscientists; the Globus data portal, which provides for interactive data staging to/from experimental facilities and computing centres, and the publication of data generated at such facilities and centres; and the Advanced Photon Source data sharing system, used to distribute data from light source experiments. We explain how the Globus APIs provide intuitive access to authentication, authorization, sharing, transfer, and synchronization capabilities. Companion iPython/Jupyter notebooks will provide application skeletons that workshop participants can adapt to realize their own research data portals, science gateways, and other web applications that support research data workflows.

High-speed networks, science DMZs, and Globus APIs together create a new research data platform on which you can create entirely new classes of scientific application. This workshop, organized in collaboration with ESnet, will provide all of the the information that you need to imagine and create modern research data portals that leverage this platform to advance discovery.

Workshop Agenda

The presentation, links to source code, and documentation are available on the workshop materials page.

Wednesday, April 20
07:30—17:00	registration desk open
7:30—8:30	breakfast Savoy
8:30—10:00	Introduction, Concepts, and Components Imperial I Led by: Steve Tuecke We will introduce the Modern Research Data Portal and set the context for how Globus and the ScienceDMZ combine to deliver unique data management capabilities. This will include: Overview of use cases: Common patterns like data publication/distribution, orchestration of data flows, etc. Overview of the Globus platform: Architecture and brief overview of available services Introduction to the Globus Auth API: Authenticating and authorizing a client Introduction to the Globus Transfer API: Make your first call and move data with Globus Introduction to the Python SDK for using Globus Auth and Transfer
10:00—10:30	beverage break
10:30—12:00	Hackathon Part I Imperial I Led by: Steve Tuecke You will create your first simple web application using the Globus Auth and Globus Transfer APIs. You may bring your own portal/gateway/web app code, or use a sample application we provide. You will learn how to register the application with Globus Auth, authenticate using Globus Auth's OpenID Connect API, and call the Globus Transfer API to do a directory listing.
12:00—13:30	lunch Savoy
13:30—15:00	Globus Transfer API Deep Dive Imperial I Led by: Steve Tuecke We will explore the Globus Transfer API in depth and demonstrate how to directly access files from an endpoint using the Globus Connect HTTPS Endpoint Server.
15:00—15:30	beverage break
15:30—17:00	Hackathon Part II Imperial I Led by: Steve Tuecke Extend your web app to use additional capabilities of the Globus Transfer API including initiating asynchronous file transfers, checking transfer status, managing shared endpoint ACLs, and using direct HTTPS file access.
Thursday, April 21
7:30—17:00	registration desk open
7:30—8:30	breakfast Imperial
8:30—10:00	Globus Auth API Deep Dive Walnut Ballroom Led by: Steve Tuecke We will explore the Globus Auth in depth. You will learn how to extend the Globus platform with your own services, using Globus Auth to authorize calls to your REST API.
10:00—10:30	beverage break
10:30—12:00	Hackathon Part III Walnut Ballroom Led by: Steve Tuecke Extend your web app with a REST API and grant access to resources using linked identities. We will also hold an open discussion and gather feedback on the Globus platform to inform future development and product direction.
12:00	workshop adjourns; move to GlobusWorld 2016 plenary sessions