The movement of data on the internet is a major challenge for researchers in BIG Sciences. Over the past decade, we can see that it is not only in BIG Science that generates large amount of data, but also Internet of things (IoT) as standards of the data resolution being captured continue to evolve.
We have also seen increases of network bandwidth, from mega to Giga, and 100s of Gbps. Despite this high bandwidth capacity, researchers are still not experiencing the corresponding increase in the “speed” of data transfer. This is due to multiple factors: server, transport protocol, etc.
Move That Data! is the first-ever Data Mover challenge to be held on a global scale and the inaugural edition will be held in Singapore. The challenge aims to bring together experts from industry and academia, in a bid to test their software across servers located in various countries (USA, Singapore, Australia, Japan) that are connected by 100G international networks.
Dates to note
1. Objective of the data mover challenge
2. The Challenge
- To showcase challengers by having them compete in deploying the best software tools on Data Transfer Nodes (DTNs) that are set up within existing international networks across the globe.
- To bring together a community of experts in DTNs deployment and operations.
- Each team will be given 1 week to deploy the software on globally distributed nodes.
- The transport protocol must minimally support a data transfer rate of 10Gbps or higher.
Server Hardware sponsorship
- Teams must leverage on the Global Research & Education network infrastructure for the data mover challenge.
- The deployed DTNs must connect all locations including Australia, Japan, Singapore and USA.
- Teams must determine the connection type that will be used, for example, point-to-point, multipoint or distributed.
- Multiple Virtual Local Area Networks (VLANs) will be set up, per location, to provide inter-connectivity.
- Network monitoring must be set up to capture data transfer statistics.
- e.g. 100/10 Gb capability, local storage, connection to high performance file systems
- Purpose built DTN servers will be provided by partners based on baseline minimum specifications
- Existing DTN systems at each partner sites must be used.
- CentOS7 Operating System
- Data transfer tools & software will be provided by the participants
5. Judging Criteria
- This data mover challenge is open to all organisations, companies, research institutions, academia, researchers, post-graduate students and undergraduate students.
- Representatives of all partnering organisations for the challenge should not participate as team member throughout the competition.
1) Maximum sustainable transfer rate from disk to disk.
- A scoresheet with specific data transfer scenarios will be provided during the competition.
- Judges will consider the following factors:
2) Usability of the software
6. Results Announcement
- Innovativeness in usage. The competitor can use an application to demonstrate their software across multiple sites.
The winning team will be announced at the SupercomputingAsia 2019 (SCA19) conference in March 2019 in Singapore (www.sc-asia.org), and will have the opportunity to showcase a live demonstration of their winning solution.
- Return coach airfare for team leader of the winning team to Singapore from the country he/she is based in.
- Subsistence Allowance (including accommodation) for SCA19
- Complimentary all-access conference pass for SCA19
- Recognition of achievement at SCA19
Registration opens from 1 through to 30 Nov 2018. The registration site is as follows:
Please contact firstname.lastname@example.org for any questions or enquiries.