A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions


Presentation held at Workshop on Grids, Clouds, and P2P Computing (CGWS), 2012
Rhodes Island, Greece – Euro-Par 2012

Abstract – Archives of distributed workloads acquired at the infrastructure level reputably lack information about users and application-level middleware. Science gateways provide consistent access points to the infrastructure, and therefore are an interesting information source to cope with this issue. In this paper, we describe a workload archive acquired at the science-gateway level, and we show its added value on several case studies related to user accounting, pilot jobs, fine-grained task analysis, bag of tasks, and workflows. Results show that science-gateway workload archives can detect workload wrapped in pilot jobs, improve user identification, give information on distributions of data transfer times, make bag-of-task detection accurate, and retrieve characteristics of workflow executions. Some limits are also identified.

 

Related Publication

  • [PDF] [DOI] R. Ferreira da Silva and T. Glatard, “A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-steps, and Workflow Executions,” in Euro-Par 2012: Parallel Processing Workshops, I. Caragiannis, M. Alexander, R. Badia, M. Cannataro, A. Costan, M. Danelutto, F. Desprez, B. Krammer, J. Sahuquillo, S. Scott, and J. Weidendorfer, Eds., , 2013, vol. 7640, pp. 79-88.
    [Bibtex]
    @incollection{ferreiradasilva-cgws-2013,
    year = {2013},
    booktitle = {Euro-Par 2012: Parallel Processing Workshops},
    volume = {7640},
    series = {Lecture Notes in Computer Science},
    editor = {Caragiannis, Ioannis and Alexander, Michael and Badia, RosaMaria and Cannataro, Mario and Costan, Alexandru and Danelutto, Marco and Desprez, Fr\'ed\'eric and Krammer, Bettina and Sahuquillo, Julio and Scott, StephenL. and Weidendorfer, Josef},
    doi = {10.1007/978-3-642-36949-0_10},
    title = {A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-steps, and Workflow Executions},
    author = {Ferreira da Silva, Rafael and Glatard, Tristan},
    pages = {79--88}
    }

 

192 views

Continue Reading