Accurately Simulating Energy Consumption of I/O-intensive Scientific Workflows

While distributed computing infrastructures can provide infrastructure-level techniques for managing energy consumption, application-level energy consumption models have also been developed to support energy-efficient scheduling and resource provisioning algorithms. In this work, we analyze the accuracy of a widely-used application-level model that have been developed and used in the context of scientific workflow executions. To this end, we profile two production scientific workflows on a distributed platform instrumented with power meters. We then conduct an analysis of power and energy consumption measure- ments. This analysis shows that power consumption is not linearly related to CPU utilization and that I/O operations significantly impact power, and thus energy, consumption. We then propose a power consumption model that accounts for I/O operations, including the impact of wait- ing for these operations to complete, and for concurrent task executions on multi-socket, multi-core compute nodes. We implement our proposed model as part of a simulator that allows us to draw direct comparisons between real-world and modeled power and energy consumption. We find that our model has high accuracy when compared to real-world execu- tions. Furthermore, our model improves accuracy by about two orders of magnitude when compared to the traditional models used in the energy- efficient workflow scheduling literature.

Per-task power (top) and total energy (bottom) consumption measurements for the Epigenomics map task and the SoyKB haplotype caller and indel realign, as well as estimated with traditional methods (estimation) and our proposed model (wrench-*)

Reference to the paper:

  • [PDF] [DOI] R. Ferreira da Silva, A. Orgerie, H. Casanova, R. Tanaka, E. Deelman, and F. Suter, “Accurately Simulating Energy Consumption of I/O-intensive Scientific Workflows,” in Computational Science – ICCS 2019, 2019, p. 138–152.
    [Bibtex]
    @inproceedings{ferreiradasilva-iccs-2019,
    author = {Ferreira da Silva, Rafael and Orgerie, Anne-C\'{e}cile and Casanova, Henri and Tanaka, Ryan and Deelman, Ewa and Suter, Fr\'{e}d\'{e}ric},
    title = {Accurately Simulating Energy Consumption of I/O-intensive Scientific Workflows},
    booktitle = {Computational Science -- ICCS 2019},
    year = {2019},
    pages = {138--152},
    publisher = {Springer International Publishing},
    doi = {10.1007/978-3-030-22734-0_11}
    }


701 views

You may also like

Leave a Reply

Your email address will not be published. Required fields are marked *