Directed Research

Marina del Rey, CA - View from ISI

Are you looking for research experience? I am always looking for good students that want to conduct small research projects in the area of distributed systems, with emphasis in scientific workflows, cloud computing, containers, data and resource management, and computational simulation.

If you have interest in any of the research projects below, please contact me ( rafsilva@…). In addition, please visit the USC’s guidance for Directed Research students for additional information on how to proceed.

We are located outside campus at the Information Sciences Institute (Marina del Rey, CA). USC provides continuous free shuttle services between the campus and ISI. Please, visit the USC’s transportation website for the schedules.

Project #1: Building tools to support research in Scientific Workflows


Overview —
A significant amount of recent research in scientific workflows aims   to develop new techniques, algorithms and systems that can overcome the challenges of efficient and robust execution of ever larger workflows on increasingly complex distributed infrastructures. Since the infrastructures, systems and applications are complex, and their behavior is difficult to reproduce using physical  experiments, much of this research is based on simulation. Workflow execution traces and synthetic workflow generators are the main tools used by researchers to evaluate their novel algorithms and mechanisms. Although several trace archives are available for the community, there is a lack of a standard mechanism to represent them in a consistent manner.

Summary of Activities — The student will extend the tools available as part of the WorkflowHub project to parse Pegasus’ output logs, and create JSON trace files that will be made available online via GitHub. The student will also implement a Python version of a Workflow Generator (based on our previous work), that will also be made publicly available in the WorkflowHub project website.

Required Skills: Python, Git
Desired Skills: JSON, Unix

Project #2: Enabling Scientific Workflow executions via Jupyter Notebooks

Overview — The Jupyter Notebook is a web application that allows users to create and share documents that contain live code, equations, visualizations and explanatory text. Its flexible and portable format resulted in a rapidly adoption by the research community to share and interact with experiments. Jupyter Notebooks has a strong potential to reduce the gap between researchers and the complex knowledge required to run large-scale scientific workflows via a programmatic high-level interface to access/manage workflow capabilities.

Summary of Activities — The student will extend/develop a Pegasus’ Python API for Jupyter to manage workflow executions (submission and monitoring), as well as collecting workflow performance information (e.g., statistics).

Required Skills: Python, Git
Desired Skills: Jupyter, Unix