A Heterogeneous Computing System for Data Mining Workflows

摘要: The computing-intensive Data Mining (DM) process calls for the support of a Heterogeneous Computing (HC) system, which consists multiple computers with different configurations, connected by high-speed LAN, increased computational power and resources. DM can be described as multi-phase pipeline process, in each phase there could many optional methods. This makes workflow very complex modelled only Directed Acyclic Graph (DAG). An HC system needs an effective efficient scheduling framework, orchestrates all computing hardware to perform competitive workflows. Motivated need practical solution problem workflow, this paper proposes dynamic DAG algorithm according characteristics execution time estimation model jobs. Based on approximate job time, first maps jobs machines decentralized diligent (defined paper) manner. Then performance initial mapping improved through migrations when necessary. heuristic used it considers factors both minimal completion criterion critical path DAG. We implement established Multi-Agent System (MAS) environment, reuse existing algorithms is achieved encapsulating them into agents. Practical classification problems are test measure performance. detailed experiment procedure result analysis also discussed paper.