An Integrated Framework for Data Mining and Distributed Database Optimization in Resource-Constrained Network Environments

Abstract

The widespread availability of distributed computing environments in industrial and organizational structures has led to an increasing need for efficient data management mechanisms and mechanisms for extracting knowledge. A holistic framework is presented in this paper to overcome the performance problem of the resource constrained network environments by using optimized data mining techniques along with distributed database management strategies. The presented framework is based on the principles of database partitioning, query optimization, association rule mining and network-aware scheduling, and includes a layered structure that can minimize the latency of query execution while preserving the data consistency of the distributed nodes. The framework has been evaluated on simulated distributed environments and has proven to improve the throughput and resource usage in comparison with traditional single-node mining strategies. The outcome indicates a high degree of relevance to enterprise information systems, industrial data repositories and emerging network-integrated computing platforms where volume and distribution of data are operation challenges that remain constant.