Communications in Science and Technology
Vol 2 No 1 (2017)

Comparison of Distributed K-Means and Distributed Fuzzy C-Means Algorithms for Text Clustering

I Made Artha Agastya (Universitas Gadjah Mada)
Teguh Bharata Adji (Universitas Gadjah Mada)
Noor Akhmad Setiawan (Universitas Gadjah Mada)



Article Info

Publish Date
01 Jun 2017

Abstract

Text clustering has been developed in distributed system due to increasing data. The popular algorithms like K-Means (KM) and Fuzzy C-Means (FCM) are combined with MapReduce algorithm in Hadoop Environment to be distributable and parallelizable. The problem is performance comparison between Distributed KM (DKM) and Distributed FCM (DFCM) that use Tanimoto Distance Measure (TDM) has not been studied yet. It is important because TDM’s characteristics are scale invariant while allowing discrimination collinear vectors. This work compared the combination of TDM with DKM (DKM-T) and TDM with DFCM (DFCM-T) to acquire performance of both algorithms. The result shows that DFCM-T has better intra-cluster and inter-cluster densities than those of DKM-T. Moreover, DFCM-T has lower processing time than that of DKM-T when total nodes used are 4 and 8. DFCM-T and DKM-T could perform clustering of 1,400,000 text files in 16.18 and 9.74 minutes but the preprocessing times take hours.

Copyrights © 2017






Journal Info

Abbrev

cst

Publisher

Subject

Engineering

Description

Communication in Science and Technology [p-ISSN 2502-9258 | e-ISSN 2502-9266] is an international open access journal devoted to various disciplines including social science, natural science, medicine, technology and engineering. CST publishes research articles, reviews and letters in all areas of ...