Image processing technology has a wide range of applications, such as in the medical, military, surveillance, and robotics industries. Analyzing objects in images is crucial when it comes to image processing. This study focuses on image processing to track objects of red, green, and blue (RGB) colors through the utilization of a camera. There are two research schemes: image processing and data classification. The data classification method used is the fuzzy and adaptive neuro-fuzzy inference system (ANFIS). The methods of image subtracting and region properties are commonly utilized for image processing. Based on the classification data results, the fuzzy logic classification demonstrated a higher accuracy rate of 86% when compared to Anfis' 65%. This was observed when both classification models were tested using a random sample. The value of Anfis is small due to the limited size of the training data used. As a result, it is recommended to use a fuzzy classifier for object color tracking for good performance.