Cocoa is an important agricultural commodity in Indonesia which contributes to the economy with a production share of 15.68%. Cocoa diseases, such as Black Pod Rot and Pod Borer, are very detrimental to farmers. So it is necessary to build a recognition model that can classify automatically with high performance. Unfortunately the collected dataset is very unbalanced, and this is an additional challenge as it can reduce recognition performance. This study proposes disease recognition in cocoa images using the EfficientNet-B4 Convolutional Neural Network (CNN) model with fine-tuning. In this study also used seven kinds of data augmentation. The result is that the proposed CNN model has a high accuracy of 97.3% which is an increase of about 7.4% compared to the original model, at relatively few epochs. In addition, the proposed model is compared with other CNN models such as Xception, InceptionV3, ResNet, DenseNet, and EfficientNet, using the same approach, namely fine-tuning and epoch. The result is that the proposed method is superior to other models. This confirms that the proposed CNN model can also work better on unbalanced data.