Object detection can determine the existence of an object, scope and image. Object detection begins with the introduction of an object. This method can be used to automate the process of sorting and grading oil palm fresh fruit bunches (FFB) at palm oil mills, which are still done manually. Image annotations are needed in building the software so that the software can identify object features in an image, especially imager in video frames. This study aims to annotate images of oil palm FFB into 2 categories, namely normal palm and abnormal palm. This category is the standard regulation of the Minister of Agriculture No. 14 of 2013. Image acquisition is carried out by varying the position of each oil palm FFB with the top and bottom position of the fruit which is then augmented 4 times which function to multiply the image data model to be annotated. Annotation is done using the python program application, namely Labelimg. The amount of image data that has been annotated is 200 images consisting of 100 normal palm images and 100 abnormal palm images.