Initial results of the development of intelligent noninvasive phenotyping of raspberries using machine learning and 3D imaging

To distinguish candidate cultivars in fruit breeding, the characteristics of several thousand seedlings must be described and evaluated, which is done visually. Fruit counting and measuring in the field is currently the most used method. However, it is tedious and time-consuming and requires abundant manpower. In addition, visual evaluation is relatively subjective, and results may vary from different evaluators. The solution is to develop a machine vision-based method to automate and to intensify the fruit breeding process. In Latvia, the fruit breeding programme focuses on four fruit crops, including raspberries. Inclusion of raspberries (Rubus idaeus) in the breeding programme is related to the fact that there are no cultivars suitable for Latvian agroclimatic conditions. Breeding experience shows that the selections have insufficient ecological phenotype plasticity, which is a risk for cultivation of these cultivars in Latvian climate. In raspberries, yield depends on the sum of yield components, and each component is evaluated separately. The most important yield components are: number of laterals per shoot, number of flowers/fruits per lateral shoot, and average fruit mass. With the evaluation of these traits, image-based approaches to raspberry phenotyping are gaining momentum and provide fertile ground for non-invasive raspberry detection and categorization. The objective of this research is to develop a methodology and tools for non-invasive phenotyping of raspberry yield components based on red, green, blue (RGB) image colour value and 3D images, as well as provide descriptive and inferential statistics of raspberry cultivars. We propose a manually annotated 2D raspberry data set with ground truth region of interest (ROI)classifying labelled into five classes: “Buds”, “Flowers”, “Unripe Berries”, “Ripe Berries”, and “Damaged Berries” to verify and evaluate the raspberry detection problem using real-time deep neural network Yolo5. We also present three algorithms for 3D computational image processing to create 3D bounding boxes for raspberry parametrizing.