AI-Based Image Processing Systems in Agriculture

Albert Van Breemen explains in the podcast that cucumber cultivation in the Netherlands relies heavily on manual labor, especially when regularly cutting off the leaves at the base of each plant. This work is essential to optimize the yield and energy distribution of the plant.

To reduce this reliance on labor, the company VBTI in cooperation with the Dutch company Cropteq has developed a robotic system based on AI-based machine vision. A decisive factor in this automation is the precise recognition and differentiation of plant structures. Due to the natural variations in plant shape, this is a major challenge.

Traditional image processing technologies have difficulty recognizing irregular shapes. Deep learning offers a promising solution: by analyzing large data sets and training neural networks with a large number of images, the system can accurately identify leaves and cucumbers even in the presence of deviations.

This technological breakthrough helps to overcome long-standing bottlenecks in agricultural automation and offers a more precise and efficient alternative to manual labor.

However, the development of AI-controlled automation for agriculture is not just about data acquisition. Complex challenges in the areas of perception and control must also be solved. Van Breemen points out the difficulties of developing a leaf-cutting robot that is able to accurately recognize plant structures, determine their 3D position and precisely control the robot's movements.

However, robot technology is not limited to cucumber cultivation. It can also be transferred to other agricultural applications such as harvesting tomatoes or peppers. By customizing training data and robot actions, the same AI framework can be applied to different agricultural processes, making it a scalable solution for automation in agriculture.

While deep learning is excellent at object recognition, traditional search algorithms show their strengths in structural analysis. Deep learning identifies plant nodes and stems, while heuristic algorithms map their connections and create a structured plant model.

This hybrid approach, which combines AI with traditional algorithms, enables significantly higher prediction accuracy and thus increases the efficiency of automation in agriculture.

One example of VBTI's development work that Van Breemen presents in the podcast is the detection of asparagus. An impressive accuracy of 95 percent was achieved in the detection of asparagus in the field.

During a three-month harvest season, the machines collected more than four million data points, which were used to continuously improve the model. With the expanded data set, the accuracy is expected to increase to 98 to 99 percent.

The development of a precise AI model requires extensive data collection. However, as Van Breemen explains in the podcast, transfer learning makes it possible to achieve high accuracy with much smaller data sets. Instead of training a deep learning model from scratch - which usually requires millions of images - VBTI uses pre-trained models that were originally developed for other applications such as vehicle recognition.

By modifying these models and fine-tuning them with just a few thousand greenhouse images, the AI could be adapted to recognize cucumbers and plant structures with a high degree of accuracy.

To create the required dataset, the VBTI manually captured images with handheld cameras in the greenhouses. These images were then annotated - a process in which humans mark important elements such as cucumbers, leaves and stems. This annotated dataset formed the basis for training AI models and allowed deep learning algorithms to learn and improve with a much smaller amount of data than would otherwise have been the case.

Camera variations play a crucial role in the training of AI models. For four years, VBTI collected data with five different camera systems, each offering a unique look. As Albert Van Breemen explains in the podcast, this diversity strengthens the generalization ability of the model.

Deep learning benefits enormously from a wide range of input data, so a model that has been trained with different camera data can cope better with varying environments and imaging conditions. The result? A more robust model that works reliably in real-world applications.

One of the biggest challenges in the development of AI models is to ensure that they also work reliably in new environments. In a greenhouse where the system has not yet been tested, conditions may prevail that are not represented in the original data set.

Van Breemen emphasizes in the podcast how important it is to start with an initial, albeit small, data set and continuously improve the model by integrating new data - a process known as bootstrapping. It is even more important to integrate data collection directly into the machines.

For example, VBTI's leaf-cutting robots are equipped with functions that enable them to collect new images independently. This innovative approach reduces the cost of data acquisition in the long term, even if labeling the data remains a challenge and a significant cost factor.

The continuous monitoring and adaptation of AI systems is crucial for their long-term performance in agriculture. To ensure this, VBTI has developed the OneDL platform, which enables regular model improvements through an iterative process called AI Gym.

This innovative approach ensures that AI models are always fed with up-to-date data to adapt to ever-changing agricultural conditions. As plant structures and growing environments change over time, continuous data collection and regular training are necessary to maintain constant precision in tasks such as cutting cucumber leaves or harvesting asparagus.

VBTI's OneDL platform plays a central role in data collection, re-learning AI models and adapting them to different agricultural environments. It enables experimentation and comparison of models, as the performance of a model is highly dependent on the underlying data set.

With support for various deep learning frameworks and an abstraction layer to simplify integration, the platform offers a flexible solution that can be deployed in the cloud or on-premise as required. It works with different camera types, including RGB, RGBD and hyperspectral cameras, and supports ONNX format models, making it ideal for various hardware solutions.

In this way, the OneDL platform ensures that AI models are always up to date and can be flexibly adapted to changing requirements in agriculture.

As AI-driven greenhouse automation advances, researchers are looking for innovative ways to increase efficiency and reduce reliance on manual data entry. One promising approach is basic AI models that enable accurate object recognition with a minimum of labeled data.

Although these models are often too large for embedded systems, they serve as a starting point for the development of smaller, more efficient solutions specifically tailored to agricultural tasks.

Another interesting trend is the increasing importance of generative AI. These technologies could significantly reduce the effort and cost of image labeling, leading to even faster and more cost-effective automation. In the podcast, Albert Van Breemen explains that VBTI plans to expand its AI capabilities beyond computer vision* and integrate the analysis of time series data.

While VBTI is currently focused on vision-based automation, these ongoing developments open up new possibilities to make the future of agricultural technology even more efficient and innovative.

* Machine vision is a specialized application of computer vision in industrial automation, while computer vision in general encompasses the processing and analysis of images and video using artificial intelligence.