【2020 Application Example】 Automatic fruit screening system: A solution that uses neural networks, AI, and automation to improve fruit screening efficiency by 10 times, increase output value by NT$1.7 billion, and significantly improve quality with 93% accuracy
34Taiwan is located in the subtropics and has a diverse geographic environment that is very suitable for growing fruit. Bananas and pineapples were once extremely popular export commodities that we are proud of. However, farmers in consuming countries gradually obtained the excellent seeds of Taiwan’s fruits, and were able to grow the same quality fruit but at a more affordable price, causing our fruit exports to face a major crisis! At present, although Taiwan's fruits such as mango and guava still have certain competitive advantages, if they fail to make further progress compared with other countries, they will still encounter the same problem over time and cannot be ignored! Fruit quality and brand value are the only ways for Taiwan's fruit industry to remain competitive internationally.
Fruit screening is the main link in fruit production and marketing that determines quality. Currently, the industry is highly dependent on aging rural manpower, resulting in rising fruit screening costs due to labor shortage and making it extremely difficult to maintain stable yield. Therefore, the automation of fruit screening work has become a very important and urgent issue. Professor Chi-Chun Lee at the Department of Electrical Engineering of National Tsing Hua University led a team to develop an automatic fruit screening system that combines cameras, conveyor belts, and AI. The system currently has an accuracy reaching 93%. One production season can increase the output value of mango by NT$1.7 billion. With the gradual development of the AI system, the accuracy is expected to improve in the future, and the same system can also be applied to other fruits, further promoting traceable fruit and driving the technological upgrading of Taiwan's fruit industry.
Fruit screening relies heavily on scarce manpower, and the aging of the rural population makes the situation even worse
Professor Chi-Chun Lee learned about the fruit industry’s dilemma from his classmate Yu (alias), who had studied together in the United States. Yu is the young second-generation successor of one of Taiwan's largest fruit import and export companies. According to Yu's observations in the industry over numerous years, Taiwan's fruit production and export usually generated good profits at first, but after fruit farmers in the consuming countries obtained the seeds, they will often attempt to grow the fruit locally to reduce costs and obtain greater profits. If Taiwanese fruits cannot surpass the products of fruit farmers in consuming countries in terms of quality or brand value, they will be eliminated because competitors' costs are indeed lower.
Fruit screening is used to divide fruits according to quality. If they cannot pass the minimum specification, they will be discarded as waste products. In practice, the work of screening fruits will be carried out by farmers' goods yards and distributor' packaging yards respectively. However, if it is not properly handled by the collection freight yards and the packaging yards do not do a good job in sampling in the early stage, it will result in a loss for distributors and cause 30% of A/A+ grade fruits to be eliminated.
This job relies heavily on experienced fruit screeners. More experienced fruit screeners can not only control the quality and reduce the chance of fruit damage in the fruit screening process, but also have the ability to pick out about 10% more A+ grade fruits, which adds great value. What worries the industry is that experienced fruit screeners are gradually decreasing due to the aging population in rural areas, making them a very rare resource. Such rare human resources are often in high demand during busy farming periods. Farmers or distributors who fail to hire experienced fruit screeners have to settle for less experienced one, taking on the risk of additional losses and paying greater costs. The most unfortunate situation suffering a loss of 30% mentioned above.
▲ Fruit screening is an important process in the later stages of fruit production when packaging and selling. Failure to properly control quality will result in huge losses.
AI is very suitable for assisting in fruit screening, but it is difficult to obtain data sets
After understanding Yu's difficulties, Professor Lee found that this was a problem that could be solved using AI - although fruit screening relies heavily on experienced fruit screeners, it is a highly repetitive task. Handling repetitive tasks with a large amount of data has always been a strength of AI.
However, the first problem appeared even before research and development work started: Which fruit do we start with?
First of all, a suitable fruit must reach a certain export volume, and the fruit must still have considerable room for growth. For some fruits that lack international competitiveness, such as bananas and pineapples, companies no longer have the ability to invest more funds to purchase equipment, let alone sponsor R&D or assist the R&D team in experiments.
When you have an idea, you need to pick up the pace and put it into practice as soon as possible! Therefore, Irwin mango, which still has a certain advantage in terms of scale, was selected as the first experimental subject of the automatic fruit screening system.
The first step after harvesting mangoes is to screen the fruits for the first time at the goods yard. After the fruits are screened, they are sent to the packaging yard for fumigation and disinfection, and preparation for sale or loaded into containers for export. However, exporters with a deeper understanding of the target market will have stricter quality requirements and will often screen the fruit again to ensure the quality of the fruit before fumigation at the packaging site. Since employees at the goods yard are paid based on the number of mangoes screened rather than on the quality of the mangoes, they focus on quantity when working. As a result, to ensure the quality of the selected fruits, the subsequent packaging factory has to screen the fruit again, increasing labor. The solution seems simple and clear - A camera, machine conveyor belts for grading and sorting, and an AI that can distinguish the quality of mangoes from their appearance are all that are needed to achieve automatic fruit screening. However, the hard part is how can AI distinguish the quality of mangoes? That’s right, you must start by establishing a training data set! In order to create the data set, Professor Lee's team established a website that allows anyone to upload photos of mangoes and rate them. Once the data sets are refined, they can be used to train AI.
▲ The fruit screening machine developed by Professor Lee's team uses AI image recognition to select the best looking mangoes.
The accuracy of the trained AI reaches 93%, which can increase the output value by NT$1.7 billion in one season.
In 2019, the assistance of the Industrial Development Bureau (now the Industrial Development Administration of the Ministry of Economic Affairs) and AI HUB accelerated the verification of the technology.
Professor Lee's team accumulated 100,000 entries of data during the 2-month empirical period, and the accuracy of the trained AI reached 93%! This is far higher than the manual screening accuracy of 70%, resulting in a clear difference in quality. In terms of export value, the output value of mango is expected to be increased by NT$1.7 billion in one season! It can also reduce labor costs by NT$1.866 million and avoid the seasonal labor shortage problem mentioned above.
In addition, since it is no longer necessary to screen the fruit once at the goods yard and packaging yard each, it also reduces losses caused by human error in the fruit screening process. When the technology becomes more mature, the same system can be applied to other fruits exported by Taiwan, such as wax apple and guava, in the future, taking Taiwan's fruit industry to the next level.
Since it is AI, accuracy can be improved through continuous training, and continuous adjustment of algorithms and cooperation with equipment manufacturers can significantly improve production capacity. In addition, Professor Lee is also organizing the AI Cup competition with the sponsorship of manufacturers and the government, allowing more teams to use the same data set to continue to develop the algorithm, in hopes of facilitating further cooperation with companies that are interested.
Irwin mango grade identification system on AI HUB
Professor Lee's team hopes to use the power of AI to achieve complete traceability of fruits from production to packaging and transportation, thereby increasing the brand value of Taiwan's fruits! Besides hoping to allow Taiwan's fruits to seize a place in the fiercely competitive foreign markets, with high-quality supply, Taiwan's fruits can also shine internationally and become the pride of Taiwan.
▲ Taiwan's fruits still have certain competitive advantages in the international market, but they also face competitive pressure from fruit farmers in consuming countries as they are exported.
▲ Easily save NT$1.866 million per mango season and significantly improve quality.
