:::

【2020 Application Example】 Automatic fruit screening system: A solution that uses neural networks, AI, and automation to improve fruit screening efficiency by 10 times, increase output value by NT$1.7 billion, and significantly improve quality with 93% accuracy

Taiwan is located in the subtropics and has a diverse geographic environment that is very suitable for growing fruit. Bananas and pineapples were once extremely popular export commodities that we are proud of. However, farmers in consuming countries gradually obtained the excellent seeds of Taiwan’s fruits, and were able to grow the same quality fruit but at a more affordable price, causing our fruit exports to face a major crisis! At present, although Taiwan's fruits such as mango and guava still have certain competitive advantages, if they fail to make further progress compared with other countries, they will still encounter the same problem over time and cannot be ignored! Fruit quality and brand value are the only ways for Taiwan's fruit industry to remain competitive internationally.

Fruit screening is the main link in fruit production and marketing that determines quality. Currently, the industry is highly dependent on aging rural manpower, resulting in rising fruit screening costs due to labor shortage and making it extremely difficult to maintain stable yield. Therefore, the automation of fruit screening work has become a very important and urgent issue. Professor Chi-Chun Lee at the Department of Electrical Engineering of National Tsing Hua University led a team to develop an automatic fruit screening system that combines cameras, conveyor belts, and AI. The system currently has an accuracy reaching 93%. One production season can increase the output value of mango by NT$1.7 billion. With the gradual development of the AI system, the accuracy is expected to improve in the future, and the same system can also be applied to other fruits, further promoting traceable fruit and driving the technological upgrading of Taiwan's fruit industry.

Fruit screening relies heavily on scarce manpower, and the aging of the rural population makes the situation even worse

Professor Chi-Chun Lee learned about the fruit industry’s dilemma from his classmate Yu (alias), who had studied together in the United States. Yu is the young second-generation successor of one of Taiwan's largest fruit import and export companies. According to Yu's observations in the industry over numerous years, Taiwan's fruit production and export usually generated good profits at first, but after fruit farmers in the consuming countries obtained the seeds, they will often attempt to grow the fruit locally to reduce costs and obtain greater profits. If Taiwanese fruits cannot surpass the products of fruit farmers in consuming countries in terms of quality or brand value, they will be eliminated because competitors' costs are indeed lower.

Fruit screening is used to divide fruits according to quality. If they cannot pass the minimum specification, they will be discarded as waste products. In practice, the work of screening fruits will be carried out by farmers' goods yards and distributor' packaging yards respectively. However, if it is not properly handled by the collection freight yards and the packaging yards do not do a good job in sampling in the early stage, it will result in a loss for distributors and cause 30% of A/A+ grade fruits to be eliminated.

This job relies heavily on experienced fruit screeners. More experienced fruit screeners can not only control the quality and reduce the chance of fruit damage in the fruit screening process, but also have the ability to pick out about 10% more A+ grade fruits, which adds great value. What worries the industry is that experienced fruit screeners are gradually decreasing due to the aging population in rural areas, making them a very rare resource. Such rare human resources are often in high demand during busy farming periods. Farmers or distributors who fail to hire experienced fruit screeners have to settle for less experienced one, taking on the risk of additional losses and paying greater costs. The most unfortunate situation suffering a loss of 30% mentioned above.

▲ Fruit screening is an important process in the later stages of fruit production when packaging and selling. Failure to properly control quality will result in huge losses.

AI is very suitable for assisting in fruit screening, but it is difficult to obtain data sets

After understanding Yu's difficulties, Professor Lee found that this was a problem that could be solved using AI - although fruit screening relies heavily on experienced fruit screeners, it is a highly repetitive task. Handling repetitive tasks with a large amount of data has always been a strength of AI.

However, the first problem appeared even before research and development work started: Which fruit do we start with?

First of all, a suitable fruit must reach a certain export volume, and the fruit must still have considerable room for growth. For some fruits that lack international competitiveness, such as bananas and pineapples, companies no longer have the ability to invest more funds to purchase equipment, let alone sponsor R&D or assist the R&D team in experiments.

When you have an idea, you need to pick up the pace and put it into practice as soon as possible! Therefore, Irwin mango, which still has a certain advantage in terms of scale, was selected as the first experimental subject of the automatic fruit screening system.

The first step after harvesting mangoes is to screen the fruits for the first time at the goods yard. After the fruits are screened, they are sent to the packaging yard for fumigation and disinfection, and preparation for sale or loaded into containers for export. However, exporters with a deeper understanding of the target market will have stricter quality requirements and will often screen the fruit again to ensure the quality of the fruit before fumigation at the packaging site. Since employees at the goods yard are paid based on the number of mangoes screened rather than on the quality of the mangoes, they focus on quantity when working. As a result, to ensure the quality of the selected fruits, the subsequent packaging factory has to screen the fruit again, increasing labor. The solution seems simple and clear - A camera, machine conveyor belts for grading and sorting, and an AI that can distinguish the quality of mangoes from their appearance are all that are needed to achieve automatic fruit screening. However, the hard part is how can AI distinguish the quality of mangoes? That’s right, you must start by establishing a training data set! In order to create the data set, Professor Lee's team established a website that allows anyone to upload photos of mangoes and rate them. Once the data sets are refined, they can be used to train AI.

▲ The fruit screening machine developed by Professor Lee's team uses AI image recognition to select the best looking mangoes.

The accuracy of the trained AI reaches 93%, which can increase the output value by NT$1.7 billion in one season.

In 2019, the assistance of the Industrial Development Bureau (now the Industrial Development Administration of the Ministry of Economic Affairs) and AI HUB accelerated the verification of the technology.

Professor Lee's team accumulated 100,000 entries of data during the 2-month empirical period, and the accuracy of the trained AI reached 93%! This is far higher than the manual screening accuracy of 70%, resulting in a clear difference in quality. In terms of export value, the output value of mango is expected to be increased by NT$1.7 billion in one season! It can also reduce labor costs by NT$1.866 million and avoid the seasonal labor shortage problem mentioned above.

In addition, since it is no longer necessary to screen the fruit once at the goods yard and packaging yard each, it also reduces losses caused by human error in the fruit screening process. When the technology becomes more mature, the same system can be applied to other fruits exported by Taiwan, such as wax apple and guava, in the future, taking Taiwan's fruit industry to the next level.

Since it is AI, accuracy can be improved through continuous training, and continuous adjustment of algorithms and cooperation with equipment manufacturers can significantly improve production capacity. In addition, Professor Lee is also organizing the AI Cup competition with the sponsorship of manufacturers and the government, allowing more teams to use the same data set to continue to develop the algorithm, in hopes of facilitating further cooperation with companies that are interested.

Irwin mango grade identification system on AI HUB

Professor Lee's team hopes to use the power of AI to achieve complete traceability of fruits from production to packaging and transportation, thereby increasing the brand value of Taiwan's fruits! Besides hoping to allow Taiwan's fruits to seize a place in the fiercely competitive foreign markets, with high-quality supply, Taiwan's fruits can also shine internationally and become the pride of Taiwan.

▲ Taiwan's fruits still have certain competitive advantages in the international market, but they also face competitive pressure from fruit farmers in consuming countries as they are exported.


▲ Easily save NT$1.866 million per mango season and significantly improve quality.

 

 

 

 

 

 

Recommend Cases

這是一張圖片。 This is a picture.
CCTV Intelligent Video Search System

Search for a specific person, find someone with a suitcase entering the factory in Gao'an area Color features of the person and the object confirmed, person in blue and black top, suitcase in black color, throughCCTV the intelligent video search system, by setting object and color retrieval conditions, it can successfully locate three video clips containing the target subject This greatly aids operational staff in finding the target items, and through this system, search speed can far surpass manual effort6fold Pain Points The CSE-Kaohsiung Plant is densely equippedCCTVto monitor every corner of the plant area, but when an incidenthappens, it's impossible within a limited time throughCCTVvideo playback to find the incident, the implications and risks behind this are self-evident Many areas that are usually unmanned can easily become security blind spots Thus, how to monitor a vast plant area more intelligently and effectively is one of the crucial aspects of building a smart plant for the semiconductor industry The AES Plant in Kaohsiung covers a vast area, with many important sites requiring monitoring of personnel movements to ensure corporate secrets and employee safety 1 Automated production lines and warehouses In semiconductor enterprises’ automated production lines and warehouses, oftenAGV(Automated Guided VehicleAGVs automated guided vehicles travel at high speeds if plant personnel inadvertently enterAGVthe moving area and cannot issue a warning to the person, then the regrettable accidents that occur will be too late to reverse 2 Material and product storage areas Materials used in semiconductor-related processes are costly if areas storing materials or products are breached, there is a risk of loss of high-value materialsproducts 3 High-security areas Trade secrets relate to the core technological competitiveness of semiconductor-related enterprises if someone breaches the high-security areas, there is a risk of corporate secrets being leaked The safety of trade secrets has always been one of the most critical issues for semiconductor enterprises 4 Loading docks At AESLButthe dock area often has loading vehicles coming and going if someone intrudes into the dock area, there is a risk of vehicle collisions and accidents Additionally, goods awaiting shipment at the dock area could be stolen or potentially damaged from collisions, thus causing significant reputation and financial losses for the company, further leading to production and shipping inconvenience When an abnormal event occurs, how to quickly search for the relevant key footage from massive data Many important locations within the AES Kaohsiung Plant need to be equippedCCTVfor safety checks, butCCTVWith thousands to tens of thousands of cameras, manually searching through footage for an event requires laborious frame-by-frame review which is time-consuming and inefficient In light of advancements in computer vision, it's beneficial to utilizeAIto replace manual playback and searching Problem Scenario Object Detection The data source for object detection comprises two parts Open-source datasetsOIDv4and AES Kaohsiung PlantCCTVImage files For these files, search for usable data, specificallyOIDv4image files For these files, extract the defined nine major categories of objects for training data among them, two object categories, knives and gasoline barrels, were not found inOIDv4found usable data for knives and gasoline barrels, while the remaining seven categories of objects are available fromOIDv4useful training data found for the remaining seven categories of objects, all marked Regarding the Kaohsiung PlantCCTVimage files, select some frames Frame of the footage, and manually annotate the objects to be_detected for training and testing data Nine Major Objects Color Recognition The data source for color recognition is divided into two partsInternet image screenshots, and Kaohsiung PlantCCTVimage files Currently, no publicly available open-source datasets specifically for color recognition applications have been found, so images are collected from the web Search the web for images of the defined nine major object categories, save the images after separating the objects from the background, keeping only the object sections, and mark the images according to color Additionally, for the Kaohsiung PlantCCTVimage files, use the already-markedbounding boxextractCCTVimage files from variousFramesections of objects identified by color, and finally, visually identifiable images are marked according to color Each object category has its specific color definition, depending on the usual colors seen in these objects in real life Dynamic Ignore during Training FromOIDv4during the training of the object detection pilot model, since each image in this dataset is only marked for a single category, but the image may contain other desired detection categories unmarked For such cases, dynamic ignore techniques will be employed during training to avoid confusion Next, use the extracted training data from the Kaohsiung Plant toFine-Tuneenhance the detection rate of the object in specific designated areas Finally, select the model that computes the lowest loss value in the test set during the training process as the main object_detection model Dynamic Ignoring AIHelp You View CCTV The intelligent video search system primarily serves as an assistive system for searching surveillance footage, capable of speeding up the process of finding target events by setting search conditions for objects By simply defining the search conditions, you can quickly produce thumbnails of critical objects and playback for review, shortening the time required for manual case retrieval of the past The search time is quickly6doubled, allowing the front-end security unit to use this platform to strengthen the first line of risk management supervision and take timely preventive measures 「Translated content is generated by ChatGPT and is for reference only Translation date:2024-12-12」

【導入案例】巨量遙測空間數據AI分析雲端服務平台 使衛星遙測影像順利落地應用
AI Analysis Cloud Service Platform for Remote Sensing Big Data Enables the Smooth Application of Satellite Remote Sensing Images

Although satellite remote sensing images can make all surface objects visible, it still requires a lot of time and manpower to be truly applied to the industry In order to effectively solve the problems that customers face in digesting huge amounts of image data and eliminate technical obstacles for cross-domain users to process satellite remote sensing images, ThinkTron has developed an "AI Analysis Cloud Service Platform for Remote Sensing Big Data" as a new beginning for cross-domain AI applications for spatial information In recent years, in response to the impact of industrial globalization, Taiwan's agriculture has continued to transition towards technology-based and higher quality, improving the yield and quality of crops by solving problems, such as microclimate impacts and pest and disease control The demand of agriculture on images has expanded endlessly to accurately grasp the growing environment of crops In the early years when UAVs unmanned aerial vehicles were not yet popular, manual field surveys were the most basic but most labor-intensive work With the emergence of UAV drones, aerial photography operations might not be difficult, but the range that can be photographed is limited Furthermore, surveying expertise is required to accurately capture spatial information At this time, the use of satellite remote sensing data may break away from the past imagination of using image data Taiwan Space Agency TASA ODC data warehouse services In the past ten years, with the breakthrough of modern satellite remote sensing application technology, Digital Earth has become a new trend in global data collection Countries have developed data cube image storage technology, and the development of smart agriculture has become one of the largest image users Determining planting distribution is the first step in understanding crop yields Free satellite remote sensing images, powerful data warehousing support, and the team's robust image recognition technology are important supports for accelerating agricultural transformation Using satellite remote sensing image data can accelerate the development of smart agriculture However, in the past, it was difficult to extract large-area crop distribution through satellite remote sensing images, not to mention the cost If you wanted to use free information, you had to visit all websites of international space agencies, look through the wide variety of satellite specifications, and carefully evaluate the sensor specifications, image resolution, and revisit cycle After finding suitable images, you had to look at them one by one to filter the ones you need Next is downloading dozens of images that are often several hundreds of Megabytes MB each, which might exceed the capacity of your computer Also, after overcoming image access and preparing data, you must then start to confirm the downloaded image products and which bands you want, because the image you see is not just an image file jpg or png, but rather complex multi-spectral information, attribute fields and coordinate information It takes a lot of effort just to confirm the correct information Facing GIS software packages with complex functions is the start of another trouble The complex image pre-processing process and the inflexible machine learning package greatly reduce the efficiency of analyzing data After finally getting the results of crop identification, you might find that the best time for using map information may have already passed The above-mentioned complex and time-consuming satellite image processing problems are precisely the pain points of the market ThinkTron expanded from traditional machine learning to modern deep learning applications, and developed an "AI Analysis Cloud Service Platform for Remote Sensing Big Data" under the GeoAI framework, breaking through the constraints of details in the spatial information for customers Differences between the process before and after introducing the AI analysis cloud service platform ThinkTron said that Taiwan's ODC Open Data Cube system has been completed and began providing services after years of efforts from the Taiwan Space Agency TASA, formally becoming aligned with international trends The powerful warehousing technology allows users to easily capture and use image data of a specific time and spatial range according to their needs The warehouse stores multiple satellite image resources from international space agencies, including the ESA's Sentinel-1 one image every 6 days, Sentinel-2 one image every 6 days, USGS's Landsat-7 one image every 16 days, Landsat-8 one image every 16 days, and the domestic Formosat-2 one image every day and Formosat-5 one image every 2 days ThinkTron develops satellite image recognition tools based on Python Breaking free from the limitations of GIS Geographic Information System software packages, ThinkTron integrated GDAL Geospatial Data Abstraction Library based on Python, and considered computing efficiency and parallel processing when developing all tools required for satellite image processing and image recognition modeling, including coordinate system and data format conversion, grid and vector data interaction, and data intra-difference and normalization All of the tools are designed with AI applications in mind, and some commonly used tools are packaged into an open source package under the name TronGisPy to benefit the technical community ThinkTron utilized the team's understanding of satellite remote sensing images and the collected tagged data crop distribution maps to preset the image recognition modeling process, the required training data specifications, and dataset definitions This is imported into the machine learning LightGBM or deep learning CNN framework that was completed in advance, and the entire training process to be performed in the Web GIS interface, providing users with partial flexibility to freely filter images, confirm spatial and temporal ranges, select models, and adjust hyperparameters In addition to the operation of training models, it also provides historical models to output identification results, and finally displays the identification results of crop distribution on the Web GIS map In fact, agriculture is not the only industry that needs satellite remote sensing applications AI applications of spatial information have also appeared in various fields as companies in different industries aim to enhance their global competitiveness For example, surveying and mapping companies that have a large amount of map data can use the AI analysis cloud service platform to store map data while also accelerating the efficiency of digital mapping Under the severe global climate change and the risk of strong earthquakes, there is a wide variety industrial insurance, agricultural insurance, financial insurance, or disaster insurance are all inseparable from spatial information The use of remote sensing image recognition to understand insurance targets has long been an international trend AI Analysis Cloud Service Architecture for Remote Sensing Big Data

這是一張圖片。 This is a picture.
AI-Based PCBA Surface Defect Detection Improvements

With the introduction of theAOIAIWith the introduction of the system, we can improve product yield, reduce costs, and from a business perspective, increase customer trust and sales revenue Moreover, AIit has advantages that are difficult to imitate, unlike other equipment that can be bought with money, making it hard for our competitors to catch up with us Our company's current development We are committed toIOTsmart manufacturing our systems already include smart materials systems, environmental humidity control systems, anti-miscarriage systems, smart procurement computation systems, smart inventory systems, solder paste management systems, and production management systems We have asked other manufacturers about the possibility ofAIinspectingPCBAsurface defects, each hoping that we would purchase their equipment, but none were effective upon verification After discussing with IT service providers, we defined it asAOIAIa feasible operational model Tzuhong Technology has invested inAOIAIan inspection plan to checkSMTtext on components, solder joints, polarity, missing partsand usingAIto replace manual learningAOIand define the 'potentially defective' parts, enhancing productivity and reducing misjudgment rates Industry pain points Taiwan faces a severe labor shortage, especially those willing to perform visual inspections are few and typically older, increasing the frequency of missed inspections Thus, the most critical bottleneck in the pursuit of high-quality electronics has become post-production inspections Previous consumer products with undetected anomalies were acceptable within a certain ratio However, in the automotive industry today, undetected defects could lead to fatalities hence, the automotive industry has extremely high quality demands To survive in the automotive supply chain, we must address the issue of undetectable anomalies Moreover, as wages in Taiwan continue to rise, we can only endeavor toAIreplace traditional manpower with technology, otherwise, even if the anomaly leakage problem is resolved, the relatively high labor costs will still prevent competitiveness in this industry Application technology and explanation Initially,Figure 1,PCBUpon emerging,Reflowsystem, it will undergoAOIwill undergo inspection, dividing into 'suspected defective' and good products At this point, the 'suspected defective' portion accounts for20manual review for these20parts, further classifying the 'suspected defective' portion into good and defective products With We aim to leverageAItechnology, to shift from manual re-inspection of these20technology, we aim to replace manual review of 'suspected defective' products withAIand after review, the results still yield 'good' and 'suspected defective' products, but now 'suspected defective' comprises only3thus reducing the workload of Tzuhong's employees from20down to only3In theory, it isAOIIn theory, after inspection, it is further reviewed byAIbut it appears to go throughAOIonly, so we call this technologyA0IAIDetectionFigure 2。 The original AOI inspection process The operator will place the testPCBboard intoAOIthe inspection equipment, outputtingAOI information on defective products, then manually re-inspect one by one to determine if they are defective AOIAI inspection process The operator will place the testPCBboard intoAOIthe inspection equipment, outputtingAOIinformation on defective products after, then proceed byAIfirst performingAOIre-assessment of defective products, outputtingAIinformation on defective products afterward, then manually re-inspect one by one to determine if they are defective Process differences By introducing theAOIAIsystem, not only can we enhance the efficiency and yield of visual inspection personnel, we also have this timeAIexperience in system introduction, we will also incorporateAIthe use of big data into Tzuhong's existing smart manufacturing systems, further enhancing the performance of our smart manufacturing systems and reducing the pressure on employees Difference between pre and post-introduction Promotion strategy 1 Similar field diffusion allSMTmanufacturers face bottlenecks in inspections leading to shipment delays introducing this system can solve the severe labor shortage issue and enhance shipment speed and quality, allowing self-promotion to customers or through equipment dealers to cater to relevant needs 2 Cross-industry expansion plans negotiate withAOImanufacturers to directly integrateAIthe system intoAOItheir systems, enhancing their market competitiveness Profit strategy 1 In collaboration withAOImanufacturers, collect licensing fees 2 Direct sales toSMTthe manufacturing industryAIsystems 3 ProvideSMTmanufacturing industryAOIAIsystem subscription model「Translated content is generated by ChatGPT and is for reference only Translation date:2024-12-09」