:::

【2019 Application Example】 How can public restrooms rely on IoT and cloud technology to become cleaner, solve 70% of customer complaints, and increase efficiency by 120 times?

IoT smart restroom: A revolution of clean, power-saving, and convenient new smart restrooms

Six sensors are used to detect toilet paper, hand soap level, water leakage, odor, people flow, and toilet usage conditions, and combined with NBIoT transmission, cloud system, and LINE robot. It greatly reduces customer complaints and improves the efficiency of replenishing consumables in restrooms. Coupled with real-time notifications, it can prevent illegal smoking in restrooms and improve safety. Users will no longer face the dilemma of wet, dirty, smelly restrooms, or toilet paper running out, greatly upgrading their experience.

What is your impression when you walk into a public restroom in a popular tourist area? No hand soap? No toilet paper? Or even a dirty, smelly, and leaking restroom? The IoT big data smart restroom solution of the Institute for Information Technology (III) solves all inconveniences of restrooms all at once.

According to statistics of the Environmental Protection Administration (EPA), Executive Yuan, there were more than 43,000 public restrooms registered and managed in Taiwan as of the end of September 2019, but the entire EPA only had over 34,000 people. Cleaning and managing such a large number of sites is obviously not an easy task. Coupled with the inevitable arrival of an aging society, the number and quality of personnel cleaning restrooms will inevitably encounter unprecedented bottlenecks. The introduction of effective service processes and assistance of technologies has become a major issue that must be faced sooner or later.

The IoT smart restroom service solution demonstrated by the III at over 20 restrooms around Taiwan may provide a good direction for us to solve this problem.

Overwhelming number of customer complaints, four major problems, and three solutions of the III

In 2016, when the MRT Songshan Station, which is connected to the train station, was officially opened, the public restrooms that were already at full capacity resulted in serious customer complaints due to the overwhelming use. Songshan Train Station, which originally had an average daily passenger volume of only 40,000, was already near a bottleneck in service capacity. After the connected MRT Songshan Station was opened, the number of passengers increased to 70,000. The restrooms that were already near full capacity were completely unable to cope with the additional passenger volume after the MRT station was opened.

Cao Xueqin once wrote a classic line that touched people's hearts in the novel "A Dream of Red Mansions": "When a wall is about to collapse, everybody gives it a shove" may be able to describe this phenomenon: The toilet paper and hand soap in each restroom was never replenished in time, the sinks were dirty, and the toilets could never be cleaned in time. There was an overwhelming number of customer complaints about the restrooms as a result. In addition, the public restrooms of Songshan Train Station are closer to the main passageways of passengers than the public toilets of MRT Songshan Station. At this point Songshan Train Station had to face and solve this problem.

Since Songshan Train Station has worked with the III for a long period of time, it commissioned the III to help solve this troublesome problem.

Edison has a famous saying: "If I find 10,000 ways something won't work, I haven't failed. I am not discouraged, because every wrong attempt discarded is another step forward." The first thing that the III needs to do is conduct pain point analysis and think about the underlying problem. After reviewing customer complaints and discussing and analyzing them with front-line cleaning service companies, the III found four problems and three solutions:

The four problems are: Toilet paper and hand soap are not promptly replenished, sinks are damp, and the space has a foul smell.

The three solutions correspond to these four problems respectively: 1. Delicacy management of consumables such as toilet paper and hand soap. 2. Digitize the key performance indicators (KPI) in the service process, such as the dampness of the sink, or the odor concentration in the space. 3. Use new Internet of Things (IoT) technology to implement the first two solutions, and assist big data and cloud technology in achieving efficient site cleaning management.

"Technology features and R&D process"

The combination of six key sensors with IoT cloud motherboard and big data, thoroughly resolving 70% of customer complaints and increasing efficiency by 120 times.

I. Delicacy management of consumables

To achieve delicacy management of toilet paper and hand soap, the first step is to develop sensors to detect these two consumables.

Starting in 2017, the III began to design the first infrared toilet paper detection module. The module mainly uses the physical characteristics of toilet paper usage habits for detection: Under normal use, toilet paper is placed on an iron drum holder, and its thickness slowly becomes thinner as it is used.

This module requires the combination of a position sensitive detector (PSD ), infrared emitting diode (IRED), and signal processing circuit (SPC) to effectively determine the length of toilet paper with accuracy reaching one decimal place.

When the detection module was first developed, there were no designs that could be referenced, so sensor selection, circuit board designing and planning, sensor programming, and even the light-cured 3D printed casing design were all completed in the III.

Public restroom field detector demonstration image


However, despite overcoming all the difficulties in designing and producing the toilet paper sensor, there was no way to foresee that fixing the sensor in place would be the most difficult problem.

Public restroom sensor display picture


The III team shared with us: “At first, we used hot melt adhesive to fix it in place, but cleaning personnel needed to open and close it every time they replenished toilet paper. The sensor fell due to too much vibration and not being firmly fixed in place.

The worst situation was in the women's restroom: When a female passenger was using the toilet, the sensor was not properly fixed in place and fell. Don’t you think this sensor looks like a pinhole camera? If something like this suddenly falls on the ground in the women's toilet, how bad do you think it will be? (laughs)

Fortunately, our superiors supported us, and we continued to develop the technology until we were able to successfully fix the sensor firmly in place. Otherwise, this project would have been aborted a long time ago."

Sensor with line interactivity demonstration


After the toilet paper detection module was launched, an inspection of toilet paper usage that once took cleaning personnel 15-20 minutes to complete now only takes 10 seconds by opening the app. This greatly improved efficiency by 120 times.

Now that the consumption of toilet paper has been solved, the next problem is detection when hand soap is at a low level.

Unlike toilet paper, the amount refilled each time for hand soap isn't always the same. Because the design philosophy was to use the lowest cost and most stable components to complete this function to facilitate future scaling, a common Hall sensor was chosen. It was mounted on the exterior of the soap dispenser to achieve the detection of low soap levels.

The principle is actually very simple. Once the liquid level is lower than a certain percentage, the Hall effect sensor can sense the change in voltage from electromagnetic induction of the liquid level. The sensor sends a signal to the back-end cloud server, and then the server then sends a message to cleaning personnel the same as the toilet paper sensor.

II. Digitization of key performance indicators (KPIs) in service processes

If the sink is wet, water will often seep onto the floor. In addition, the bottom of passengers’shoes will inevitably carry dust, so the floor will become dirty when they step on the wet floor. Visually, this will give people a sense that the “restroom is dirty." However, it is impossible to have cleaning personnel on duty in the restroom at all times, so a special sensor is needed to detect this situation.

The III uses the resistance characteristics of thin film resistors. When there is liquid on the surface of the thin film resistor, it will lower the overall resistance value and further change related values of the analog signal output. In this way, moisture can be detected by simply laying thin film resistors on surfaces that often become wet. For example, next to the windowsill or on the sink.

However, since sensors are relatively expensive and scratches will damage the performance of the sensors, this moisture detection sensor is only used in specific public restrooms.

Apart from looking dirty, if a foul smell comes from a public restroom, people will think it is dirty even if it looks bright and clean.

However, odor detection is not that easy to solve.

At first, we searched all kinds of sensors in Taiwan and overseas to find this "electronic nose." We eventually found a suitable MEMS chip in the product line of a major Japanese manufacturer that specializes in the production of gas sensors.

The III started from breadboard testing, circuit design drawings, to outsourced chip production, taking nearly six months to complete the design of the sensor.

Furthermore, in the process of developing smart  restrooms, we also received requests to develop other modules, such as people flow detection and usage detection.

感測器配置於洗手台下方呈現


During the development process, we found that users may accidentally close the door of some accessible toilets after use and forget to turn off the lights, so it seems as if the toilet has been occupied all day long. However, people who really need to the toilets are blocked outside the door of accessible toilets that are actually vacant. This problem was relatively simple. The engineer found a ready-made people flow sensor module and installed it under the sink, and the problem was easily solved.

In addition, environmental protection and carbon reduction requirements are hard to meet for some remote public restrooms, such as Tri-Mountain (Lishan) National Scenic Area. Due to the remote location, responsible personnel must turn on the lights every day at work and turn off the lights when they get off work. Sometimes not many tourists use the public restroom all day long, but all the lights and equipment are still on all day long, which is a waste of electricity.

Generally, commercially available sensors are very dull and will turn off the power as soon as the set time of 30 seconds to 10 minutes is up. Such a sensor may be adequate at home when only one person uses the toilet. However, in a restroom that can easily reach 60 ping or above, several detectors will be needed to work together to ensure whether there are still users in the restroom. This is another problem without a commercially available solution. The III had no choice but to integrate multiple sensors and develop algorithms on the MCU to solve this problem.

III. The introduction of new IoT, cloud, big data, and 5G NBIoT technologies

On the path of innovation, there are always difficulties waiting for engineers to overcome. In the process of solving problems as they come, we also refined the solution step by step, making it cheaper, more reliable, and more convenient.

After the sensors described above were completed, the system gradually generated new problems for the III to solve. For example, the barrier of user habits, power consumption issues, cost issues, etc.

The app was changed to a LINE group robot to become more aligned with user habits

透過電信商、技術商、服務商擴散至場域的 Line 服務擴散系統圖

When the public restroom of about 60 ping at Songshan Train Station was completed for the first time in 2017, MCU and WIFI communication were used to monitor and transmit data to the server around the clock. After the system determines an abnormality, it uses the mobile app developed by the III to notify cleaning personnel.

This design seems to be impregnable at first glance. However, the average age of on-site cleaning personnel is over 50 years old, no one used the dedicated app, and front-line personnel often deleted the program within a few days of use. There is a whole set of sensors monitoring, but no cleaning personnel actually use it. User habits are often the biggest obstacle to the introduction of new technologies.

After conducting user interviews we found that the cleaning personnel of every public restroom have a LINE group.

廁所設備 LINE 群組溝通使用圖

The III team mentioned: "Knowing that they ( cleaning personnel ) have a LINE group makes things easier!

At first, we cautiously asked the cleaning personnel if they would invite a robot "new colleague" to help inspect toilet paper and determine abnormalities in the restrooms.

At the beginning, the cleaning ladies were a little skeptical. When they discovered that this robot "new colleague" was very useful, they fell in love with it."

Due to cost, environmental protection, and convenience issues, WIFI was upgraded to NBIoT communication protocol.

WIFI is fast and has wide bandwidth. A restroom has a men's room and women's room, which requires two separate systems for monitoring, and each system needs an independent 4G network to connect to the cloud system. Therefore, the construction and communication costs are relatively high, and the power consumption is also relatively high.

At this point, readers may have questions: Public restrooms are all set up in public spaces. Is there no public WIFI network available?

The III team gave us a very in-depth answer: "Actually, almost every public space has a WIFI network that can be used. However, sharing WIFI with other people is prone to interference, and IoT devices are simple and lack security control mechanisms. If you use public WIFI, there is a certain degree of security. risk.

Therefore, in our solution, we still designed a closed WIFI communication system to solve the communication problem.

In addition, since a WIFI base station can only support 20-30 nodes, a women's room with 18 toilets requires a separate systems. Coupled with the fact that it is separated by a concrete wall, the signal will be very weak and even affect the stability of the signal. Therefore, a public restroom installing two systems is mainly due to stability considerations rather than cost considerations."

In densely populated areas, using WIFI to transmit data to the server is not too troublesome. However, when smart restroom systems are beginning to be applied to restrooms in remote areas, such as Lishan, Guguan, Shitoushan and other public restrooms of national park visitor centers, maintaining network connection is indeed a difficult problem.

Fortunately, new generation mobile communication networks of 5G includes narrow-band Internet of Things (NBIoT) specially designed for the Internet of Things. The III is the first in Taiwan to develop Taiwan's first NBIoT MCU control system designed for smart restrooms using the NBIoT chipset of a domestic chip manufacturer.

In addition to the significant cost reduction, this system is also very energy efficient, requiring only 1/6 of the power of the original WIFI system. The most important thing is that compared to traditional WIFI, which requires a relatively stable 4G signal connection, this system has wider coverage and allows communication deep in the mountains and out in the wild. This allows wider coverage of smart restrooms in the future without being limited by network signals.

IV. "Effect Analysis and Future Outlook"

IoT smart toilet: A revolution of clean, power-saving, and convenient new smart toilets!

As the complete set of sensors, cloud system, NBIoT, and LINE robot are gradually launched, the benefits are clear.

In the case of public restrooms at Songshan Train Station, from being overwhelmed at first to greatly reducing the number of customer complaints by 70%, the time required to inspect toilet paper use was shortened from the original 15-20 minutes to only 10 seconds. Once an abnormal situation occurs, it has gone from being undetected to the prompt notifications today.

Interestingly and unexpectedly, this entire system also brings the added benefits of safety and thorough enforcement of tobacco hazards prevention laws. When a toilet is occupied for more than 40 minutes, a warning will be sent to the cleaning personnel group. Hence, when a user occupies a toilet for too long, cleaning personnel will knock on the door. This greatly improves safety.

In addition, odor detectors are also very sensitive to the smell of smoke. Since smoking is prohibited in national parks, tourists sometimes sneak into public restrooms in remote areas to smoke. In public restrooms of national parks, once the odor detector detects the smell of smoke, it will play a voice message about the Tobacco Hazard Prevention Act to let tourists clearly know that smoking in public restrooms will result in a fine of NT$2,000 to NT$10,000. Since the installation of odor detectors, the number of users smoking secretly in public restrooms has significantly decreased.

The "smart public restrooms" at Songshan Train Station won the "Golden Way Award" from the Ministry of Transportation and Communications for overcoming various difficulties, which made it famous. From a constant stream of customer complaints to model public restrooms that the public sector has enthusiastically visited, the additional workload on the case officer from handling group visits is actually a luxury to be worrying about.

Future Outlook

The system has proven its stability and cost effectiveness during the three years of R&D and field experiments, and has now been successfully transferred to domestic system integration companies. The III also hopes that this system can be expanded in the future, and the technology can even be transferred to Europe and the United States.

In addition, on the basis of stable and reliable data flow and communication connections, the introduction of big data for analysis may make the deployment of manpower more delicate, and the problem of uneven work distribution can be expected to be fundamentally corrected.

Facing the arrival of an aging society, NBIoT communication systems, combined with various IoT sensors, may be able to bring us a healthier and safer living environment. Some repetitive tasks that traditionally relied heavily on manpower can also use technology to greatly improve efficiency.

Recommend Cases

【導入案例】無人智慧販賣機 黑沃咖啡一分鐘打造精品咖啡
Unmanned Intelligent Vending Machines - Black Wo Coffee Creates Boutique Coffee in a Minute

Technology also carries the aroma of coffee Situated on Gaogong Road in the Southern District of Taichung, the original Black Wo Coffee store covers a space of 28 ping, filled with the scent of coffee mixed with cultural creativity and technology Since its establishment in October 2016, Black Wo Coffee has expanded to 7 directly managed stores and 28 franchise stores across Taiwan Among the 15,000 coffee sellers nationwide, Black Wo Coffee has risen uniquely through the use of AI technology to create an unmanned intelligent vending machine that brews exquisite and aromatic coffee in just one minute Black Wo Coffee's physical store creates a culturally creative and fashionable atmosphere Image Black Wo Coffee official website According to the International Coffee Organization ICO, Taiwanese people consume 285 billion cups of coffee annually, with the market size exceeding 70 billion NT dollars Ambitiously, as per Starbucks' survey, by 2018 the overall Taiwanese coffee market reached 72 billion NT dollars and rose to 90 billion by 2020 Over the last five years, the Taiwanese coffee market has expanded annually by about 20, showing remarkable growth potential Coffee demand presents incredible business opportunities, growing at a rate of 20 annually With coffee now being a symbol of fashionable consumption in Taiwan, aside from first-tier coffee shops like Starbucks and Louisa, there are convenience stores like 7-11 and FamilyMart, and numerous boutique coffee houses scattered through the streets and alleys How to capture consumer attention and stand out in the 'red ocean' of the coffee market requires flexibility and creativity, understanding consumer needs and tastes, which are also essential for cultivating brand loyalty Beyond physical storefronts, Black Wo Coffee is also actively developing digital channels Its ecommerce platform includes the official website, PChome, momo, and group-buying hosts, providing multiple channels and ensuring steady growth in performance Even so, the founder of Black Wo Coffee, Lin Pei-ni, continually seeks innovation Due to the passive and scattered situation with franchise stores in the first three years, it was difficult to actively grasp market trends and the company noticed a certain lag in communicating with consumers and keeping up with brand dynamics, making it challenging to cultivate loyal brand advocates Artisan boutique coffee is deeply beloved by consumers Image Black Wo Coffee official website Through the AI Eagle Eye System, market intelligence costs are significantly reduced To address the dual challenges of not being able to quickly capture market trends and high market research costs, Black Wo Coffee introduced the AI Eagle Eye System in 2020 to scout market intelligence By comprehensively crawling articles from social websites, news platforms, and forums, automatically tagging, and suitably filtering, this system scanned 4,858 articles using 24,290 keywords, enabling precise insights into consumer preferences at minimal costs At the same time, after launching new products, not only can franchise stores be notified promptly, but the acceptance level of consumers can also be assessed through social platforms It serves as a reference for whether to promote aggressively Through the collection of data and analysis by AI algorithms, consumer-preferred flavors are selected, reducing the risks associated with new launches and increasing the success rate of new products Therefore, in 2021, Black Wo Coffee boldly explored new markets by introducing the world's first AIoT smart coffee innovative concept in collaboration with Pxmart for the first 'Intelligent Supermarket', integrating Black Wo Coffee to create an unmanned intelligent hand-drip coffee machine for consumers to enjoy a unique flavor experience Insight into consumer tastes leads to the creation of AIoT Unmanned Intelligent Vending Machines Taiwan's first Pxmart 'Intelligent Supermarket' in Neihu, Taipei introduces the world's first AIoT smart coffee concept store, able to interact with the AI smart coffee vending machine, AI hand-washing coffee machine, and AI vacuum cold brew machine through a mobile app, meeting three different coffee technology experiences in one place The self-service area features the only unmanned intelligent coffee vending machine in Taiwan that uses chilled milk to make milk foam, selecting Black Wo's 5A grade milk, and completing the payment, grinding, and brewing all within one minute The first Pxmart 'Intelligent Supermarket' was established on Ruiguang Road in Neihu District, Taipei Image Pxmart FB fan page The Pxmart Intelligent Supermarket features an AI smart coffee vending machine, which is operated using an app to enjoy aromatic coffee Image Pxmart FB fan page Now, with the addition of AI technology elements, drinking coffee is not just about having coffee it also brings more brand-new tech experiences and conveniences to consumers「Translated content is generated by ChatGPT and is for reference only Translation date:2024-05-19」

【解決方案】佐翼科技無人機導入高爾夫球場域 可節省一半人力
Droxo Tech Applies Drones in Golf Courses to Reduce Manpower by Half

For most golf courses, the operations and management is a headache "Golf courses are selling turf and need to be properly taken care of," a golf course manager bluntly pointed out Facing the market pain points of labor shortage, aging population and high cost, the use of AI drones for pesticide spraying and pest control will reduce labor costs by more than half and greatly improve the overall operational efficiency At noon in early summer, an AI drone is slowly taking off at the Taipei Golf Club in Taoyuan Its main task is to test AI drone fertilizing and pesticide spraying on the golf course In fact, drones of Droxo Tech, the company performing this task, are widely used for fertilization, pesticide spraying, and pest and disease control for rice, bananas, and tea trees For golf courses with turfs that often cover tens to hundreds of hectares, AI drones are needed to assist in turf maintenance Data collection, development of pesticide spraying AI models, and multispectral image analysis and testing will be carried out in the current stage In the future, large-scale technology implementation and verification will be carried out to set an example for applying drones to golf courses Using AI drones to fertilize and spray pesticides can reduce the manpower required by half The traditional way of maintaining the turf in golf courses is to carry spray buckets or drive spraying vehicles to spray areas one by one "Domestic golf courses began to plant ultra-dwarf Bermuda grass in 2001 This grass species prefers a cool climate and is not suitable for Taiwan's hot and humid weather" Droxo Techrsquos CEO further pointed out that to prevent turf from pests and diseases, pesticide spraying is necessary For an 18-hole golf course, it is equivalent to spraying pesticides once a week, and the T-ground and fairways are sprayed every two months For golf courses, spraying pesticides is time-consuming and labor-intensive It is important to note that large-scale spraying will increase the risk of personnel poisoning and increase the amount of pesticide used Benefits of applying agricultural drones to golf courses According to Droxo Techrsquos research, golf course pests include Spodoptera litura, which comes out at night to look for food, so pesticide spraying must be carried out in the evening According to the traditional method, pesticide spraying requires two vehicles and three personnel for a total of 45 hours If AI drones are used for fertilizing and pesticide spraying, it only takes one operator to spray 08 hectares of land in 20 minutes, saving about two-thirds of the manpower and reducing operating costs by about 30 Using AI drones to fertilize and spray pesticides on golf courses can reduce the manpower required by half In addition to the significant benefits of using agricultural drones for golf course turf maintenance, Droxo Tech also specially introduced AI multispectral image recognition for NDVI Normalized Difference Vegetation Index analysis "The so-called multispectral is to direct light with different wavelengths on the turf, and the reflected images are collected for analysis" Droxo Tech CEO Liu continued to explain that each plant absorbs light with different wavelengths, so multispectral imaging can determine the growth status of grass species At the same time, combined with AI image recognition, the distribution of pests and diseases can be accurately detected, and the amount of pesticide used is determined on this basis Cross-domain collaboration to build a multi-source turf image databasenbsp Using AI multispectral image recognition technology, Droxo Tech will collect visible light, multispectral, thermal images, and hyperspectral images to establish a multi-source turf image database to fully understand the growth cycle of Bermuda grass Droxo Tech has accumulated rich experience in agricultural AI drone pesticide spraying , but there are still many problems that need to be overcome to implement AI solutions in golf courses For example, it is necessary to establish a new pesticide spraying model and test flight methods, especially the application of multispectral image recognition PoC is not difficult, but actual implementation requires more test evidence, repeated inferences, and collaboration with plant experts This part must rely on the cross-domain integration of legal entities such as the Institute for Information Technology III, gathering more fields for verification, and creating a paradigm before it can be more widely adopted by golf courses There are not many international cases on the application of AI drones in golf courses During the verification process, it is not yet known whether it can be quickly copied to the next golf course However, Droxo Tech CEO Liu believes that through cross-domain collaboration, clearly defining the problems and listing them one by one, supply and demand parties can reach a consensus, propose solutions to each problem, and seek cooperation with internal and external resources Only then will we be able to gradually achieve the goal of making golf courses smarter and smoothly assist the industry with transformation Zuoyi Technology's CEO, Liu Junlin 「Translated content is generated by ChatGPT and is for reference only Translation date:2024-05-19」

【導入案例】赫銳特科技VCSEL封裝元件瑕疵導入AOI檢測 提升產能效率20
HRT Technology Improves Production Efficiency by 20% Through AOI Detection of Defects in VCSEL Packaging

In 2017, the launch of the iPhone X made 3D sensor technology used in Face ID highly popular, which drove the development of VCSEL, a core component in the 3D sensor module In the detection of defects in incoming packaged VCSEL, the use of AI inference models can solve the industry's issue with low yield and improve reliability to 95 VCSEL technology currently can be used in many applications and various end consumer markets, including robots, mobile devices, surveillance, drones, and ARVR VCSELs are a good solution in applications that require high-speed modulation capabilities, such as cameras and biometrics VCSEL technology has a wide range ofnbsp applications, including in drones Pictured Zoyi Technology's Agricultural Drone VCSEL technology has a wide range of applications, AI technology assists in defect detection HRT Technology stated that the packaged VCSEL market is also facing strong price competition from competitors, and needs to further reduce costs and enhance product competitiveness One of the key problems is the replacement of glass lens with epoxy resin lens The production of traditional glass lenses has high yield, but the cost is higher than that of epoxy resin lenses Due to the cutting process of epoxy resin, the side wall of cutting lines can easily have rough edges, causing it to be oversized The release of stress caused by heat during the mounting process will directly cause the optical lens to break HRT Technology pointed out that the incoming inspection of VCSEL epoxy resin lenses is very important Under the constraints of packaging space, the space for fitting the package and optical lens is limited Moreover, the optical lenses will be confined to a metal frame If the dimensional tolerances are properly controlled, stress release due to heat during mounting can easily cause the optical lens to break, resulting in a yield loss of up to 10 in the VCSEL package reliability verification, resulting in an increase in production costs In order to solve the problems above, HRT Technology hopes to use AI to monitor the size and appearance defects of epoxy resin components in the VCSEL epoxy resin lens incoming stage, verifying whether their dimensions meet specifications, whether the cutting edges are smooth, and whether there are any defects in their appearance Since traditional incoming material inspection requires a rough visual inspection by humans to distinguish the quality The problem of image collection needs to be solved first to successfully collect image data Therefore, HRT Technology first developed an Automated Optical Inspection AOI device, which includes X, Y, Z three-axis motion, high-resolution cameras, and related control software to automatically record images After collecting the image data, opencv aligns the test image and a normal image to determine differences between the two images, and then pixel mapping is used to compare the pixel area to complete initial screening Manual labeling is carried out according to the image classification above, including samples that are normal, have defects in appearance, or have different shape characteristics, and then algorithm training and verification is carried out Residual neural network ResNet or other related algorithms are used for deep learning to identify the quality of lenses Implementation of AOI inspection improves production efficiency by 20 and above Comparing the differences before and after the implementation of AI image inspection, the incoming VCSEL lens inspection before implementation only involved manual inspection of the appearance The lens is packaged on the VCSEL package that has completed die bonding After passing the general light up test, the final reliability test high temperature reflow is performed Failed samples go into the rework process However, after the implementation of AOI inspection, it can screen defective lenses sooner and reduce the cost of subsequent materials input, it can also reduce the need for rework due to failure, improving yield to 95 and above in the reliability verification This is expected to help companies reduce production costs by 10 and increase production efficiency by 20 and above The difference before and after implementing AI image detection HRT Technology pointed out that this technology is an AI application developed based on tiny images It uses deep learning algorithms to identify defects in the images The trained network automatically classifies image data to predetermined categories Defect categories can be determined through reference images, so cumbersome programming is not required In the industrial machine vision environment, deep learning is mainly used for classification tasks in applications, such as inspection of industrial products or identification of parts In the future, with the development of IoT wearable devices and the trend of energy saving, the size of optoelectronic components will continue to shrink This technology can be applied to the detection of defects in the appearance of other tiny optoelectronic components in the future