⚡ The main thing in 60 seconds (TL;DR)
⏱ ~ 13 мин чтения- Computer vision models can produce impressive results in controlled environments. Turning them into reliable embedded products is mainly a matter of camera design and...
- In the laboratory, images are uniform and lighting is controlled.
- Some systems had to keep up with the fast production line.
Computer vision models can produce impressive results in controlled environments. Turning them into reliable embedded products is mainly a matter of camera design and model hardware.
Yalantis is a compliance-focused, full-service technology partner with deep IoT and computer vision expertise spanning software, embedded, hardware, and artificial intelligence development. This article uses four Yalantis projects to show how this work is done in practice, from optics and image correction to power design and field testing.
Why machine vision models running in the lab don't work well on real devices
A computer vision model that works well in a lab often performs poorly on a real device because the lab hides problems created by the physical system. In the laboratory, images are uniform and lighting is controlled. Usually there is enough processing power to perform the inference without worrying about the rest of the system.
The model is then moved into the real world.
In a factory, reflections from metal surfaces can appear as defects, and parts on an assembly line leave only a short window for inspection. These issues are easy to underestimate when developing a model.
At Yalantis, we've encountered them in a variety of embedded computer vision projects: inspecting automotive electronics, detecting cracks in brake calipers, monitoring inventory inside commercial refrigerators, and analyzing microplastic particles in flowing water.
Constraints varied from project to project. Some systems had to keep up with the fast production line. Others had to run on batteries for years or make measurements that were difficult to verify at all.
All four projects led to the same conclusion: a reliable machine vision system depends on the design of everything associated with the model, from the first image the camera captures to the decision the device makes.
This work often starts with the camera.
Why image correction precedes model change
Many vision errors occur before the image reaches the model, so the first thing to look at is the camera and image processing. When a computer vision system starts making mistakes, it is tempting to blame the model and turn to more training data or a larger network.
We've seen this on the automotive electronics line. The manufacturer used automated optical inspection (AOI) to inspect engine controller and airbag circuit boards, and the AOI flagged too many good boards.
Most of the problems arose due to lead-free soldering. Their uneven, reflective surfaces created patterns that AOI had difficulty distinguishing from real defects, so operators spent hours each shift rechecking the boards, which were often fine.
Replacing the AOI hardware would be expensive, so we developed a secondary inspection system to test boards tagged by the original machine. Before selecting a model, our engineers spent two weeks in the factory analyzing tagged images. It turned out that most of the false positives were due to lighting changes and solder reflections, so the images needed to be normalized before output.
A similar problem occurred in another automotive project, this time with cast aluminum brake calipers. The manufacturer needed to detect microcracks in polished metal parts, but the standard vision system no longer worked as reflections and changing ambient light hid the cracks.
The team started with how the images were captured. Multi-angle dome lighting with polarizing filters suppressed specular reflections, while high-resolution monochrome sensors with telecentric lenses produced stable images without perspective distortion.
When optics are present, a custom U-Net segmentation model running on NVIDIA Jetson Orin can identify potential vulnerabilities. There were few real examples of rare defects, so the training set was supplemented with synthetic images.
Both projects achieved the same goal using different methods. The PCB system captured image consistency during preprocessing, and the caliper station captured it using lighting and optics. Both approaches relieved the load on the neural network.
This is why image acquisition deserves the same engineering attention as output. The model can't reconstruct details that the camera never captured, and can't reliably separate a crack from the highlights that hide it.
Practical advice: Before changing the model, study a sample of real fault images. If errors are due to glare or uneven lighting, they are usually cheaper to correct in camera or in image processing.
A good image is only the first step. It still needs to be processed and turned into a solution that reaches its desired destination, and on the production line all this must happen quickly.
How to ensure the speed of integrated vision for a production line
To keep up with the production line, each step from image capture to output to the controller must fit within the line's time budget. The brake caliper project shows how quickly computer vision becomes a timing problem.
The parts moved along the line at a speed of 1.2 meters per second. Once the caliper reached the inspection point, the system had a short window to find the crack and send a command to remove the part. The correct prediction obtained after the part has passed the rejection mechanism has no practical value.
Thus, the system was designed all the way from image capture to physical action. The camera captures the part and Jetson Orin runs the segmentation model. The application then reports the result to the line PLC via OPC UA, and the pneumatic manipulator rejects any part with a bang.
The project reported a 15% increase in line speed due to the new inspection approach, demonstrating why vision quality should be assessed as part of the manufacturing process.
The PCB project also reached its deadline, but at a different stage in the chain. Once the team realized that the images needed to be normalized, software correction was too slow for the line, so the work was moved to dedicated equipment.
The system runs on NXP i.MX 8M Plus processor. Its built-in image signal processors process the corrections, and a neural processor runs the defect detection model. This model is based on MobileNet SSD and trained on the line defect classes: solder bridges, tombstones, insufficient wetting, and cold joints. NXP's eIQ toolkit optimizes it for the target hardware.
With dedicated resources for image preparation and output, the system keeps up with the pipeline. Each verdict is sent to the PLC as a GPIO pass/fail signal, so only confirmed defects are passed on for manual inspection.
The project reported a reduction in false rejections of more than 35% without any changes to the existing AOI. Since go-live, no actual defects have been reported to customers.
Both production line designs show that fast output is only part of the time budget. The rest of the system adds its own delays: camera interfaces, memory transfers, frame queues, and communication with external equipment.
These delays also shift as the device heats up and frames accumulate in buffers.
For industrial computer vision, an important measurement is the total response time, from the moment the camera is triggered until the result reaches the controller. If this is followed by a physical action, the response of the actuator must fit within the same window.
This changes the way equipment is selected. In addition to AI performance claims, engineers need to evaluate camera interfaces, ISP capabilities, memory bandwidth and thermal performance, and connectivity to surrounding hardware.
Some embedded vision systems face the opposite problem: they have to use as little power as possible while still doing their job.
How to create a machine vision device that will run for years on batteries
Running on batteries for years means spending power only on important images and leaving some of the workload to a low-power processor. Consider a small chamber installed inside a commercial refrigerator. It takes photos of shelves when the door opens and sends actionable images to a cloud platform where grocery and convenience stores remotely monitor inventory (Figure 1).

Here the hard constraint is power. The device must last at least three years on a single set of non-rechargeable batteries, and this requirement changes the entire architecture.
Yalantis designed the camera hardware and firmware to work with the client's existing cloud platform.
The first prototype captured and loaded an image every time a door was opened, and many of those images were useless. Some were blurred by motion, while others were blocked by the buyer's hand. Downloading all of them wasted energy and filled cloud storage with images no one needed.
The team changed the capture workflow. Each door opening now triggers two frames one second apart, and the on-device model selects the best frame. Only this frame is loaded.
The model's only task is to select the frame that is worth sending. This makes it part of the power structure.
Fewer downloads solved only part of the power problem. Cold and humidity reduce battery capacity, and measurements of the first units released showed higher consumption than the main processor datasheet suggested.
The team redesigned the hardware architecture. The second low-power processor became the main controller and took on part of the workload. The body has also undergone several changes to accommodate long-lasting batteries while keeping the camera small enough to remain unnoticeable.
Preliminary measurements have shown that the revised architecture could achieve a three-year lifespan, although testing is still ongoing.
The image quality also needed improvement. The model was trained on images from commercial refrigerators, and in a test refrigerator with different lighting, it periodically missed obstacles. This made verification at target stores necessary. At the reporting stage, one unit was working in a street cafe, and another 20 prototypes were being prepared for field testing before mass production.
Comparison with automotive projects is indicative. On the production line, work focused on processing images quickly enough to allow action before parts passed the rejection point. The refrigerator focused on deciding when to shoot and what was worth sending while maintaining battery life.
This is one of the most important lessons of embedded systems design: the best architecture is the one that meets the tightest application constraints.
Sometimes the hardest constraint is proving that the system's measurements can be trusted at all.
How to Test an Integrated Vision System in the Field
Field validation means testing the system under the conditions in which it will actually operate and finding a way to verify the results when no reference method exists. The microplastics monitoring project shows how difficult this can be. The goal was to create devices that count and size microplastic particles for municipal and industrial applications.
Two devices were used: an inline module that displayed an image of water flowing past a viewing window in a pipe, and a benchtop analyzer for detailed testing of samples. Both submitted their results to a web dashboard designed to provide early warning of rising particle levels.
Yalantis developed the electronics and software for both devices, using housings designed by the manufacturer's engineers.
Here the camera looks at an environment that is constantly changing. The turbidity and color of the water affected the image quality, and the flow rate was determined by the main pipeline. Lighting was the main image parameter that the device could control, so the optical setup and processing pipeline had to be designed together.
The system combined classical image processing with a machine learning model trained on a large set of manually labeled particle images. The fixed deadline for the live demo did not leave time for a separate prototyping phase, so the visualization chain was set up as one integrated system from the very beginning.
The most difficult problem was verification. The project did not have a reference method for directly counting microplastics in flowing water, and laboratory methods work with filtered samples, which are difficult to compare with continuous in-line measurements.
Therefore, the team assessed the results using theoretical calculations and expert assessments of the resulting images. Detection and rejection thresholds remained configurable while formal acceptance criteria were not yet defined.
Seven months after development began, both devices were demonstrated at a major industry exhibition. Long-term measurement accuracy had yet to be proven under production conditions.
Reliability means something different for each type of device. In an inspection system, it covers false rejections and missed defects under constant production load. For a measuring instrument, this depends on checking the results against relevant reference criteria.
Testing should reflect how the device will actually be used. A model that performs well on a test set may still fail when lighting changes, and a processor that meets its timing budget in a short test may begin to throttle after a few hours.
Engineers also need to plan for failures. What should the review system do when a verdict is reached too late or when the image is too poor to make a reliable decision? The answers determine whether the surrounding equipment can be affected by the system.
What it takes to turn a demo into an embedded product
Turning a proof of concept into an embedded product requires designing the entire system to meet the most challenging constraints. In these projects, the most important decisions were made outside the neural network: optics, image correction, power calculations, and verification method.
Taken together, these projects point to one way to approach the implementation of embedded computer vision (Figure 2). Start by identifying the constraint that has the most impact on what the device is supposed to do in the real world. Then design the entire chain around it: from the camera to the system interfaces.

On a fast production line, this means minimizing end-to-end latency so that every solution reaches the controller on time. When using a battery-powered camera, this means every frame needs to be trimmed and there is no need to download product.
Creating these products also requires multiple disciplines working together. The PCB verification project involved a solution architect, a machine learning engineer, embedded and backend developers, a quality assurance engineer, and a project manager. Other projects added optical and power work to the usual embedded product development steps.
This combination of disciplines turns a promising computer vision demonstration into a working embedded product. The deployed system must collect the necessary information and provide a result that another system can rely on, within realistic hardware limitations.
The real engineering challenge is making computer vision work in environments where the decision actually matters.

About the author
Mikhail Maidan is the technical director of Yalantis, an engineering company developing embedded and peripheral artificial intelligence products. The four projects discussed in this article are based on work done by Yalantis engineering teams.
The post Making Embedded Computer Vision Work in the Real World first appeared on Embedded.
💬 Discussion 0
BBCode & Emoji supportedParticipate in discussions, ask authors questions, and share benchmarks by joining the ProArena engineering community.