Quality control

→ Classical computer vision

The main difference between classical and machine learning based computer vision is the feature definition. The goal of any classifier is to detect the features which are useful to differentiate the target classes. In the case of quality assurance this can be viewed as a list of criteria which a good piece needs to fulfil (e.g., the size within boundary) or, in contrast, criteria that classify a product as defective (e.g., a scratch).

In classical computer vision the feature engineering is performed manually, i.e., classification criteria need to be defined and afterwards hardware-software solutions need to be defined to observe the features on the production line. For example, to check for the correct size, the piece could stop in front of a camera and boundaries defined within which the piece must fit on the produced image. The engineers would then need to make sure that the contrast between the piece boundaries and the background enable to take a decision based on the pixel values.

Classical machine learning can be a very potent solution for some use cases, especially if the number of important features is limited and after the initial feature engineering, the solutions can be easily replicated on identical or similar production lines. The feature engineering stage can, however, quickly become overwhelming for advanced applications. In addition, the solutions are very sensitive to exterior influences (e.g., changing illumination) and therefore need to be perfectly isolated to provide satisfactory performance.

In this context, DataThings participates to the European H2020 research project InterQ geared towards “Zero-defect manufacturing” by leveraging the data available throughout the entire manufacturing chain to improve (sometimes enforce) the overall quality. With this in mind, we have been tasked to provide a computer vision approach to identify and measure cutting tool wear from pictures. This measurement is important for the manufacturing operations since we can observe a direct relationship between product quality and cutting tool quality.

Currently, the analysis of these images is done manually by metrologists. They must define manually the wear area within a software to obtain a measurement. This approach has the disadvantage that it is done after the process, so some cuts may be of poor quality and could have been avoided if the tool had been replaced earlier.

For this use case, we decided to follow a traditional computer vision approach since the environmental setup is controlled. Indeed, a particular attention was given to the constraints of lighting and shadows during the positioning of the camera. In addition, the camera is firmly fixed allowing us to have the cutting tool to analyze always in the center of the image. The prototype developed integrates a contour identification algorithm based on the open source OpenCV library.

It identifies in most cases correctly the worn area and gives measurements at different points. Thus, the operators can immediately observe the evolution of the wear during the process. The evaluation of the outcomes by the metrologists themselves showed that the computer vision aided measurements were obtained much faster and more frequently and were sometimes even more accurate than manual measurements. The next iteration on this will target a classification of the defects, to complement the detection of the area already done. For this, a classification approach based on machine learning will be attempted.

→ Semi-supervised learning

While carefully trained supervised machine learning models provide great robustness and alleviate the need of manual feature engineering. But the need for high quality and quantity labeled datasets is a big challenge to real world applications. From our own experience with this project, the data gathering has been by far the most time-consuming task to finally evaluate our prototype.

With the prototype in place, gathering unlabeled data was fully automated and only limited by the speed of the production line. Labeling, however, required the input of domain experts and an individually planned collection session during which the production had to be stopped to allow us to put the collected pieces through the prototype setup.

Recently, researchers have started to investigate ways to reduce the required number of labels for machine learning. So-called semi-supervised learning approaches try to use only a limited number of labels and a much greater number of unlabeled samples during the training stage.

State-of-the-art methods [4] show impressive results, coming close to fully supervised baseline while using only a fraction of the available labels.

Motivated by these results we are currently investigating semi-supervised learning on our Cebi use case.