2.1.

Process Monitoring Hardware

For each of the three types of optical signals, a photodiode for the corresponding wavelength range is employed. A Si pin photodiode from Hamamatsu (S1223 01) with a sensitivity range between 350 and 1100 nm is used to detect optical emissions in the visible wavelength range. It is protected from the laser radiation by a short pass filter with a cut-off wavelength of 750 nm. Two photodiodes of the same type, G12181 010K from Hamamatsu, with a spectral sensitivity range between 950 and 1850 nm are used for detecting both the laser reflection and the infrared emission. The employed protective filters are a 1064-nm bandpass and a long pass filter with a cut-off wavelength of 1100 nm, for the laser reflection and the IR emission, respectively. Additional lenses with focal lengths of 30 mm direct the light toward the corresponding detectors.

The electronic preamplification of the photodiode signals is performed with a logarithmic amplifier. The integration of the sensor system was carried out on a LASER P 1000 U machine from AgieCharmilles. The machine features a femtosecond laser from Edgewave (FX200 1 GF) with a pulse duration of 800 fs and a maximum average power of 75 W. An f-Theta lens is used as the focusing optic. The lens has a focal length of 125 mm, which leads to a spot size of $29\mu \mathrm{m}$. For the beam deflection, a Scanlab galvanometer scanner intelliSCAN III 14 is used. The sensors are integrated around the f-Theta lens, as shown in Fig. 1.

Fig. 1

Schematic representation of the detectors mounted around the focusing optic.

To ensure repeatability of the monitoring, a holder for the sensors was designed to be mounted on the fixed housing of the optic. Therefore, the detectors are always at an exact same height above the surface of the workpiece and aligned with the center of the workpiece. Moreover, special attention was paid to the reproducibility of the position of the sensors during assembly and disassembly. The sensors are each mounted in an extended cylindrical enclosure with four screws each. Therefore, the inclination around their longitudinal axis is suppressed so that the same angle relative to the surface is always maintained, ensuring the alignment with the center of the processing field even after the sensor system had been dismantled.

2.2.

Signal Acquisition Hardware

The signal acquisition and analysis are based on the UltraZed SOM board from Avnet. This is a hybrid system that consists of two subsystems: a field programmable gate array (FPGA) and a soft processor, combining advantages from both worlds in one chip. Among other things, the deterministic real-time capability of the FPGA is essential for this work and the processor enables a flexible implementation of the network communication, e.g., to a personal computer. The FPGA is sufficiently large to implement the needed data acquisition as well as many additional real-time analyses. The UltraZed SOM board also includes memory devices and interfaces necessary for the data acquisition and forwarding to a personal computer or a database. The 2 GB large main memory allows buffering of the sensor data for nearly 10 min. The UltraZed card is extended by a custom-made carrier board to implement the hardware connectors for communication with the laser source, scanner, machine control, and sensors.

For the analog to digital (A/D) conversion, boards based on an LTC2351 14 chip that can sample 12 analog channels at up to 250 kHz simultaneously with a resolution of 14 bits are used. To minimize the noise immunity, all digital signals are transmitted differentially between the carrier card and the A/D converter board. The information about the processing position is read directly at the communication line between the scanner and its controller at the sampling rate of 100 kHz.

3.1.

Data Pool Creation – Machining Trials

The purpose of the conducted experiment was to enable a comprehensive search for correlations between the recorded secondary optical emissions and the properties of the workpiece surface. This is a general investigation of the possibilities of the monitoring system, with the aim to show that the information about the surface state is contained in the collected data. In consequence, the defect-localization method developed and described in Sec. 4 is then applicable to any use case solely by tuning the analysis parameters.

Unfortunately, the intensities of the optical emissions are not decoupled from the changes in processing parameters. Furthermore, just like the processing result, the collected optical emissions are influenced by fluctuations of the laser characteristics. However, more detailed analyses of these dependencies will not be performed within the scope of this investigation since they had already been carried out and thoroughly described previously.³⁵ During both the experiment design and the data analysis, it is, therefore, essential to differentiate between the changes in the recorded signals caused by the processing parameters from the ones that originate from the surface. An analysis of an ill-designed data set potentially leads to a false conclusion, e.g., a detection of changing surface roughness induced solely by increasing the laser power. The rise in laser power would lead to an increase in the secondary emissions intensity. However, this does not mean that they result from the higher surface roughness, although the correlation would be present in many cases.

For this reason, during the machining trials described in this work, both the starting conditions and the processing parameters were varied. Overall, 99 different parameter combinations were used yielding areas with many different surface roughness on tool steel surfaces. The same experiment was repeated on a total of 28 specimens. They were obtained pretreated in 14 variations of machining processes, all of them being repeated twice in the sample set. The pretreatment processes include milling (23, 18, and $12\mu \mathrm{m}$), polishing (0.2, 0.1, 0.05, and $0.025\mu \mathrm{m}$), electrical discharge machining (6.3, 3.2, 1.6, 1.0, 0.8, and $0.4\mu \mathrm{m}$), and grinding ($0.6\mu \mathrm{m}$). The USP-laser machining took place at constant feed rate of 3.5 m/s and constant pulse overlap of 76%. The number of pulses per burst (PPB) was varied between 1 and 10 and the single pulse peak fluence between 0.25 and $4\mathrm{J}/\mathrm{c}{\mathrm{m}}^2$. The repetition frequency was kept constant at 500 kHz. The number of structured layers was set for each structured area individually, in order to reach the same depth for every structure.

The data acquired during these trials was used to fulfill two of the objectives defined in the previous section. Every of the 99 first layers over each of the differently pretreated samples was analyzed in order to investigate the potential of measuring the initial surface condition. The much larger spectrum of the resulting surface roughness, due to combining the parameter and initial surface variations, was used to investigate the detection of changes during the machining.

3.2.

Detection of the Initial Surface Roughness

To directly compare the data of the detected optical emissions with the roughness values, different characteristics were extracted from each of the time series acquired during the machining of every first structured layer. They were calculated for each sensor individually from the recorded emission intensities during the processing of the entire layer and only within the laser-on-times. To compensate for the logarithmic amplifier, prior to computing the characteristics, the raw data was linearized using exponential functions with experimentally determined factors. Therefore, the parts of the time series collected during the jumps as well as the acceleration and deceleration phases of the scanner were not taken into account.

The following approach is motivated by the assumption that the intensity of the emissions recorded by detectors at a fixed position is directly influenced by the fact that the light scatters more diffusely from rougher surfaces. For this reason, the variation of the raw data within the whole first textured layer was closely examined. To additionally achieve the independence from the processing parameters, the relative standard deviation was calculated using the equation

Figure 2 shows the in this way extracted relative standard deviation of the secondary process emissions over the initial surface roughness for all the specimens. In all four graphs, the multiplication of previously individually calculated characteristics of all three sensors for PPB = 1 and PPB = 10 is shown.

Fig. 2

(a) Relative and (b) absolute standard deviations of the sensor data during the machining of the first layers over surfaces with varying initial roughness and for changing fluences and for PPB = 1 (top graphs) and PPB = 10 (bottom graphs).

As expected, the relative standard deviation is dependent on the surface condition. Despite its clearly visible correlation with the initial surface roughness, this feature also apparently depends on both varied processing parameters. Furthermore, even with parameters known in advance, the roughness is not clearly identifiable. While the surfaces can be differentiated from each other at a constant one pulse per burst with a known single pulse peak fluence below $F_0=1\mathrm{J}/{\mathrm{cm}}^2$, this is no longer the case at 10 pulses per burst, especially at higher fluences.

In contrast to the relative, the absolute standard deviation of the back reflection, shown on the right-hand side of Fig. 2, makes the original state detection possible for all but the three roughest surfaces preprocessed by milling. Nevertheless, the relative standard deviation is still more relevant for differentiating among the three milled surfaces. However, especially for PPB = 10, none of the features enable the distinguishing of the milled surfaces from the rest. In addition, regardless of the parameters, it is also impossible to detect the subtle differences, especially at higher laser powers. An example of this is well presented by the data of PPB = 1 and $F_0=4\mathrm{J}/{\mathrm{cm}}^2$ shown in blue.

In conclusion, considering the relative standard deviation alone is insufficient for a parameter-independent detection of the surface condition. However, a differentiation between largely different roughness regions is in most cases possible when the processing parameters are known in advance.

Due to the less promising results, the other extracted features will not be discussed separately. It is nevertheless worth mentioning that the maxima of all-optical emissions also showed a dependence on the initial surface roughness, which would be efficient to implement in an online analysis. In contrast, and as expected, the mean intensities for all three sensors were strongly dependent on the laser fluence only. Furthermore, none of the other characteristics of the individual emissions has shown such a pronounced correlation with the surface roughness as the standard deviation of the back reflection.

3.3.

Monitoring of Changes during the Machining

The goal of this investigation was to examine how effectively the surface roughness that results from the USP processing can be detected from the secondary emissions. Therefore, analogously to the previous section, various characteristics of the detected secondary emissions were calculated from the same machining trials and compared with the surface roughness. This time, however, the data from the last machining layer and the roughness measured after the completed machining were used.

In contrast to the measurement investigated in Sec. 3.2, the surface roughness now depends on both the initial surface condition and the machining parameters. Thorough inspections of the resulting surface roughness as a function of the initial condition and the processing parameters have shown that the original condition in fact represents the main influence. Following the analysis results described in the previous section, it is to be expected that the differentiation between the markedly rough and smooth resulting surfaces based on the data of the last machining layer is possible. This expectation was indeed confirmed by the analogous analyses of the last machining layers.

The next step was the investigation of the possibility to detect changes in the roughness induced by the USP processing. For this, only the surfaces with the same initial state were compared among each other. In Fig. 3, an example of the direct comparison of data collected during the first and the last layer on two specimens polished with $S_a=0.025\mu \mathrm{m}$ over the resulting roughness is given.

Fig. 3

Multiplied relative standard deviations of all signals within (a) the first and (b) last layer in dependence of the resulting roughness for two specimens with initial surface roughness $S_a=0.025\mu \mathrm{m}$ and total fluences [$\mathrm{J}/{\mathrm{cm}}^2$].

Here, each of the 99 processing parameters is represented by the total fluence, which considers the pulse overlap of two consecutive bursts assuming a pulse overlap of 1 within a burst:³⁶

In the final step, the same characteristic was determined for each layer individually. The goal of this analysis is to explore the possibilities for monitoring relative changes during the machining. Examples of two groups of initial surfaces treated with PPB = 1 and $F_0=4.14\mathrm{J}/{\mathrm{cm}}^2$ are shown in Fig. 4. To eliminate the dependency on the scan direction, this illustration only considers every other machining layer. Both the increase in surface roughness for the polished surfaces and its decrease for the milled ones are reflected in the relative standard deviation of the optical emissions.

Fig. 4

Relative standard deviation of optical emissions within each layer over the number of layers for two sets of initial surfaces: (a) a polished surface and (b) a milled surface.

Table 1 gives an overview of the surface roughness before and after machining for the four specimens from Fig. 4.

Table 1

Surface roughness of specimens from Fig. 4.

Generally, it can be concluded that the method described here is not sufficient for the detection of the subtle differences between the absolute values of the surface roughness. However, the change in surface condition during machining can be tracked through the changes in relative standard deviation. This result is used as a basis for implementing the analysis for the defect detection and localization.

4.1.

Experiment Design for Online Defect Detection

To accelerate the initial development of an automatic defect detection, a demonstrative parameter set was chosen that provokes holes on a workpiece surface after only a few machined layers. Therefore, the sample structure from Fig. 5 was processed with pulse overlap of 96%, feed rate of 0.5 m/s, PPB = 1 and a fluence of $2.5\mathrm{J}/{\mathrm{cm}}^2$.

Fig. 5

Microscopic images of the sample structure (a) defect-free, after five layers; and (b) defect-containing, after 15 layers.

Figure 6 shows 2D false-color representations of the intensities of all optical emissions acquired during the machining of both 5th and 15th layer. A qualitative comparison with the microscopic images from Fig. 5 already confirms the plausibility of the defect detection from the recorded emissions.

Fig. 6

2D representation of the raw data during the machining of the structure from Fig. 5. (a) During the fifth layer and (b) during the 15th layer for each of the three sensors indicated by the column name.

Moreover, the data depicted in Fig. 6 reveal how each recorded wavelength range adds an additional aspect to the knowledge about the surface state. While the point-shaped defects are most clearly distinguishable from their surroundings in the raw data of the recorded laser reflection, the line-shaped anomalies are more vividly visible in the images of both IR- and VIS-emissions. These horizontal lines originate from the surface topography prior to laser structuring, which can be seen around the sample structures in Fig. 5. Therefore, other than the point-like defects, they are already visible in the emission data of the first few layers, as shown in Fig. 6 and are highly dependent on the choice of the scan direction.

4.2.

Data Analysis for Offline Defect Detection

From the beginning of this analysis, the future objective of implementing an online detection and feedback was considered. One of the main requirements for this is the sequential analysis able to run in parallel with the data acquisition. Furthermore, due to the limited memory capacity of the FPGA board, only as short as possible time series should be processed at any given moment. Another important goal was the development of a universal error detection, which can be transferred to machining with other processing parameters and different machines. The evident error detection based solely on emission intensities that follows from Fig. 6 is insufficient, because it is dependent on the absolute values of the process emissions and thus is not generally applicable.

In the first step, the defect detection was analyzed using the complete time series data of one processing layer. All analyses were performed on the one-dimensional data of each emission type and the results were subsequently mapped to a 2D plane for verification. Figure 7 shows the analysis procedure on an example of VIS emissions for a defect-free and a defect-containing layer.

Fig. 7

Demonstration of the analysis procedure on an example of VIS-emission raw data during a single scan line for (a) one defect-free and (b) one defect-containing layer; here, the thresholds are symmetrically calculated by computing $I_{\text{filtered}}\pm 0.025·\mathrm{mean}(I_{\text{filtered}})·\mathrm{std}(I_{\text{filtered}})$ within one layer.

For each pair of coordinates, an individual calculation was performed to determine whether the corresponding optical emissions indicate a surface defect. To achieve this, the deviation of every data point from the signal’s mean value was observed. The mean was calculated from the entire layer, but the laser-off areas as well as 80 data points after the rising and before the falling edge of the laser modulation signal were omitted. To differentiate between the deviations that indicate a defect and the ones originating from the noise, a tolerance band was defined, within which no decision can be made. The limits of this tolerance band are calculated for each layer separately. For its determination, the standard deviation of the emission intensities within that layer was taken into account. Moreover, the VIS emissions were low-pass prefiltered by forming a moving average with five datapoints since they contain high-frequency disturbances that do not originate from the machining.

In contrast to a detection based on absolute values that follow from Fig. 6, this method is transferrable to a broader range of use cases. The comparison with the mean value contributes to the independence of the processing parameters. The tolerance band limits that depend on the standard deviation within one layer, enable the detection of defects regardless of how pronounced they are.

The results of this analysis are presented in Fig. 8. For all three emission signals, the defects are manifested by a decrease in signal levels and are highlighted in red. This behavior can be explained by a local rise of the surface roughness. In these areas, the diffuse reflection increases, whereas the directional reflection decreases. This can lead to less light deflection toward the detectors, resulting in negative deviations from the mean value of the optical process emissions.

Fig. 8

2D representation of the analyzed raw data of the defect-containing layer for each of the detectors: [(a)–(c) IR-emission, laser reflection, and VIS-emission]. The outliers are classified by checking whether the filtered data cross the upper and lower thresholds from Fig. 7.

In the next step, the analysis was further improved to achieve a data reduction as well as rapid response times during the future closed loop control. Instead of the data of one entire layer, it now executes on time series collected within a 1-ms long rolling window. At the constant sampling rate of 100 kHz used in this case, this window length corresponds to a set of 100 data points. The results of the analysis performed on the time series before being mapped on a 2D plane are depicted in Fig. 9.

Fig. 9

Analyzed data of the defect-containing layer for each of the detectors [(a)–(c) IR-emission, laser reflection, and VIS-emission]. The outliers are classified by checking whether the filtered data cross the thresholds determined according to Fig. 7; instead of computing the thresholds within a whole layer, here, they are calculated in a 1-ms-long rolling window.

To achieve these results, two further adjustments had to be made compared to the analysis within the whole layer. First, the areas close to the laser modulation signal edges are not omitted. This is not required anymore, because the local calculation of the tolerance limits insures more stringent criteria for the differentiation between defects and noise that originates from the signal edges. In addition, the factors for calculating the tolerance limits had to be correspondingly adjusted.

Although the results from Fig. 9 show successful detection of most defects, this approach is not free from disadvantages. For example, the line-shaped defects that spread parallelly to the scan direction are not visible if their size exceeds the window length. Moreover, even the point-shaped defects are detectable up to a maximum size, which is also determined by the window length. Beside the size of the defects, the scanning speed imposes another limitation to the detection for further applications. This is because the spatial resolution is directly dependent on the scanning speed due to the constant data acquisition rate.

4.3.

Automation of the Defect Detection

The aim of this analysis step was to define a factor that describes the extent of defects and can be used as a criterion for terminating the machining. Similar to the previous section, a sequential analysis on a small sample plays an essential role here as well. This is because the processing is to be interrupted as quickly as possible after the extent of errors has exceeded a predefined limit.

As the starting point, the results of the analysis within a rolling window were used. In this case, simply counting the negative outliers would be insufficient since they are present even in the data that correspond to the defect-free surfaces. However, it is evident that they become increasingly organized into larger clusters as actual defects appear on the workpiece surface. From this observation, a factor was derived that is proportional to both the number and the area of the red regions from Fig. 9.

In order to calculate this measure of defects directly from the one-dimensional data, the number of negative outliers within a 10-datapoint long rolling window is sought. The derived measure corresponds to the total number of such windows that contain at least nine negative outliers. This factor was calculated for each of the 15 layers during the structuring of the geometries from Fig. 5. In Fig. 10, it is plotted over the layer’s number. The figure depicts both the factors determined from the individual sensor signals as well as the overall measure derived from their multiplication.

Fig. 10

Automatically determined error measures over the ordinal number of the structured layer. They represent the number of adjacent negative outliers from Fig. 9 inside a 10-datapoint rolling window along the scan direction. (a) For each individual sensor and (b) multiplied and normalized error measures from the left graph.

The multiplied error measures from Fig. 10 right is a unitless quantity. To improve the clarity of the representation, its y-axis is divided by a constant factor. It increases with the increasing number of layers, which stands in agreement with the microscope images. On the other hand, the factor derived from the VIS emissions shows a decrease between the layers 5 and 10. Therefore, omitting the VIS emissions would in this case improve the accuracy of the detection. However, this would actually impair the accuracy for the last four layers of the same structure. There, the differentiation based on the analysis results of VIS emissions is even finer than based on both other emissions.

The error measure is proportional to the number and the area of the red areas from Fig. 9. From both Figs. 9 and 10, it follows that the VIS-photodiode is most sensitive to this kind of disturbances. However, due to calculating the overall error measure as a multiplication of all, the absolute value of each of them does not play a role.

A possible improvement of the algorithm that follows this would be to introduce weighting factors for the individual sensors. They must be previously determined depending on the significance of the corresponding data and the superimposed noise. Furthermore, an additional analysis perpendicular to the scan direction would allow the detection of defects that spread along the second axis.

4.4.

Implementation of the Online Analysis

Using the results of the offline analyses, the FPGA-based data acquisition system, described in Sec. 2.2, was extended to detect the defects in parallel to the machining process. The analysis algorithm had to be adjusted to better fit the online analysis requirements. For example, instead of a rolling window for the calculation of the mean value and the extraction of deviations, a sliding 1-ms long window was used. In contrast to the case of offline analysis, this has no negative effect on the speed of the online evaluation. The sliding window analysis even has an important advantage. For each data point, the change in emission intensities at the rising edge of the laser modulation is automatically considered separately and is therefore more accurate. Moreover, to omit the areas where the laser was idle, its modulation signals are monitored continuously and processed by the same controller.

Another adjustment in comparison to the analysis from Sec. 4.2 is the linearization of the raw data, which was excluded in this implementation. To compensate for the altered conditions, the threshold values for the outlier detection were redetermined. Nevertheless, they are still calculated relative to the mean value and as a function of the standard deviation within 1 ms.

To implement the feedback directed to the user, a threshold was introduced, above which the determined number of defects is considered unacceptable. The calculation of this threshold occurs automatically and is based on the number of defects extracted during the processing of the first layer. It is therefore reset at the beginning of the machining of a new structure.

Once the threshold is reached during the processing, a message is forwarded to the PC. In the case of processing with an unchanged parameter set, this is assumed to be an irreversible state. Therefore, this message is only reset at the beginning of the processing of a new structure or when a parameter change is made.

Following its implementation in the FPGA, the error detection and feedback were validated using the set of parameters from Sec. 4.1 and both the sample structure from Fig. 5 as well as a rectangular area. The processing was performed on a polished surface with two different orientations of the scan direction relative to the surface texture induced by the preprocessing. As a result, the error message was generated without an exception. However, the moment of its generation was dependent on both the structured geometry and its orientation relative to the initial state of the surface.

To investigate the intermediate results more closely, the FPGA design was temporarily modified so that it forwards the internally determined factors to the PC. An example of the error measures extracted parallelly to the processing are given in Fig. 11. The horizontal line denotes the automatic ally determined termination criterion, i.e., the moment at which the error message was generated. In contrast to the offline analysis, the characteristic of the general error measure is linear due to its calculation by adding instead of multiplying the individual factors.

Fig. 11

Automatically determined error measures over the ordinal number of the structured layer, (a) for each individual detector and (b) addition of the error measures from the left graph. Here, the termination threshold is the first value that exceeds the five times error measure computed from the data collected during the first layer.

Based on these results, the extension of the monitoring system to terminate the machining immediately after the defect detection is realizable by modulating one of its digital outputs. Long-term developments include an automatic triggering of one or more additional intermediate processing steps in order to flatten the defects, e.g., by means of the already investigated USP-laser polishing.³⁷ Further specific strategies to automatically control the processing parameters in order to remove or prevent the defect emergence are equally possible.