2.1.

Study Area

The study area was in the town of Oakville, Ontario (Fig. 1). All ash trees within the township boundary were part of the study. However, due to data limitations, particularly related to ground data acquisition, we focused mainly on six sites within Oakville, which are indicated in the final infestation map. Oakville is the first town in Canada to use aerial-based imagery to compile and inventory public and private ash tree damage.¹¹ Oakville is located in the EAB infestation zone, and has multiple confirmed cases of EAB infestation. Therefore, this region has great interest in research into early detection of EAB for both environmental and economical reasons.

Fig. 1

Location of the town of Oakville, Ontario, Canada, and the ground survey results of individual tree distribution. Numbers in the inset are the IDs of trees with confirmed infestations.

2.2.

Airborne and Ground Hyperspectral Data

Two sets of commercial hyperspectral remote sensing data were used:

2.3.

High Spatial Resolution Aerial Imagery

High spatial resolution aerial imagery was provided by the York University library. The data were collected during the winter of 2008, having 1 m resolution, and are orthorectified with blue, red, and green in three spectral bands. However, the absence of leaves at the time of image acquisition limited the use of these data in this study. We, therefore, only use the images to validate the locations of trees and to confirm tree species identification.

2.4.

Google Earth Aerial Imagery

The employment of Google Earth (GE) aerial imagery in scientific studies poses a dilemma. GE imagery is not quality assured with any known or documented geometric or radiometric corrections. However, the data are free and easy to access, and are potentially very powerful, containing, for example, multiple years of data from the same location. In the Oakville area, GE imagery has an average spatial resolution of 13–15 cm over our test sites. Before making use of the GE imagery, we made precise calculations of scale using known objects, such as rooftops. We also validated the geometric locations of trees, and the differences in the locations of our sampled trees. From this, we found that the GE imagery and the orthorectified images were within 0.5 m of the tree centers. We did not validate or calibrate the radiometric values of the GE imagery, as we were only concerned with the relative changes in the pixels.

2.5.

Ground Truthing

We used two sources of ground-based information to determine the likelihood of infestation from nearby infested or healthy trees, and to support ground validation. First, most infestations were confirmed using a previously documented branch sampling approach.⁷ Two branches, approximately 50 cm long and no less than 6 cm in diameter, of each targeted ash tree were obtained, their bark was peeled off, and the underlying cambium inspected for evidence of EAB infestation, such as serpentine galleries, larvae, and young beetles (Fig. 2).

Fig. 2

The branch sampling approach for EAB detection. A larva is found in this sample. Copyrighted at AMEC Inc., used with permission.

Second, nine community maps showing street and park tree layouts were supplied by the town of Oakville. In these maps, the infestation states of individual trees were recorded as “healthy,” “low,” “medium,” and “high” infestation states, as indicated by ground-based survey. We manually transferred these locations and condition information into our imagery data.

3.1.

Segmentation and Identification of Individual Ash Trees

To ensure the accuracy of the locations and boundaries of individual ash tree crowns, multiscale automatic segmentation¹⁸ results from both the hyperspectral and high spatial resolution images were manually checked and corrected. In the hyperspectral image, the pixels within each tree crown were identified and classified into the same crown. Using ground surveys and the community maps provided by Oakville, all ash trees used in the following training and validation studies were checked and confirmed.

3.2.

Information Retrieval

3.2.4.

Spatial pattern feature: longitudinal profile for tree crown delineation

Spatial information within a tree crown is useful in evaluating early leaf loss and shape change of the crown. The textural or structural anomalies in an infested crown during the early infestation stage are difficult to detect using traditional statistical textural analysis.⁸ Instead, in this study, a longitudinal profile along the solar principal plane, end to end on the tree crown passing through the tree center, as developed by Zhang and Hu, was adopted.¹⁷ A sample longitudinal profile is given in Fig. 4. In this figure, the tree crown boundary has already been segmented out, and the longitudinal profile is the line made up of digital values (e.g., reflectance) along the sun’s illumination direction. To further reduce potential spectral variation and to minimize uncertainties in the GE image, the normalized relative differential values were used, wherein the differences in neighboring pixel values were divided by the average of all spectra within the crown. The numbers of significant drops in these longitudinal profiles were used to determine the possible infestation stage.

Fig. 4

A sample longitudinal profile. The tree crown is segmented out and the profile is a line made up of digital values (e.g., reflectance) along the sun’s illumination direction, end to end through the tree center.

In this study, we used the following criteria to determine the infestation stage: crossing from the center of the tree, if there was at least one major drop in values on both sides, it was considered a low infestation state, with a score of 0.5; if there were at least four drops on both sides combined, it was classified as medium infestation, with a score of 1.5; and any case with more than six such drops was considered highly infested with a score of 2.5. The necessary size and significance of these drops were determined by a threshold value derived from training.

3.3.

Information Conversion and Final Evaluation

4.1.

Prior-Processing Studies

4.1.1.

Ground level hyperspectral signature separability assessment

The ground level hyperspectral signature separability assessment is critical in spectral-based studies. As mentioned earlier, if no significant differences are observed between high detail (e.g., ground level) spectra of trees at different stages of infestation, then the likelihood of low detail (e.g., canopy level) spectra exhibiting sufficient differences for classification is very low. A set of sample reflectances for ash leaves from trees at different stages of infestation are shown in Fig. 5. The figure shows that there are clear differences in leaf-level hyperspectral measurements throughout the spectral range. Therefore, it may be expected that information retrieved from spectral data, particularly entire spectra, will be informative.

Fig. 5

Average ASD measurements of leaves (each from 19–21 samples) from ash trees that are healthy, and which have low, medium, and highly infested states. There are clear differences between the different health states throughout the entire spectral range. However, we also note that the differences in different bands are not equal and some signals may be caused by noise.

4.1.2.

Contribution sensitivity analysis and performance evaluation for selection of vegetation indices

To determine bands that are potentially more successful in the VI study, the leaf-level contribution sensitivity analysis was calculated based on the PROSPECT model, which was used to evaluate the performance of various vegetation indices. The contribution analysis calculates a CI that provides quantitative measurements of the “contribution” of each observation to the final retrieval of the targeted parameter. Therefore, a higher CI represents a higher correlation. As a result, those bands with high CI should be preferred during VI band selection. We calculated a series of CI values for different leaf chlorophyll contents from the simulation studies of PROSPECT, which are shown in Fig. 6. The outcomes of these CIs show strong agreement with the findings in Sims et al.²² and Pontius et al.⁹ Figure 6 illustrates the following: (1) there are two regions, 500–600 and 700–750 nm, showing significant increase with the increase in chlorophyll content; (2) at wavelengths of less than 500 nm, the CI only changes for cases with low-chlorophyll content, typically less than $35\mu {\mathrm{gcm}}^{-2}$; and (3) the region from 600–700 nm has a moderate effect on the CI and decreases with increasing chlorophyll content. From this, it may be expected that vegetation indices within the ranges of 500–600 and 700–800 nm will perform better than indices spanning other ranges, such as PRI, SR680 (Table 2).

Fig. 6

Calculated contribution index based on a PROSPECT simulation at different leaf chlorophyll contents. Leaf chlorophyll content varies from 15 to $75\mu {\mathrm{gcm}}^{-2}$. The other parameters used by PROSPECT were fixed, with at leaf structure parameter $(N[])=1$ to 4, equivalent water thickness $(\mathrm{Cw}[\mathrm{cm}])=0.025$, and leaf dry matter per area $(\mu {\mathrm{gcm}}^{-2})=0.005$.

4.2.

Vegetation Indices and Their Infestation Level Scores

Besides the CI preference, the final selection of vegetation indices considered the index performance in each infestation state. We used information from trees of a known infestation state to compare the performances of the various vegetation indices, as presented in Fig. 7. In this figure, all VI values were normalized by their averages. Greater variation between different infestation stages was considered as representative of a good indicator. As such, PRI and CSc were clearly the two highest rankings, while NPQI, SR680, and TCARI/OSAVI all showed a similar level of performance. TCARI/OSAVI incorporated prefixed coefficients, which may need to be calibrated for different data sets, which is a relatively complex process. NPQI was less preferred by the CI measurements, but the bands of this index are in a different region of the spectra. The use of NPQI would increase the stability of this study, and hence it was chosen, but is given half the weight of CSc and PRI in the final function.

Fig. 7

Performance of selected vegetation indices. T/O indicates the TCARI/OSAVI index and AMEC is the index proposed by AMEC. For the infestation stages, 1 is healthy, 2 represents low infestation, 3 is medium infestation, and 4 represents high infestation.

After examining each of the final selected vegetation indices, we found they are in good agreement with previous reports in the literature:

All results from the vegetation indices were converted into normalized scores ranging from 0 to 5. Their functions were derived from correlations with the data of known state trees using third-order polynomial or step linear functions. The CSc, PRI, and NQPI conversion functions can be respectively given as

Vegetation indices have provided a convenient and robust methodology for EAB detection. However, calibrations are still required for each data set. Some indices, such as NQPI, suffer from relatively low-separation performance, and thus should be utilized with reduced weighting. We also compared pixel-based spectra against the tree crown average and found the shadow effect to cause major spectral variations in near infrared (NIR) band-related indices. This also corroborates the fact that the spectral-based approach alone cannot sufficiently confirm the degree of EAB infestation.

4.3.

Retrieved Leaf Chlorophyll Content and its Infestation State Score

The absolute leaf chlorophyll content values were obtained by PROSAIL model inversion with a CI implemented LUT approach. Hence, the hyperspectral spectra derived from the hyperspectral imagery were compared to the PROSAIL simulated spectra with CI as a weighting in the LUT inversion approach. In this study, we found that the leaf structure parameter N, ranging 1.7–2, and LAI, returning values of 3 and above, were slightly higher than reported elsewhere in the literature (e.g., Refs. 35, 36, and 37). However, they are still considered to be within a reasonable range, as most of the ash trees in our study sites were mature. We found that leaf chlorophyll content retrieved from physical models has a much higher variation than the vegetation indices. However, relative decreases in leaf chlorophyll content compared with neighboring healthy trees were found in both approaches. As a result, we did not use a regression function as in the case of the vegetation indices, and instead employed a more general classification. We used two threshold values to separate the results into three classes. The threshold values used in this study were 65 and $37\mu {\mathrm{gcm}}^{-2}$, meaning that if the leaf chlorophyll content is greater than $65\mu {\mathrm{gcm}}^{-2}$, then a score of 0 is assigned, if the value is between 37 and $64\mu {\mathrm{gcm}}^{-2}$, it receives a score of 1, and if it is less than $36\mu {\mathrm{gcm}}^{-2}$, then the score is 2.

Physical model inversion is a time-consuming process, and is prone to observation noise, thus it has not been adopted in other ash EAB infestation studies.¹¹ In this study, however, since it is an object-oriented approach, average tree spectra were used instead of individual pixels, significantly reducing computation time and increasing the signal-to-noise ratio. More importantly, the model inversion approach evaluates the overall spectra, taking into consideration the quality of observations, including uncertainties, and providing the absolute values of target parameters. Although the parameter we calculated in this way is leaf chlorophyll content, it still can be considered as a different approach and a fresh perspective compared to vegetation indices method, which focuses on only a few bands. We implemented CI as a weighting in the merit function to improve the retrieval accuracy. Hence, retrieving leaf chlorophyll content through physical model inversion can provide important information to be used in the final evaluation. Ideally, we seek consistent results with regard to leaf chlorophyll changes from both methods. We did confirm that the infested trees have lower leaf chlorophyll content than nearby healthy ones; however, the model inversion approach suffers from high saturation rates, which means it is fairly easy for results to converge on to the minimum or maximum values of the preset range.

4.4.

Spatial Patterns and their Infestation State Scores

We used a normalized differential value of 0.2 as the threshold value to determine major declines in the tree crown. We found that the normalized differential longitudinal profiles show better performance with regard to resistance to uncertainties, and partially to background and shadow effects. This method simply involves counting the number of significant drops across the tree top crown profile. In Fig. 8, all trees are resampled to 30 pixels in length to illustrate them together as a normalized size. This conversion ensures low levels of infestation can be detected, while highly infested interpretations should require further supportive evidence. We confirmed that the spatial patterning within the tree crown is a strong indicator of infestation, as previously reported.^5,13

Fig. 8

Normalized differential longitudinal profiles for different representative trees. Any declines that cross the two threshold lines are considered significant. The number of significant drops is used to form the infestation estimation.

Figure 8 illustrates some typical examples for trees of different infestation stages. The healthy, or likely healthy, trees have a much smoother crown curve due to the healthy growing conditions, whereas the infested tree shows a clear alternating gap pattern. This pattern is due to the leaf decline within the crown, which has been reported in Pontius et al.⁹ However, spatial pattern can be normally detected when the infestation is in the dieback stage. It is necessary to determine the optimal spectral band or VI from which this pattern is to be obtained, as it is also prone to variations in solar illumination conditions.

4.5.

Complete Sample Calculation Using the Operational Workflow

Let us perform a simple but complete calculation step by step to illustrate the operational workflow introduced in this study. After segmenting each individual tree, regardless of whether this is done automatically or manually, then a spectral profile, like that shown in Fig. 9, can be obtained. These spectra can be significantly altered if unnecessary pixels are included, and care must be taken to include all pixels in the sunlit portion of the tree crown, even the dark pixels.

Fig. 9

A typical hyperspectral profile from the sunlit portion of the tree crown.

By calculating the selected VI from the spectrum in Fig. 9 and applying the expression functions listed in Eq. 3, we find that $\mathrm{Csc}=0.129$, giving a score of 2.25, $\mathrm{PRI}=-0.241$, which gives a core of 0.48; and NPQI is $-0.201$, with a score of 2.398. If this spectrum is input into the PROSAIL model, with three variables and the remaining parameters set to the values mentioned in the method section, then the inversion result, giving leaf chlorophyll content, is 65. Therefore, according to our threshold values, the score from model inversion is 0.

From Fig. 10, it can be seen that there is only one major decline that crosses our threshold value. Therefore, the assigned score from the spatial pattern longitudinal profile is 0.5.

Fig. 10

Normalized differential longitudinal profile for the same tree crown as in Fig 9. The threshold value in this study is 0.2, which means any declines that have an absolute value greater than 0.2, either positive or negative is considered major and significant.

With regard to the distance from trees of known infestation state, this tree has a known medium infested tree nearby along the same street, and is spaced four trees away from it, while the nearest known healthy tree is across the street. These relationships are illustrated in Fig. 11, in which M, H, and X indicate the trees known to be medium infested, healthy, and the target tree, respectively. The target tree X is the fifth tree from the infested tree, and there is no tree between X and H; therefore, H is considered to be the sixth tree.

Fig. 11

A sample street scenario for distance constant calculation. M is a medium infested tree, H is a known healthy tree, and X is the targeted tree. The numbers indicate the order of trees involved in this calculation.

The final score for this individual tree is given as the solution from a linear function $y=ax+b$, with the two known points (1,2.5) and (6,0) at $x=5$, which equates to 0.5.

Finally, the final estimation score of this individual tree, considering the score values and weightings, is given as

Since the score is less than 1, this tree is graded as currently healthy. Considering the typical green spectral signature attributes of a healthy living tree, the green peak, made by the green band being higher than both the blue and the red bands, and the red edge, from the NIR band being much higher than the red band, resulting in a high NDVI value, are both clearly presented. In addition, the nearby neighboring tree is currently known to be healthy. Thus, from raw observations, this tree is most likely currently healthy, which supports our calculation result. However, since there is an infested tree within the spread radius, this tree is potentially in high risk of future infestation. This is not in conflict with our score findings, since all of southern Ontario has been marked as a high risk zone. The score simply reflects the health state of the tree at the time of data acquisition.

4.6.

Map of Estimated Ash Tree Health

Using this workflow to calculate infestation scores for all trees within our study area, the final results are collected and illustrated with ENVI (Exelis, Visual Information Solution, Boulder, Colorado, USA). The map of final results is shown in Fig. 12, where different colors indicate different estimated levels of infestation. A comparison of 40 infested trees and 40 healthy trees for assessment of estimation accuracy is shown in Table 3. Columns show the predicted result (Pd), and the rows list data from ground truthing. The overall accuracy, including healthy trees, is 70%, and 62.5% when healthy trees are excluded. The healthy tree omission error is 22.5%, and commission error is 18.5%.

Fig. 12

Color-coded result map, where red indicates a score of 4 or higher, and highly infested trees; yellow has a score of 2.5 to 4 and represents a medium infested tree; blue, with a score of 1 to 2.5, shows low or early infested trees; and green indicates a score of less than 1 and a current healthy tree.

Table 3

Accuracy assessment of for estimated and ground observed tree health states for 80 sample trees.