Noise reduction for near-infrared spectroscopy data using extreme learning machines

Highlights

• There are many pre-processing techniques for NIR data to choose from.

• We propose to avoid the pre-processing by using a novel algorithm called C-PL-ELM.

• C-PL-ELM uses two Lagrange multipliers as optimization constraints.

• Results for regression and classification tasks confirm the advantages of C-PL-ELM.

Abstract

The near infrared (NIR) spectra technique is an effective approach to predict chemical properties and it is typically applied in petrochemical, agricultural, medical, and environmental sectors. NIR spectra are usually of very high dimensions and contain huge amounts of information. Most of the information is irrelevant to the target problem and some is simply noise. Thus, it is not an easy task to discover the relationship between NIR spectra and the predictive variable. However, this kind of regression analysis is one of the main topics of machine learning. Thus machine learning techniques play a key role in NIR based analytical approaches. Pre-processing of NIR spectral data has become an integral part of chemometrics modeling. The objective of the pre-processing is to remove physical phenomena (noise) in the spectra in order to improve the regression or classification model. In this work, we propose to reduce the noise using extreme learning machines which have shown good predictive performances in regression applications as well as in large dataset classification tasks. For this, we use a novel algorithm called C-PL-ELM, which has an architecture in parallel based on a non-linear layer in parallel with another non-linear layer. Using the soft margin loss function concept, we incorporate two Lagrange multipliers with the objective of including the noise of spectral data. Six real-life dataset were analyzed to illustrate the performance of the developed models. The results for regression and classification problems confirm the advantages of using the proposed method in terms of root mean square error and accuracy.

Introduction

Near-infrared (NIR) (Pierna et al., 2011) spectroscopy are mainly used to measure light absorption of the so-called mid-infrared light, in order to identify and quantify various materials. Spectroscopy in combination with varied multivariate algorithms has played an important role for fast and nondestructive analysis in petrochemical, agricultural, medical, and environmental sectors (Liu et al., 2015a, Luypaert et al., 2007, Kim et al., 2010, Park et al., 2012).

According to the Beer–Lambert law, the absorption of light in a medium is proportional to the path length and the concentration of the absorbing agent. That is, there is a linear relationship between absorbance and concentration when the path length remains constant, which motivated the use of linear multivariate calibration techniques, such as multiple linear regression (MLR), principal components regression (PCR) (Keithley et al., 2009) and partial least squares regression (PLS) (Wilcox et al., 2016).

However, the linearity of the Beer–Lambert law is limited by chemical and instrumental factors, such as, deviations in absorptivity coefficients at high concentrations, non-symmetrical chemical equilibrium, intermolecular reactions, existence of humidity inducing hydrogen bonding, changes in temperature, non-monochromatic radiation, scattering of light, fluorescence or phosphorescence of the sample, stray light, nonlinear detector response (Despagne and Luc Massart, 1998), etc. When the system exhibits strong nonlinear behaviors, classical linear methods may not completely identify the relationship between the spectra and corresponding concentrations and thus would produce large errors in regression and classification problems. Many nonlinear techniques have been developed, such as, artificial neural network (ANN) (Hajnayeb et al., 2011), support vector machine (SVM) (Li et al., 2009), nonlinear partial least squares (Rosipal and Trejo, 2001), etc. These methods may perform well on nonlinear data but are computationally more complex than linear methods and have the limitation of being prone to overfitting (Peng et al., 2013).

In the case that the experimental measures deviate from the Lambert–Beer law, a suitable pre-processing must be considered to compensate for this nonlinear behavior. The disadvantage of including such additional factors is an increase in the complexity of the model and, in turn, it is likely to have a reduction of the robustness of the model for future predictions. All pre-processing techniques have the aim to reduce the noise in the data with the purpose of improving the characteristics looked for in the spectra. However, there is always the danger of choosing the wrong type or applying a pre-processing that is too severe that you may (unintentionally) delete valuable information. This problem is described in detail by Rinnan et al. (2009).

For near-infrared spectroscopy data, it generally contains linear and non-linear components in regression or classification problems. Linear methods may have difficulty obtaining a good performance, since the non-linearity is usually modeled in a limited way. However, the linear methods are more simple and stable. The non-linear methods can provide better performance than the linear methods, but are more complex. Therefore, a simple, fast, precise and effective method is required.

In the early 1990s, different authors (Schmidt et al., 1992, Pao et al., 1994) independently proposed feedforward neural networks comprising randomly initialized and untrained connections between the input layer and a hidden layer of non-linear neurons. Then, in the 2000s these type of networks were revisited under the name of Extreme Learning Machine (ELM) (Huang et al., 2004). Recently, a comparison of these neural networks with random weights, for classification and regression problems, was carried out by Henríquez and Ruz (2018a), also, in Zhang and Suganthan (2016b), a survey on randomized algorithms for training neural networks was presented. In this paper, we will use the term ELM as a reference for randomized feed-forward neural networks (Henríquez and Ruz, 2018b).

ELM has been successfully applied in many fields (Samat et al., 2014, Cao et al., 2016). Especially, for NIR data, ELM combined with feature selection techniques has been used to determine amino acid nitrogen in soy sauce (Ouyang et al., 2013), total acid content in vinegar (Chen et al., 2012), pear internal quality attributes (Jiang and Zhu, 2013), etc. Recently, in Li et al. (2016) was analyzed the feasibility of Fourier transform infrared transmission (FT-IR) spectroscopy to detect talcum powder illegally added in tea. In Yang and Sun (2016) the abilities of six popular multivariate classification techniques are compared, including ELM. In Bian et al. (2017) ELM was used for near-infrared spectral quantitative analysis of diesel fuel and edible blend of oil samples.

A main difficulty when working with NIR data is the fact that there are many pre-processing techniques from which to choose from (more details in Section 2), and even combinations of them, where different researchers use arbitrarily such methodologies, in order to achieve good performance in classification and prediction problems. With this motivation, there is a need for a robust methodology to solve this problem. Therefore, we propose to reduce the noise in NIR data by using a novel algorithm called C-PL-ELM, which has a parallel architecture. This algorithm has a non-linear layer in parallel with another non-linear layer, generating a more powerful nonlinear mapping. Using the soft margin loss function concept (Bennett and Mangasarian, 1992), we incorporate two Lagrange multipliers as optimization constraints (similar to the concept of support vector regression (Drucker et al., 1997)) with the objective of including the noise in spectroscopy data, thus avoiding the pre-processing of the experimental measures.

The remaining of the paper is organized as follows: In Section 2, we briefly review some of the most popular pre-processing techniques. The detail of the proposed C-PL-ELM algorithm is presented in Section 3. Simulation results and comparisons are provided in Section 4. Section 5 presents discussions on the performance of C-PL-ELM with respect to the different datasets. Conclusions are drawn in Section 6.