Determination of moisture content in corn samples: a critical evaluation of standard normal variate preprocessing for NIR spectral data

Background and objective: Standard normal variate (SNV) preprocessing is widely applied to spectroscopic data prior to multivariate modeling, under the assumption that it mitigates undesired background variations while preserving analyte signal information. However, the scaling step in SNV, where each spectrum is divided by its standard deviation, could potentially distort the covariance between spectral intensities and component concentrations. This study systematically evaluates the effect of SNV preprocessing on the ability to develop accurate quantitative models for determining analyte concentrations in mixtures, with a focus on the determination of moisture in corn using NIR spectroscopy.Materials and methods: Simulations were performed to generate single and multi-component spectroscopic datasets with varying levels of multiplicative scatter and baseline offset effects. Additionally, an experimental near-infrared (NIR) dataset for corn samples with reference moisture values was utilized. Partial least squares (PLS) regression was employed to model the simulated and experimental data, with and without SNV preprocessing. Model calibration and prediction performance metrics were assessed.Results and conclusion: For simulated datasets without background interferences, SNV preprocessing eliminated useful concentration-related variations by forcing all sample spectra to equal lengths, severely degrading PLS calibration and prediction abilities. In scenarios with multiplicative/additive perturbations, while SNV mean-centering helped mitigate these undesired effects, the subsequent scaling step obscured analyte concentration information in the spectral intensities. PLS models built from raw corn NIR spectra provided excellent determination of moisture in corn using NIR spectroscopy, whereas SNV preprocessing led to significantly higher prediction errors. The findings demonstrate that indiscriminate application of SNV can be detrimental for precise quantitative spectroscopic analysis by disrupting the covariance between signals and analyte levels. Therefore, preprocessing strategies should be judiciously evaluated based on the specific data characteristics and modeling objectives.

کلیدواژه ها:

Corn ، Moisture ، Partial Least Squares ، pre-processing ، Standard Normal Variate