Potential issues with data that has not been preprocessed

Through performance parameter testing, we can obtain raw parameter data, which may exhibit the following issues:

(1) Does not belong to the same dimension. That is, when the features have different scales, they cannot be compared directly.

(2) Information redundancy For certain quantitative features, the meaningful information they convey is inherently interval‑based; for example, academic performance. If one is only interested in whether a student has passed or failed, it is necessary to convert the quantitative scores into binary values—“1” for pass and “0” for fail.

(3) Qualitative features cannot be used directly. Certain machine learning algorithms and models can only accept quantitative features as input, so qualitative features must be converted into quantitative ones. The simplest approach is to assign a numerical value to each category; however, this method is overly flexible and increases the burden of hyperparameter tuning.

(4) There are missing values. Missing values need to be imputed.

(5) Low information utilization rate Different machine learning algorithms and models exploit information in the data in distinct ways. As mentioned earlier, in linear models, dummy encoding of categorical features can yield non-linear effects. Similarly, polynomial transformation of quantitative variables, or other types of transformations, can also achieve non-linearity.

Data preprocessing methods

Dimensionless normalization transforms data of different scales into a common scale. Common dimensionless normalization methods include standardization and min–max scaling. Standardization assumes that the feature values follow a normal distribution; after standardization, the data are transformed into a standard normal distribution. Min–max scaling leverages boundary values to rescale the range of feature values to a specific interval, such as [0, 1].

The difference between standardization and normalization is that standardization processes data column‑wise, using the z-score transformation to bring feature values into a common scale. Normalization, on the other hand, processes data row‑wise, with the goal of ensuring that sample vectors share a uniform scale—i.e., they are all converted into “unit vectors”—when computing similarity via dot products or other kernel functions.

The core of quantile-based binarization lies in setting a threshold: values greater than the threshold are assigned 1, while those less than or equal to the threshold are assigned 0.

Typically, dummy coding is used to convert categorical features into quantitative ones: suppose there are N distinct categories; the original feature is expanded into N separate features, with the i-th expanded feature taking a value of 1 when the original feature corresponds to the i-th category and 0 otherwise. Compared with direct assignment, dummy coding avoids additional hyperparameter tuning, and for linear models, it can produce non-linear effects. For example, to examine the impact of the four seasons, one might use the following encoding scheme:

There are five commonly used methods for handling missing values.

(1) Case-removal method If any variable contains missing data, the corresponding case is excluded from the analysis. This approach is particularly effective when the proportion of missing values is relatively small. However, there is no consensus in the academic community regarding what constitutes a “small” proportion—some suggest 5%, while others propose 20%. A key limitation is that reducing the sample size to ensure data completeness can lead to substantial resource waste and the loss of valuable information embedded within those cases. Moreover, when missing data are non‑random, this method may introduce bias into the dataset, potentially resulting in incorrect conclusions.

(2) Mean Replacement Method The attributes of a variable are categorized into numerical and non-numerical types, and these are handled separately. If the missing values are numerical, they are imputed by replacing each missing value with the mean of that variable across all other observations. If the missing values are non-numerical, they are imputed using the mode—the most frequently occurring value of that variable among all other observations. However, this approach introduces bias: it does not affect the estimation of the variable’s mean, but it relies on the assumption of completely random missingness and tends to reduce the variable’s variance and standard deviation.

(3) Thermal Card Filling Method For a variable containing missing values, the hot‑deck imputation method identifies the most similar observation in the dataset and replaces the missing value with the corresponding value from that similar case. Different problems may employ different criteria to assess similarity. The most common approach is to use a correlation matrix to determine which variable (e.g., variable Y) is most closely related to the variable with missing values (variable X). The cases are then ranked according to the magnitude of Y’s observed values, and the missing values in variable X are imputed using the data from the case immediately preceding the missing value in the ranking.

(4) Return substitution method The regression imputation method first requires selecting several predictor variables to forecast the missing values, then constructing a regression equation to estimate those missing values—essentially replacing the missing data with their conditional expectations. Compared with earlier imputation techniques, this approach leverages as much information as possible from the dataset. However, its drawbacks include: (a) it tends to overlook random error, leading to underestimation of standard deviations and other unknown measurement characteristics; moreover, this issue becomes more pronounced as the amount of missing data increases. (b) Researchers must assume a linear relationship between the variable containing missing values and the other variables, an assumption that often does not hold in practice.

(5) Multiple Replacement Method Multiple imputation is a data‑imputation and statistical analysis method developed by Rubin et al. in 1987, representing an improvement over simple imputation. First, the multiple imputation technique replaces each missing value with a set of plausible values, thereby accounting for the uncertainty inherent in the imputed data. Subsequently, standard statistical procedures are applied to analyze the resulting multiple datasets. Finally, the statistical estimates from each dataset are combined to yield an overall estimate of the population parameter. Because multiple imputation does not substitute a single value for missing data but instead generates a random sample of possible imputations, it captures the uncertainty arising from missing data and enables more robust statistical inference. By employing this approach, researchers can readily infer the unknown characteristics of missing data without discarding any observations.

Common data transformations include polynomial-based, exponential-based, and logarithmic-based methods. For univariate functions, data transformations can be implemented using a unified approach.

Table 1: Summary of Data Preprocessing Methods

Feature selection methods

Once data preprocessing is complete, we need to select meaningful features and feed them into reliability analysis algorithms and models for training.

Feature selection and dimensionality reduction share some similarities: both aim to reduce the number of attributes (or features) in a feature set. However, they differ in their approaches: dimensionality reduction typically leverages relationships among attributes—such as combining multiple attributes to create new ones—thereby transforming the original feature space. In contrast, feature selection involves selecting a subset from the original feature set, maintaining an inclusion relationship without altering the original feature space.

The goal of feature selection is: “A sound mathematical model should be simple in form. The purpose of constructing machine learning models is to uncover the underlying structure and essence of the problem from the original feature dataset; accordingly, the selected features should provide a more informative interpretation of the problem.”

(a) Improve the accuracy of forecasts;

(b) Construct faster, lower‑cost prediction models;

(c) Enables a better understanding and interpretation of the model.

Generally, feature selection is guided by the following two considerations, as shown in Table 2.

Table 2: Criteria for Feature Selection

(Part of this article is original and intended for educational reference only. If you wish to reprint it, please be sure to cite the source.)

Author Introduction