Development and Validation of a Computer Vision-Based Tool for Automated 3D Dental Model Arch Prediction

2.2.1 Utilization of 3M morphometric arch form templates

In this study, we used 3M arch form templates designed to classify dental arches into three main shapes: ovoid, tapered, and square (Fig. 1).

Figure 1: 3M morphometric arch form templates: standardized shapes for orthodontic assessment, showcasing tapered, square, and ovoid forms for classifying patient arch types

2.2.2 Majority voting method

The majority voting method is an approach for aggregating assessments by multiple evaluators into a consensual decision. This method excels in scenarios with evaluators of varied experience levels and mitigates subjectivity in decision-making. By employing this technique, the collective judgment of the evaluators was harnessed for a more precise classification of dental arch shapes (ovoid, tapered, and square) as commonly carried out in orthodontics.

Majority vote consensus for dental arch classification. During the dental arch shape classification, the evaluators independently assessed each model and cast their vote for one of the three shapes. The classification receiving the majority of votes was designated as the winner for that particular model. Mathematically, the consensual decision C for a sample can be represented as follows:

where E_i denotes the classification chosen by evaluator i, and ¥ is an indicator function equal to 1 if the internal condition is true (i.e., if evaluator i chooses classification k), and 0 otherwise. The classification k accumulating the highest total sum of indicator function values among all evaluators is selected as the consensus classification.

Weighted majority vote. To further refine the technique and account for the varying levels of experience among evaluators, a weighted majority vote was implemented. Here, the votes of each evaluator were assigned a weight reflecting their level of expertise, thereby enhancing the reliability of the decision-making process based on experience. The weighted consensus decision C _w was determined by

where w_i represents the weight given for experience, which is attributed to the vote of evaluator i, wherein more experience carries more weight. This approach ensures that the final classification decision is fair by the evaluators and thus yields a more reliable result.

The use of majority vote consensus and weighted majority voting methods provides reliability to the classification of dental arch shapes, integrating the collective expertise of the evaluators.

2.3.1 First phase: automatic dental alignment

This phase is based on the 3D to 2D projection and allows applying 2D image processing techniques. This not only simplifies the spatial complexity inherent in 3D shapes but also aligns the models to the center of the image in order to ensure a uniform orientation across all dental models.

Figure 2: Automatic alignment overview: transformation and alignment of 3D dental models to 2.5D for consistent analysis orientation

Transformation of images from 3D to 2D. The transformation of 3D point clouds into 2D images facilitates the use of established image processing techniques by encoding spatial information into a two-dimensional format. This is achieved through a projection mechanism that maps 3D coordinates onto a 2D plane, thus simplifying the representation of spatial structures.

The projection is executed using a virtual camera system, which is conceptually placed on a plane parallel to the XY plane of the 3D space. The orientation of the camera is defined such that its viewing direction is perpendicular to this plane, aligned along the negative Z-axis in order to capture the depth of the points as variations in pixel intensity.

The mathematical expression for this is given by

where P _3D (x, y, z) represents the coordinates of a point within the three-dimensional space, while P _2D (x,y) corresponds to the resulting coordinates on the two-dimensional flat image. The projection function, denoted as Project, maps each point in the 3D space (P _3D ) to a pixel on the 2D image (P_2D). This mapping process encodes the depth information of the Z-axis of each 3D point into the grayscale intensity of the 2D image pixels. By adopting this approach, we preserve the spatial relationships of the original 3D structure, allowing for further analysis and visualizations within a computationally more accessible two-dimensional framework.

When two or more points from the 3D space are projected onto the same point in the 2D space, their centroid is calculated. The centroid represents the average position of all the points, effectively summarizing their collective position in the two-dimensional representation. This approach helps to maintain the integrity of the spatial information despite the projection from 3D to 2D, ensuring that overlapping points are represented by a single, meaningful location on the 2D image.

Distance transformation. The concept of the distance transform (DT) fundamentally deals with calculating the distance from each point on a plane to a specified subset within it. Consider a map I : Ω ⊂ℤ ² - [0,1] where the domain Q is discretely defined as {1,n} x {1,n}. This mapping I is analogous to a digital binary image, with 0 denoting black and 1 representing white.

The foreground set Ow ∈ Ω is defined as the collection of all pixel locations in Ω where the image value equals 1 (white pixels):

In contrast, the background set Ow^C ∈ Ω constitutes the complement of Ow, encompassing all pixels not in the foreground.

The DT then generates a map DT : Ow →ℝ, where each pixel p ∈ Ow is assigned the shortest distance to the set Ow^C. This is formally expressed as follows:

Here, d(p, q) signifies a distance metric between points p and q, with the goal of identifying the nearest point to p in Ow^C. To this effect, the Euclidean distance is employed for d, thereby designating DT as the Euclidean distance transform (EDT).

To enhance the utility of the EDT, we incorporate three subsequent refinement steps: i) identifying the maximum EDT value along the Y-axis, ii) conducting a simple linear regression analysis, and iii) aligning the model with the centerline of the Cartesian plane, which serves as a standardized reference in our methodology.

Midline detection. The final step in our methodology involves aligning the dental model with the Cartesian plane's centerline. This is achieved by calculating the angle between the detected dental midline and a standard 90° reference line, utilizing the slope ß ₁ derived from our regression analysis. The angle 0, representing the deviation from the vertical, is determined as follows:

This equation accommodates the orientation of the gradient ß₁, ensuring that the angle 0 finely tunes the model's alignment to the vertical axis of the Cartesian plane by standardizing the reference alignment for all models.

2.3.2 Second phase: cusp detection

Figure 3: Cusp detection. This approach transforms a 3D dental model into a 2D representation, detects cusps separately on each half along the Y-axis, integrates these detections, and further applies cusp detection along the X-axis, resulting in a detailed cusp image.

This phase begins by transforming the projections from 3D to 2D and then duplicating the 2D image to identify the maximum intensity values along both the X- and Y-axes. It is important to note that this 2D representation is not a simple flat image; it serves as a false-color depth map, where the depth information is encoded in the image's colors

For the Y-axis, an additional process is carried out: the image is bisected to locate the maximum values or cusps. Once these maximum values are identified, the images are merged to produce a complete representation of the cusps in the digital dental model.

During this phase, the following steps were taken:

1. Image halving along the Y-axis. The projected image is divided along the X-axis, enabling a targeted search for maximum pixel values, which enhances cusp detection precision.

2. Maximum value search along the Y-axis. Within each half, an exhaustive search is conducted to find the maximum values along the Y-axis. The corresponding equation is given by:

3. Maximum value search along the X-axis. Unlike that for the Y-axis, the search along the X-axis spans the entire width of the image, so it is not necessary to divide the image to search for the maximum intensity pixels on this axis. The corresponding equation is given by:

4. Image integration. The previously found maxima are merged into an image that contains all the cusps or maximum intensities along the X- and Y-axes. The process is mathematically represented as:

This formula illustrates the merging of the maximum intensity values found in the left and right halves of the image (along the Y-axis) with those found on the X-axis. The Merge function integrates these individual findings into a single image, highlighting the cusps in the digital dental model.

This strategy enables the localization of essential markers on the dental arch, facilitating a representation of the cusps to be analyzed in subsequent phases.

2.3.3 Third phase: curve fitting

In the third phase, the focus is on selecting mathematical models to accurately describe the shape of the dental arch. This phase utilizes 21 reference points, which are based on morphometric templates and arranged following the Fibonacci series, for mathematical approximation. Various mathematical models were evaluated, but a sixth-order polynomial model was chosen due to its balance between adaptability and the right mix of computational efficiency and precision in modeling the dental arch form ^23-25.

Figure 4: Third phase: curve fitting. This diagram demonstrates how the models for the dental arch shapes are created using templates (tapered, square, ovoid) with 21 reference points. These points enable precise curve fitting with a sixth-order polynomial, representing each arch.

Annotation with reference points. Each template is annotated with 21 reference points to represent the geometry of the dental arch. This includes a central point, complemented by ten points on each side, placed according to the Fibonacci sequence or golden ratio, thus reflecting the natural proportions observed in biological forms.

Exploration of mathematical models. A variety of mathematical models are evaluated, including polynomial functions, Fourier series, exponential models, and Gaussian distributions, in order to identify the most suitable representation of the dental arch shape.

Application to main arch forms. Our curve-fitting process is applied to morphometric templates representing the three main arch forms in orthodontics (ovoid, square, and tapered). This categorization is applied to both the lower and upper jaw, thereby recognizing the unique characteristics and clinical significance of each arch form in the dental model. This approach ensures that the developed mathematical models are broadly applicable and reflect the diverse arch geometries observed in clinical practice.

Selection of the sixth-order polynomial model. The sixth-order polynomial model was chosen for its ability to capture the curvatures of dental arch shapes, offering a balance between model complexity and computational tractability. This decision was supported by extensive literature documenting the effectiveness of polynomial models in modeling dental forms. The model is defined as follows:

where f (x) represents the curve fitting function, with the coefficients p₁,... ,p₇ optimized to align with the arch shapes.

2.3.4 Fourth phase: optimization of arch shape selection through model comparison

This phase is characterized by a detailed comparative analysis between the cusp images and the sixth-order polynomial models fitted based on the 21 reference points annotated on the morphometric templates.

Figure 5: Comparative analysis of dental arch shapes. This diagram illustrates the optimization process of selecting the most accurate mathematical model for each dental arch shape, using error metrics (SSE and RMSE) against reference points from the cusp image.

The comparative analysis was conducted using advanced curve-fitting techniques, where the similarity between theoretical models and empirical data was assessed using error metrics such as the sum of squared errors (SSE) and the root mean square error (RMSE). These metrics provide a quantitative measure of the models' fidelity in replicating the actual shapes of the arch as outlined by the cusp images.

Below is a description of the aforementioned error metrics used to quantify the alignment between the mathematical models and the cusp images:

• Sum of squared errors (SSE). This metric aggregates the squared differences between the observed cusp locations in the images and the corresponding predictions made by the mathematical model.

where y_i,actual are the actual cusp positions and y_i,model are the positions predicted by the model.

• Root mean square error (RMSE). This metric provides a measure of the average magnitude of the error, offering a more interpretable assessment of model performance.

Interpretation of SSE and RMSE in arch shape comparison. The interpretation of the SSE and the RMSE in the context of dental arch shape comparison extends beyond mere numerical values. While high SSE and RMSE values might initially indicate discrepancies between the mathematical models and the actual cusp images, these metrics play a broader and combined role in our analysis:

• Rather than outright disqualifying models due to high error values, the SSE and RMSE provide insights into which arch shape (ovoid, tapered, or square) best fits the three-dimensional structure of the dental arch under study.

• Through comparative analysis, these metrics enable the evaluation of relative accuracy among models, identifying the one that most accurately reflects the specific geometries of the empirical data.

Therefore, the essence of SSE and RMSE in our research is to enable a comparative evaluation, highlighting the arch shape model that most faithfully represents the arch form:

Here, Δ_model signifies the model with the lowest RMSE, hence the closest approximation to the actual dental arch shape, emphasizing comparative rather than absolute error analysis to identify the anatomically most accurate model.

For a better understanding of accuracy compared to the other models, we calculated the normalized error percentage for each model while considering the total SSE (the sum of the SSE across all models) and adjusting each model's error in relation to this total:

This percentage reflects the model's error relative to the collective error, where values closer to 100 indicate higher model precision. This numerical representation offers an understandable metric for selecting the most accurate model, which would be the one closest to 100.

2.4.1 Attribute agreement analysis

Attribute agreement analysis examines the consistency of assessments by Evaluators 1, 2, 3, and our tool, both within and against a known standard. Fleiss's kappa quantifies this agreement:

where P _o is the observed agreement, and P _e is the expected agreement by chance.

2.4.2 Statistical validation metrics

As previously mentioned, in order to quantify prediction accuracy, we used the RMSE and SSE, i.e., as expressed in Eqs. (13) and (12), respectively.

2.4.3 Gage R&R study - XBar/R method

The Gage R&R study assesses the measurement system's variability, which is crucial for understanding the sources of variability in the measurements:

Combining these methods ensures a comprehensive and multi-faceted validation of the model's performance, affirming its reliability for clinical application.

Comparative Gage R&R study for lower and upper jaw evaluations

The Gage repeatability and reproducibility (R&R) study, utilizing the XBar/R method, was conducted to assess the measurement systems for both lower and upper jaw analyses, showcasing the dynamics between repeatability, reproducibility, and part-to-part variability.

Figure 7: Run chart comparing expert evaluations for upper jaw arch form at two different time points. Arch forms are coded as 1 (square), 2 (ovoid), and 3 (tapered), with evaluators represented by color: Evaluator 1 (blue), Evaluator 2 (red), and Evaluator 3 (green).

Figure 8: Bar chart illustrating the components of variation in Gage R&R studies for the lower and upper jaws, where blue bars indicate the % contribution and red bars represent the % study variation

3.2.1 Attribute agreement analysis

In the above-presented attribute agreement analysis, the consistency of three evaluators and our method was assessed for both the lower and upper jaws. The analysis aimed to measure the reliability of the evaluators' measurements against each other and against a standard measurement.

Figure 10: Comparison of intra- and inter-rater reliability in dental arch assessments for lower and upper jaws. The left panels display the within-evaluator agreement (i.e., Eva 1, Eva 2, Eva 3, Our Method) with the corresponding percentages and 95% CIs. The right panels show evaluator vs. standard agreement, including the percentage and the 95% CIs for both jaws.

Each evaluator vs. the standard. This portion assessed the agreement level for each evaluator, providing kappa statistics, standard errors, Z-scores, and p-values for each response and overall (Table III).

All evaluators vs. the standard. This portion summarized the collective agreement, offering kappa statistics for each response and overall, along with standard errors, Z-scores, and p-values, demonstrating the group's consistency with the standard (Table III).

3.2.2 Mathematical model fit and comparison summary

The mathematical models for predicting dental arch shapes were evaluated with a focus on polynomial and non-linear models. The comparison was based on the SSE and the RMSE, aiming to identify a model that accurately captured the complexity of dental arches while maintaining a balance to prevent overfitting (Table IV).