ABSTRACT

Introduction

Although the Central Physician Appointment System (MHRS) implemented in Türkiye in 2010 improved access to healthcare services, patients cannot directly schedule radiological imaging appointments. This study aims to predict the likelihood of radiology appointment cancellations at city and state hospitals using machine learning algorithms.

Methods

Radiology appointment data recorded between January 2021 and June 2024 at a city hospital and a state hospital were analyzed. The dataset contained more than 3 million rows, with independent variables including age, gender, International Classification of Diseases-10 diagnosis codes, and imaging type. Logistic regression, random forest, and support vector machine (SVM) models were developed and evaluated using accuracy, precision, recall, and F1-score.

Results

Appointment cancellations were less frequent at the city hospital than at the state hospital. Logistic regression achieved 82% accuracy and demonstrated the best performance among the conventional models. Following additional validation procedures with different train-test splits, random forest and SVM achieved accuracies of approximately 97.66% and 97.69%, respectively. Higher cancellation rates were observed among patients aged 18-30 and for non-urgent procedures. Most cancellations occurred either 3-5 days before the scheduled date or on the same day.

Conclusion

Predicting radiology appointment cancellations with machine learning supports efficient use of hospital resources. Developing tailored strategies based on hospital type can improve capacity management, patient satisfaction, and overall service quality.

Introduction

Digital transformation enhances the quality of healthcare services and has become an essential component of modern healthcare management. Compared to traditional appointment scheduling methods, online appointment systems provide faster and more practical solutions for both patients and healthcare providers (1, 2). Studies indicate that web-based services are increasingly preferred over conventional methods. Traditional appointment systems generally involve direct interaction with scheduling personnel via phone or in person. While these methods offer flexibility, they are time-consuming and constrained by staff availability. On the other hand, online appointment systems automate scheduling and integrate the process into a single accessible platform. This not only reduces the workload on administrative staff but also minimizes patients’ waiting times (3).

Online appointment systems are widely used in healthcare to improve scheduling coordination and facilitate patient access to services. Compared with traditional appointment methods, these systems may reduce administrative workload, support more efficient resource utilization, and improve operational workflow through centralized scheduling and automated processes (4-7).

Recent studies have demonstrated that machine learning models can effectively predict radiology appointment no-shows with moderate to high accuracy, typically achieving area under the curve (AUC) values ranging from approximately 0.75 to 0.93 depending on the dataset and clinical context (8-13).

However, evidence also suggests that model performance may vary across healthcare settings and often declines when transitioning from retrospective analyses to real-world deployment, highlighting the need for context-specific evaluation and continuous recalibration (8).

Prior research consistently identifies several key predictors of appointment cancellations, including previous no-show behavior, scheduling characteristics such as lead time and appointment timing, imaging modality and exam type, and socioeconomic factors such as insurance status and access to care (9-13). These findings suggest that both patient-related and system-level variables play a critical role in shaping attendance behavior.

The adoption of patient-centered applications has been shown to reduce appointment no-show rates. Allowing patients control over appointment scheduling enhances their commitment to the system. However, online appointment systems are not equally accessible to all demographic groups (8). Particularly, older adults, those with lower levels of education and income, and those residing in rural areas often face challenges in effectively utilizing digital healthcare services. Consequently, many still rely on phone-based appointment systems (9).

Managing waiting times in healthcare is a critical issue that affects patient satisfaction and access to care. The increasing demand for medical services, coupled with the limitations of traditional appointment scheduling methods, has made waiting times a major concern for both patients and healthcare providers (10). The implementation of technology, particularly online appointment systems, has introduced new ways to improve patient access and reduce waiting times in hospitals. The advantage of leveraging online appointment systems lies in their potential to streamline operations, enhance the efficiency of healthcare service delivery, and improve overall patient experience (11).

Globally, various countries have implemented digital appointment management platforms to facilitate patient access to healthcare services and improve scheduling efficiency. Examples include Türkiye’s Central Physician Appointment System (MHRS), the United Kingdom’s NHS online services, and widely used patient portal systems such as MyChart and Zocdoc (12-19). These platforms primarily serve as operational tools for appointment coordination and digital access management within healthcare systems.

It is well established that online appointment systems significantly reduce patient waiting times (14). These systems allow patients to efficiently schedule and manage doctor appointments while also enabling healthcare providers to better organize patient flow. Previous studies have shown that digital appointment systems may contribute to improved scheduling coordination, reduced administrative burden, and more efficient management of healthcare resources (14-17). These systems are particularly valuable in high-demand clinical settings where workflow optimization and appointment management are critical for operational efficiency.

A study conducted by AlHajeri et al. (20) in 2021 demonstrated that the adoption of online patient appointment systems significantly improved the quality of healthcare services. These findings align with other studies reporting a decrease in average patient waiting times (14). For instance, a study conducted by Maeder and Martin-Sanchez in 2012 highlighted that web self-service applications in primary healthcare settings led to improved patient perceptions and reduced waiting times (18).

Previous studies have suggested that digital appointment systems may improve scheduling coordination and operational workflow in healthcare settings (20, 21). The study, conducted across multiple geographical regions, further supported the benefits of such systems. Cao et al. (22) also demonstrated that online appointment systems decreased hospital congestion and improved patient flow, leading to a noticeable reduction in waiting times. These findings underscore the potential of online appointment systems in improving healthcare service efficiency and minimizing patient waiting times (23).

Considering the benefits of online appointment systems, they could be equally useful for scheduling not only medical consultations but also radiological imaging appointments (e.g., MRI, CT, X-ray, angiography). The method of scheduling radiological imaging appointments varies depending on the country, the hospital’s digital infrastructure, and the overall healthcare system. In Türkiye, radiological imaging requests are typically scheduled in person at hospital information desks or appointment offices. Hospitals process appointment requests and subsequently transfer the information to the MHRS, where appointments are scheduled. However, patients are unable to directly initiate or view these appointment requests through MHRS.

A study conducted in Tuscany, Italy, found that MRI and CT scans were not optimally managed, leading to excessively long patient waiting times. The study identified both underutilization and overutilization due to a lack of organized scheduling. Policy-makers emphasized the need to investigate the causes of long waiting times and to implement strategies to reduce the burden on the healthcare system (24). Another study assessing the financial impact of MRI and CT waiting times on the healthcare system focused on developing a decision-support system to optimize scheduling. This system aimed to minimize financial losses and reduce patient waiting times for diagnostic imaging services (25, 26).

A study conducted at a Canadian hospital found that radiology appointment requests often faced scheduling challenges across the emergency, outpatient, and inpatient services. To address these issues, the hospital implemented measures to enhance efficiency during peak hours, streamline workflows, and reduce waiting times. Evaluating patient and workflow processes was shown to improve efficiency in radiology departments (26).

Increasing waiting times also have a direct impact on clinical outcomes. Studies suggest that prolonged waiting times may exacerbate patient symptoms, deteriorate medical conditions, and lead to poorer clinical outcomes (1). A recent study demonstrated that increased waiting times for CT imaging were associated with tumor growth and cancer progression in lung cancer patients, ultimately affecting clinical outcomes (26).

Despite these advancements, limited evidence exists on how predictive models perform across different hospital types within the same healthcare system. Variations in infrastructure, patient profiles, and operational workflows may influence model performance and generalizability, necessitating further investigation across diverse clinical environments (27-31).

This study aims to predict radiology appointment cancellations using data obtained from a city hospital and a state hospital, and to compare the performance of multiple machine-learning models across these two public healthcare settings. By identifying key predictive factors associated with appointment cancellations, the study seeks to provide practical insights for improving scheduling efficiency and resource utilization in radiology services.

Methods

Radiological imaging appointments are scheduled through hospital secretariats. However, as of December 2024, secretariats process these requests in the MHRS system and create appointment schedules. Scheduling is based on the type of imaging requested by the patient. This process, however, poses certain challenges. For instance, if a patient visiting an otolaryngology clinic is referred for an MRI by their physician, they cannot obtain the MRI appointment date without consulting the hospital secretariat. This creates a gap in the scheduling system, particularly for canceled appointments.

This study was approved by the Ethics Committee of Alanya University on February 4, 2025 (decision number: 02). Because the study used retrospective, anonymized hospital records, the committee waived the requirement for individual patient consent. All data were anonymized prior to analysis to ensure confidentiality and protect patients’ privacy.

In this study, data from a randomly selected city hospital and a state hospital in Türkiye were analyzed. The aim was to predict which patients were more likely to cancel their radiological imaging appointments (e.g., magnetic resonance imaging, computed tomography, colonoscopy, and endoscopy) and to determine which type of hospital was more prone to cancellations. This predictive analysis provides hospitals with an opportunity to optimize their operational processes.

The dataset used in this study comprises data collected from a state hospital and a city hospital accredited as approved in 2024 stages 6 and 7 in Türkiye. The state hospital serves a population of 594,053 at the district level, whereas the city hospital serves a population of 420,699 at the provincial level. In this study, the term “city hospital” refers to a large-scale, tertiary healthcare facility with high patient capacity and advanced medical infrastructure. The term “state hospital” refers to a secondary-level public healthcare institution that provides general medical services and has relatively low patient volume and resource capacity. These terms are used to distinguish among different types of public healthcare institutions within the national healthcare system.

Additionally, the difference in the number of radiological imaging devices available at the two hospitals is relatively small. The city hospital has 113 imaging devices, while the state hospital has 106; this discrepancy is primarily due to the presence of Holter monitoring devices. Therefore, the number of imaging devices is not expected to have a significant impact on appointment scheduling.

The dataset includes all radiological imaging requests recorded between January 2021 and June 2024 at these hospitals. Table 1 provides a detailed breakdown of the dataset contents. The dataset includes demographic information (e.g., age and gender), types of imaging procedures (e.g., MRI, X-ray), diagnosis codes [International Classification of Diseases (ICD) (ICD-10 codes)], appointment dates, appointment cancellation records, and the time elapsed before cancellation.

Independent variables selected for analysis include age, gender, type of imaging procedure, and diagnosis codes, which are likely to influence appointment cancellations. A feature-selection process guided the choice of variables to enhance model accuracy.

Statistical Analysis

In Table 1, a comparative analysis of the city and state hospitals revealed that the state hospital serves a larger patient population and offers a wider range of services. The state hospital has a high patient density, particularly among individuals under 18 years and those aged 30-65 years. In contrast, the city hospital is preferred more frequently by adult and elderly patients. Although both hospitals provide a broad range of imaging services, the state hospital offers additional procedures, such as electroencephalography, echocardiography, electrocardiography, and Holter monitoring, which contribute to its more comprehensive healthcare service portfolio.

All the collected data were transferred to PostgreSQL, creating a database containing approximately 3 million rows. The database was then connected to Python for further analysis. In this study, the Pandas, NumPy, and Scikit-learn libraries were used. These libraries, commonly used for data analysis and prediction in Python, enable normalization of values to a defined range and offer various useful functions. Following this step, missing values in the dataset were identified and imputed using the means of their respective columns.

Data preprocessing was conducted to prepare hospital data for analysis. The analysis was performed using Python version 3.11.4, with data processing nodes such as Number Filter.

• Data Cleaning and Filtering (Number Filter): In this step, unnecessary spaces within the text and irrelevant numerical values were removed.

• Categorical Variable Transformation: Categorical variables were converted into numerical values to ensure compatibility with machine learning models.

• Handling Missing Values: The dataset was checked for missing values, and any missing values were replaced with the mean value of the respective column. This method preserved the dataset’s integrity while preventing missing data from negatively impacting analysis results. Consequently, a stable analysis was conducted without data loss during the model training and prediction phases.

Feature Selection and Model Training

To develop prediction models, the dataset was first structured as input data, and feature selection was applied to identify key predictors. The feature selection method was applied to 3,222,251 rows of data to enhance algorithm performance. Subsequently, the dataset was divided into 70% training data and 30% test data for model evaluation.

• Training Data: Used to support the learning process of the models.

• Test Data: Used to assess the prediction accuracy of the models.

The dataset was partitioned into training (70%) and test (30%) subsets. Additional analyses using alternative train–test splits were also conducted to evaluate the stability and robustness of model performance across different data partitions.

To ensure methodological rigor and to prevent potential data leakage, the dataset was partitioned using a patient-level splitting strategy. Specifically, all records belonging to the same patient were assigned to either the training or the test set, thereby preventing overlap between the two sets. This approach ensures that the model does not learn patient-specific patterns that could artificially inflate performance.

In addition, a temporal validation strategy was applied by using earlier records for model training and more recent records for testing. This approach was intended to more closely simulate real-world clinical implementation scenarios, where models are applied to unseen future data.

Furthermore, only variables available prior to the appointment date were included in the predictive modeling process. Variables that could directly or indirectly encode the outcome (such as cancellation timing or post-appointment information) were excluded to ensure that the model predictions were based solely on information available at the time of appointment scheduling.

The final set of input variables for the predictive models included age, gender, imaging procedure type, and ICD-10 diagnosis codes. These variables were selected based on both domain knowledge and their potential relevance to appointment cancellation behavior.

To ensure compatibility with machine learning algorithms, categorical variables, particularly ICD-10 diagnosis codes, were transformed into numerical representations using one-hot encoding. This approach preserves the categorical structure of the data and enables the models to effectively utilize diagnostic and procedural information.

Feature engineering was designed to maintain clinical relevance by including only pre-appointment information, thereby enhancing the applicability of the models in real-world healthcare settings.

Feature selection was conducted to reduce dimensionality and improve model interpretability and generalization performance. Variables with low variance or limited contribution to predictive performance, as identified through exploratory statistical analyses and model-based importance evaluations, were excluded during preprocessing. The final feature set—age, gender, imaging procedure type, and ICD-10 diagnosis codes—was selected based on both domain knowledge and their relevance to appointment cancellation behavior identified during exploratory analyses.

Model Performance Evaluation

To measure the accuracy of the developed models, predictions were compared with actual values. Several evaluation metrics were employed, including:

• Accuracy: The proportion of correctly predicted instances relative to the total dataset, reflecting overall model performance.

• Precision: Indicates how precise the model’s positive predictions are.

• Recall (Sensitivity): Measures how well the model captures actual positive instances.

• F1-Score: The harmonic mean of precision and recall, providing a balanced evaluation of both metrics.

These metrics assess how accurately the model predicts both positive and negative cases by comparing predicted values with actual values.

Because the dataset exhibited class imbalance between canceled and non-canceled appointments, model performance was evaluated using multiple complementary metrics, including precision, recall, F1-score, and receiver operating characteristic (ROC)-AUC, rather than relying solely on accuracy values.

In addition to accuracy, precision, recall, and F1-score, model performance was further evaluated using the ROC curve analysis and the AUC metric. These metrics provide a more comprehensive assessment of model performance, particularly in classification tasks with potential class imbalance. These metrics are particularly important in healthcare datasets, where class imbalance may affect the reliability of accuracy-based evaluations.

Confusion matrices were also generated to examine the distribution of true positives, true negatives, false positives, and false negatives, enabling a more detailed evaluation of model predictions.

Furthermore, model calibration was considered to assess the agreement between predicted probabilities and observed outcomes, which is particularly important for clinical applicability and decision-making processes.

To further assess the robustness of model performance, additional analyses were performed using different train-test splits. In each iteration, the models were retrained and re-evaluated to determine whether performance remained stable across alternative data partitions. This procedure served as an additional safeguard against overfitting and data leakage, thereby supporting the generalizability of the findings.

To minimize the risk of data leakage, only variables available prior to the scheduled appointment date were included in the predictive modeling process. Variables directly associated with appointment outcomes or post-appointment processes were excluded from model development. Furthermore, all records for the same patient were assigned exclusively to either the training or the test dataset to prevent patient-level overlap between data partitions.

Application of Machine Learning in Healthcare

In recent years, machine learning models have been increasingly utilized to optimize healthcare system operations and improve service delivery. In particular, predictive modeling approaches have been widely applied to analyze appointment scheduling and no-show behaviors in radiology settings. Previous studies have demonstrated that machine learning models can predict radiology appointment no-shows with moderate to high accuracy, typically achieving AUC values ranging from approximately 0.73 to 0.93 depending on the dataset, model type, and implementation context (8-10,31-34).

Appointment scheduling and cancellation behaviors are critical factors that affect hospital resource utilization and operational efficiency (26-38). However, model performance may vary across healthcare settings and often declines when transitioning from retrospective analyses to real-world deployment, highlighting the importance of context-specific model development and validation (8, 31).

In this study, multiple predictive models, including logistic regression, random forest, support vector machines (SVMs), and linear regression, were applied to predict appointment cancellations in two different healthcare environments—a city hospital and a state hospital. Linear regression was included as a baseline model to provide a reference point for evaluating the performance of more advanced machine learning algorithms.

The performance of these models was systematically compared, and the findings were used to derive practical recommendations aimed at improving scheduling efficiency and resource allocation in healthcare systems.

Results

Data Preprocessing

The preprocessing steps applied to the dataset are as follows:

• Handling Missing Data: Missing data were handled separately for numerical and categorical variables to ensure methodological appropriateness. Numerical variables were imputed using mean values, while categorical variables, such as ICD-10 diagnosis codes, were imputed using the mode or treated as a separate category where appropriate. This approach ensured the integrity of categorical information and prevented the introduction of invalid numerical representations for non-numeric variables. By applying variable-specific imputation strategies, the robustness and reliability of the dataset were improved for subsequent modeling. Although more advanced imputation techniques were considered, the selected approach was deemed appropriate given the scale and structure of the dataset.

• Outliers were identified using the interquartile range method. Values falling outside the lower and upper 5% thresholds were adjusted by replacing them with the nearest boundary values.

• Feature Selection: To enhance model performance, the most relevant features were selected. The feature selection process identified age, gender, and diagnosis codes as significant predictors.

• Continuous variables were scaled using minimum-maximum scaling to ensure compatibility across machine learning algorithms and to reduce the effect of differences in variable magnitudes during model training.

As shown in Table 2, the dataset consisted of canceled and non-canceled appointments, with an observed class distribution of approximately 18% canceled and 82% non-canceled cases, indicating class imbalance.

To address this issue, model performance was evaluated using multiple metrics beyond accuracy, including precision, recall, F1-score, and ROC-AUC, which provide a more reliable assessment in imbalanced classification settings.

Calculation of Time Difference

The number of days between a patient’s request date and appointment date was calculated. This computation was performed in a Pandas DataFrame, facilitating the analysis of time gaps between the two dates.

Figures were generated from cleaned, outlier-free data to assess the workflow efficiency in both hospitals and to identify potential areas for improvement.

Figure 1 presents a comparative analysis of the time difference, in days, between appointment and radiology imaging dates in city and state hospitals. Each data point represents an individual patient.

• In the city hospital, the time difference is predominantly between 0 and 10 days, indicating that procedures are completed swiftly and that the hospital follows an efficient process management system. Notably, a peak in density between 0 and 6 days suggests that patients experience minimal delays between appointment scheduling and imaging procedures.

• As of January 2025, the state hospital serves 594,053 individuals at the district level, whereas the city hospital provides healthcare to 420,699 individuals at the provincial level. The city hospital’s smaller patient population may contribute to faster, more efficient service delivery owing to a reduced patient load.

• In contrast, the state hospital’s graph displays a broader range of time differences, spanning 0-80 days. While a significant concentration of cases occurs within the 0–20-day range, a substantial number of cases exceed 20 days, suggesting delays in certain processes.

This variation may be attributable to higher patient volumes in state hospitals, which lead to resource constraints and extended waiting times. Increased patient density often results in longer processing times, particularly when hospital resources are limited, causing fluctuations and inefficiencies in workflow management.

The city hospital, benefiting from lower patient density, completes patient appointments more quickly and demonstrates more consistent performance. In contrast, the state hospital’s larger patient population may be one of the main causes for process variability and prolonged processing times. This situation may result from differences in resource management, workflow processes, and organizational structure. When the operations of both hospitals are evaluated, the impact of population size on process efficiency is evident.

Table 3 presents a comparison of patient cancellation behaviors in radiological imaging requests based on hospital type. The machine learning models used to predict patient appointment cancellations are: linear regression, logistic regression, random forest, and SVM. During the model selection process, the fundamental characteristics of each algorithm, data structure, and study objective were considered.

To further enhance interpretability, ROC curves were generated for the random forest model (Figure 2), providing a visual representation of classification performance across different thresholds.

In addition, feature importance analysis was conducted to identify the most influential predictors of appointment cancellation (Figure 3).

Linear Regression

Linear regression is a fundamental method used to model the relationship between dependent and independent variables. This model is known for its strong performance on continuous variables; however, its effectiveness in classification problems is expected to be limited. In this study, linear regression was selected as a baseline comparison model and used as a reference for identifying indirect effects such as appointment cancellations (27).

The linear regression model achieved 78.82% accuracy on city hospital data. The model reasonably predicted the linear relationship between continuous independent variables and the dependent variable. However, the error rate was 21.18%, indicating that the linear model may not fully capture the complexity of healthcare data.

For the state hospital, the linear regression model achieved an accuracy of 75.60% and an error rate of 24.40%. The lower accuracy rate, compared with that of the city hospital, suggests that the state hospital dataset contains greater variability. While this model provided a limited solution for classification problems, it served as a fundamental method for understanding the data structure.

Logistic Regression

Logistic regression is a widely used classification model for binary dependent variables. It has a strong theoretical foundation for addressing binary classification problems, such as appointment cancellations. This model predicts cancellation probabilities using a linear combination of independent variables and normalizes the results to lie between 0 and 1. The compatibility of Logistic Regression with the dataset is attributed to the fact that appointment cancellations can largely be predicted based on independent variables (28).

The logistic regression model demonstrated effective performance on data from a city hospital, achieving an accuracy rate of 82.46% and an error rate of 17.54%. The precision and recall were 83%. The model provided a highly successful solution for predicting appointment cancellations and effectively handled the binary nature of the dependent variable.

For the state hospital, the logistic regression model achieved an accuracy rate of 80.50% and an error rate of 19.50%. Precision 81%, while recall was 80%. Although the model performed slightly worse than the city hospital, it remained an effective tool for predicting appointment cancellations.

Random Forest

Random forest is an ensemble-based model composed of multiple decision trees and is known for its high performance on complex datasets, particularly when non-linear relationships exist among variables. The model generates predictions by constructing multiple decision trees for each observation, and determines the final classification by majority vote. This approach effectively reduces variance and bias commonly observed in single decision-tree models. In addition, its relative robustness against overfitting makes it a reliable predictive tool, especially for large-scale datasets (29).

In this study, the random forest model initially demonstrated high accuracy. However, to ensure a more reliable and generalizable evaluation of model performance, additional analyses were conducted using different train-test data splits. Following these validation procedures, the model achieved an accuracy of approximately 97.66%.

However, these findings should be interpreted with caution, as retrospective healthcare datasets may contain structural patterns that can contribute to inflated predictive performance despite validation strategies intended to prevent this.

Results from both the city and state hospital datasets indicate that the model consistently demonstrates high performance across different data settings. Nevertheless, the initially observed extremely high accuracy values may have been influenced by dataset-specific characteristics and the separability of variables.

Support Vector Machine

SVMs are powerful classification algorithms capable of handling high-dimensional datasets and non-linear classification problems. The fundamental principle of SVM is to identify an optimal hyperplane that maximizes the margin between different classes, thereby improving classification performance. Although originally developed for linear classification, SVM can also model non-linear relationships through the use of kernel functions, which transform the input space into higher-dimensional feature spaces (30).

Due to their ability to generalize well, particularly in complex and high-dimensional data, SVM algorithms have been widely used in healthcare applications, such as disease diagnosis and prognosis. In addition, SVM can be extended to multi-class classification problems through one-vs.-one and one-vs-all strategies. In this study, the dataset was modeled using binary classification for each class pair within the SVM framework (31, 32).

In the present study, the SVM model demonstrated high classification performance across both hospital datasets. Following additional validation procedures using different train–test splits, the model achieved an accuracy of approximately 97.69%, indicating strong predictive capability. These findings suggest that SVM is an effective and robust classifier for predicting appointment cancellations in city and state hospitals.

To evaluate whether model performance depended on a single data split, repeated train–test analyses were conducted. Similar performance across these alternative splits indicated that the results were stable and not solely attributable to a specific partitioning of the dataset.

Although logistic regression exhibited a slightly lower accuracy in the state hospital than in the city hospital, it still proved to be a robust and effective classification model. Its ability to handle binary dependent variables effectively and to provide easily interpretable results makes it a valuable tool for enhancing operational processes in healthcare services.

While linear regression is inherently limited for classification problems, it served as a valuable comparative tool for understanding the effects of independent variables at a basic level. The model’s linear, assumption-based structure did not fully capture the complex relationships in healthcare data, but it provided a useful starting point for examining general trends in the independent variables.

The random forest and SVM models demonstrated very high predictive performance on both hospital datasets. While these models achieved near-perfect classification results, these findings should be interpreted with caution. Given the complexity and structure of large-scale healthcare datasets, the observed high performance may be influenced by underlying data characteristics. To address this concern, additional validation strategies and strict data partitioning methods were applied to minimize the risk of overfitting and data leakage. These precautions enhance the robustness and reliability of model performance and support the validity of the findings.

An analysis of appointment cancellations revealed notable patient groups and behavioral patterns. Patients aged 18-30 and individuals scheduling non-urgent procedures had higher cancellation rates. Additionally, most appointment cancellations occurred 3 to 5 days before the scheduled date or on the same day. These findings provide valuable insights for both understanding patient behavior and proactively managing hospital workflows.

Discussion

This study evaluated the performance of multiple machine-learning models for predicting radiology appointment cancellations using large-scale patient appointment data from a city and a state hospital in Türkiye. The findings demonstrate that machine learning approaches can support the prediction of appointment cancellations and may help improve operational planning in radiology services.

Among the evaluated models, random forest and SVM demonstrated the strongest predictive performance across both hospital datasets. These findings are generally consistent with previous studies reporting that machine learning approaches can effectively predict radiology appointment no-shows and missed appointments in healthcare settings (31-36). Previous studies conducted in radiology departments have shown that predictive analytics can support scheduling optimization, improve resource utilization, and identify patients with higher cancellation risk (31, 34). Similarly, Chong et al. (32) demonstrated that artificial intelligence–based predictive approaches may improve outpatient MRI scheduling efficiency and operational planning. The present study extends the existing literature by comparing predictive model performance across two public hospital settings within the same national healthcare system.

The relatively high predictive performance observed in the present study may be associated with several factors. First, the dataset included more than three million appointment records collected over an extended period, which may have enabled the models to capture recurring behavioral and operational patterns associated with appointment cancellations. Second, the use of structured hospital information system data, including imaging procedure types and ICD-10 diagnosis categories, may have contributed to improved classification performance. Previous studies have also emphasized that appointment-related variables, procedure characteristics, and patient history are important predictors of no-show behavior (9-13,33-37). Despite the preventive validation strategies implemented in this study, retrospective healthcare datasets may contain hidden structural patterns that contribute to elevated predictive performance. Therefore, the findings should be interpreted with caution until validated in independent datasets.

To reduce the risk of overfitting and unintended data leakage, several methodological precautions were applied during model development. Patient-level train-test separation, temporal validation strategies, and repeated analyses using alternative train-test splits were implemented to improve methodological robustness. In addition, only variables available before the appointment date were included in the predictive modeling process. These precautions were intended to better simulate real-world implementation conditions and to improve the reliability of the findings.

The findings also provide clinically and operationally relevant insights regarding appointment cancellation behavior. Higher cancellation rates were observed among younger patients and individuals scheduled for non-urgent procedures. Similar findings have been reported in previous studies, suggesting that younger age groups may exhibit greater schedule variability and lower perceived urgency regarding healthcare appointments (11, 13, 35, 37). In addition, the observation that many cancellations occurred either several days before the appointment or on the same day suggests that appointment behavior may be influenced by scheduling flexibility and patient decision-making processes.

From an operational perspective, predictive models may support more efficient appointment management in radiology departments. Identifying patients with a high likelihood of cancellation could enable healthcare administrators to implement targeted reminder systems, dynamic scheduling approaches, or controlled overbooking strategies for high-risk appointment groups. Such interventions may contribute to reducing unused imaging capacity and improving workflow efficiency, particularly in high-demand healthcare environments.

The comparison between the city hospital and the state hospital also revealed differences in appointment workflow characteristics. The city hospital demonstrated shorter waiting periods and more consistent appointment completion times, whereas the state hospital exhibited greater variability in scheduling intervals. These differences may be associated with patient density, operational workload, and institutional workflow structures. Such findings suggest that hospital-specific operational characteristics should be considered when implementing predictive scheduling systems.

Despite the promising findings, several limitations should be acknowledged. First, the study was conducted using retrospective data from a single city hospital and a single state hospital, which may limit the generalizability of the findings. Second, external validation using independent datasets was not performed. Therefore, model performance should be interpreted within the context of the current dataset and healthcare settings. Third, although multiple validation strategies were applied, the possibility of dataset-specific structural regularities influencing predictive performance cannot be completely excluded. Future studies should include multicenter datasets, prospective validation approaches, and external testing procedures to further evaluate the generalizability and clinical applicability of predictive scheduling models.

Overall, the findings suggest that machine learning–based predictive approaches may support appointment management and operational planning in radiology services. However, broader validation studies are required before such systems can be generalized across different healthcare environments.

Study Limitations

This study used data from a single city hospital and a single state hospital, which may limit the generalizability of the findings. Therefore, the results should be interpreted within the context of the selected healthcare settings. However, this study can be considered an initial pilot analysis aimed at exploring the potential of machine learning models to predict radiology appointment cancellations using large-scale real-world data.

Future studies should include data from multiple hospitals and diverse healthcare settings to validate and generalize the findings. In addition, external validation using independent datasets would further strengthen the robustness and applicability of the proposed models. Despite this limitation, the large-scale nature of the dataset provides valuable insights into appointment cancellation behavior and supports the robustness of the findings.

Conclusion

This study demonstrated that machine learning models can predict radiology appointment cancellations from large-scale patient appointment data. Among the evaluated algorithms, random forest and SVM models demonstrated the strongest predictive performance in both hospital datasets.

The findings suggest that predictive modeling approaches may support appointment management and operational planning in radiology services by helping identify patients with a higher likelihood of cancellation. In particular, younger patients and individuals scheduled for non-urgent procedures exhibited higher cancellation tendencies, indicating the potential value of targeted reminder and scheduling strategies for high-risk groups.

Despite the promising findings, the results should be interpreted within the context of the current study. External validation using broader and more diverse datasets is necessary to further evaluate the generalizability and real-world applicability of these predictive models in different healthcare environments.