Building a Prediction Model for Radiographically Confirmed Pneumonia in Peruvian Children

Background Community-acquired pneumonia remains the leading cause of death in children worldwide, and current diagnostic guidelines in resource-poor settings are neither sensitive nor specific. We sought to determine the ability to correctly diagnose radiographically confirmed clinical pneumonia when diagnostics tools were added to clinical signs and symptoms in a cohort of children with acute respiratory illnesses in Peru. Methods Children < 5 years of age with an acute respiratory illness presenting to a tertiary hospital in Lima, Peru, were enrolled. The ability to predict radiographically confirmed clinical pneumonia was assessed using logistic regression under four additive scenarios: clinical signs and symptoms only, addition of lung auscultation, addition of oxyhemoglobin saturation (Spo2), and addition of lung ultrasound. Results Of 832 children (mean age, 21.3 months; 59% boys), 453 (54.6%) had clinical pneumonia and 221 (26.6%) were radiographically confirmed. Children with radiographically confirmed clinical pneumonia had lower average Spo2 than those without (95.9% vs 96.6%, respectively; P < .01). The ability to correctly identify radiographically confirmed clinical pneumonia using clinical signs and symptoms was limited (area under the curve [AUC] = 0.62; 95% CI, 0.58-0.67) with a sensitivity of 66% (95% CI, 59%-73%) and specificity of 53% (95% CI, 49%-57%). The addition of lung auscultation improved classification (AUC = 0.73; 95% CI, 0.69-0.77) with a sensitivity of 75% (95% CI, 69%-81%) and specificity of 53% (95% CI, 49%-57%) for the presence of crackles. In contrast, the addition of Spo2 did not improve classification (AUC = 0.73; 95% CI, 0.69-0.77) with a sensitivity of 40% (95% CI, 33%-47%) and specificity of 72% (95% CI, 68%-75%) for an Spo2 ≤ 92%. Adding consolidation on lung ultrasound was associated with the largest improvement in classification (AUC = 0.85; 95% CI, 0.82-0.89) with a sensitivity of 55% (95% CI, 48%-63%) and specificity of 95% (95% CI, 93%-97%). Conclusions The addition of lung ultrasound and auscultation to clinical signs and symptoms improved the ability to correctly classify radiographically confirmed clinical pneumonia. Implementation of auscultation- and ultrasound-based diagnostic tools can be considered to improve diagnostic yield of pneumonia in resource-poor settings.

Pneumonia remains the most common infectious cause of morbidity and mortality in young children worldwide. 1 Most of the 1.1 million deaths in children < 2 years of age occur in resource-limited settings. 2 Early and accurate diagnosis of bacterial pneumonia presents a major challenge toward successful treatment. Current international guidelines rely on clinical presentation and physical examination, with imaging used in ambiguous or severe cases. 3,4 A lack of trained physicians and access to diagnostic tools, such as laboratory tests and imaging, make it difficult to follow international guidelines in resource-limited settings.
The World Health Organization (WHO) developed a pneumonia case management algorithm for resourcelimited settings, allowing diagnosis based on symptoms and clinical signs. 5 This algorithm has been shown to have low diagnostic specificity. [6][7][8] Furthermore, no individual clinical features, including those in the WHO case management algorithm, were sufficient to reliably predict radiographically confirmed pneumonia. 9 Although chest radiography (CXR) is a standard diagnostic tool for the identification of pneumonia, it has poor validity. 10 Without a standardized training approach, such as the WHO CXR methodology, CXR also has high interobserver variability, and current clinical guidelines do not require a CXR for the diagnosis of pneumonia. 4 There is evidence that lung auscultation and pulse oximetry improve the ability to correctly identify pneumonia 11,12 ; however, a recent prospective study had mixed results with pulse oximetry improving diagnosis but not auscultation. 13 Lung ultrasound (LUS) has been shown to have good sensitivity and specificity compared with CXR. [14][15][16][17] Previous studies have attempted to develop predictive models in children with suspicion of pneumonia. [18][19][20][21][22] Current algorithms have not been reliable and have been limited by small samples or the exclusion of common pediatric respiratory diseases such asthma or bronchiolitis. 21,22 We sought to assess the diagnostic value of clinical prediction models based on lung auscultation, pulse oximetry, and LUS to identify radiographically confirmed clinical pneumonia in Peruvian children < 5 years of age. This assessment may elucidate the value of implementing these clinical tools where CXR may not be available or appropriate in the diagnosis of pneumonia.

Study Design
We consecutively enrolled children aged 2 to 59 months presenting to the ED, inpatient wards, and outpatient clinics with an acute respiratory illness at the Instituto Nacional de Salud del Niño in Lima, Peru, between January 2012 and September 2013. 23 The Instituto Nacional de Salud del Niño is the largest freestanding pediatric hospital in Lima. It is a public government-run hospital (www.insn.gob.pe) serving predominantly low-income populations, and it is also a national referral center. We excluded children with a history of significant heart disease or chronic respiratory disease other than asthma and children who required invasive airway management. We also recruited 230 children without any acute illness, 23 but we limited the use of their data to oxyhemoglobin saturation (SpO 2 ) values in this analysis. This cohort of children was used in a previous LUS validation study, 16 and study protocol was published elsewhere. 23 The study was approved by the institutional review board committees of the Instituto Nacional de Salud del Niño (Lima, Peru) (No. CL-4311), A.B. PRISMA (Lima, Peru) (No. CE1457.11), and the Johns Hopkins School of Medicine (Baltimore, MD) (No. 64148). Written informed consent was obtained from a parent or guardian prior to enrollment into the study.

Data Collection
Child participants who met inclusion criteria underwent a standard clinical assessment, after consent was obtained from their parents, for signs and symptoms, including lung auscultation, pulse oximetry, and imaging. Clinical assessment, including auscultation, was conducted by the treating pediatrician. A study team member recorded the clinical by the Doris Duke Charitable Foundation Clinical Research Fellowship.
findings, including auscultation findings, vital signs, presenting history, and SpO 2 . 23 All children underwent LUS and had an anterior-posterior CXR taken. SpO 2 was assessed using pediatric probes on either Rad 5v pulse oximeters (Masimo Corp) or, in few instances, the peripheral pulse oximeters available at Instituto Nacional de Salud del Niño. Lung auscultation was performed on the anterior and posterior zones of the thorax, with the patient supine or upright, as previously described. 23 Pediatricians conducting auscultation were asked to report the presence of the following findings: crackles, wheeze, decreased breath sounds, or bronchial breath sounds. Ausculatory findings were obtained with acoustic stethoscopes.

Definitions
We used WHO growth standards to define wasting (weight-for-height z score < À2 SD), stunting (height-for-age z score < À2 SD), and severe malnutrition (weight-for-height z score of # À3 SD). 24 We used age-specific respiratory rate cutoffs to define tachypnea: $ 50 breaths/min for children 2 to 11 months of age and $ 40 breaths/ min for children 12 to 59 months of age. 4 Tachycardia was defined as $ 190 beats/min for children 2 to 11 months of age and $ 140 beats/min for children 12 to 59 months of age. 4 Pulse oximetry was included as a continuous variable, but we conducted sensitivity analyses with SpO 2 cutoffs of # 92% and # 95%.

Pneumonia
The definitions of clinical pneumonia, asthma, bronchiolitis, or an upper respiratory tract infection were based on standard of care, using patient history and physical examination, including SpO 2 and CXR results. A clinical diagnosis was made by the treating pediatrician. Children had WHO-defined pneumonia if they had an acute presentation of either cough or difficulty breathing and also had either lower chest wall indrawing or age-specific tachypnea. 4 Severe pneumonia or very severe disease was defined as WHO pneumonia with at least one of the following danger signs: persistent vomiting, convulsions, lethargy, no oral intake, stridor, or severe malnutrition. 25 Severe clinical pneumonia was defined as a clinical diagnosis of pneumonia, by the treating pediatrician, and the presence of at least one of the danger signs previously listed. 25 LUS Study children received a complete LUS using a MicroMaxx portable ultrasound machine (Sonosite/FujiFilm) with an HFL38/13-6 MHz linear transducer. LUS assessment was conducted by one of three trained general practitioners following a standardized protocol developed using international recommendations. 16,26 Interpretation and conduct of LUS were performed independent of clinical evaluation and CXR findings. 16 We defined pneumonia on LUS as the presence of a hypoechoic area consistent with a consolidation and occupying of more than one intercostal space in longitudinal view, or a smaller consolidation with a pleural effusion, and interstitial abnormalities was defined as three or more B lines within a single acoustic window. We required agreement by two of three ultrasound readers for a final LUS diagnosis. 16

Radiographic Pneumonia
We obtained anteroposterior CXR on all children with an acute respiratory illness. Radiographic pneumonia was defined as the presence of a lobar consolidation with or without pleural effusion. 27 All chest radiographs were reviewed by two members of a team of three expert pediatric radiologists blinded to clinical information and results from LUS. 16 Radiographic diagnosis was made as a consensus of the team using a standardized protocol, as previously described. 16

Biostatistical Methods
Our primary objective was to assess the ability of different diagnostic algorithms to correctly classify children diagnosed with clinical pneumonia that is corroborated by the finding of a lobar consolidation on CXR. As such, we compared each additive clinical scenario against radiographically confirmed clinical pneumonia. We evaluated the following four additive scenarios: WHO-defined pneumonia, 5 addition of lung auscultation findings, addition of SpO 2 by pulse oximetry, followed by the addition of LUS findings. We used multivariable logistic regression to model the presence of radiographically confirmed clinical pneumonia as a function of the four additive scenarios, adjusted for malnutrition (both wasting and stunting), having tachycardia, and a having previous history of pneumonia.
We used logistic regression to calculate a concordance statistic (C statistic), which is statistically equivalent to the area under the curve (AUC). 28 Models with a higher AUC did better at identifying radiographically confirmed clinical pneumonia. Analyses were performed using STATA version 13 (Stata Corp) and R (The R Foundation).

Participant Characteristics
There were 832 children recruited to the study and who underwent diagnostic imaging for pneumonia. Two children were missing clinical data (< 1%) and were excluded from the analysis. We summarized participant characteristics in Table 1. Mean participant age was 21.3 months, 59% of which were boys, 8% were wasted, and 17% were stunted. The study primarily included an inner city population that is low to middle income. We summarized socioeconomic status in Table 1. Overall, final clinical diagnoses, as reported by the treating pediatrician, were as follows: 453 (55%) had clinical pneumonia, 133 (16%) had asthma, 103 (12%) had bronchiolitis, and 143 (17%) had an upper respiratory infection. Radiologists identified 221 consolidations (27%) and 264 interstitial opacities (32%) on chest radiographs in children. A total of 191 children (23%) met criteria for radiographically confirmed clinical pneumonia and 429 children (51.6%) met criteria for WHO-defined pneumonia.

Distribution of SpO 2 by Pneumonia Status
We plotted the distribution of SpO 2 values by categories of acute respiratory illness, ranging from none to having severe pneumonia, and stratified by CXR findings (Fig 1). Mean SpO 2 was lowest in children with a clinical diagnosis of pneumonia, followed by children with either asthma, bronchiolitis, or an upper respiratory tract infection. It was highest among children without an acute illness. Overall, mean SpO 2 was lower in children with clinical pneumonia than in those who did not have clinical pneumonia (95.9% vs 97.1%, respectively; P < .001). There was no difference in SpO 2 values chestjournal.org

Classification of Radiographically Confirmed Pneumonia
We summarized the diagnostic validity for each individual clinical tool ( Table 2) and plotted AUCs (Fig 2) and corresponding receiver operating characteristic curves (Fig 3) for the different diagnostic tools based on the described four additive scenarios to classify pneumonia when using radiographically confirmed clinical pneumonia. WHO-defined pneumonia had a 66% sensitivity and 53% specificity for correctly identifying radiographically confirmed clinical pneumonia. The presence of cough or shortness of breath with intercostal indrawing or age-specific tachypnea, crackles on auscultation, and decreased breath sounds on auscultation had sensitivities > 60%, whereas decreased breath sounds on auscultation, SpO 2 # 92%, and consolidation on LUS had specificities > 90% (Table 2). Consolidation on LUS had the highest positive likelihood ratio for radiographically confirmed clinical pneumonia, whereas consolidation on LUS and presence of crackles had the lowest negative likelihood ratio. As noted in Figure 2, the use of WHO-defined pneumonia was limited in its ability to classify radiographically confirmed clinical pneumonia. The addition of lung auscultation improved the classification of radiographically confirmed clinical pneumonia, with decreased breath sounds, presence of crackles, and absence of wheezes independently associated (Fig 4).
LUS contributed to the largest improvement in the classification of radiographically confirmed clinical pneumonia. When LUS alone, without auscultation and SpO 2 , was added to WHO-defined pneumonia, it improved the classification of radiographically confirmed clinical pneumonia (AUC ¼ 0.82; 95% CI, 0.78-0.85). Consolidation on LUS was associated with radiographically confirmed clinical pneumonia (Fig 4). Finding interstitial abnormalities on LUS was indicative of not having radiographically confirmed clinical pneumonia. The addition of SpO 2 and LUS, without lung auscultation, improved classification of radiographically confirmed pneumonia beyond clinical signs and symptoms (AUC ¼ 0.82; 95% CI, 0.79-0.86). However, the model that included lung auscultation, with SpO 2 and LUS, had better discrimination (AUC ¼ 0.85 vs AUC ¼ 0.82).

Subgroup Analyses
In children 12 to 59 months of age, the use of WHOdefined pneumonia resulted in poor classification of radiographically confirmed clinical pneumonia (Fig 2). The addition of lung auscultation improved classification (Fig 2), with crackles, decreased breath sounds, and absence of wheezes independently associated with radiographically confirmed clinical pneumonia (Fig 4). The addition of pulse oximetry to identify hypoxemia did not improve classification (Fig 2) and was not independently associated with radiographically confirmed clinical pneumonia (Fig 4). LUS contributed the largest improvement in the classification of radiographically confirmed clinical pneumonia.
In infants 2 to 11 months of age, the use of WHOdefined pneumonia also resulted in poor classification (Fig 2). The addition of lung auscultation improved classification, albeit less so than in children 12 to 59 months of age (Fig 2). When the contributions of lung auscultation were also assessed independently, we found that the presence of crackles and absence of wheezes were not associated with radiographically confirmed clinical pneumonia, whereas decreased breath sounds were associated with radiographically confirmed clinical pneumonia (Fig 4). The addition of pulse oximetry resulted in an improvement in the classification of radiographically confirmed clinical pneumonia (Fig 2). A lower SpO 2 was associated with a higher odds of having radiographically confirmed clinical pneumonia (Fig 4). Finally, LUS again was associated with the largest improvement in the classification of radiographically confirmed clinical pneumonia (Fig 2).

Discussion
We found that the WHO definition of pneumonia based on clinical symptoms and signs alone had poor discrimination for radiographically confirmed clinical pneumonia among Peruvian children who presented with an acute respiratory illness. Although children with pneumonia had lower SpO 2 , the use of pulse oximetry to identify hypoxemia did not add value, above WHO-defined pneumonia, to the classification of radiographically confirmed clinical pneumonia in the overall population. In contrast, both the use of auscultation or LUS improved the classification of children with radiographically confirmed clinical pneumonia.
Lung auscultation remains an important component of pneumonia diagnosis with more predictive accuracy than an initial clinical assessment alone. In the study population, the presence of crackles, decreased breath sounds, and absence of wheezes were all important predictors for radiographically confirmed clinical pneumonia. This is consistent with other studies where auscultation is a useful predictor of radiographically confirmed pneumonia. 29 Presence of physicians, trained personnel, or even a device 30 that identifies lung sounds may be critical for increasing the accuracy of a clinical algorithm for the diagnosis of pneumonia in resourcelimited settings.  We plotted AUCs and corresponding 95% CIs derived from multivariable logistic regression models for each additive clinical scenario (described in y axis), for all children in the sample (first panel) and stratified by age group: 2 to 11 mo of age (middle panel) and 12 to 59 mo of age (bottom panel). We also present numerical AUCs and 95% CIs for each row. LUS ¼ lung ultrasound; SpO 2 ¼ oxyhemoglobin saturation; WHO ¼ World Health Organization.

chestjournal.org
The use of pulse oximetry to identify pneumonia is supported by studies showing that a low SpO 2 may help to identify more cases of pneumonia than a clinical approach alone. 12,31 Our findings in Peru demonstrated that although SpO 2 was significantly lower in children with pneumonia when compared with those who did not have pneumonia, it did not add value to other diagnostic tools to identify radiographically confirmed clinical pneumonia. One possibility that may explain our findings is that our study was conducted in a tertiary referral hospital, where physicians were able to more easily recognize hypoxemia without having to use a pulse oximeter. The sample size may also limit our ability to provide adequate inferences because only 73 children (9%) had an SpO 2 # 92%. Our analysis suggests that SpO 2 may aid in the diagnosis of pneumonia when assessing infants 2 to 11 months of age, possibly because clinical signs of hypoxemia are more difficult to ascertain in this age group.  Figure 4 -Forest plot of the odds of having radiographically confirmed clinical pneumonia using four types of clinical tools. We plotted the adjusted OR of having radiographically confirmed clinical pneumonia for each four clinical tools in an additive scenario using overall study sample, and then stratified by age groups (2-11 and 12-59 mo of age). Adjusted ORs are represented with diamonds and 95% CIs are represented by horizontal lines. We show the additive scenarios on the y axis. Four sets of logistic regression models were built. The first model included a composite for variables to WHO pneumonia, and adjusted for confounders (medical history of pneumonia, age-specific tachycardia, and weight-for-height and height-for-age z scores). The second model included WHO pneumonia, three auscultatory variables (absence of wheezes, presence of crackles, and decreased breath sounds), and vide supra confounders. The third model was a set of models that included WHO pneumonia, the three auscultatory variables, SpO 2 expressed three different ways, and vide supra confounders. Specifically, we ran three independent models with SpO 2 as a continuous variable, and SpO 2 with the thresholds of # 95% and # 92%. The fourth model included WHO pneumonia, the three auscultatory variables, continuous SpO 2 , two lung ultrasound variables (interstitial abnormalities or consolidation), and vide supra confounders. Adjusted ORs and 95% CIs are also presented numerically for each row. See Figure 2 legend for expansion of abbreviations.
Finally, consolidation on LUS, when added to a clinical model, pulse oximetry, and lung auscultation, had the strongest prediction values for radiographically confirmed clinical pneumonia. Children with interstitial opacities alone on LUS were less likely to have radiographically confirmed clinical pneumonia. LUS has recently been proposed as an alternative to CXR because of its high accuracy to diagnose pneumonia in both adults 14 and children, 15 with pediatric studies confirming these findings in low-resource communities. 16 In addition, studies have shown that LUS is a safe alternative to CXR in children with suspected pneumonia. 32,33 Our data support these safety and efficacy trials in the use of LUS as a good predictor of radiographically confirmed clinical pneumonia. Moreover, LUS could be a substitute for CXR in settings that do not have the capability or resources to manage a CXR system.
Our study has several strengths. First, we obtained data on clinical signs and symptoms, auscultation, SpO 2 , and imaging in a large number of children with acute respiratory symptoms. Second, LUS was interpreted by practitioners blinded to clinical or CXR information to avoid potential biases in the interpretation of LUS images. Finally, this study included a variety of acute lower respiratory conditions that could be confused with pneumonia.
Our study also has some potential shortcomings. First, the study population was mostly derived from a tertiary referral center. Although children were recruited from outpatient clinics and not all children were referred for respiratory illness, the generalizability of our findings may be limited to children seeking care at tertiary medical centers. Second, we excluded children with chronic lung disease other than asthma and congenital cardiac diseases from the study, further limiting generalizability. Third, we only conducted longitudinal scans when performing LUS. It is possible that the addition of transverse scanning would have resulted in higher diagnostic performance for pneumonia. 34 Finally, a gold standard for the diagnosis of pediatric pneumonia is not well defined, and we acknowledge that the interobserver variability in the interpretation of CXR, especially on absence of clinical findings, is high.

Conclusions
Different algorithms, including use of signs and symptoms, laboratory data, and imaging, have been proposed to better diagnose pneumonia in children. Still, there is no consensus of which predictors have the highest yield for discrimination of radiographically confirmed clinical pneumonia, and results may vary depending on population, age, and setting. Our analysis found that lung auscultation and LUS may improve diagnosis of pediatric pneumonia, beyond clinical signs and symptoms. The next steps should be validation studies to assessing utility, ease of use, and feasibility of auscultation and LUS tools in resource-limited settings, and their impact on clinical outcomes.

Acknowledgments
Author contributions: W. C. had ultimate oversight over study conduct, analysis, and interpretation of results. F. P. contributed to design of manuscript, conducted the data analysis and interpretation, and was responsible for writing the manuscript. M. A. C. was responsible for supervision of data gathering and ultrasound, participated in sonographic grading, contributed equally to interpretation of findings, and contributed to writing of the manuscript. L. E. E. contributed to the study design, was responsible for the conduct of the study, was responsible for supervision of data gathering and ultrasound, participated in sonographic grading, contributed to the analysis and interpretation, and contributed to writing of the manuscript. M. G. contributed to data analysis and interpretation and contributed to writing of the manuscript. R. H. G. contributed to study design, contributed to analysis and interpretation, and contributed to writing of the manuscript. C. H. M. contributed to data interpretation and contributed to writing of the manuscript. D. F.-Q. was responsible for the conduct of the study and contributed to writing of the manuscript. P. C.-C. was responsible for radiographic analysis for chest radiographs and contributed to writing of the manuscript. J. M.-C. was responsible for supervision of data gathering and ultrasound, participated in sonographic grading, and contributed to writing of the manuscript. E. D. M. contributed equally to data interpretation and contributed to writing of the manuscript. W. C. conceived the study design, contributed to analysis and interpretation, and was responsible for writing of the manuscript.

Role of sponsors:
The sponsor had no role in the design of the study, the collection and analysis of the data, or the preparation of the manuscript.