Baseline Characteristics of Study Population
From May 1,2006 to December 31,2019, a total of 1036 patients with 1039 breast cases (3 of whom had simultaneity bilateral breast malignancies) met the inclusion criteria. 698 (696 patients) were diagnosed by frozen sections and 341 (340 patients) by paraffin sections. Based on the above data, we calculated that the diagnostic sensitivity of frozen section was 67.18%, the FNR was 32.82%.
After removing patients with incomplete image data, 876 patients (876 cases) had complete image data and were selected for logistic regression analysis and nomogram construction, and randomly assigned to the training set and testing set in a ratio of 7:3. The characteristics of the patients are shown in Table 1.
Table 1
Baseline characteristics of the training set and the testing set
Patient Characteristic
|
No. of Patients
|
Total, n = 876
|
%
|
Training Set, n = 613
|
%
|
Testing Set, n = 263
|
%
|
Age at diagnosis
|
Median(y)
|
53(28–86)
|
|
54(30–86)
|
|
51(28–83)
|
|
༜45y
|
183
|
20.9%
|
127
|
20.7%
|
56
|
21.3%
|
45 ~ 59y
|
410
|
46.8%
|
286
|
46.6%
|
124
|
47.1%
|
≥ 60y
|
283
|
32.3%
|
200
|
32.6%
|
83
|
31.6%
|
ultrasonographic features
|
Ultrasound echo image
|
cystic-solid echo
|
82
|
9.4%
|
55
|
9.0%
|
27
|
10.3%
|
solid echo
|
764
|
87.2%
|
538
|
87.8%
|
226
|
85.9%
|
glands echo disorderly
|
9
|
1.0%
|
7
|
1.1%
|
2
|
0.7%
|
normal echo
|
21
|
2.4%
|
13
|
2.1%
|
8
|
3.0%
|
Ultrasonic maximum diameter
|
≤ 10mm
|
111
|
12.7%
|
75
|
12.2%
|
36
|
13.7%
|
༞10mm
|
765
|
87.3%
|
538
|
87.8%
|
227
|
86.3%
|
DPSE on ultrasonic image
|
Yes
|
298
|
34.0%
|
201
|
32.8%
|
97
|
36.9%
|
No
|
578
|
66.0%
|
412
|
67.2%
|
166
|
63.1%
|
US-BI-RADS
|
1 ~ 3
|
29
|
3.3%
|
18
|
2.9%
|
11
|
4.2%
|
4a or 0
|
192
|
21.9%
|
136
|
22.2%
|
56
|
21.3%
|
4b
|
344
|
39.3%
|
240
|
39.2%
|
104
|
39.5%
|
4c or 5
|
311
|
35.5%
|
219
|
35.7%
|
92
|
35.0%
|
mammographic features
|
microcalcifications on MG
|
Yes
|
420
|
47.9%
|
297
|
48.5%
|
123
|
46.8%
|
No
|
456
|
52.1%
|
316
|
51.5%
|
140
|
53.2%
|
MG-BI-RADS
|
1 ~ 3
|
71
|
8.1%
|
45
|
7.3
|
26
|
9.9%
|
4a or 0
|
269
|
30.7%
|
182
|
29.7%
|
87
|
33.1%
|
4b
|
247
|
28.2%
|
171
|
27.9%
|
76
|
28.9%
|
4c or 5
|
289
|
33.0%
|
215
|
35.1%
|
74
|
28.1%
|
physical examination symptoms
|
whether the mass could be palpable
|
Palpable
|
812
|
92.7%
|
571
|
93.1%
|
241
|
91.6%
|
unpalpable
|
64
|
7.3%
|
42
|
6.9%
|
22
|
8.4%
|
nipple discharge
|
Yes
|
96
|
69
|
11.3%
|
27
|
10.3%
|
No
|
780
|
89.0%
|
544
|
88.7%
|
236
|
89.7%
|
pathological features
|
PL-CNB
|
Yes
|
252
|
28.8%
|
174
|
28.4%
|
78
|
29.7%
|
No
|
624
|
71.2%
|
439
|
71.6%
|
185
|
70.3%
|
SA-CNB
|
Yes
|
61
|
7.0%
|
45
|
7.3%
|
16
|
6.1%
|
No
|
815
|
93.0%
|
568
|
92.7%
|
247
|
93.9%
|
Frozen section assessment
|
Failure diagnosis
|
283
|
32.3%
|
203
|
33.1%
|
80
|
30.4%
|
Correct diagnosis
|
593
|
67.7%
|
410
|
66.9%
|
183
|
69.6%
|
Abbreviation: DPSE, dense punctate strong echo; US-BI-RADS, the category of the BI-RADS on ultrasonography; MG, mammography; MG-BI-RADS, the category of the BI-RADS on mammography; PL-CNB, core needle biopsy contained papillary lesions; SA-CNB, core needle biopsy contained sclerosing adenosis. |
Logistic Regression Analysis
In the training set, of 613 patients, 205 (33.44%) were false negative. In the univariate logistic regression analysis, for patients who were sixty years and older (OR, 1.653; 95% CI, 1.029–2.686; P = 0.0396), patients who have PL-CNB (OR, 5.037; 95% CI, 3.468–7.366; P < 0.0001), or patients who have SA-CNB (OR, 2.133; 95% CI, 1.161–3.917; P = 0.014), the FNR of frozen section was higher, but lower for those that showed solid image on ultrasonography (OR, 0.286; 95% CI, 0.158–0.505; P < 0.0001), DPSE on ultrasonic image (OR, 0.205; 95% CI, 0.128–0.319; P < 0.0001), US-BI-RADS 4C-5 (OR, 0.273; 95% CI, 0.101–0.737; P = 0.0094), clustered microcalcifications on mammography (OR, 0.203; 95% CI, 0.138–0.294; P < 0.0001), and MG-BI-RADS 4C-5 (OR, 0.203; 95% CI, 0.190–0.750; P = 0.0049).
In multivariate logistic regression analysis with backward stepwise selection, US-BI-RADS 4C-5 (OR, 0.250; 95% CI, 0.081–0.777; P = 0.0157), clustered microcalcifications on mammography (OR, 0.345; 95% CI, 0.216–0.543; P < 0.0001) were associated with lower FNR, but for DPSE on ultrasonic image, the correlation is slightly weaker (OR, 0.595; 95% CI, 0.335–1.044; P = 0.0727). On the contrary, PL-CNB (OR, 4.251; 95% CI, 2.804–6.492; P < 0.0001) and SA-CNB (OR, 3.727; 95% CI, 1.897–7.376; P = 0.0001) were associated with higher FNR (show in Table 2).
Table 2
Multivariate Logistic Regression Analysis of Factors Associated with the diagnostic failure risk of frozen section in the Training Set, n = 613
Variable
|
FN/total
|
FNR
|
OR
|
95% CI
|
P-value
|
DPSE on ultrasonic image
|
|
|
|
|
|
No
|
176/412
|
42.7%
|
Reference
|
-
|
-
|
Yes
|
27/201
|
13.4%
|
0.595
|
0.335–1.044
|
0.073
|
US-BI-RADS
|
|
|
|
|
|
1–3
|
9/18
|
50.0%
|
Reference
|
-
|
-
|
0 or 4a
|
61/136
|
44.9%
|
0.601
|
0.195–1.861
|
0.372
|
4b
|
88/240
|
36.7%
|
0.434
|
0.143–1.321
|
0.137
|
4c-5
|
45/219
|
20.5%
|
0.250
|
0.081–0.777
|
0.015
|
microcalcifications on MG
|
|
|
|
|
|
No
|
155/316
|
49.1%
|
Reference
|
-
|
-
|
Yes
|
48/297
|
16.2%
|
0.345
|
0.216–0.543
|
< 0.001
|
PL-CNB
|
|
|
|
|
|
No
|
98/439
|
22.3%
|
Reference
|
-
|
-
|
Yes
|
105/174
|
60.3%
|
4.251
|
2.804–6.492
|
< 0.001
|
SA-CNB
|
|
|
|
|
|
No
|
180/568
|
31.7%
|
Reference
|
-
|
-
|
Yes
|
23/45
|
51.1%
|
3.727
|
1.897–7.376
|
< 0.001
|
Abbreviation: DPSE, dense punctate strong echo; US-BI-RADS, the category of the BI-RADS on ultrasonography; |
MG, mammography; PL-CNB, core needle biopsy contained papillary lesions; |
SA-CNB, core needle biopsy contained sclerosing adenosis; FN, false-negative; FNA, false-negative rate; OR, odds ratio. |
Nomogram Development
On the basis of results from multivariable logistic regression analysis, a nomogram was developed to predict the FNR of frozen section. In the nomogram, the total score is calculated by using clinical and pathologic features, contain BI-RADS category on ultrasonography, DPSE on ultrasonic image, clustered microcalcifications on mammography, PL-CNB, and SA-CNB. This total score can then be used to assign a probability of FN to individual patient using the scale at the bottom of Fig. 1.
Nomogram Validation
The resulting nomogram was internally validated using the bootstrap method. We use formula to determine the cutoff value of the validation: \(f\left(x\right)=\frac{nx(1-FNRx)}{N-nx(1-FNRx)}\), x represents the total score, nx represents the number of patients with this score and below, FNRx represents the actual false negative rate of patients with this score and below, and N represents the total number of patients. The best cutoff value is obtained at the peak of the formula value curve, that represents the best clinical utility. The cutoff value we set was the total score 135 points (Fig. 2), the prediction model had an AUC of 0.794 (95% CI: 0.756–0.831) in the training set, indicating that the multivariate logistic regression model had potentially promising predictive power (Fig. 3A). The model demonstrated an adequate level of accuracy for predicting the FNR of frozen section.
The independent testing set of 263 patients also showed good discriminatory ability, with an AUC of 0.800 (95% CI: 0.736–0.865), indicating that the multivariate logistic regression model in a separate, individual data set of patients had potentially promising predictive power (Fig. 3B).
The calibration was good for the training and testing cohorts and showed no significant difference between the predicted and observed probabilities of failure diagnosis (P = 1.000), indicating that the nomogram was well calibrated (Fig. 4).
On the basis of the predicted probability of FN, we calculated the practical FNR of different cutoff points in total patients (876 patients). When predicting the probabilities of patients who were more likely to be FN, the patients with practical FNR accounted for 10% and 10.16% of those who had a predicted probability of FN ≤ 10% and ≤ 15%, respectively. Among patients with a predicted probability of FN ≥ 60%, ≥ 70%, and ≥ 80%, the practical FNR accounted for 71.7%, 73.4%, and 87.5%, respectively (show in Table 3).
Table 3
The Nomogram Applied to Data Sets at Different Predicted Probability Cutoff Values
Cutoff Value
|
No. of Patients (%) |
FNR, %
|
Total score (points)
|
Predicted probability of FN
|
Total
|
False negative
|
≤ 40
|
≤ 10%
|
220(25.1)
|
22(7.8)
|
10.0%
|
≤ 76
|
≤ 15%
|
246(28.1)
|
25(8.8)
|
10.2%
|
≤ 135
|
≤ 30%
|
459(52.4)
|
52(18.4)
|
11.3%
|
> 135
|
> 30%
|
417(47.6)
|
231(81.6)
|
55.4%
|
≥ 190
|
≥ 50%
|
235(26.8)
|
152(53.7)
|
64.7%
|
≥ 220
|
≥ 60%
|
159(18.2)
|
114(40.3)
|
71.7%
|
≥ 251
|
≥ 70%
|
60(6.8)
|
44(15.5)
|
73.3%
|
≥ 288
|
≥ 80%
|
8(0.9)
|
7(2.5)
|
87.5%
|
Abbreviation: FN, false negative; FNR, false negative rate |
These results demonstrated that the individual probability of FN of frozen section could be predicted accurately by combining information from routinely available clinicopathologic variables.