Healthcare has been one of the most affected sectors during the coronavirus disease 2019 (COVID-19) pandemic. The utilization of related services for non-COVID-19 diseases fell dramatically following the point at which the virus broke out; however, little is known about whether this observed decline in healthcare use was due to voluntary behaviors or enforced measures. This paper quantifies the spontaneous change in healthcare utilization during and after the pandemic. We utilize a county-by-week-level dataset from Taiwan's National Health Insurance (NHI) record, covering the entire Taiwanese population, and a difference-in-differences design. Our results indicate that even if there were no human mobility restrictions or supply-side constraints, people voluntarily reduced their demand for healthcare, due to fears of contagion, or COVID-related precautionary behaviors. We find that the number of outpatient visits (inpatient admissions) decreased by 21% (11%) during the pandemic period (February to May 2020). Furthermore, the demand response of healthcare for infectious diseases (e.g., flu) was much greater and more persistent than for non-infectious diseases, thereby suggesting that the substantial decline in accessing healthcare was induced by positive public health externality of prevention measures for COVID-19. Finally, we find that the demand for healthcare services did not get back to the pre-pandemic baseline, even when there were no local coronavirus cases for 253 consecutive days (mid-April to December 2020) in Taiwan.