Health-related physical fitness (HRPF) has demonstrated high clinical relevance, and its level is associated with the ability to perform activities of daily living with vigor and with lower risk of chronic diseases. Measuring and tracking HRPF often requires specialized equipment and personnel, which are expensive and less applicable for the general population. Wearables may mitigate this issue by providing useful estimates of the HRPF. The aim of this study was to estimate the fitness level for all HRFP domains using smartwatch data. Remote data were obtained from 120 participants (n=98, 22 drop-outs), which wore the smartwatch for a period of 30 days and were divided into three groups: healthy control (n=18), athletes (n=12), and participants with chronic disease (n=46). The fitness level for each HRPF domain was obtained in laboratory following the American College of Sports Medicine (ACSM) guidelines to obtain ground-truth values used to fit multivariable regression models for the cardiorespiratory endurance, body composition, muscular strength, muscular endurance, and flexibility domains. This study demonstrated the feasibility of fitness level prediction for all HRPF domains from the smartwatch, and that normalized scores from these predictions can be as effective as scores from ground-truth measurements from specialized clinical equipment for distinguishing athletes, healthy and diseased participants.