Spatial light modulators have become an essential tool for advanced microscopy enabling breakthroughs in 3D, phase, or super-resolution imaging. However, continuous spatial-light modulation without diffraction artifacts, polarization dependence, and able to capture sub-ms microscopic motion is challenging. Here we present a photothermal spatial light modulator (PT-SLM) enabling the fast wavefront shaping free of diffraction artifacts, having a high transmissivity and modulation efficiency independent of light polarization. It is based on the microscopic heating of a thin layer of thermo-optic material confined between the photothermal heat-source and a transparent heatsink. We achieve a phase-shift > π with a response time as short as 70 µs with a theoretical limit in the sub-µs range. The combination of the PT-SLM with an interferometric scattering microscope (iSCAT) allowed us to perform quantitative phase imaging of sub-diffractional scatterers and decipher the 3D nanoscopic displacement of microtubules matching closely with control data from atomic force microscopy.