Physics-based molecular representations are the cornerstone of all modern machine learning techniques applied to solve chemical problems. While several approaches exist to design ever more accurate fingerprints, the majority resolves in including more physics to construct larger and more complex representations. Here, we present an alternative approach to harness the complexity of chemical information into a lightweight numerical form, naturally invariant under real-space transformations, and seamlessly including the information about the charge state of a molecule. The Spectrum of Approximated Hamiltonian Matrices (SPAHM) leverages the information contained in widely-used and readily-evaluated ``guess'' Hamiltonians to form a robust fingerprint for quantum machine learning. Relying on the origin of the SPAHM fingerprints and a hierarchy of approximate Hamiltonians, we analyze the relative merits of adding physics into molecular representations and find that alternative strategies, focusing more on the machine learning task, represent a clear route towards direct improvements.