Nondestructive viscoelastic characterization of soft elastomers and biological materials is crucial to the development of functional elastomeric devices or understanding physiological responses of biomaterials to mechanical stimuli. Optical coherence elastography (OCE) methods based on stress waves have been demonstrated to be capable of measuring viscoelastic properties of soft materials at several kHz. However, most studies rely on single relaxation rheological models to account for viscoelasticity, which may not accurately describe material behaviors over a large frequency range. This work aimed at developing an optical coherence elastography (OCE) based method to quantify shear storage and loss moduli of soft membrane materials without assuming rheological models. The guided elastic wave displacement field in a silicone elastomer (Ecoflex™ 00-10) membrane subjected to monochromatic excitation from 1 kHz to 5 kHz was measured by the OCE method. A guided-wave finite element model was fitted to the frequency-dependent elastic wave speeds and amplitude decay coefficients to estimate frequency-dependent storage and loss shear moduli. The estimated shear storage and loss moduli of the sample were 67-95 kPa and 7.5-25 kPa in the frequency range of 1-5 kHz. This work demonstrated that the proposed OCE method is ideal for nondestructive viscoelastic characterization at frequencies between the operational range of conventional rheometers and high frequency (MHz) ultrasound methods.