When conductors differ from their mirror image, their chirality is reflected in the electron states and their dynamics. Conventionally, mirror symmetries are broken by the atomic structure of crystals and imprint their low symmetry onto the itinerant electron states. Unusual transport properties are a hallmark of chiral crystals, such as a non-linear electric response known as electronic magneto-chiral ansiotropy (eMChA). Here we report such a transport signature in the centro-symmetric layered Kagome metal CsV3Sb5, observed via second harmonic generation under magnetic field. We observe the signal in a charge-ordered phase of the material, which has been proposed to be a loop-current phase with spontaneously broken mirror symmetries. While charge ordering sets in at TCDW ≈ 94 K, the eMChA transport becomes significant only at temperatures below T'≈ 34 K, indicative of further evolutions of this ordered phase upon lowering the temperature. Crucially, we demonstrate that the nonlinear magnetotransport can be switched by small out-of-plane fields, in accordance with recent tunneling microscopy experiments. Unlike structurally chiral materials, our results demonstrate how the electronic chirality of a metal can be tuned and even abruptly switched by small changes in magnetic fields alone -- a prerequisite for their applications in chiral electronics.