Chest X-rays are playing an important role in the testing and diagnosis of COVID-19 disease in the recent pandemic. However, due to the limited amount of labelled medical images, automated classification of these images for positive and negative cases remains the biggest challenge in their reliable use in diagnosis and disease progression. We applied and implemented a transfer learning pipeline for classifying COVID-19 chest X-ray images from two publicly available chest X-ray datasets {https://github.com/ieee8023/covid-chestxray-dataset},{https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia}}. The classifier effectively distinguishes inflammation in lungs due to COVID-19 and pneumonia (viral and bacterial) from the ones with no infection (normal). We have used multiple pre-trained convolutional backbones as the feature extractor and achieved an overall detection accuracy of 91.2% , 95.3%, 96.7% for the VGG16, ResNet50 and EfficientNetB0 backbones respectively. Additionally, we trained a generative adversarial framework (a cycleGAN) to generate and augment the minority COVID-19 class in our approach. For visual explanations and interpretation purposes, we visualized the regions of input that are important for predictions and a gradient class activation mapping (Grad-CAM) technique is used in the pipeline to produce a coarse localization map of the highlighted regions in the image. This activation map can be used to monitor affected lung regions during disease progression and severity stages.