Microbiome sequencing data are very complex. In order to simplify analyses, researchers often perform unsupervised clustering to identify naturally occurring clusters and then investigate the clusters’ associations with various characteristics of interest. However, clustering performance and related conclusions can vary depending on the algorithm or beta diversity metric used. To improve microbiome analysis methods, a new study tested the performance of several metrics on four datasets with well-separated groups and a clinical dataset with less-clear group separation. None of the metrics was universally superior, but certain metrics underperformed under certain conditions. For example, the Bray-Curtis metric performed poorly in a dataset with rare high-abundance OTUs (groups of related bacteria), while the unweighted UniFrac metric performed poorly in a dataset with prevalent low-abundance OTUs. Tweaking the properties of these datasets improved these two metrics’ performance, which inspired the researchers to create a new combined metric. The novel metric performed well in all datasets, including in comparison with another combined metric, the generalized UniFrac distance. Although tests on more datasets are needed, the results emphasize the lack of a one-size-fits-all metric for microbiome studies while providing a new, high-performance metric combining the strengths of two popular clustering metrics.