Primate cerebral cortex is highly convoluted with much of the cortical surface buried in sulcal folds. The origins of cortical folding and its functional relevance has been a major focus of systems and cognitive neuroscience. Stereotyped patterns of cortical folding across individuals and multiple primate species indicate common evolutionary pressures in their development. However, foundational questions regarding organizing principles shared across species remain unanswered. Taking a cross-species comparative approach with a careful consideration of historical observations, we investigate cortical folding within the calcarine sulcus, a primary fold in primates. We identify two macroanatomical structures – the retrocalcarine and external calcarine sulci – in 24 humans and 6 macaque monkeys. We show that within species, these sulci are identifiable in all individuals, fall on a similar part of the V1 retinotopic map, and thus, serve as anatomical landmarks predictive of functional organization. Yet, across species, the actual underlying visual field representations differ strikingly across humans and macaques. Thus, the structure-function correspondence for an evolutionarily old structure like V1 is species-specific and suggests intriguing differences in developmental constraints across primates.