The extraordinary variation in male guppy coloration has been a powerful model for studying the interplay of natural and sexual selection. Many guppy populations exhibit substantial Y-linkage of color traits, and this has hampered the identification of the genetic architecture underlying male guppy color, as well as clouded our understanding of how this exceptional level of diversity is maintained. Here we used a population with low levels of Y-linkage for color variation to identify the heritability and genetic basis of male color variation using convolutional neural networks for high-resolution phenotyping coupled with selection experiments and controlled pedigrees and whole-genome resequencing for Genome Wide Association Study (GWAS) of colour. Our phenotypic and genomic results converge to show that color patterning in guppies is a combination of many heritable features, each with a partially overlapping genetic architecture. Unusually, our GWAS results suggest that copy number variation (CNV) is responsible for much of the variation in colour in guppies, providing a potential mechanism for the maintenance of variation of this classic model trait.