Multiple maxima of likelihood in phylogenetic trees: An analytic approach

Maximum likelihood (ML) is a widely used criterion for selecting optimal evolutionary trees. However, little is known on the nature of the likelihood surface for trees, especially as to the frequency of multiple optima. We initiate an analytic study for identifying sequences that generate multiple optima. We report a new approach to calculating ML directly, which we have used to find large families of sequences that have multiple optima, including sequences with a continuum of optimal points. Such datasets are best supported by different (two or more) phylogenies that vary significantly in their timings of evolutionary events. Some standard biological processes can lead to data with multiple optima and consequently the field needs further investigation. Our results imply that hill climbing techniques, as currently implemented in various software packages, cannot guarantee to find the global ML point, even if it is unique.