Effectively using unsupervised machine learning in next generation astronomical surveys

I. Reis*, M. Rotman, Dovi Poznanski, J. X. Prochaska, L. Wolf

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


In recent years many works have shown that unsupervised Machine Learning (ML) can help detect unusual objects and uncover trends in large astronomical datasets, but a few challenges remain. We show here, for example, that different methods, or even small variations of the same method, can produce significantly different outcomes. While intuitively somewhat surprising, this can naturally occur when applying unsupervised ML to highly dimensional data, where there can be many reasonable yet different answers to the same question. In such a case the outcome of any single unsupervised ML method should be considered a sample from a conceivably wide range of possibilities. We therefore suggest an approach that eschews finding an optimal outcome, instead facilitating the production and examination of many valid ones. This can be achieved by incorporating unsupervised ML into data visualization portals. We present here such a portal that we are developing, applied to the sample of SDSS spectra of galaxies. The main feature of the portal is interactive 2D maps of the data. Different maps are constructed by applying dimensionality reduction to different subspaces of the data, so that each map contains different information that in turn gives a different perspective on the data. The interactive maps are intuitive to use, and we demonstrate how peculiar objects and trends can be detected by means of a few button clicks. We believe that including tools in this spirit in next generation astronomical surveys will be important for making unexpected discoveries, either by professional astronomers or by citizen scientists, and will generally enable the benefits of visual inspection even when dealing with very complex and extensive datasets. Our portal is available online at galaxyportal.space.

Original languageEnglish
Article number100437
JournalAstronomy and Computing
StatePublished - Jan 2021


FundersFunder number
Applied Artificial Intelligence Initiative by UC Santa Cruz
Instituto de Astrof?sica de Canarias
Israeli Science Foundation
Kavli Institute for the Physics and Mathematics of the Universe
Leibniz Institut f?r Astrophysik Potsdam
MPIA Heidelberg
Max-Planck-Institut f?r Astronomie
Max-Planck-Institut f?r Astrophysik
Max-Planck-Institut f?r Extraterrestrische Physik?
National Astronomical Observatories of China
SciPy2013, 2000
U.S. Department of Energy Office of Science
University of Wisconsin
U.S. Department of Energy
National Aeronautics and Space Administration
Alfred P. Sloan Foundation
Yale University
United States - Israel Binational Science Foundation
Lawrence Berkeley National Laboratory
Vanderbilt University
Ohio State University
University of Utah
University of Washington
Johns Hopkins University
Carnegie Mellon University
University of Notre Dame
Pennsylvania State University
University of Virginia
University of Portsmouth
New Mexico State University
Maritime and Port Authority of Singapore
United States-Israel Binational Science Foundation541/17
Australian Institute of Physics
Israel Science Foundation
University of Tokyo


    Dive into the research topics of 'Effectively using unsupervised machine learning in next generation astronomical surveys'. Together they form a unique fingerprint.

    Cite this