FEDEX: An Explainability Framework for Data Exploration Steps

Daniel Deutch, Amir Gilad, Tova Milo, Amit Mualem, Amit Somech

Research output: Contribution to journalConference articlepeer-review

Abstract

When exploring a new dataset, Data Scientists often apply analysis queries, look for insights in the resulting dataframe, and repeat to apply further queries. We propose in this paper a novel solution that assists data scientists in this laborious process. In a nutshell, our solution pinpoints the most interesting (sets of ) rows in each obtained dataframe. Uniquely, our definition of interest is based on the contribution of each row to the interestingness of different columns of the entire dataframe, which, in turn, is defined using standard measures such as diversity and exceptionality. Intuitively, interesting rows are ones that explain why (some column of ) the analysis query result is interesting as a whole. Rows are correlated in their contribution and so the interesting score for a set of rows may not be directly computed based on that of individual rows. We address the resulting computational challenge by restricting attention to semantically-related sets, based on multiple notions of semantic relatedness; these sets serve as more informative explanations. Our experimental study across multiple real-world datasets shows the usefulness of our system in various scenarios.

Original languageEnglish
Pages (from-to)3854-3868
Number of pages15
JournalProceedings of the VLDB Endowment
Volume15
Issue number13
DOIs
StatePublished - 2022
Event48th International Conference on Very Large Data Bases, VLDB 2022 - Sydney, Australia
Duration: 5 Sep 20229 Sep 2022

Funding

FundersFunder number
National Science FoundationIIS-2008107, IIS-1703431, IIS-1552538
Horizon 2020 Framework Programme804302
European Research Council
United States-Israel Binational Science Foundation
Israel Science Foundation
Tel Aviv University

    Fingerprint

    Dive into the research topics of 'FEDEX: An Explainability Framework for Data Exploration Steps'. Together they form a unique fingerprint.

    Cite this