First Trimester Gaze Pattern Estimation Using Stochastic Augmentation Policy Search for Single Frame Saliency Prediction

Elizaveta Savochkina*, Lok Hin Lee, Lior Drukker, Aris T. Papageorghiou, J. Alison Noble

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

While performing an ultrasound (US) scan, sonographers direct their gaze at regions of interest to verify that the correct plane is acquired and to interpret the acquisition frame. Predicting sonographer gaze on US videos is useful for identification of spatio-temporal patterns that are important for US scanning. This paper investigates utilizing sonographer gaze, in the form of gaze-tracking data, in a multi-modal imaging deep learning framework to assist the analysis of the first trimester fetal ultrasound scan. Specifically, we propose an encoder-decoder convolutional neural network with skip connections to predict the visual gaze for each frame using 115 first trimester ultrasound videos; 29,250 video frames for training, 7,290 for validation and 9,126 for testing. We find that the dataset of our size benefits from automated data augmentation, which in turn, alleviates model overfitting and reduces structural variation imbalance of US anatomical views between the training and test datasets. Specifically, we employ a stochastic augmentation policy search method to improve segmentation performance. Using the learnt policies, our models outperform the baseline: KLD, SIM, NSS and CC (2.16, 0.27, 4.34 and 0.39 versus 3.17, 0.21, 2.92 and 0.28).

Original languageEnglish
Title of host publicationMedical Image Understanding and Analysis - 25th Annual Conference, MIUA 2021, Proceedings
EditorsBartłomiej W. Papież, Mohammad Yaqub, Jianbo Jiao, Ana I. Namburete, J. Alison Noble
PublisherSpringer Science and Business Media Deutschland GmbH
Pages361-374
Number of pages14
ISBN (Print)9783030804312
DOIs
StatePublished - 2021
Externally publishedYes
Event25th Annual Conference on Medical Image Understanding and Analysis, MIUA 2021 - Virtual, Online
Duration: 12 Jul 202114 Jul 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12722 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th Annual Conference on Medical Image Understanding and Analysis, MIUA 2021
CityVirtual, Online
Period12/07/2114/07/21

Funding

FundersFunder number
Engineering and Physical Sciences Research CouncilEP/T028572/1, EP/R013853/1
NIHR Oxford Biomedical Research Centre

    Keywords

    • Data augmentation
    • Fetal ultrasound
    • First trimester
    • Gaze tracking
    • Single frame saliency prediction
    • U-Net

    Fingerprint

    Dive into the research topics of 'First Trimester Gaze Pattern Estimation Using Stochastic Augmentation Policy Search for Single Frame Saliency Prediction'. Together they form a unique fingerprint.

    Cite this