Matters of ethics, trust, and potential liability for autonomous systems

J. Christopher Brill, James P. Bliss, Peter A. Hancock, Dietrich Manzey, Joachim Meyer, Alison Vredenburgh

Research output: Contribution to journalConference articlepeer-review

Abstract

The objective of this panel was to discuss issues related to the development and use of autonomous systems, with specific focus on the overriding themes of ethical considerations and potential liability for Human Factors and Ergonomics (HF/E) professionals who are involved in their development. Chris Brill provided opening remarks to frame the discussion and introduce the panelists. James Bliss discussed legal implications related to our collective penchant for developing conservative, false-alarm prone automation. Peter Hancock advocated for human-centered constraints on autonomous systems, as they may, one day, pose an existential threat to humanity. Dietrich Manzey discussed ethical considerations for autonomous systems, including how design can encourage ethical user behavior. Joachim Meyer argued that HF/E professionals have an obligation to help designers understand the ethical implications of poor design, particularly in the context of autonomous systems. Lastly, Alison Vredenburgh provided thoughts on potential liability for HF/E professionals, particularly in light of the relative newness of autonomous systems. The panel then turned to facilitated discussion with panelists and audience members. Specific themes included the boundaries of our responsibilities as HF/E professionals for ill-conceived or morally-objectionable systems, potential implications of manipulating user trust through design, cross-cultural perspectives on public acceptance and legal peril, and how concerns might differ by domain (e.g., medical vs.combat vs. manufacturing). The session concluded with panelists summarizing how ethics influence design and recommendations for how HF/E professionals can potentially protect themselves from legal liability for mishaps involving autonomous systems they helped develop.

Original languageEnglish
Pages (from-to)308-312
Number of pages5
JournalProceedings of the Human Factors and Ergonomics Society
DOIs
StatePublished - 2016
EventHuman Factors and Ergonomics Society 2016 International Annual Meeting, HFES 2016 - Washington, United States
Duration: 19 Sep 201623 Sep 2016

Fingerprint

Dive into the research topics of 'Matters of ethics, trust, and potential liability for autonomous systems'. Together they form a unique fingerprint.

Cite this