Background: The potential for catastrophe resulting from anesthetic equipment failure and the failure of training programs to adequately prepare residents to detect and manage equipment failure prompted the Israel Board of Anesthesiologists to include simulation-based testing in the Objective Structured Clinical Evaluation component of the Israeli Board Examination in Anesthesiology. Methods: We used simulation-based scenarios to measure the performance of residents while (a) checking the anesthesia machine before the first morning case, (b) checking the anesthesia machine between cases, (c) managing an oxygen pipeline failure, and (d) managing an expiratory valve failure. Results: During board examination, 3 of 28 examinees failed to correctly check at least 70% of the items on the anesthesia machine checkout list before the first morning case and 3 of 30 failed to correctly check 70% of the items between cases. Although all examinees recognized inadequate oxygen cylinder pressure and a malfunctioning valve, 1 of 31 examinees failed to open the O2 cylinder, 6 of 31 did not disconnect the anesthesia machine from the central oxygen supply, 14 of 31 could not explain how to minimize the use of oxygen, 2 of 30 failed to find the faulty valve, and 15 of 30 could not give the correct differential diagnosis. Conclusions: During simulation-based board examination most senior anesthesia residents became aware of equipment failures but many failed to correctly diagnosis and manage the failure.