We had previously shown that endoplasmic reticulum (ER)-associated degradation (ERAD) of glycoproteins in mammalian cells involves trimming of three to four mannose residues from the N-linked oligosaccharide Man 9GlcNAc2. A possible candidate for this activity, ER mannosidase I (ERManI), accelerates the degradation of ERAD substrates when overexpressed. Although in vitro, at low concentrations, ERManI removes only one specific mannose residue, at very high concentrations it can excise up to four α1,2-linked mannose residues. Using small interfering RNA knockdown of ERManI, we show that this enzyme is required for trimming to Man 5-6GlcNAc2 and for ERAD in cells in vivo, leading to the accumulation of Man9GlcNAc2 and Glc1Man 9GlcNAc2 on a model substrate. Thus, trimming by ERManI to the smaller oligosaccharides would remove the glycoprotein from reglucosylation and calnexin binding cycles. ERManI is strikingly concentrated together with the ERAD substrate in the pericentriolar ER-derived quality control compartment (ERQC) that we had described previously. ERManI knockdown prevents substrate accumulation in the ERQC. We suggest that the ERQC provides a high local concentration of ERManI, and passage through this compartment would allow timing of ERAD, possibly through a cycling mechanism. When newly made glycoproteins cannot fold properly, transport through the ERQC leads to trimming of a critical number of mannose residues, triggering a signal for degradation.