Many manufacturing processes incorporate various factors which affect the quality of the products, and the analysis and optimization of those factors are critical activities of engineers. Although there have been a lot of research on statistical methods to investigate the factorial effects on the quality metrics, these statistical methods are not always applied due to problems in data integrity, lack of control / measurement, or technical / administrative constraints. On the other hand, conventional heuristic methods regarding the selection of critical quality factors are mostly devoid of metrics which can be examined objectively . This study, therefore, implemented AHP for the quantitative prioritization of control factors involved in a flat end milling manufacturing process. In order to validate the metrics synthesized from the experience of skilled workers, the decision making has been followed by multivariate analysis of variance based on GLM. The results elucidate that AHP is able to provide fairly reliable metrics about the contribution of process parameters and group-wise judgment of qualified experts can improve the consistency of the prioritization.