The polar code proposed by Arıkan is the first error correction code that can achieve the symmetric capacity of discrete memoryless channel. The polar code was adopted as a channel coding scheme of the 5G standard and has been actively studied. The successive cancellation flip (SCF) algorithm has been proposed and it shows the better error correction performance of SC algorithm. However, the SCF algorithm has a disadvantage in that the average decoding time step increases significantly in a channel environment with low reliability. This paper proposes a critical set based non-uniform segmentation method for SCF decoder. We analyze the critical set distribution and segment the entire codeword based on the critical set. As a result, the latency is reduced without degradation of the error correction performance. Compared to the conventional SCF and segmented SCF decoder, the proposed algorithm reduces 66.79% and 10.77% decoding time steps at SNR 1.0dB, respectively.