High Efficiency Video Coding (HEVC) adopts a new technique, called quad-tree based coding. However, deciding the optimal size of CU, the basic unit of the quad-tree structure, is very time-consuming process. As a result, huge computational complexity of the CU size decision process makes it difficult to be used in resource-constrained environments. In this thesis, we propose a fast CU size decision algorithm for inter prediction. As frame rate goes higher today, texture information between adjacent frames is becoming similar. Thus, optimal CU depth level is highly correlated with the CU of the previous frame. Proposed algorithm skips some unnecessary depth by employing these correlations. Proposed algorithm shows a reasonable performance in terms of reducing the complexity of the CU size decision process and still maintaining comparable RD performance. Experimental results demonstrate that the proposed algorithm can save the total encoding time up to 40% while maintaining similar PSNR and RD performance compared to the HEVC reference model (HM).