Commit 4f95a7fe authored by Yaowu Xu's avatar Yaowu Xu
Browse files

Fix the setting of mode_skip_mask

In mode selection loop, once mode_index pass mode_skip_start, all
modes with a different reference frame from current best mode are
masked out using mode_skip_mask.

However, the setting of mode_skip_mask may use an invalid mode if
there is no mode tested yet. This commit fixes the issue by making
sure a mode has been tested and selected. Otherwise, no mode will be
masked out because of their reference frame.

Change-Id: Ib0009e8a96836a65cf5347440fff8a2e1a67f29f
parent e7e426b2
......@@ -3304,7 +3304,7 @@ int64_t vp9_rd_pick_inter_mode_sb(VP9_COMP *cpi, MACROBLOCK *x,
// Look at the reference frame of the best mode so far and set the
// skip mask to look at a subset of the remaining modes.
if (mode_index == mode_skip_start) {
if (mode_index == mode_skip_start && best_rd < INT64_MAX) {
switch (vp9_mode_order[best_mode_index].ref_frame[0]) {
case INTRA_FRAME:
break;
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment