Commit 7380b25e authored by Rupert Swarbrick's avatar Rupert Swarbrick Committed by Debargha Mukherjee
Browse files

Round up subsampled frame size in av1_loop_restoration_corners_in_sb

The previous code converted a frame_w (say) of 1 to zero for a plane
where subsampling was enabled, causing a division by zero in
av1_get_rest_ntiles. This doesn't match the spec, which says
subsampling rounds up.

The patch adds the rounding, and also adds an assertion to
av1_get_rest_ntiles to help diagnose any other broken callsites.

Change-Id: Ia6c249fa935c3a16d122ba6e7b450fe99f412fde
parent 7a5587a8
......@@ -1463,8 +1463,11 @@ int av1_loop_restoration_corners_in_sb(const struct AV1Common *cm, int plane,
const int denom = 1;
#endif // CONFIG_FRAME_SUPERRES
const int ss_frame_w = frame_w >> (plane > 0 && cm->subsampling_x != 0);
const int ss_frame_h = frame_h >> (plane > 0 && cm->subsampling_y != 0);
const int ss_x = plane > 0 && cm->subsampling_x != 0;
const int ss_y = plane > 0 && cm->subsampling_y != 0;
const int ss_frame_w = (frame_w + ss_x) >> ss_x;
const int ss_frame_h = (frame_h + ss_y) >> ss_y;
int rtile_w, rtile_h, nvtiles;
av1_get_rest_ntiles(ss_frame_w, ss_frame_h,
......
......@@ -194,6 +194,8 @@ static INLINE int av1_get_rest_ntiles(int width, int height, int tilesize,
int tile_width_, tile_height_;
tile_width_ = (tilesize < 0) ? width : AOMMIN(tilesize, width);
tile_height_ = (tilesize < 0) ? height : AOMMIN(tilesize, height);
assert(tile_width_ > 0 && tile_height_ > 0);
nhtiles_ = (width + (tile_width_ >> 1)) / tile_width_;
nvtiles_ = (height + (tile_height_ >> 1)) / tile_height_;
if (tile_width) *tile_width = tile_width_;
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment