Make the CDEF RDO handle 4:2:2 properly
This fixes an assert: av1/common/cdef_block.c:561: cdef_filter_fb: Assertion `bsize == BLOCK_8X8 || bsize == BLOCK_4X4' failed. The RDO simply assigned a strength of 0 in the 4:2:2 case and called cdef_filter_fb(), but cdef_filter_fb() will complain about 4:2:2 even if the strength is 0. The fix assigns a chroma mse of 0 when the the subsampling is different for x and y rather than to call the filter. This is faster also. The mse isn't really 0, but calculating the actual chroma mse doesn't change result. BUG=aomedia:881 Change-Id: I6154e21ddcca30e51baf805684dace10459c3350
parent
146a060a
No related branches found
No related tags found
Loading
Please register or sign in to comment