• Steinar Midtskogen's avatar
    Enable CDEF for chroma for 4:2:2 and 4:4:0 · ab6c9c77
    Steinar Midtskogen authored
    This enables CDEF for chroma when the horizontal and vertical
    subsampling differs.  Since the direction search is only performed in
    luma, the following corrections to the direction index are done for
    chroma to correct the distortion resulting from the subsampling:
    
    4:2:2:   4:4:0:
    0 -> 7   0 -> 1
    1 -> 0   1 -> 2
    2 -> 2   2 -> 2
    3 -> 4   3 -> 2
    4 -> 5   4 -> 3
    5 -> 6   5 -> 4
    6 -> 6   6 -> 6
    7 -> 6   7 -> 0
    
    This improves the chroma PSNR BDR by about 2% for 4:2:2 content at
    cpu-used=4, low delay configuration.  4:2:0 and 4:4:4 content is
    unchanged.
    
    Change-Id: Iee92d3697bc5a6fc9b1f340a63243a334935b433
    ab6c9c77
bitstream.c 171 KB