• Yushin Cho's avatar
    [dist-8x8] Bug fix - exclude invisible pixels · 2f025aae
    Yushin Cho authored
    There were two places where invisible pixels are included
    when calculating distortion with dist-8x8 is turned on.
    Those are bugs and fixed by this commit.
    1) When av1_dist_8x8() is called in dist_8x8_sub8x8_txfm_rd()
    - New assert() are also added to detect any failures in the future.
    2) Inside av1_dist_8x8() and av1_dist_8x8_diff(), with tune-metric == PSNR.
    Change-Id: I078d1feb3bb010c0e34041640bb69a400125817a
rdopt.c 480 KB