• Yaowu Xu's avatar
    fixed the scaling in 8x8 trellis quant · 7f33be9e
    Yaowu Xu authored
    This commit has a few minor fixes to the 8x8 trellis quant, so to
    make it work regardless if extend_qrange is enabled or not. It also
    borrowed adaptive RDMULT constants from 4x4 trellis that was missed
    in the 8x8 trellis quant.
    
    Change-Id: I60d7769071f102c699b5084597e62bca87a1f759
    7f33be9e
encodemb.c 40.9 KB