• Yue Chen's avatar
    Disable residue hash feature on cross-border blocks · 25dc0701
    Yue Chen authored
    Disable this feature unless the entire block is within the frame.
    The reason is, rd decisions in mbmi, e.g. inter_tx_block[][], made
    for blocks partially out of the border can be partly nonsense
    therefore cannot be reused by blocks at other locations.
    It caused an infinite loop when encoding a clip with repetitive
    patterns. A cross-border block has an invalid big tx stored
    in inter_tx_block[0][1] and the other block (same residue, within
    frame) reused this mbmi, which makes encoder never reach the
    termination condition when tx blocks are being recursively
    Change-Id: Id25a1dbc4a68b5136f6bdf9f6b5811b7ec6920b0
rdopt.c 477 KB