-
Yushin Cho authored
Previously, for block >=8x8, and tx < 8x8, we skipped setting the early-exit flag in block_rd_txfm() because distortion for sub8x8 tx block is from MSE but reference (best) is from daala-dist. However, not setting early-exit flag turned out to be the reason for a regression in MSE probe mode of daala-dist because it loses the chance to set rd_stats properly. On the other hand, there is still a small regression, say 0.05% psnr bd-rate, which seems to occur in the case that a tx block in a partition has chosen the skipped rd_cost since it is smaller than non-skip rd_cost and set the early-exit flag to 0 (so, not exit), but the daala-dist applied to the whole partition cannot access the same info but can choose from two kinds of rd_costs: 1) all tx blocks are skipped (even if a tx block has non-zero coeff) and 0 bits 2) sum of final distortion of all tx blocks (i.e. non-zero coeff decoded) and bits to encode coeffs. Change-Id: I2ec69972aa1f22d465293cb9e8d5e18ef2c6f7f3
04eb9594