Commit 930118c5 authored by Guillaume Martres's avatar Guillaume Martres Committed by Guillaume Martres
Browse files

PVQ: Fix incorrect calculation of rd_stats

When PVQ is on, we reencode at the end of choose_tx_size_type_from_rd to
get the entropy contexts right, previously this was done using
txfm_rd_in_plane but this is different from the encodes done in the loop
which use txfm_yrd, the result is that rd_stats is set incorrectly at
the end of choose_tx_size_type_from_rd when PVQ is on.

Results on objective-1-fast with --limit=5:

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.5803 | -1.0598 | -1.4565 |  -0.3377 | -0.8153 | -0.5934 |    -0.9943

See https://goo.gl/Hvv0E2

Change-Id: Iccc7b0afaff849f959a0084eb48dbb838bc3cb1a
parent 6dbacf28
......@@ -1818,8 +1818,7 @@ static void choose_tx_size_type_from_rd(const AV1_COMP *const cpi,
#endif
#if CONFIG_PVQ
if (best_rd != INT64_MAX) {
txfm_rd_in_plane(x, cpi, rd_stats, ref_best_rd, 0, bs, best_tx,
cpi->sf.use_fast_coef_costing);
txfm_yrd(cpi, x, rd_stats, ref_best_rd, bs, best_tx_type, best_tx);
}
#endif
}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment