This prepares for the next refactoring to support 2x2 transform
block sizes.

Change-Id: Ia06bc487da34e853ef9323cd13e3d482e819db43

Move computing the kf_y_mode_cdf tables per coded intra mode symbol to
 computing them only when the probabilities are updated.

Change-Id: I5999447050c2f7d5dbccde80bee05ecd1c5440ab

Remove the separate implementations of set_contexts() in encoder
and decoder.

Change-Id: I9f6e9b075532faae0f74f885d9443589254258a7

Plus a small code clean up. The experiment of EXT_REFS, compared against
the baseline, using Overall PSNR, now obtains a gain on lowres as:
Avg: -5.818; BDRate: -5.653

Compared against the previous EXT_REFS results on lowres, a tiny gain is
obtained as:
Avg: -0.047, BDRate: -0.063

(1) 780952 Add encoder first pass support to bi-prediction in EXT_REFS
(2) f91498 Add pred prob handling for new references in EXT_REFS
(3) e91472 Add decoder support for bi-direct prediction in EXT_REFS
(4) 0dbac9 Add encoder support to new references in EXT_REFS
(5) ad70cc Remove hard-coded number for EXT_REFS
(6) 9c1e2f Add the use of new reference frames at encoder in EXT_REFS
(7) 6d4fde Add the experiment flag of EXT_REFS

Change-Id: I26f7ca45b9ede7579fdb9d0d6a1a91f4334599bd

Example performance: 1.8% bit rate savings using
the AQ test mode aq-mode=4 :
./aomenc --codec=av1 --ivf --tile-columns=1 --tile-rows=1 \
                 --kf-max-dist=1000 --kf-min-dist=1000 --cpu-used=0 \
                 --passes=1 --threads=1 --lag-in-frames=0 \
                 --end-usage=q --limit=600 --cq-level=42 \
                 --aq-mode=4 --error-resilient=1 out.bits FourPeople_1280x720_60.y4m

Change-Id: Iba01cf2732a57f3c27481ac2a3c8fc37bb9e5533

Change-Id: I4128af44776d1f361bddc1fdffb75ed2224dbfa5

Change-Id: I97487bf353471bf9d245cd620780adfb1d3fc2b1

This flag was already added to aomedia/master, so bringing it back to
webm/nextgenv2, as part of an effort to get the two codebases in sync.

Change-Id: I2b933a6a160e4210d1411a9e7978149eb8553205

This reverts commit 9b25f306 to
reinstate the reverted commit with fixes that solved the build issues
when --enalbe-clpf is used in configure.

Change-Id: I15447cae7fa9b3deb27976345dc3db230a4a7a60

This reverts commit 97535038.

Change-Id: I9f8e891739352ca2bde4b294e37c85a668f416e0

These signals were in the uncompressed frame header (as a temporary
hack), which caused two problems:

* We don't want that header to be duplicated in the slice header
* It was necessary to signal the number of bits to transmit up front

However, the filter size can be 128x128 which is greater than the SB
size, and a decoder wouldn't be able to know whether to read a bit or
not until the final SB of that 128x128 block has been decoded
(depending on whether the 128x128 is all skip or not).  Therefore the
signalling was changed for 128x128 blocks so that every top left SB of
a 128x128 filter block contains a signal regardless of whether the
block is all skip or not.  Also, all the MB's of 128x128 block are
filtered even if they are skip MB's.  This gives the signal a purpose
even when the 128x128 block is all skip, and it also gives a slight
coding gain as it leaves a way to filter skip blocks, which was
previously forbidden.

Low latency:
PSNR YCbCr:     -0.19%     -0.14%     -0.06%
   PSNRHVS:     -0.15%
      SSIM:     -0.13%
    MSSSIM:     -0.15%
 CIEDE2000:     -0.19%

High latency:
PSNR YCbCr:     -0.03%     -0.01%     -0.09%
   PSNRHVS:      0.04%
      SSIM:      0.00%
    MSSSIM:      0.02%
 CIEDE2000:     -0.02%

Change-Id: I69ba7144d07d388b4f0968f6a53558f480979171

Objective quality impact (low latency):

PSNR YCbCr:      0.13%     -1.37%     -1.79%
   PSNRHVS:      0.03%
      SSIM:      0.24%
    MSSSIM:      0.10%
 CIEDE2000:     -0.83%

Change-Id: I8ddf0def569286775f0f9d4d4005932766a7fc27

This commit ports a CLPF change from aom/master by manually
cherry-picking:
7560123c

Change-Id: I61eb08862a101df74a6b65ece459833401e81117

Change-Id: I9152f898dfacdf3877ed719f193bb1e0dbee0a1a

Includes a major refactoring/enhancement to support
tile-adaptive switchable restoration. The framework can be
readily extended to add more restoration schemes in the
future. Also includes various cleanups and fixes.

Specifically the framework allows restoration to be conducted
on tiles such that each tile can be either left unrestored, or
use bilateral or wiener filtering.

There is a modest improvemnt in coding efficiency (0.1 - 0.2%).

Further enhancements will be added subsequently to improve coding
efficiency and complexity.

Change-Id: I5ebedb04785ce1ef6f324abe209e925c2d6cbe8a

this matches style guidelines and stabilizes successive runs of
clang-format across the tree. remaining types should be address in
successive commits.

Change-Id: I6ad3f69cf0a22cb9a9b895b272195f891f71170f

Change-Id: I2b2b70e756b7eb9611b7b33b7d5f19b3b30e0a50

Cherry-Picked the following commits:
0defd8f2 Changed "WebM" to "AOMedia" & "webm" to "aomedia"
54e66767 Replace "VPx" by "AVx"
5082a369 Change "Vpx" to "Avx"
7df44f17 Replace "Vp9" w/ "Av1"
967f722f Remove kVp9CodecId
828f30ce Change "Vp8" to "AOM"
030b5ffc AUTHORS regenerated
2524caee Add ref-mv experimental flag
016762be Change copyright notice to AOMedia form
81e55269 Replace vp9 w/ av1
9b94565b Add missing files
fa8ca9f2 Change "vp9" to "av1"
ec838b76  Convert "vp8" to "aom"
80edfa01 Change "VP9" to "AV1"
d1a11fb9 Change "vp8" to "aom"
7b582513 Point to WebM test data
dd1a5c8d Replace "VP8" with "AOM"
ff00fc0f Change "VPX" to "AOM"
01dee0bb Change "vp10" to "av1" in source code
cebe6f0c Convert "vpx" to "aom"
17b05679 rename vp10*.mk to av1_*.mk
fe5f8a8a rename files vp10_* to av1_*

Change-Id: I6fc3d18eb11fc171e46140c836ad5339cf6c9419

Manually cherry-picked commits:
ceef0583 libvpx->libaom part2
3d26d916 libvpx -> libaom
cfea7dd7 vp10/ -> av1/
3a8eff73 Fix a build issue for a test
bf4202ed Rename vpx to aom

Change-Id: I1b0eb5a40796e3aaf41c58984b4229a439a597dc

Frame can be split into rectangular tiles for application of separate
bilateral or Wiener filters per tile. Some variable names changed for
better readability.

Change-Id: I13ebc4d0b0baf368e524db5ce276f03ed76af9c8

Adapt rectangular txfm experiment to syntax/tokenization/loopfilter
framework of VAR_TX

Change-Id: Idcb005ecf5b3712de3e1cccb0d811ca16d87af24

Mannually cherry-picked:
15791332 Use OD_DIVU for small divisions in temporal_filter.
03122298 Replace divides by small values with multiplies.
9c48eec7 Removing divisions from od_dir_find8()
0950ed82 Merge "Port active map / cyclic refresh fixes to vp10."
efefdad7 Port active map / cyclic refresh fixes to vp10.
1eaf748c Port switch to 9-bit rate cost to aom.
0b1606e7 Only build deringing code when --enable-dering.
e2511e15 Deringing cleanup: don't hardcode the number of levels
8fe5c5d6 Rename dering_in to od_dering_in to sync with Daala
4eb1380d Makes second filters for 45-degree directions horizontal
7f4c3f58 Removes the superblock variance contribution to the threshold
3dc56f93 Simplifying arithmetic by using multiply+shift
cf2aaba9 Return 0 explicitly for OD_ILOG(0).
49ca22aa Use the Daala implementation of OD_ILOG().
85187243 Fix compiler warning in od_dering.c.
485d6a69 Prevent multiple inclusion of odintrin.h.
51b7a998 Adds the Daala deringing filter as experimental

Note that a few of the changes were already in libvpx codebse.

Change-Id: I1c32ee7694e5ad22c98b06ff97737cd792cd88ae

Manually cherry-picked following commits from AOMedia git repository:
bb2727c2 Sort includess for "clpf.h"
c297fd00 Add quantisation matrix range parameters.
0527894f Add encoder option and signaling for quant matrix control.
4106232b Turn off trellis coding for quantization matrices.
4017fca3 Modify tests to allow quantization matrices.
1c122c24 Add quant and dequant functions for new quant matrices.
95a89994 Enable CLPF
f72782bb Fix a build issue
73bae509 Add quantisation matrices and selection functions
33208d20 Added support for constrained low pass filter (CLPF)

Change-Id: I60fc1ee1ac40e6b9d1d00affd97547ee5d5dd6be

clang-format authored Aug 11, 2016 and James Zern committed Aug 12, 2016

Change-Id: I01d8241eba3ccaf4d06c00a51df2d17c126f6f9d

We have renamed following Macros to avoid name confusion:

REFS_PER_FRAME --> INTER_REFS_PER_FRAME
(= ALTREF_FRAME - LAST_FRAME + 1)
MAX_REF_FRAMES --> TOTAL_REFS_PER_FRAME
(= ALTREF_FRAME - INTRA_FRAME + 1)

INTER_REFS_PER_FRAME specifies the maximum number of reference frames
that each Inter frame may use.
TOTAL_REFS_PER_FRAME is equal to INTER_REFS_PER_FRAME + 1, which
counts the INTRA_FRAME.

Further, at the encoder side, since REF_FRAMES specifies the maximum
number of the reference frames that the encoder may store, REF_FRAMES
is usually larger than INTER_REFS_PER_FRAME. For example, in the
ext-refs experiment, REF_FRAMES == 8, which allows the encoder to
store maximum 8 reference frames in the buffer, but
INTER_REFS_PER_FRAME equals to 6, which allows each Inter frame may
use up to 6 frames out of the 8 buffered frames as its references.
Hence, in order to explore the possibility to store more reference
frames in future patches, we modified a couple of array sizes to
accomodate the case that the number of buffered reference frames is
not always equal to the number of the references that are being used
by each Inter frame.

Change-Id: I19e42ef608946cc76ebfd3e965a05f4b9b93a0b3

Change-Id: Ic90ebe6136f4b75645ba699d49c0bcb3764ddccf

68e7e4d0 Remove VP9_CAP_POSTPROC
0738390c Remove vp9_temporal denoise
b89861a4 Remove vp9-postproc

Change-Id: I4ecaa0ac83a519c8174a494378fc23df610ff2a8

This commit makes the encoder to use different frame context index
for different frame types. In the baseline setting, it sets the
frame context index of the overlay frame to be different from other
regular inter frames. In the ext-refs setting, it further allows
the backward reference frame to use a different index.

It improves the compression performance for both settings.

Baseline
lowres  0.12%

ext-refs
lowres  0.50%
midres  0.56%

Change-Id: I7c63ddec9fc296c56a86353cf2c661a740b97a97

Currently nothing is implemented to compute GM parameters, this
just adds the capability to send them in the bitstream if they
were computed. Still need to implement the reconstruction
based on the parameters in reconinter.

Change-Id: I72aea3c6a9de9f5a40f96da76c82b54a52781fe2

Add the ability to pick between 3 quantization profiles.
The profile is chosen based on the entropy context at the
block level.

Change-Id: Iaea0485798441b7d635962c2563f3a477f582dac

This patch removed the experiment of BIDIR_PRED and merged the feature
into the experiment of EXT_REFS:

(1) Each frame now has up to 6 reference frames, namely
    LAST_FRAME, LAST2_FRAME, LAST3_FRAME, GOLDEN_FRAME, (forward) and
    BWDREF_FRAME, ALTREF_FRAME (backward);
    LAST4_FRAME has been removed;
(2) First pass still keeps the 8 updates:
    KF_UPDATE, LF_UPDATE, GF_UPDATE, ARF_UPDATE, OVERLAY_UPDATE, and
    BRF_UPDATE, LAST_BIPRED_UPDATE, BI_PRED_UPDATE;
(3) show_existing_frame==1 is supported in the experiment of EXT_REFS;
(4) New encoding modes are added for both single-ref and compound cases,
    through the use of the 2 extra forward references (LAST2 & LAST3)
    and the 1 extra backward reference (BWDREF).

RD performance wise, using Overall PSNR: Avg/BDRate
        Bipred only      Prev EXT_REFS    Current EXT_REFS with bipred
lowres: -3.474/-3.324    -1.748/-1.586    -4.613/-4.387
derflr: -2.097/-1.353    -1.439/-1.215    -3.120/-2.252
midres: -2.129/-1.901    -1.345/-1.185    -2.898/-2.636

If in vp10/encoder/firstpass.h, change BFG_INTERVAL from 2 to 3, i.e. to
use 2 bi-predictive frames than 1, a further improvement may be
obtained:
                 Current EXT_REFS with bipred
        1 bi-predictive frame    2 bi-predictive frames
lowres: -4.613/-4.387            -4.675/-4.465
derflr: -3.120/-2.252            -3.333/-2.516
midres: -2.898/-2.636            -3.406/-3.095

Change-Id: Ib06fe9ea0a5cfd7418a1d79b978ee9d80bf191cb

This experiment implements non-uniform quantization where
the width of the bins increases gradually to more closely
match a laplacian distribution of the coeficcients.

Performance Gain:
derflr: 0.15%
hevcmr: 0.675%

Change-Id: I25234244e3bcd94b87c1f77cf682190b61c8ef94

This is for the bidir-pred experiment. Previously the length of the
bi-predictive frame group interval is fixed at 2, i.e. one
bi-predictive frame may be inserted every other frame. This patch
makes the length adjustable, i.e. any positive number may be
specified, but the use of the backward ref will be turned off if the
bi-predictive frame group interval is larger than the golden frame
group.

Further, an additional rate factor level has been added:
INTER_LOW
, which applies to LAST_BIPRED_UPDATE frames that are not used as
references.

Change-Id: I5514d34a64dd486bbb5756c2d0612946f598a789

Major parts have been implemented as follows:
(1) Added BRF_UPDATE, LASTNRF_UPDATE, and NRF_UPDATE in firstpass.c;
(2) Added the handling for the scenario of
"cpi->common.show_existing_frame == 1" at the encoder;
(3) Added a new reference frame of BWDREF_FRAME;
(4) Have bwd-ref work with upsampled references.

Note that when the experiment of "ext_refs" turned on, this experiment
will be turned off automatically currently.

RD performance in Overall PSNR has been improved, compared against the
VP10 baseline:

lowres: Avg -3.312; BDRate -3.154
derflr: Avg -1.927; BDRate -1.176
midres: Avg -2.149; BDRate -2.001
hdres : Avg -0.567; BDRate -0.588

Change-Id: I4c06ff51cc20194bffbd4d2346e57ba3dcf6b62c

When not using ext-tile, there were still dependencies between tile
rows due to various tools (eg intra predictors) relying on the above
row or above mode info, which can be in the above tile. This is now
broken (the same way as it was when ext-tile is enabled) by fixing
the appropriate predicates.

Change-Id: I107dd0d8481775a792f14e05cfbbd761f16cdc1e

In VP10, REFRESH_FRAME_CONTEXT_OFF mode is only set when the error
resillient mode is on. Instead of being used to decide how to update
the frame contexts, it is used to decide if or not to reset the
frame contexts.

To verify, ran borg test on lowres set. The result is neutral.
Overall PSNR: -0.006%; SSIM: -0.006%.

Change-Id: Ic48265cf7488e80c6f5aab3eef7ba1c273506419

This will facilitate bringing the zero node into the token set while
allowing its probability to vary independently.

Change-Id: I57b44c0fce44debb8e612021e44713b229d1b3cf

The uncompressed frame header contains a bit to signal whether the
frame is encoded using 64x64 or 128x128 superblocks. This can vary
between any 2 frames.

vpxenc gained the --sb-size={64,128,dynamic} option, which allows the
configuration of the superblock size used (default is dynamic). 64/128
will force the encoder to always use the specified superblock size.
Dynamic would enable the encoder to choose the sb size for each
frame, but this is not implemented yet (dynamic does the same as 128
for now).

Constraints on tile sizes depend on the superblock size, the following
is a summary of the current bitstream syntax and semantics:

If both --enable-ext-tile is OFF and --enable-ext-partition is OFF:
     The tile coding in this case is the same as VP9. In particular,
     tiles have a minimum width of 256 pixels and a maximum width of
     4096 pixels. The tile width must be multiples of 64 pixels
     (except for the rightmost tile column). There can be a maximum
     of 64 tile columns and 4 tile rows.

If --enable-ext-tile is OFF and --enable-ext-partition is ON:
     Same constraints as above, except that tile width must be
     multiples of 128 pixels (except for the rightmost tile column).

There is no change in the bitstream syntax used for coding the tile
configuration if --enable-ext-tile is OFF.

If --enable-ext-tile is ON and --enable-ext-partition is ON:
     This is the new large scale tile coding configuration. The
     minimum/maximum tile width and height are 64/4096 pixels. Tile
     width and height must be multiples of 64 pixels. The uncompressed
     header contains two 6 bit fields that hold the tile width/heigh
     in units of 64 pixels. The maximum number of tile rows/columns
     is only limited by the maximum frame size of 65536x65536 pixels
     that can be coded in the bitstream. This yields a maximum of
     1024x1024 tile rows and columns (of 64x64 tiles in a 65536x65536
     frame).

If both --enable-ext-tile is ON and --enable-ext-partition is ON:
     Same applies as above, except that in the bitstream the 2 fields
     containing the tile width/height are in units of the superblock
     size, and the superblock size itself is also coded in the bitstream.
     If the uncompressed header signals the use of 64x64 superblocks,
     then the tile width/height fields are 6 bits wide and are in units
     of 64 pixels. If the uncompressed header signals the use of 128x128
     superblocks, then the tile width/height fields are 5 bits wide and
     are in units of 128 pixels.

The above is a summary of the bitstream. The user interface to vpxenc
(and the equivalent encoder API) behaves a follows:

If --enable-ext-tile is OFF:
     No change in the user interface. --tile-columns and --tile-rows
     specify the base 2 logarithm of the desired number of tile columns
     and tile rows. The actual number of tile rows and tile columns,
     and the particular tile width and tile height are computed by the
     codec ensuring all of the above constraints are respected.

If --enable-ext-tile is ON, but --enable-ext-partition is OFF:
     No change in the user interface. --tile-columns and --tile-rows
     specify the WIDTH and HEIGHT of the tiles in unit of 64 pixels.
     The valid values are in the range [1, 64] (which corresponds to
     [64, 4096] pixels in increments of 64.

If both --enable-ext-tile is ON and --enable-ext-partition is ON:
     If --sb-size=64 (default):
         The user interface is the same as in the previous point.
         --tile-columns and --tile-rows specify tile WIDTH and HEIGHT,
         in units of 64 pixels, in the range [1, 64] (which corresponds
         to [64, 4096] pixels in increments of 64).
     If --sb-size=128 or --sb-size=dynamic:
         --tile-columns and --tile-rows specify tile WIDTH and HEIGHT,
         in units of 128 pixels in the range [1, 32] (which corresponds
         to [128, 4096] pixels in increments of 128).

Change-Id: Idc9beee1ad12ff1634e83671985d14c680f9179a

Change-Id: I3a2fe4073bb94c5afc24d9274e6edcdb3aed934f

Change-Id: Idcb1e94298d4b7d8832d285548ec2d2ced4b2988