Adapt rectangular txfm experiment to syntax/tokenization/loopfilter framework of VAR_TX Change-Id: Idcb005ecf5b3712de3e1cccb0d811ca16d87af24