Skip to content
  • David Barker's avatar
    Fix an edge case with global motion / warped motion · 94b876f4
    David Barker authored
    There is a rare edge case in ransac(), causing the tests
    AV1/DatarateTestLarge.ChangingDropFrameThresh/{4,5,6}
    to fail when global motion is enabled. The sequence of events is:
    
    * Since GLOBAL_MOTION_TYPES = 3, we try to generate a ROTZOOM model
    * A model is generated at ransac.c:215, and its first 4 components
      are copied to best_params. The last four components of best_params
      are left as {0, 1, 0, 0}.
    * We then finish the trial_count loop, and call find_transformation
      one final time to generate a refined model. But this fails, and
      best_params is not overwritten.
    * get_wmtype decides that this is an AFFINE model, since
      wmmat[4] != -wmmat[3] and wmmat[5] != wmmat[2].
    * We try to encode this global motion model. But the entropy coder
      cannot encode an AFFINE model, and crashes at daalaboolwriter.h:74
      (with OD_ASSERT(symb != -1))
    
    The fix is to copy 8 entries to best_params regardless of the
    model type we are generating, in line with the changes in
    https://aomedia-review.googlesource.com/#/c/5589/
    
    Change-Id: I6dbdfb997924f8ddf7ea3a1d557463264ea63cbe
    94b876f4