1. 11 Jan, 2018 1 commit
    • Angie Chiang's avatar
      Move rect txfm's shift setting into config · a4c80485
      Angie Chiang authored
      For 2:1 txfm, move the sqrt2 scaling to the end of the fwd txfm
      function
      
      This only affects the txfm accuracy slightly but simplifies the
      range management a lot.
      
      Change-Id: I2eef8c15b7e8958a1fb0f2ae632604fdbab86046
      a4c80485
  2. 05 Jan, 2018 1 commit
  3. 04 Jan, 2018 1 commit
  4. 23 Dec, 2017 1 commit
    • Sarah Parker's avatar
      Replace hbd adst4 with lbd adst4 · 95f52605
      Sarah Parker authored
      0.05% drop in performance for 10 bit
      0.03% drop in performance for 12 bit
      
      Updated relevant tests:
      - Use the fadst4 function from VP9 as the reference.
      - Update some max/avg error thresholds
      
      Change-Id: Ic8c5b591eea3309427d2bb42828d44e640f718a1
      95f52605
  5. 16 Dec, 2017 1 commit
  6. 14 Dec, 2017 2 commits
    • Urvang Joshi's avatar
      reference_hybrid_2d: Scaling within the function. · ec6acb2b
      Urvang Joshi authored
      Earlier the scaling was done outside the function. But now, we do that
      within the function itself, so that the reference output can be directly
      compared (without scaling) to integerized transform output.
      
      BUG=aomedia:1114
      
      Change-Id: I67feaf43c22c2893300336e0a9d7fa6eb2a184d9
      ec6acb2b
    • Urvang Joshi's avatar
      AV1FwdTxfm2d.RunFwdAccuracyCheck: Add rect txfms. · 5fb50f89
      Urvang Joshi authored
      - Added all 1:2 and 1:4 transforms through a dynamically generated list.
      - Reworked the code to support testing these rectangular transforms.
      
      BUG=aomedia:1114
      
      Change-Id: I7e83b48f02a530716d5e30103780c5c4f450cbbd
      5fb50f89
  7. 12 Dec, 2017 1 commit
  8. 30 Nov, 2017 1 commit
    • Urvang Joshi's avatar
      Reduce cos bits for 64x64. · 09191ca2
      Urvang Joshi authored
      This is to get the high range within 32 bits.
      
      The compression performance is neutral.
      
      Change-Id: I475b2694f7cc7d786478c8821cf17982281dabce
      09191ca2
  9. 15 Nov, 2017 1 commit
  10. 02 Nov, 2017 1 commit
    • Sebastien Alaiwan's avatar
      Remove experimental flag of EXT_TX · 3bac9928
      Sebastien Alaiwan authored
      This experiment has been adopted, we can simplify the code
      by dropping the associated preprocessor conditionals.
      
      Change-Id: I02ed47186bbc32400ee9bfadda17659d859c0ef7
      3bac9928
  11. 22 Aug, 2017 1 commit
  12. 21 Aug, 2017 1 commit
  13. 09 Aug, 2017 2 commits
    • Angie Chiang's avatar
      Add additional txfm config check · d9e0e0be
      Angie Chiang authored
      Change-Id: I0b67ad4f37a22812472b191e0a6f1d218430c454
      d9e0e0be
    • Angie Chiang's avatar
      Add txfm config test · 9c7089a9
      Angie Chiang authored
      This test makes sure two things:
      1) txfm stage range is within desired limit
          (lbd:16 bits hbd:32 bits)
      2) txfm stage range + cos bits is within desired limit
          (lbd:32 bits hbd:32 bits)
      
      Change-Id: Ie2cc3c8265810e034c1461def4717fa9d4c29945
      9c7089a9
  14. 12 Apr, 2017 1 commit
  15. 29 Nov, 2016 1 commit
  16. 01 Sep, 2016 2 commits
  17. 12 Aug, 2016 1 commit
  18. 13 May, 2016 2 commits
  19. 11 May, 2016 1 commit
  20. 10 May, 2016 1 commit
  21. 22 Apr, 2016 1 commit
    • Yi Luo's avatar
      Change hybrid transform function argument from TXFM_2D_CFG* to int · cf7f0069
      Yi Luo authored
        Unit test shows manually developed SSE4.1 code would performs ~30%
        better if TXFM_2D_CFG configuration is set in lower level. This
        change only updates function signature. There is no performance
        impact.
      
      Change-Id: I62692bd50a21ffc8a944bbd6c155c0a2020ad77b
      cf7f0069
  22. 19 Apr, 2016 1 commit
  23. 06 Apr, 2016 1 commit
  24. 05 Feb, 2016 1 commit
  25. 09 Nov, 2015 1 commit
    • Yaowu Xu's avatar
      Get test to build with MSVC · 4bc259db
      Yaowu Xu authored
      Added _USE_MATH_DEFINES to make sure M_PI can be refered to definitions
      in math.h for MSVC
      
      Change-Id: Idca128910384593a002eb08bae72c739fb998e19
      4bc259db
  26. 04 Nov, 2015 1 commit