Skip to content
  • Deb Mukherjee's avatar
    Refinements on modelcoef expt to reduce storage · 07443f15
    Deb Mukherjee authored
    Uses more aggrerssive interpolation to reduce storage for the
    model tables by almost more than half. Only 48 lists of probs are
    stored (as opposed to 128 before), corresponding to ONE_NODE
    probabilities of:
    3, 7, 11, ..., 115, 119,
    127, 135, ..., 247, 255.
    Besides, only 1 table is used as opposed to 2 before. So the overall
    memory needed for the tables is just 48 * 8 = 384 bytes.
    The table currently used is based on a new Pareto distribution with
    heavier tail than a generalized Gaussian - which improves results on
    derf by about 0.1% over a single table Generaized Gaussian.
    Results overall on derfraw300 is -0.14%.
    Change-Id: I19bd03559cbf5894a9f8594b8023dcc3e546f6bd