• Jingning Han's avatar
    Singularity handling in Gaussian elimination · 041c67b9
    Jingning Han authored
    When the fwd Gaussian elimination process encounters diagonal
    element as zero value, the linear equation does not have unique
    solution. Return the linear solver state as unsolvable in such
    case. This resolves a floating point exception issue due to divided
    by zero in the loop restoration filter.
    
    BUG=aomedia:437
    
    Change-Id: I3c67525691a3003f9f470e8a0d5b4ee187cba963
    041c67b9
pickrst.c 43.9 KB