Skip to content
Snippets Groups Projects
Commit f7c9abd4 authored by Petter Reinholdtsen's avatar Petter Reinholdtsen
Browse files

Make sure difference in encfrag.c is signed when taking absolute value.

Switching type of cd from unsigned to int make sure the difference
passed to abs() wil be a signed value.

As dc is calculated to at most ((8 * 8 * (2^8-1)) + 32) / 64, ie 255
and the highest value it can hold before it is shifted 6 bit positions
to the right is (8 * 8 * (2^8-1)) + 32) ie. 16352, it should fit nicely
in an integer on any platform where int is at least 16 bit.

Fixes this clang message:

../../../lib/encfrag.c:104:26: warning: taking the absolute value of unsigned type 'unsigned int' has no effect [-Wabsolute-value]
    for(j=0;j<8;j++)sad+=abs(_src[j]-dc);
                         ^
../../../lib/encfrag.c:104:26: note: remove the call to 'abs' since unsigned values cannot be negative
    for(j=0;j<8;j++)sad+=abs(_src[j]-dc);
                         ^~~
parent 4b90527c
No related branches found
No related tags found
Loading
Checking pipeline status
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment