* Remove some unused variables
* Reduce need for casts by typing intermediate buffers appropriately
* Avoid copying data which is never modified; use the original data
  instead.
* Reduce number of intermediate buffers required, saving allocations
  of 576KiB in the decoder and ~1MiB in the encoder

No effect on performance

Change-Id: I55243904dd8e818fb6d43fa431903736475d23ff