Enable reference frame masking for rt mode

Reference frame masking helped good quality mode to gain about 5% in
encoding speed, this commit enable it for rt mode to gain the speed
improvement.

In addition, this commit move the speed feature setup to a separate
function.

Change-Id: I015e8f78bbb21dd43ae183b9b9355bea2ccda9c5