Removed assertions were not properly set up and may cause decoding failure when USE_CASUAL_CTX is enabled. This CL does not change bitstream.

Change-Id: Ib9193cbda32f342335a79aca39e9cc49204a0ec9