Add assertions for band size restrictions.

The way folding is implemented requires two restrictions:
1. The last band must be the largest (so we can use its size to
 allocate a temporary buffer to handle interleaving/TF changes).
2. No band can be larger than twice the size of the previous band
 (so that once we have enough data to start folding, we will always
 have enough data to fold).

Mode creation makes a heuristic attempt to satisfy these
 conditions, but nothing actually guarantees it.
This adds some asserts to check them during mode creation.
They current pass for all supported custom modes.