The direction inversion semantics are now handled by simply
using the registers for in/out available, no need to keep
track of inversion semantics exmplicitly anymore.

Reviewed-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
Reviewed-by: Jan Kotas <jank@cadence.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>