This will simplify drivers which would only unregister the clk in their
remove() op.

Signed-off-by: Michael Walle <michael@walle.cc>
Link: https://lore.kernel.org/r/20201108185113.31377-6-michael@walle.cc


Signed-off-by: Stephen Boyd <sboyd@kernel.org>