This will simplify drivers which would only unregister the clk in their
remove() op.

Signed-off-by: Michael Walle <michael@walle.cc>
Link: https://lore.kernel.org/r/20201105192746.19564-3-michael@walle.cc


Signed-off-by: Stephen Boyd <sboyd@kernel.org>