The driver forgets to unregister the notifier in remove.
Add the call to fix it.

Signed-off-by: Chuhong Yuan <hslester96@gmail.com>
Signed-off-by: Sebastian Reichel <sebastian.reichel@collabora.com>