Otherwise the remove function will be called twice leading to bad effects. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>