When removing empty directories do not remove the base directory. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>