Instead of making a pure 32bit write to a read/modify/write
operation with sr32 use writel directly. This saves a few bytes
of binary space.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>