That's normal because the shutter speed is faster (which allows less light through) and the wide angled lens has a smaller sensor than the main lens, the quality will be much worse (hence the grain/noise). The effect happens when it's 120fps FHD too but not to the same extent.
What is a problem that I have just noticed is auto focus hardly works on slow mo 240fps. Even if I tap to focus it doesn't work. Works fine in 120fps, normal and pro video mode though. It could be because I'm trying it indoor at night (even though all my lights are on so it's not dark).