I wouldn’t do that, I see no benefits, that shouldn’t take a lot of space, as by definition these are empty. You would gain more space by deleting a MP3 file. On the other hand, it could be a problem, potentially, I don’t know but if this is a risk for basically no benefit, why would you?