There is various system parameters which influence the time taken to flush the queue to the device.
If in doubt issue a sync command in the terminal - when the prompt returns - it is safe to remove the device.
A couple of threads on the subject - there is more to find if you search for it