That happens, yes, but then what you have to do is right-click the icon and select “Search for updates”. This will then force the update notifier to reload, and in seeing that there are no new updates anymore, then depending on your system tray settings, it’ll turn green again or ─ as @KesLE6 noted ─ disappear.
Normally you wouldn’t run into that situation because the updates are usually bundled and will encompass a newer kernel, which means that you must reboot after updating. But sometimes there are only a few packages to update ─ e.g. an urgent security fix ─ and then it may indeed happen that the update notifier icon remains red.
But even that is not permanent, because it’ll automatically and periodically poll for updates anyway, and so it would then discover on its own account that there are no new updates anymore. The only thing is that depending on how often it is set to poll for updates, it might still take hours before it realizes that you’re already up-to-date.