Upgrade to version 8.6.2 stalls at 100% for > 13 hours

[#1]

Last night we upgraded our NIOS grid from 8.5.1 to 8.6.2. We have a small grid with 2 HA Grid Masters and 4 grid members all running in VMWare. 

The upgrade appeared to go well last night, and all nodes are green in dashboard. From 'show status' in the CLI on each node, all report to be running 8.6.2 now. 

On the Grid > Upgrade page, Grid Upgrade Status shows 100%, followed by 'Upgrade to 8.6.2-49947-c076923293a0 in progress' on the message bar, and this has remained since last night about 15 hours ago. 

As part of the upgrade, as is expected, the Active role for our Grid Master failed over to the secondary node where it still remains. I was under the impression that Active would fail back to the primary node upon completion, but that may be a separate issue. 

Is it normal for NIOS to perform cleanup or other admin tasks that take more than 15 hours following what appears to be 100% completion of an upgrade? Are there any logs I can refer to in the CLI or GUI to gain insight into what might be going on and whether I should keep waiting or take other action? 

[#2]

It sounds like the upgrade is completed, and you're still within the partial revert window.  For the 24 hours after a scheduled migration is completed, it is still possible to roll back an individual appliance if needed.  The grid limits the sorts of changes that can be performed during this window.

After 24 hours that message will go away, and after that only a complete rollback would be possible.  (note that a complete rollback also means rolling back to the data & configuration from the previous version)

[#3]

We have seen this happen as well.  Working with support, the fix was to run from the grid master "set grid_upgrade forced_end" to finalize the upgrade process.

In our case this was from a defunct "ghost" Reporting Appliance member stuck in our grid, so it was endlessly waiting for that to report in before marking the upgrade complete.

Another possible cause is referenced in:  https://docs.infoblox.com/space/nios90/239206467/Managing+Upgrade+Groups