I came across a GPO that had a few proxy server related settings configured in "Extra Registry Settings" which I needed to remove.  The "Extra Registry Settings" configuration does not show up in GPMC for later OS versions to edit these settings.
 
Powershell can be used to remove "Extra Registry Settings" with the "Remove-GPRegistryValue" command to specify the GPO name, the registry key, and the value.