Feathering won't turn off

[Ferdrimmler]

I'm trying to crop the white border around this hat.  I used select-by-color with a very high threshold to select anything with white in it.
You see that there are some areas inside the hat selected.



I invert the selection with ctrl-I, then use "shift" to add the selected area to the selection.  (Curiously, the tool shows that it's in mode "replace", which it isn't, as you'll see in the next photo.)



The pixels inside that parallelogram have been added to the selection:


If at this time I delete the selection, notice the dark smudges inside the hat (one is the upper-left corner) where it doesn't remove all the color.
It appears to be feathering.


Undo, ctrl-I, delete the white: also deletes the same amount from the hat as was left behind in the previous delete:



This has got me totally stuck!  I've tried turning antialiasing on and off, tried using "subtract" mode instead of "add" mode--no matter what I do, I can't get GIMP to delete just the area inside the selection.  Note feathering is turned off both of all the selection tools used.
I'm on a tight deadline and have no way to proceed.