How do I fill and avoid smooth brush screwing things up?

[Ofnuts]

Not only the black shadow, you also have gaps between yellow and black.

This should be avoided by using the adequate painting technique:

• Make what you want to paint transparent (if not already so) using Color-To-Alpha or Color Erase mode.
  
• This will create edges with partially transparent pixels
  
• Fill the inside with a  paint tool in Behind mode, it will fully paint the fully transparent pixels and "complement" the partially transparent ones exactly as needed.
  

See here for a more detailed explanation.