Ninja'd while making some screenshots so here it is in pictures 
You need an alpha channel to provide 'transparency' Layer -> Transparency -> Add Alpha Channel If that entry is greyed out then you already have the alpha. An indication in the layers dock is the layer name. Bold is no Alpha, regular is with alpha.
No alpha and cutting / erasing does use the BG colour. Best if you leave it white, The colour does remain as information in a PNG (depending on export options)
Unlikely in this case but alpha can be displayed in various ways Edit -> Preferences -> display
edit: Just seen
The alpha channel is per layer for any particular image. Saved in the xcf Gimp file
You need an alpha channel to provide 'transparency' Layer -> Transparency -> Add Alpha Channel If that entry is greyed out then you already have the alpha. An indication in the layers dock is the layer name. Bold is no Alpha, regular is with alpha.
No alpha and cutting / erasing does use the BG colour. Best if you leave it white, The colour does remain as information in a PNG (depending on export options)
Unlikely in this case but alpha can be displayed in various ways Edit -> Preferences -> display
edit: Just seen
Quote:Is it just the set of images I am using now ? I thought I messed up some GIMP settings...
The alpha channel is per layer for any particular image. Saved in the xcf Gimp file