Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Question regarding Precision Setting
#2
Yes, using 32-bit floating-point takes 4 times more RAM than 8-bit. On the other hand, there is much less color loss during color operations, and you avoid round off errors than show as artefacts. The latter is fairly frequent when you use layer blend modes. Try this yourself:
  • Create a 800x800 image in 32-bit FP precision
  • On the  base layer, make a black to white horizontal gradient
  • Add a layer, make a black to white  vertical gradient
  • Put the top layer in Grain extract mode: the gray diagonal area should remain smooth
  • Use Image ➤ Precision to switch to 8-bit integer: banding/moiré appears in that diagonal area
This "precision" only applies on how the data is stored in layers. All computations are done in 32-bit FP, which means conversions happen before and after... unless you use 32-bit FP, which, not requiring conversions, can be a bit faster, too.
Reply


Messages In This Thread
Question regarding Precision Setting - by ajax - 07-02-2022, 05:38 PM
RE: Question regarding Precision Setting - by Ofnuts - 07-02-2022, 09:13 PM

Forum Jump: