Patterns of Select ➤ Distort... look similar, so like Ofnuts, I think you used this function
but you did also used the Eraser Tool or a Brush Tool depending your layers' stack, with a brush or even more different brushes, but look below with a similar one
In the end your border is at least a 2 steps process
but you did also used the Eraser Tool or a Brush Tool depending your layers' stack, with a brush or even more different brushes, but look below with a similar one
In the end your border is at least a 2 steps process