Memoji not moving from frequent used section on iPhone XR Pro keyboard

how do I get these Memoji’s out of the frequent used section on my keyboard ? They won’t move down the list AND they just popped there without me even clicking on them ! I have an iPhone XR Pro (yes I know it’s old and I will upgrade it before end of the year) but I don’t like these stickers just staying there and not moving!! If they moved towards the less used sections, then it would be fine as I know they would eventually go off the frequent used ones!! Any ideas please??