This is a known issue with the 2022 & 2023 Frontier. The handle has too much lateral play and makes contact with the key hole cover, and pushes it off with time.
My 2022 Pro4X had the cover come off completely, and the handle assembly was replaced by the dealer while still under the 36k warranty.

 

Same here bought my 23 SV new,went to pick it up at dealers .It fell off when I opened the door.

 

Thanks all.
I ended up taking the cover off and applied some JB Weld then reinstalled it. I'm happy with the results.

 

Is there a blank keyhole cover available? I installed one on my G37 to cover the drivers side up. It was a stock passenger side but was interchangeable.

 

As of about a year ago, you were not going to find just that part - at least not in a matched color. Resist pulling toward rear as you open and it may mitigate some. Fronts are different from rears in how much wiggle there is (if curious). More detail here, but whole thread is recommended

As for the "rub", I'm not sure its limited to 22/23. In contradiction to myself, I apparently said:

Observation: my '24 has lube between handle and lock cover on front doors only. My '22 didn't have any.