I feel ya. My truck's paint is completely covered with swirls and light scratches.... and I usually wash it once a week. Maybe it's the thin clear coat as IronGrave mentioned?? It seems every time I wash the pollen, dust and dirt off, there is another coat of swirl marks and random scratches...this paint definitely scratches a lot easier than my previous cars (midnight blue and black).
Hit the paint with an orbital buffer and some wax. It reduced the appearance of swirls by 75% on my paint. The scratches from all the mud and rocks flipping up are always going to be there, though.
Our 2006 Altima SE-R is black and guess who maintains it's appearance? Yep me.
Best thing you can do is hand wash it only (no automatic car washes unless they are the brushless kind).
Use a GOOD car wash soap, not the $2.99 a gallon stuff and a microfiber mitt or towel.
And don't skimp on soap when you fill up the bucket, that's what lets the mitt or towel glide over the paint and not drag.
From there all you can really do is keep it waxed and use compounds/polishes to remove anything you don't like in the paint when you find it.
It takes work, but black is a great color when it's clean: