Thanks for all the replies! Perhaps I didn't explain correctly or called it the wrong thing. Someone else who witnessed what was going on told me that the griefer more than likely built a large box and added the invisiprim texture to it.
Here is everything that happens. Hopefully this helps to clarify:
1. The griefer himself is invisivle, including name tag
2. He uses a follower so no matter where I go he is right there with me.
3. Whatever he is using also makes me invisible and anyone else in the radius. Only my eyes, ears and necklace show.
4. The first time this ever happened I didn't know what was going on but happened to click in mid air around myself and that's when I noticed that it highlighted an "object" and was not just blank air around me.
5. Sometimes the tool he is using causes other objects or backgrounds (skies, etc) to flicker in and out briefly.
Hope that gives some better insight.