There's nothing wrong with it. Some people just like to be mean. And the people who are jerks aren't worth your time anyway. I second the posts that say sometimes it's the regions, maybe there is a theme or something? When I started in SL, I only had a human avatar and I would go to furry spaces and get the cold shoulder because I didn't have a furry avatar. Same thing when I had a furry avatar, or because I have shorter avatars, sometimes they are mistaken for child avatars. But there are other times that are fine. I roam around as a wolf sometimes, and it's fun to see people say "gasp! look it's a wolf!" as if they were seeing one in the wild, it was pretty cool lol.
Just do you and be you, and if people are being jerks then just walk away and go exploring. There's a whole world out there. Don't let them get you down ♥