like anything else, it's definitely something you can outgrow.
personally, i think it's good to grow as a person and experience new things and try new hobbies and not just hunker down in purely anime
and video games and feel defensive if anyone suggests trying other things.
there was a time when i would have defined myself as 'an anime fan', but that time is over, since i'm into so many other things just as much or more, not to mention i tend to prefer manga more than anime, nowadays. that doesn't stop me from geeking out over any new anime or manga series i discover that i like just as much as ever. as you grow as a person, sometimes things other than anime and going to anime conventions become a larger priority and i don't think there's anything wrong with that.
if you're still just as into it, and you're still obsessed with anything 'anime' and going to cons is something you look forward to all year, then go nuts. it does irk me when people assume that if you don't make watching every episode of the latest over-hyped series or check out all the new anime of each season that you were never really into anime or much of a fan. i was super-into it, and i'm still totally into anything i come across that's good, but i have other things i enjoy just as much now, and a life, so i can't just spend endless hours watching every mediocre series or go broke buying badges for every single anime convention in america. i don't think that makes it a phase i've grown out of so much as it means that i've grown as a person.