How long have you been on the zoloft? I guess that's the major quiestion. If you've been on it a year or so, then yes, it's possible that you could get off it and be just fine. Just be prepared for the changes in your brain chemistry when you taper off the drug.
However, you should talk to your doctor. That's the person ultimately responsible for your safety and for what drugs you're on.
Good luck, my friend.
Dan