The only thing I would add to what's already been said is that it doesn't sound like you're abusing your Adderall. Abuse generally means that you're taking more than your prescription recommends or you're taking something you're not supposed to. If you find that your current dose is insufficient and start taking a second dose during the day, then you really have to worry about words like 'addiction' and 'abuse.'

Your withdrawal symptoms are a concern, but it sounds like you just have to work with your doctor and find the right dosage or perhaps a different medication to minimize your issues. I've never taken Adderall, but from what I've seen on this board, it does seem to be more likely to cause overt symptoms of dependence than some of the other ADD meds (Concerta, Ritalin, etc.).