Hi Sarah,
I feel your pain I really do and you aren't alone. This is a very typical scenario!
When I was eCommerce manager for a company I started their affiliate program, managed it on top of all of the other hats I wore. I did all the PPC, SEO, managed the web design devision, wrote the compnay newsletter and also managed the web customer service department. (returns, bad credit cards, customer servive email questions, etc.)
As they kept piling on more responsibilities I found I only had 10 minutes a day for the affiliate channel IF I was lucky.
So I QUIT and started my own affiliate management company. Now I don't do management any more. I specialize in recruiting, promotion and PR. But I am still a one woman show - only I am my won boss and every single thing I do helps my company grow not his! 
Not saying you should quit, just sharing my story. Do you belong to Shawns affiliatemanager.net forum? There was a similar question a few months ago and some good discussion.