i also have been at this many years, and crawls from google, yahoo and any other L.P.E. are determined by the authority of the page. Spiders crawl pages more frequentl when a page is adding pages, content and making regular changes because they learn habbits of sites. Sites that are updated frequently get crawled more often. pages that get new links get crawled more often.
When a spider hits a 404 page, it does not remove it from the index the first trip. it may not remove it for 6 or 7 crawls. so, if you have a page that has been indexed a year and has had no new links show up, and the content or design has not changed for a year, then the crawl rate for that page is going to be not so frequent and it may take 6 weeks to 3 months for a spider to crawl it. now, with a 3 month interval between crawls, 4 crawls would take a year, and with the intervals being spread so far apart, it would not be unrealistic for the server to be down 2 of the 4 yearly crawls.
also, can you drop links validating your opinion is precise?
The Engineers at the search engines are not dummies, they think through this process when setting the algo's, and it is a way to protect from removing pages that were having temp. server problems.
Then, yahoo also slowed their crawl rate because of complaints of to heavy of a crawl rate back in august of this year.
Yahoo! Search Blog: Weather Report -- Major Crawl Improvements
Want to get a page removed, post some links to it on CL. or digg.