Lessons Learned: Enumerating Large Lists Can Cause Difficult Problems Without Proper Planning

One author of this book worked with a company that had 9,000 users and was growing rapidly. This company received an average of 100 resumes for every open position that it advertised. To help its recruiters manage the heavy resume load, the company contracted with a consultant to customize a document library that would be the landing location for resumes that were submitted through the company Web site. The library had code that immediately notified the correct recruiter for each open position when a new resume was submitted.

The company did not stress test the code that came from the contractor. It simply implemented the customized document library in production and found that it worked exactly as desired. However, the company soon realized that the list of resumes was becoming increasingly difficult to enumerate. After about 1,800 resumes had been received, the list of documents in the library was no longer enumerable. No matter how the company tried to enumerate the documents, the list couldn’t be enumerated. Because the code didn’t take the recruiter directly to the individual document, the recruiters were no longer able to open new resumes, which was obviously cause for concern.

To avoid losing the documents, the company called the consultant back and asked him to pull all of the documents directly from the SQL database. Then the consultant had to re-code the library to move the documents to other multiple locations based on new design specifications he built into the revised Web part.

This customer not only learned about the difficulties of enumerating large lists of documents, but also about the value of testing custom code and stressing that code beyond expected limits.