Seminal work on colorectal cancer (CRC) incidence modeling argued that the slope of the linear log-log CRC incidence with age implies that 6 or 7 somatic mutations are required for transformation to CRC. Subsequent modeling efforts have built on this theme by using models of linear, sequential transformations for predicting CRC incidence in the population in general and specific demographic groups. These random mutation models rarely explicitly account for the intermediate adenoma stages in the adenoma to carcinoma sequence. By including the colon history of individuals, additional patient-specific information can be used to increase the resolution of the CRC models. This data is also relevant since adenoma dysphasia and numbers have been shown clinically to correlate with the likelihood of subsequent advanced adenomas - with CRC likely following thereafter. Our goals are to (1) include polyp formation in a CRC development model and (2) to account for the heterogeneous nature of polyp genetics by incorporating multiple routes of formation inherent using a mutation network model. It is then shown how the model, tuned to both polyp and CRC prevalence, can be used to refine predictions of CRC incidence based on the results of colonoscopies rather than demographics alone.