Brock, W., Hommes, C. and Wagener, F. (2005). Evolutionary dynamics in markets with many trader types Journal of Mathematical Economics, 41(1/2):7--42.

This paper develops the notion of a large type limit (LTL) describing the dynamical behavior of heterogeneous markets with many trader types. It is shown that generic and persistent features of adaptive evolutionary systems with many trader types are well described by the large type limit. Stability, bifurcation routes to instability and strange attractors in a simple evolutionary financial market model are studied. An increase in the 'intensity of adaption' or in the diversity of beliefs may lead to deviations from an unstable RE fundamental benchmark and excess volatility. A large evolutionary system may thus become unstable and complicated dynamics may arise when agents become sensitive to small differences in fitness.