[1] Craig Boutilier, ``Learning Conventions in Multiagent Stochastic Domains Using Likelihood Estimates,'' in Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence (UAI-96), edited by Eric Horvitz and Finn Jensen (Morgan Kaufmann, San Francisco, 1996).

[2] Michael Bowling and Manuela Veloso, ``Rational and Convergent Learning in Stochastic Games,'' in Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, Seattle, WA, 2001.

[3] Caroline Claus and Craig Boutilier, ``The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems,'' in Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98) and of the Tenth Conference on Innovative Applications of Artificial Intelligence (IAAI-98), Menlo Park, July 26--30, 1998 (AAAI Press).

[4] Junling Hu and Michael P. Wellman, ``Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm,'' in Proceedings of the Fifteenth International Conference on Machine Learning (Morgan Kaufmann, San Francisco, 1998).

[5] Michael L. Littman, ``Markov Games as a Framework for Multi-agent Reinforcement Learning,'' in Proceedings of the Eleventh International Conference on Machine Learning (Morgan Kaufmann, San Francisco, CA, 1994).

[6] John W. Sheppard, ``Colearning in Di#erential Games,'' Machine Learning, 33 (1998) 201.

[7] Yoav Shoham and Moshe Tennenholtz, ``On the Emergence of Social Conventions: Modeling, Analysis and Simulations,'' Artificial Intelligence, 94(1--2) (1997) 139--166.

[8] R. Sun and T. Peterson, ``Multi-agent Reinforcement Learning: Weighting and Partitioning,'' Neural Networks, 12(4--5) (1999) 727--753.

[9] Ming Tan, ``Multi-agent Reinforcement Learning: Independent vs. Cooperative Agents,'' in Proceedings of the Tenth International Conference on Machine Learning (Morgan Kaufmann, San Francisco, 1993).

[10] Kagan Tumer and David Wolpert, ``Collective Intelligence and Braess' Paradox,'' in Proceedings of the Seventh Conference on Artificial Intelligence (AAAI-00) and of the Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI-00) (AAAI Press, Menlo Park, CA, 2000).

[11] Luc Steels, ``The Origins of Ontologies and Communication Conventions in Multi-agent Systems,'' Autonomous Agents and Multi-Agent Systems, 1(2) (1998) 169--194. Complex Systems, 14 (2003) 315--334 A Distributed Learning Algorithm for Communication Development 333

[12] Ezequiel A. Di Paolo, ``Social Coordination and Spatial Organization: Steps Towards the Evolution of Communication,'' in Proceedings of the Fourth European Conference on Artificial Life ECAL'97, edited by Phil Husbands and Inman Harvey (The MIT Press, Cambridge, MA, 1997).

[13] Bruce MacLennan, ``Synthetic Ethology: An Approach to the Study of Communication,'' in Artificial Life II, volume X of SFI Studies in the Sciences of Complexity, edited by C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen (Addison-Wesley, Redwood City, CA, 1991).

[14] Jason Noble, The Evolution of Animal Communication Systems: Questions of Functions Examined through Simulation, Ph.D. Thesis, University of Sussex, Sussex, UK, November, 1998.

[15] Martin A. Nowak and David C. Krakauer, ``The Evolution of Language,'' in Proceedings of the National Academy of Sciences of the United States of America, 96 (1999) 8028--8033.

[16] Gregory M. Werner and Michael G. Dyer, ``Evolution of Communication in Artificial Organisms,'' in Artificial Life II, volume X, edited by C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen (AddisonWesley, Redwood City, CA, 1991).

[17] Aude Billard and Gillian Hayes, ``Learning to Communicate through Imitation in Autonomous Robots,'' in Proceedings of the Seventh International Conference on Artificial Neural Networks ICANN 97 (Springer, Berlin, 1997).

[18] Aude Billard and Gillian Hayes, ``Drama, a Connectionist Architecture for Control and Learning in Autonomous Robots,'' Adaptive Behavior, 7(1) (1999).

[19] H. Yanco and L. Stein, ``An Adaptive Communication Protocol for Cooperating Mobile Robots,'' in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior, edited by H. L. Roitblat, J-A. Meyer, and S. Wilson (The MIT Press, Cambridge, MA, 1993).

[20] Michael Oliphant, Formal Approaches to Innate and Learned Communication: Laying the Foundation for Language, Ph.D. Thesis, University of California, San Diego, CA, 1997.

[21] J. Hurford, ``Biological Evolution of the Saussurean Sign as a Component of the Language Acquisition Device,'' Lingua, 77 (1989) 187--222.

[22] Bart De Boer, The Origins of Vowel Systems: Studies in the Evolution of Language. oup, 2001.

[23] L. Steels, ``A Self-organizing Spatial Vocabulary,'' Artificial Life Journal, 2(3) (1996). Complex Systems, 14 (2003) 315--334 334 E. D. de Jong and L. Steels

[24] Luc Steels, ``The Synthetic Modeling of Language Origins,'' Evolution of Communication, 1(1) (1997) 1--34.

[25] David K. Lewis, Convention: A Philosophical Study (Harvard University Press, Cambridge, MA, 1969).

[26] Martin J. Osborne and Ariel Rubinstein, A Course in Game Theory (The MIT Press, Cambridge, MA, 1994).

[27] Ariel Rubinstein, ``Why Are Certain Properties of Binary Relations Relatively More Common in Natural Language?'' Econometrica, 64(2) (1996) 343--355.

[28] Thomas C. Schelling, The Strategy of Conflict (oup, New York, 1963).

[29] Edwin D. De Jong, Autonomous Formation of Concepts and Communication, Ph.D. Thesis, Vrije Universiteit Brussel, Brussels, Belgium, June 2000. Available from: http://www.cs.uu.nl/#dejong/.

[30] Karl F. MacDorman, ``Partition Nets: An E#cient On-line Learning Algorithm,'' in ICAR 99: Ninth International Conference on Advanced Robotics, 1999.

[31] Karl F. MacDorman, ``Proto-symbol Emergence,'' in Proceedings of IROS-2000: International Conference on Intelligent Robots and Systems, 2000.

[32] J. Batali, ``Computational Simulations of the Emergence of Grammar,'' in Approaches to the Evolution of Language: Social and Cognitive Bases, edited by J. R. Hurford, M. Studdert-Kennedy, and C. Knight (Cambridge University Press, 1998).

[33] Takashi Hashimoto and Takashi Ikegami, ``Emergence of Net-grammar in Communicating Agents,'' BioSystems, 38(1) (1996) 1--14.

[34] Simon Kirby, Function, Selection, and Innateness (Oxford, New York, 1999).

[35] Simon Kirby, ``Spontaneous Evolution of Linguistic Structure---An Iterated Learning Model of the Emergence of Regularity and Irregularity,'' IEEE Transactions on Evolutionary Computation, 5(2) (2001) 102--110.

[36] Luc Steels, ``The Origins of Syntax in Visually Grounded Robotic Agents,'' Artificial Intelligence, 103(1--2) (1998) 133--156.

[37] Seth Bullock, ``A Continuous Evolutionary Simulation Model of the Attainability of Honest Signalling Equilibria,'' in Proceedings of the Sixth International Conference on Artificial Life, edited by Christoph Adami, Richard K. Belew, Hiroaki Kitano, and Charles Taylor (The MIT Press, Cambridge, MA, 1998).

[38] Melanie Mitchell, An Introduction to Genetic Algorithms: Complex Adaptive Systems (The MIT Press, Cambridge, 1996). Complex Systems, 14 (2003) 315--334 A Distributed Learning Algorithm for Communication Development 335

[39] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction (The MIT Press, Cambridge, MA, 1998). Complex Systems, 14 (2003) 315--334