Joe Lau's wiki: Main/Connectionism And LOT

Readings

• Garson, James. Connectionism. The Stanford Encyclopedia of Philosophy. stanford:connectionism.
• Gary F. Marcus (2001). The Algebraic Mind : Integrating Connectionism and Cognitive Science MIT Press. isbn:0262133792
• David Chalmers. (1990). Syntactic Transformations on Distributed Representations. Connection Science, 2, Nos 1 & 2, 53-62.
• van Gelder, T. J. (1999). Distributed versus local representation. In R. Wilson & F. Keil (Eds.) The MIT Encyclopedia of Cognitive Sciences. Cambridge MA: MIT Press, 236-8.
• Ramsey, W., Stich, S. P., & Garon, J. (1991). Connectionism, eliminativism, and the future of folk psychology. In W. Ramsey, S. P. Stich, & D. E. Rumelhart, Eds. (1991). Philosophy and connectionist theory. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Fodor and Pylyshyn

• ConnectionismFodorPylyshyn

Is connectionism an alternative to LOT?

Some possibilities

1. Connectionism is a theory at the hardware level. It tells us how classical cognitive architectures are neurally implemented.
2. Connectionism tells us how a classical architecture is implemented by connectionist networks at a lower level of algorithm and representation.
3. Connectionism denies the existence of a classical architecture. The architecture at the level of algorithm and representation is a connectionist one.
4. Connectionism is a theory at the level of algorithm and representation. But the correct cognitive architecture is a hybrid one that includes both classical and connectionist architectures.

Regarding scenario #1

See the discussion in Crick, F (1989). The recent excitement about neural networks Nature, 337, 129-132. doi:10.1038/337129a0

• Most connectionist networks are biologically unrealistic in many ways.
  • Neural connections are either excitory or inhibitory, but not both.
  • Many training rules are biologically unrealistic. For example, back propagation does not scale well, and cannot deal with one-shot learning.
  • Real neural networks may have lots of recurrent connections, unlike feed-forward networks.

Regarding scenario #2

• Connectionist representations: localist vs distributed.
• Localist representation - one node for one meaning. Can it deal with systematicity and productivity?

Distributed representations

Terminology

• Spatial distribution - wikipedia:Tetaumatawhakatangihangakoauaotamateaurehaeaturipukapihimaungahoronukupokaiwhenuaakitanarahu
• Two types of distributed representations - compositional vs non-compositional.
• Structured, distributed representations can implement compositional representations.

van der Veldea & de Kamp. Neural blackboard architectures of combinatorial structures in cognition. Behavioral and Brain Sciences

• Micro-feature representations

Distributed memory in Ramsey, W., Stich, S. P., & Garon, J. (1991).

Network architecture

Input representations

Connectionism: friend or foe?

Regarding scenario #3

Distributed representations are powerful and useful. But can they explain cognition without LOT?

• Objection #1: Unstructured distributed representations cannot explain systematicity.
  • See Chalmers (1990) for a reply.
• Objection #2: Where do distributed representations come from? (e.g. RSG model, RAAM)
• Objection #3: LOT needed to explain free transformation in central cognitive processes (e.g. conscious thoughts).

Compare: "Where do zip files come from?", "How can you change one file in a zip archive without changing others?"

GodelNumberingToyExample

Regarding scenario #4

• An example of a hybrid approach - LOT in working memory vs unstructured representations in long-term memory.
• Is this an example of LOT?

Category.Mind