About
    • AMMDI is an open-notebook hypertext writing experiment, authored by Mike Travers aka mtraven. It's a work in progress and some parts are more polished than others. Comments welcome! More.
Search
Page Contents
Full

Symbol Disease

20 Sep 2025 - 13 Nov 2026
Open in Logseq
    • Or formalism disease. This is the bug of assuming that your symbolic representation somehow are an unproblematic image of some outer reality.
    • First pointed out to me by Agre and Chapman. Articulated most clearly in that McDermott paper, Artificial intelligence meets natural stupidity | ACM SIGART Bulletin
      • As a field, artificial intelligence has always been on the border of respectability, and therefore on the border of crackpottery. Many critics have urged that we are over the border.
      • Unfortunately, the necessity for speculation has combined with the culture of the hacker in computer science to cripple our self-discipline. In a young field, self-discipline is not necessarily a virtue, but we are not getting any younger.
      • Wishful Mnemonics A major source of simple-mindedness in AI programs is the use of mnemonics like "UNDERSTAND" or "GOAL" to refer to programs and data structures. This practice has been inherited from more traditional programming applications, in which it is liberating and enlightening to be able to refer to program structures by their purposes.
      • Alright I'm in danger of quoting the whole paper so will stop. Very Relevant to My Interests, and interesting to think about in the context of LLMs and vibecoding. It sounds positively quaint to think you can or should think of your program in mechanistic rather than intentional terms.
      • OK one more
    • Charniak <1972> pointed out some approaches to understanding stories, and now the OWL interpreter includes a "story-understanding module". (And, God help us, a toplevel "ego loop". <Sunguroff, 1975>)
      • God didn't help, because "agentic systems" have exactly that.
          • Ask Claude
            • What actually happened fifty years later:
            • In December 2024, Anthropic published engineering guidance that became, in their own words, the "founding document of the loop era." Its core sentence: "Agents are typically just LLMs using tools based on environmental feedback in a loop." You give one AI a goal, access to tools, and a way to see the results of its own actions — and it plans, acts, checks, and repeats.
            • Rather than a single prompt-to-response interaction, these systems operate in a continuous loop — they sense the environment, plan the next steps, act with tools, observe outcomes, and then revise their plan.
            • The cognitive core creates plans and breaks goals into sub-tasks, running in a sense–plan–act–observe–learn loop.
            • That is, structurally, the ego loop — a top-level cycle in which the system perceives its situation, deliberates, acts, and loops back to perceive again. The difference between Sunguroff's 1975 "ego loop" and a 2025 "agentic loop" is largely one of capability and scale, not architecture. McDermott's prayer went unanswered: the field did not stop to ask whether the name matched the thing. It just waited for hardware and data to close the gap — and then resumed using exactly the same vocabulary.
            • The underlying pattern McDermott identified — take a concept from cognitive science or literature, name a module after it, ship it — remains the dominant mode. The names have proliferated: reasoning core, memory, planning, reflection, ego → agent. The honest question he posed in 1976 is still the right one: does the mechanism actually do what the name claims?