Models, AI Systems, and Reasoning

Some new terms are emerging in Machine Learning and AI Engineering. In order to collaborate better with teammates, I wanted to nail down a few definitions.

ML models and AI systems are increasingly blending together. The creation of an ML Model might require an AI System (or application or data pipeline), but an AI system (which is more like an application) will certainly use one or more models.

Currently, the term “inference” is used to mean the process of producing output from an LLM (or multimodal language model). “Reasoning” is more of AI system (or application) process. ML models perform the inference, while applications wrap that inference with reasoning strategies to produce user experiences.

Inference-Time Reasoning (for LLMs, as introduced by Strawberry o1): The model reviews its own response as it is generating it, feeding back on itself to allow branching into alternative approaches. The implementation of o1 is unknown, but likely there are a couple models and some application code to orchestrate the reasoning and responses. The ability to evaluate different paths is trained using Reinforcement Learning. OpenAI categorizes progress into levels: 1) Chat, 2) Reasoners, 3) Agents, 4) Innovators, 5) Full organizations. We are at level 2—”Reasoners”—and it’s just beginning, suggesting exciting developments ahead.

Reinforcement Learning (RL): RL is a technique that can be applied to either models or agents (which are more like applications). It allows the system to learn to take actions in an environment to maximize cumulative rewards based on feedback from prior actions.

Agent: An agent is an AI system (essentially, an application on top of a model) that perceives its environment, makes decisions, and takes actions to achieve a goal. Agentic behavior means acting autonomously and purposefully. Agents themselves do not perform inference or reasoning; they rely on LLMs or other models for that.

The CoALA paper (Cognitive Architectures for Language Agents) introduced a framework for understanding and designing language agents, focusing on both application components and advanced prompt-chains to elicit reasoning-like behaviors. The term “Cognitive Architecture”, now solidified in Sequoia’s post, refers largely to these prompt-chains (e.g., Chain of Thought, Tree of Thought). These are integral to agent architectures categorized by CoALA.

I loved the “Language Agents: From Reasoning to Acting” rundown by Shunyu Yao on different cognitive architectures. Spoiler: many topics we discuss in the office are the next obvious research directions—agent-computer interfaces, benchmarks, memory, robustness, and training data.

What is Picco? Picco is a code generation engine for its own AI-first software architecture (code as tasks + skills utilizing LLMs/multimodal foundation models). Some tasks are agentic, such as self-healing capabilities and web research—where skills invoke LLMs for reasoning.

Currently, Picco does not use reinforcement learning. While it could be considered an agent, it is more accurately described as an LLM chat wrapper executing hundreds of semi-agentic tasks and skills. Some might even consider it a “multi-agent” system.