3 Credit Hours

EA

Ed Addison

Professor of the Practice

[email protected]

Governance of multi-agent AI systems is the most sought-after AI topic in industry today. An AI system is a coordinated set of models, data, agents, workflows, interfaces, and control mechanisms that together perform tasks, make decisions, or take actions—often with some degree of autonomy—within a defined environment. This course treats such systems as organizations of interacting agents, humans, workflows, and data, and addresses how they are designed, governed, monitored, and continuously controlled.

Prerequisites

EM 534 (AI for Engineering Managers) is recommended but not required. Students who have not taken EM 534 should have prior exposure to machine learning or generative AI beyond LLM prompting.

Learning Objectives

Students will be able to:

• Distinguish among AI tools, AI systems, and AI organizations
• Decompose an enterprise workflow into agents, deterministic services, and human steps, and justify each boundary
• Diagnose failure modes in multi-agent AI systems
• Construct governance frameworks for distributed agent systems
• Define authority, delegation, and decision rights for hybrid human–agent organizations
• Manage the agent lifecycle from creation through supervision, restriction, and retirement
• Design guardrails for safe, auditable execution
• Recognize and defend against adversarial behavior originating inside the agent organization
• Evaluate model capabilities, costs, and risks from a managerial perspective
• Analyze emergent behavior and cascading failure risks
• Explain how multi-agent behavior changes qualitatively with scale, and what governance of very large agent populations requires
• Design control planes for enterprise AI systems
• Distinguish risks that enterprise controls can govern from risks that originate outside the governance boundary
• Assess architectures for autonomous active defense against rogue agents, including the governance of the defenders themselves
• Explain why model-level and provider-level safety measures do not constrain adversaries operating open-weight models on private infrastructure, and evaluate the alternative control points that remain
• Lead AI incident response when intelligent systems fail in production
• Apply regulatory frameworks (EU AI Act, NIST AI RMF, ISO/IEC 42001) to real AI systems
• Conduct scenario-based stress testing
• Communicate Responsible AI strategies at the executive level

Course Requirements

All four assignments are completed in small groups and form a single sequence: teams design an agent organization, break it, govern it, and defend it before an executive audience. Each assignment builds directly on the last, so by the final presentation every team has taken one AI organization through its full life. Detailed descriptions follow the course schedule.

Textbook

No textbook purchases are required. The instructor provides all required readings—papers, standards excerpts, and book chapters—as PDFs through the course site at no cost to students. The Course Reference List at the end of this syllabus identifies the key references from which those materials are drawn; students who wish to go deeper are encouraged, but never required, to pursue the full works.

A note on three sessions. Sessions 21, 23, and 24 address problems the literature has not yet settled: governing very large agent populations, autonomous defense against rogue agents inside enterprise networks, and misuse of agent swarms outside any vendor’s control. Readings there combine adjacent literature with the instructor’s framing and present open research problems.

Created: 10/01/2026.