Content as Behavior: A Diagnostic Lens for AI Readiness

In early 2024, New York City’s MyCity chatbot told business owners they could keep employee tips, fire workers for reporting harassment, and serve rodent-contaminated food—all of which are flatly illegal. Air Canada’s chatbot invented bereavement fare policies that ultimately cost the airline a legal judgment.

These weren’t simply AI (artificial intelligence) failures. They were diagnostic outputs revealing how content systems actually behave when humans can no longer fill-in the gaps.

Traditional content audits measure what exists: metadata presence, template coverage, taxonomy completeness. These assessments quietly assume humans will continue connecting dots—reading between the lines, making judgment calls, compensating for ambiguity.

AI removes that safety net. It can’t rely on institutional memory, tribal knowledge, or human judgment to mask structural gaps. It forces content systems to prove they can be interpreted at scale on their own.

As The Content Advantage (3rd edition) by Colleen Jones demonstrates, content systems are infrastructure, not collections of discrete assets. When that infrastructure is fragile (fragmented governance, inconsistent maintenance, unclear authority), AI doesn’t create chaos. It exposes what was already there.

A Model for Diagnosing AI Readiness

To help bring an organization closer to AI readiness, I developed a model that I call SignalScale. It’s a diagnostic lens for understanding how content systems behave when AI interprets them. As a complement to maturity models like those from Content Science, it examines the relationship between content behavior and what AI reliably produces in response.

Every content system exhibits a behavior pattern shaped by consistency, structure, and coherence. When AI is introduced, those patterns become visible. AI reacts to content behavior predictably: fragmented inputs produce unstable output; structured inputs produce stable output. SignalScale makes that relationship explicit.

Across enterprise environments, three dominant behavior patterns surface when content systems are forced into machine interpretation:

• Fragmented behavior → contradictory or hallucinatory output → broken trust
• Inconsistent behavior → unpredictable output → scaled rework
• Structured behavior → stable output → automation readiness

These aren’t aspirational states. They’re observable behavior patterns that emerge the moment content is forced into machine interpretation. The AI output makes it clear which one you’re dealing with. Let’s take a closer look at three common patterns.

Pattern 1: Fragmented

The Content System Behavior

Content lives in silos. There is no unified taxonomy. Templates exist as tribal knowledge rather than enforced structure. Subject matter experts work in isolation. Metadata is inconsistent or missing entirely. Humans compensate constantly—they know which PDF supersedes which webpage, that Legal’s definition of “customer” differs from Sales’, and how to triangulate truth from contradiction.

What AI Produces

Aggressive hallucinations. Complete contradictions. Confident answers that are factually wrong. The model treats every source as equally authoritative and synthesizes meaning from the whole. When sources contradict, it guesses. When structure is absent, it invents relationships.

Real-World Case

NYC’s MyCity chatbot sourced business regulations from dozens of agency systems. Each of those systems had its own publishing cadence, terminology, and interpretation. No content model governed structure. No metadata linked related policies. No hierarchy established authority when conflicts emerged. The AI synthesized fragments into responses that sounded authoritative but violated actual law. The city added human review to every response, turning the chatbot into an expensive suggestion engine.

Risk Signal + Implication

Legal liability. Regulatory violations. Direct customer harm. Trust collapse on first contact.

If humans are compensating for content chaos today, AI will expose it immediately and at scale.

Pattern 2: Inconsistent

The Content System Behavior

Some governance exists. Some templates are used. Some reuse happens. But exceptions dominate. Projects start well, then drift. Metadata is partially complete—useful sometimes, unreliable at scale. Different departments maintain their own variations of the same content types.

What AI Produces

Uneven output. The model performs well most of the time, building confidence. Then it delivers incorrect answers in specific scenarios—and trust collapses. The problem isn’t failure; it’s unpredictability. Users can’t tell when answers will be right or wrong because the underlying content behaves inconsistently.

Real-World Case

Air Canada’s chatbot pulled from formal policies on refunds and fare rules, but content was maintained inconsistently across systems. Policies existed in multiple locations with different wording. FAQs reflected outdated interpretations. Edge cases weren’t explicitly modeled. Human agents managed inconsistency through experience. The AI solution couldn’t, so it synthesized an answer derived from company materials that was behaviorally wrong. The airline was held liable.

Risk Signal + Implication

Erosion of user trust. Increased escalation rates. Liability exposure in edge cases. Team morale collapse from managing AI cleanup.

You’re not creating structure from nothing. You’re trying to enforce the structure the organization already has on some level.

Pattern 3: Structured

The Behavior

Templates are enforced through workflow. Metadata is required and validated at publish. Taxonomy is intentional and embedded in operations. Exceptions are documented, approved, and tracked. Content is treated as product, with ownership, standards, and lifecycle management.

What AI Produces

Stable output. Reliable responses across queries because the system provides consistent signals about authority, relationships, and context. Hallucinations are rare and traceable to specific gaps rather than systemic behavior.

Real-World Case

Organizations with structured behavior share common characteristics: rich metadata explicitly defining relationships, content systems that know what needs updating when policies change through modeled dependencies, and templates that enforce consistency at creation rather than relying on review. Learn more about what organizations with a structured approach and mature content operations do differently from the in-depth research report, What Makes Content Operations Successful in the Age of AI?

Risk Signal + Implication

The primary risk is regression—”just this once” exceptions becoming permanent, new content types introduced without governance, and metadata requirements quietly becoming optional.

If you’ve achieved structured behavior, the question becomes less whether to deploy AI and more what to automate next.

These patterns aren’t maturity levels to climb or phases to complete. They are observable behaviors that surface the moment content is forced into interpretation at scale.

The Diagnosis: Three Questions Before Deployment

Before deploying AI, diagnose your system’s behavior with these questions:

1. Authority: Can your system signal trust?
   If AI encounters contradictory sources, does your content provide signals about which to trust—or must the model guess based on recency, location, or document type?
2. Consistency: Will similar questions produce similar answers?
   If users ask the same question three ways, will AI respond consistently? Or do departments use different terms for the same concept?
3. Relationships: Does your system know what connects to what?
   If a policy changes, does your content system know what else must update? Or do dependencies exist only in human memory?

Run your critical content through the model you plan to deploy. Don’t wait for launch. Observe what it produces:

• Hallucinations point to fragmented source material or unclear authority
• Contradictions surface inconsistent maintenance or incomplete metadata
• Confident but wrong answers reveal gaps in how relationships are modeled

These aren’t failures to hide. They’re diagnostic evidence. Each traces back to upstream decisions about structure, governance, relationships. The AI is showing you exactly what your content system can and cannot support under interpretation.

What Happens Next

If you exhibit fragmented behavior: Pause deployment. Establish source hierarchies. Build minimal viable taxonomies. Create consistency in core concepts. This isn’t about perfection—it’s about creating sufficient coherence for machine interpretation.

If you exhibit inconsistent behavior: Focus on enforcement. Make governance mandatory. Strengthen metadata around relationships and authority. Document exceptions intentionally. Tighten gaps between stated standards and actual practice.

If you exhibit structured behavior: Proceed—but maintain vigilance. Watch for regression as you scale. Automate validation. Keep governance embedded in operations, not treated as a completed phase.

The cost difference is stark. Fixing content behavior before AI deployment is difficult but bounded. Fixing it after is crisis management layered on trust erosion, legal exposure, and the structural work you postponed.