SAP Business AI architecture stack · 4 layers plus governance

Joule and agents sit at the top, but their reliability depends on the shaded layers beneath them and the governance that spans all four.

The scale is significant. Reporting from Sapphire describes a Knowledge Graph built from roughly 452,000 SAP tables and 7.3 million fields turned into semantics agents can reason over, plus more than 200 specialized agents in an "Autonomous Suite". (Nanonets) Forrester counts 224 agents and 51 assistants announced, while noting most are still in mixed general availability, early adopter or preview status. (Forrester)

The architectural takeaway: agents are only as good as the core, the data and the integration paths they sit on. That is where preparation pays off.

Pillar 1: Make clean core your architectural foundation

Clean core has moved from a technical guideline to a prerequisite for AI. Bain notes that simpler systems with fewer customizations are cheaper and faster to maintain, and they also give data consistency, governance and integration a more stable base for AI adoption. (Bain & Company) An agent reasoning over a heavily modified order to cash process is reasoning over logic SAP's models have never seen.

SAP has also made clean core easier to measure. It retired the original three tier extensibility model and replaced it with the clean core level concept, which grades every extension from Level A to Level D based on upgrade safety, architectural integrity and alignment with clean core principles. (SAP News) Level C is treated as conditionally clean, workable with the right governance and documentation, while Level D is not clean core at all. (ASUG)

What this means for architects:

• Classify every custom object against the A to D levels before migration, not after. Treat Level D as remediation backlog.

• Default new extensions to released APIs, using on stack ABAP Cloud for tightly integrated logic and side by side extensions on BTP for decoupled apps.

• Retire what nobody uses. Usage analysis routinely shows a large share of legacy custom code is dormant, and every retired object is one less thing an agent has to work around.

• Govern continuously with SAP Cloud ALM and quality gates so the core stays clean after go live.

Pillar 2: Build a business data foundation AI can trust

Most AI pilots on SAP stall for the same reason: the data underneath is not clean or governed enough for anyone to trust it in a real decision. (McCoy & Partners) SAP's answer is Business Data Cloud, which delivers curated data products with business context intact, and the Knowledge Graph, which maps how entities, processes and decisions connect.

BDC is also SAP's route to the rest of your data estate. Through Business Data Cloud Connect, SAP links BDC with partner platforms such as Databricks and Google Cloud for a two way flow of data products without stripping away business context. (E3 Magazine)

To prepare, focus on the unglamorous work that determines whether agents get the right answer:

• Master data quality. Duplicate vendors, inconsistent material groups and orphaned cost centers become wrong recommendations at machine speed.

• Shared semantics. Agree on one definition of margin, open order and customer across finance, sales and supply chain before you model them as data products.

• Data migration governance. Decide what history moves to the new core, what is archived and what lands in BDC for analytics. Migration is the cheapest moment to fix data, not the most convenient moment to postpone it.

• Clear ownership. Every data product needs a business owner who is accountable for its quality, not just an IT steward.

For finance teams on S/4HANA, the Universal Journal (ACDOCA) is a natural starting point: one line item table already unifies FI and CO, which gives AI a single source for margin, cost and profitability questions.

Pillar 3: Move to API first integration on SAP BTP

Integration architecture is where AI readiness and contract risk now meet. On April 27, 2026, SAP's updated API Policy took effect, requiring integrations to use Published APIs documented on the SAP Business Accelerator Hub and flagging interfaces such as ODP RFC as unpermitted. (SAPinsider) The policy also restricts third party AI systems that plan, select or execute sequences of SAP API calls unless they run on SAP endorsed paths such as Joule, Business Data Cloud and its agent gateway. (Kai Waehner)

Because the policy is referenced in SAP's cloud terms, a breach is treated as a contract issue rather than a guideline departure. (Redress Compliance) Forrester has urged CIOs to push back and to scrutinize multiyear third party AI deals that depend on SAP data. (Forrester) Whatever your position, your architecture needs to be ready for it.

Practical steps:

• Inventory every interface that touches SAP: point to point RFCs, file drops, extractors, RPA bots and any AI tool already calling SAP. SAP Note 3439624 provides a self-assessment for existing ODP RFC usage. (Forrester)

• Standardize on SAP Integration Suite and released APIs, so new integrations are policy compliant from day one.

• Plan an agent access pattern. Agent standards such as MCP and A2A are now supported around Joule, with an MCP gateway in Integration Suite positioned as the on ramp for non-SAP agents. (Mindset Consulting)

• Raise API terms at renewal. Contract renewals are the practical enforcement moment, so negotiate the rights your multi-vendor AI strategy needs.

Pillar 4: Design for Joule, agents and AI governance

In SAP's model, Joule becomes the interface, the Knowledge Graph provides context, and agents in the Autonomous Suite execute the work. (SAPinsider) For custom needs, Joule Studio is the workbench where agents, apps, extensions and workflows are built, and it enriches each request with your organization's architecture context from SAP domain models and the Knowledge Graph. (valantic)

That dependency on architecture context is the point. If your landscape is undocumented, Joule Studio has less to work with, and every agent you build carries hidden assumptions.

Three design decisions to make now:

1. Decide where agents are allowed to act. Separate agents that read and recommend from agents that post, approve or change master data. The second group needs approval steps, audit trails and clear limits.

2. Define human oversight before deployment. SAPinsider's guidance is to settle compliance, auditability and human oversight before autonomous agents go live, and to plan for ongoing monitoring once they do. (SAPinsider)

3. Register every agent. The AI Agent Hub in SAP LeanIX is now generally available, giving one place to manage agents across their lifecycle and prove the value of AI investments. (SAP) Treat an unregistered agent the way you would treat an undocumented interface.

Be realistic about timing. Many agents shown at Sapphire are still previews, so prioritize use cases that are generally available and tie them to measurable outcomes such as days to close or invoice exception rates.

Pillar 5: Make enterprise architecture a living practice

Enterprise architecture used to be a set of diagrams refreshed once a year. In an agentic landscape it becomes operational data that AI tools consume directly, which means it has to be current.

SAP now positions three tools as the backbone of an agent led transformation toolchain within the RISE with SAP methodology: SAP LeanIX for enterprise architecture, SAP Signavio for process intelligence, and digital adoption tooling. (SAP) Together with SAP Cloud ALM for implementation and operations, they give architects a connected view of applications, processes and the agents running across them.

A living architecture practice looks like this:

• Application inventory in LeanIX with owners, lifecycle dates, interfaces and the clean core level of every SAP extension.

• Process models in Signavio that show the standard process, where you deviate, and which steps are candidates for agents.

• Architecture decision records for every extension, integration and agent, so the reasoning survives staff turnover.

• An architecture board with AI on its agenda, including an AI governance lead alongside the finance, data and security architects.

Bain frames the CIO's role here as a transformation guardian who shapes architecture decisions and makes sure AI spend turns into measurable value. (Bain & Company) That role is much easier to play with an accurate, shared picture of the landscape.

A practical readiness roadmap

You do not need to finish your cloud ERP move before starting on AI. Bain observed that migration paths are becoming more flexible, letting organizations prepare the data foundation, test targeted AI use cases and move to modern ERP when value, cost, risk and timing line up. (Bain & Company) A sensible sequence looks like this:

1. Assess (first 90 days). Build the application and interface inventory, grade custom code against the clean core levels, map every SAP API and AI integration against the new API Policy, and score master data quality in your highest value domains.

2. Decide (months 3-6). Choose your target landscape, your BDC and data product strategy, your integration standard and your agent governance model. Record each choice as an architecture decision.

3. Foundation (months 6-18). Remediate Level D code, cleanse and migrate master data, move integrations onto Integration Suite and released APIs, and stand up the AI Agent Hub and architecture board.

4. Prove value (in parallel). Run two or three generally available Joule use cases with clear metrics, ideally in finance where data is already unified, and use the results to steer the next wave.

5. Scale. Extend agents across processes as your clean core score, data quality and governance mature.

One caution from Bain: SAP says AI tooling could cut ERP migration effort by about half, but AI does not remove the need for process redesign, change management, integration simplification and data remediation. Ask delivery partners to show exactly where AI will reduce cost, time or risk, and how they will measure it. (Bain & Company)

The bottom line

SAP Cloud ERP and Business AI reward organizations that treat architecture as the product. A clean core keeps agents working on standard logic. A governed data foundation gives them answers people can trust. API first integration keeps you compliant and flexible. And a living architecture practice lets you see, govern and prove the value of every agent you deploy. As one analyst observed at Sapphire, platform choices made in 2026 may set the terms of enterprise AI for the next decade. (Bain & Company