The Architecture of In-Chat AI Apps

Key takeaways

• A plain-text reply can answer a question; it cannot carry a workflow.
• In-chat apps work because they reuse one shared widget layer across different tasks, not because each app is custom-built.
• The surface, MCP-connected tool surfaces versus native discovery surfaces, should follow the audience and the governance need, not habit.
• A working demo and a production system differ mainly in what sits around the app: live data, auth, evaluation, and monitoring.

A growing share of customer journeys now begins with a question typed into an assistant, before anyone opens a website or an app. That shift moves the first point of contact somewhere most interfaces were never designed to live: inside a conversation.

This piece is about what changes when the assistant stops replying in text and starts rendering an actual interface. Not a chatbot bolted onto a product, but an application whose surface happens to be a chat window.

Act I: The ceiling on text

Why a reply cannot carry a workflow

An assistant can answer a question well. Moving someone from that question to a finished outcome, a booked appointment, a resolved dispute, a completed purchase, asks for more than text. It needs interface pieces that show options side by side, structured comparisons the user can act on, and safe actions wired to real systems.

Prose is sequential. A workflow is not. The moment a user needs to compare three options, see a live number update as they adjust a variable, or trust that clicking a button will not silently do the wrong thing, text alone runs out of road.

In practice, the entry point is moving into the assistant, and a plain reply does not travel with it.

Two production patterns

Most teams building for assistants land on one of two shapes.

The second pattern only stays affordable if the interface pieces are reusable across tasks. That reuse is the actual engineering problem.

Act II: The widget layer

What an in-chat app actually needs

Underneath a chat-native app is a small library of interface pieces, composed differently per task rather than rebuilt from zero each time:

• cards for discovery and comparison
• charts, gauges, and structured summaries for decision support
• maps for anything location-bound
• selectors, steppers, and calculators for input
• action components, checkout, booking, confirmation, that close the loop

A commerce flow, a booking flow, and a support flow look different on the surface, but they draw from the same pieces underneath. Once that shared layer exists, adding a new task is closer to composition than to a fresh build.

Choosing a surface

The same capability layer can usually be delivered through more than one surface, and the right choice depends on the job rather than on preference. Tool-connected surfaces suit governed workflows where the assistant needs deep access to documents and structured actions. Native discovery surfaces suit customer-facing journeys where reach and low-friction access matter more than depth of tool access. Picking the wrong one does not fail loudly; it just quietly limits what the app can safely do.

This is a design and governance decision, not a technical afterthought. See AI agents vs AI workflows for how that same choice plays out one layer down, at the orchestration level rather than the surface level.

Act III: Demo to production

A worked example

A concrete illustration of this pattern: Newtuple published a proof of concept spanning commerce, healthcare, finance, and real estate scenarios, each rendered as a live in-chat app inside two different assistants from one shared widget system. It is a useful reference point precisely because it shows the reuse claim above holding up across four genuinely different workflows rather than one narrow demo.

What a production build adds

A proof of concept proves the experience layer. It usually runs on mock data, with no live backend behind it. Getting from there to something real adds a predictable set of layers:

1. Live integration to real catalogs, records, and transaction systems in place of mock data.
2. Authentication and permissions, so actions run against the right account with the right scope.
3. Evaluation, so quality and safety are checked before and after release, not assumed.
4. Monitoring and human approval for the steps that change real state.

None of that is exotic. It is the same governance layer any production AI system needs, applied to a surface that happens to render inside a chat window instead of a page. For the runtime version of that governance layer, see Engineering Agentic Systems for Reliability and Structured Output and Why It Matters.

What this changes in practice

Stop treating the assistant as a place to add a chatbot. Treat it as a surface your application can render on, with the same interface discipline and the same production controls you would demand anywhere else. The conversation is the entry point; the workflow still has to be engineered.