AI Dev Tools
·5 min read·tutorial

Beyond the Chatbox: How to Build Generative UI with LLMs and TypeScript

Learn how to build generative ui with llms using TypeScript. Step-by-step guide to building a neural expressive interface that bypasses the dead chat-log pattern.

typescriptaigenerative-uireact

The standard chat-log interface—the ubiquitous ChatGPT-style stream of text bubbles—is an evolutionary dead end for complex applications. Users do not want to converse with a command-line interface wrapped in Tailwind; they want to accomplish tasks.

Google’s recent push toward "Neural Expressive" interfaces signals a paradigm shift: interfaces should dynamically morph, rendering custom interactive UI widgets based on the model's intent and the current application state.

We are building a Neural Expressive Engine in TypeScript. This system bypasses the chatbox entirely, streaming structured UI state changes from an LLM and rendering them as interactive React components in real-time.

Here is the architecture of the engine we are building:

To see how this fits into the broader architectural shift away from simple prompts, read our deep dive on building generative UI with LLMs and React beyond the chatbox.

Why the Chat Log is Dead and How to Build Generative UI with LLMs

The primary limitation of the chat-log interface is that text is a low-bandwidth, high-friction medium for state manipulation. If a user wants to edit a generated chart, change a date range, or filter a list, typing "please change the date range to last Tuesday" is an incredibly inefficient feedback loop.

When you learn how to build generative ui with llms, you transition from treating the model as an author to treating it as a dynamic UI designer. Instead of streaming markdown text, the model streams structured JSON patches that target a client-side state machine. The client maps these patches to a registry of highly interactive, pre-compiled React components.

This is the same architectural shift we saw in advanced IDE integrations, which we analyzed in our breakdown of under the hood of Cursor Composer 2.5.

To execute this pattern, we must solve three hard engineering problems:

  1. Incremental JSON parsing: Parsing incomplete JSON chunks from a readable stream without crashing.
  2. Dynamic component reconciliation: Mounting and updating React components on the fly without losing local state.
  3. Bi-directional event binding: Allowing the generated UI to send state changes back to the LLM or local database.

Step 1: Defining the Schema and Component Registry

Our canvas can render three types of widgets: a DataGrid, a MetricCard, and an InteractiveForm. We will define strict TypeScript interfaces for our UI state to ensure type safety across the network boundary.

typescript
// types/generative-ui.ts
 
export type ComponentType = 'MetricCard' | 'DataGrid' | 'InteractiveForm';
 
export interface BaseWidget {
  id: string;
  type: ComponentType;
  title: string;
  layout: {
    w: number; // grid width (1-12)
    h: number; // grid height
  };
}
 
export interface MetricCardProps extends BaseWidget {
  type: 'MetricCard';
  value: string | number;
  change: number; // percentage change
  trend: 'up' | 'down' | 'neutral';
}
 
export interface DataGridProps extends BaseWidget {
  type: 'DataGrid';
  headers: string[];
  rows: Record<string, any>[];
  pageSize?: number;
}
 
export interface InteractiveFormField {
  name: string;
  label: string;
  type: 'text' | 'number' | 'select';
  options?: string[];
}
 
export interface InteractiveFormProps extends BaseWidget {
  type: 'InteractiveForm';
  fields: InteractiveFormField[];
  submitUrl: string;
  submitMethod: 'POST' | 'PUT';
}
 
export type WidgetProps = MetricCardProps | DataGridProps | InteractiveFormProps;
 
export interface CanvasState {
  version: number;
  widgets: WidgetProps[];
}

Step 2: The Incremental JSON Parser

LLMs stream text. If we wait for the entire JSON payload to finish streaming before rendering, we defeat the purpose of real-time responsiveness. We need an incremental JSON parser that can take a partial, broken JSON string (e.g., {"widgets": [{"id": "1", "type": "Metric") and construct a valid, partial JavaScript object.

Here is a robust, lightweight implementation of an incremental JSON parser. It uses a stateful scanner to balance brackets, quotes, and braces to close uncompleted structures.

typescript
// utils/incremental-json.ts
 
export function parsePartialJSON<T = any>(partialJson: string): Partial<T> {
  const cleanStr = partialJson.trim();
  if (!cleanStr) return {} as Partial<T>;
 
  try {
    return JSON.parse(cleanStr);
  } catch (e) {
    // If standard parsing fails, we attempt to repair the JSON string
    return forceParse<T>(cleanStr);
  }
}
 
function forceParse<T>(str: string): Partial<T> {
  let openBrackets: string[] = [];
  let inString = false;
  let escaped = false;
  let repaired = str;
 
  for (let i = 0; i < str.length; i++) {
    const char = str[i];
    if (escaped) {
      escaped = false;
      continue;
    }
    if (char === '\\') {
      escaped = true;
      continue;
    }
    if (char === '"') {
      inString = !inString;
      continue;
    }
    if (inString) continue;
 
    if (char === '{' || char === '[') {
      openBrackets.push(char);
    } else if (char === '}' || char === ']') {
      openBrackets.pop();
    }
  }
 
  // If we are currently inside an unfinished string, close it
  if (inString) {
    repaired += '"';
  }
 
  // Remove trailing commas which are invalid in JSON
  repaired = repaired.replace(/,\s*$/, "");
 
  // Close open brackets in reverse order
  while (openBrackets.length > 0) {
    const last = openBrackets.pop();
    if (last === '{') {
      // If we ended right after a key colon, append a null value first
      if (repaired.trim().endsWith(':')) {
        repaired += 'null';
      }
      repaired += '}';
    } else if (last === '[') {
      repaired += ']';
    }
  }
 
  try {
    return JSON.parse(repaired);
  } catch (finalError) {
    // Fallback: strip back to the last successfully closed object/array element
    return fallbackRegexParse(str);
  }
}
 
function fallbackRegexParse(str: string): any {
  // Ultra-fallback: extract whatever key-value pairs we can find via regex
  // useful for rendering extremely early optimistic UI states
  const match = str.match(/"type"\s*:\s*"([^"]+)"/);
  if (match) {
    return { type: match[1] };
  }
  return {};
}

Step 3: Stream Processing Hook

Now we build a React hook that manages the stream consumption. It calls our LLM endpoint, parses the Server-Sent Events (SSE) stream, runs the partial JSON parser, and updates the local state on every micro-tick.

To handle high-throughput states without causing UI stuttering, we must optimize how our state updates. For advanced asynchronous patterns in high-throughput applications, check our guide on beyond Promise.all in TypeScript.

typescript
// hooks/useGenerativeCanvas.ts
import { useState, useEffect, useRef } from 'react';
import { CanvasState, WidgetProps } from '../types/generative-ui';
import { parsePartialJSON } from '../utils/incremental-json';
 
export function useGenerativeCanvas() {
  const [canvasState, setCanvasState] = useState<CanvasState>({ version: 1, widgets: [] });
  const [isStreaming, setIsStreaming] = useState(false);
  const controllerRef = useRef<AbortController | null>(null);
 
  const streamCanvasUpdate = async (prompt: string) => {
    if (controllerRef.current) {
      controllerRef.current.abort();
    }
 
    controllerRef.current = new AbortController();
    setIsStreaming(true);
    setCanvasState({ version: 1, widgets: [] });
 
    try {
      const response = await fetch('/api/generate-ui', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ prompt }),
        signal: controllerRef.current.signal,
      });
 
      if (!response.body) throw new Error('No body in response');
      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      let buffer = '';
 
      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
 
        buffer += decoder.decode(value, { stream: true });
        
        // Process SSE formatted chunks (data: ...)
        const lines = buffer.split('\n');
        buffer = lines.pop() || ''; // Keep the last incomplete line in the buffer
 
        for (const line of lines) {
          const cleanLine = line.replace(/^data:\s*/, '').trim();
          if (!cleanLine || cleanLine === '[DONE]') continue;
 
          // Parse whatever JSON we have accumulated so far
          const parsed = parsePartialJSON<CanvasState>(cleanLine);
          if (parsed && Array.isArray(parsed.widgets)) {
            setCanvasState(prev => ({
              version: prev.version + 1,
              widgets: parsed.widgets as WidgetProps[]
            }));
          }
        }
      }
    } catch (err: any) {
      if (err.name !== 'AbortError') {
        console.error('Generative UI Streaming Error:', err);
      }
    } finally {
      setIsStreaming(false);
    }
  };
 
  return { canvasState, isStreaming, streamCanvasUpdate };
}

Step 4: The Dynamic Canvas and Component Registry

We must now build the renderer. We map the type field of our dynamic state to concrete, interactive React components.

A major gotcha here is re-rendering performance. If the parent canvas re-renders every time a new character streams in, we will drop frames. We prevent this by assigning stable keys based on widget IDs and using React.memo on the individual components.

tsx
// components/GenerativeCanvas.tsx
import React, { useTransition } from 'react';
import { WidgetProps, MetricCardProps, DataGridProps, InteractiveFormProps } from '../types/generative-ui';
 
// Component 1: Metric Card
export const MetricCard = React.memo(({ title, value, change, trend }: MetricCardProps) => {
  return (
    <div className="p-6 bg-slate-900 border border-slate-800 rounded-xl shadow-lg transition-all duration-200 hover:border-slate-700">
      <p className="text-sm font-medium text-slate-400 uppercase tracking-wider">{title || 'Loading...'}</p>
      <div className="mt-2 flex items-baseline justify-between">
        <span className="text-3xl font-bold tracking-tight text-white">{value ?? '---'}</span>
        {change !== undefined && (
          <span className={`inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium ${
            trend === 'up' ? 'bg-emerald-500/10 text-emerald-400' : 'bg-rose-500/10 text-rose-400'
          }`}>
            {trend === 'up' ? '↑' : '↓'} {Math.abs(change)}%
          </span>
        )}
      </div>
    </div>
  );
});
 
// Component 2: Dynamic Data Grid
export const DataGrid = React.memo(({ title, headers, rows }: DataGridProps) => {
  if (!headers || !rows) return <div className="p-4 bg-slate-900 animate-pulse rounded-xl h-48" />;
 
  return (
    <div className="p-6 bg-slate-900 border border-slate-800 rounded-xl shadow-lg overflow-hidden">
      <h3 className="text-lg font-semibold text-white mb-4">{title}</h3>
      <div className="overflow-x-auto">
        <table className="w-full text-left border-collapse">
          <thead>
            <tr className="border-b border-slate-800">
              {headers.map((h, i) => (
                <th key={i} className="pb-3 text-xs font-semibold uppercase tracking-wider text-slate-400">{h}</th>
              ))}
            </tr>
          </thead>
          <tbody className="divide-y divide-slate-800/50">
            {rows.map((row, rIdx) => (
              <tr key={rIdx} className="hover:bg-slate-800/20 transition-colors">
                {headers.map((h, cIdx) => (
                  <td key={cIdx} className="py-3 text-sm text-slate-300">{row[h] ?? '---'}</td>
                ))}
              </tr>
            ))}
          </tbody>
        </table>
      </div>
    </div>
  );
});
 
// Component 3: Dynamic Form with Callback binding
export const InteractiveForm = React.memo(({ title, fields, submitUrl, submitMethod }: InteractiveFormProps) => {
  const [formData, setFormData] = React.useState<Record<string, any>>({});
  const [status, setStatus] = React.useState<'idle' | 'submitting' | 'success'>('idle');
 
  const handleSubmit = async (e: React.FormEvent) => {
    e.preventDefault();
    setStatus('submitting');
    try {
      await fetch(submitUrl, {
        method: submitMethod,
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify(formData),
      });
      setStatus('success');
    } catch (err) {
      console.error(err);
      setStatus('idle');
    }
  };
 
  return (
    <div className="p-6 bg-slate-900 border border-slate-800 rounded-xl shadow-lg">
      <h3 className="text-lg font-semibold text-white mb-4">{title}</h3>
      <form onSubmit={handleSubmit} className="space-y-4">
        {fields?.map((field, i) => (
          <div key={i} className="flex flex-col gap-1.5">
            <label className="text-xs font-medium text-slate-400 uppercase">{field.label}</label>
            {field.type === 'select' ? (
              <select
                onChange={(e) => setFormData(prev => ({ ...prev, [field.name]: e.target.value }))}
                className="bg-slate-950 border border-slate-800 rounded-lg px-3 py-2 text-sm text-white focus:outline-none focus:border-indigo-500"
              >
                <option value="">Select option...</option>
                {field.options?.map((opt, oIdx) => (
                  <option key={oIdx} value={opt}>{opt}</option>
                ))}
              </select>
            ) : (
              <input
                type={field.type}
                placeholder={`Enter ${field.label.toLowerCase()}...`}
                onChange={(e) => setFormData(prev => ({ ...prev, [field.name]: e.target.value }))}
                className="bg-slate-950 border border-slate-800 rounded-lg px-3 py-2 text-sm text-white placeholder-slate-600 focus:outline-none focus:border-indigo-500"
              />
            )}
          </div>
        ))}
        <button
          type="submit"
          disabled={status === 'submitting'}
          className="w-full bg-indigo-600 hover:bg-indigo-500 text-white font-medium py-2 px-4 rounded-lg text-sm transition-colors disabled:opacity-50"
        >
          {status === 'submitting' ? 'Processing...' : status === 'success' ? 'Submitted!' : 'Execute Action'}
        </button>
      </form>
    </div>
  );
});
 
// Component Registry Mapping
const Registry: Record<string, React.ComponentType<any>> = {
  MetricCard,
  DataGrid,
  InteractiveForm,
};
 
// Main Canvas Renderer
export function GenerativeCanvas({ state }: { state: CanvasState }) {
  const [, startTransition] = useTransition();
 
  return (
    <div className="grid grid-cols-12 gap-6 p-8 bg-slate-950 min-h-screen">
      {state.widgets.map((widget) => {
        const Component = Registry[widget.type];
        if (!Component) {
          return (
            <div key={widget.id} className="col-span-12 p-4 bg-red-950/20 border border-red-900 rounded-xl text-red-400 text-sm">
              Component "{widget.type}" not found in registry.
            </div>
          );
        }
 
        // Determine grid layout
        const colSpan = widget.layout?.w ? `col-span-${widget.layout.w}` : 'col-span-12';
 
        return (
          <div key={widget.id} className={`${colSpan} transition-all duration-300`}>
            <Component {...widget} />
          </div>
        );
      })}
    </div>
  );
}

Step 5: Server-Side Streaming Endpoint

To supply structured JSON states to the client, the backend must use a model capable of reliable structured output (like gpt-4o or claude-3-5-sonnet) and force the model to output JSON that adheres to our component schema.

This is an implementation using Node.js/Next.js Route Handlers.

typescript
// app/api/generate-ui/route.ts
import { NextRequest } from 'next/server';
import { OpenAI } from 'openai';
 
const openai = new OpenAI();
 
export const runtime = 'edge';
 
export async function POST(req: NextRequest) {
  const { prompt } = await req.json();
 
  const systemPrompt = `
    You are a Neural Expressive interface engine. Instead of answering the user with text, you output a structured UI configuration to fulfill their request.
    
    You must output a single JSON object matching this schema:
    {
      "widgets": Array<{
        "id": string (unique),
        "type": "MetricCard" | "DataGrid" | "InteractiveForm",
        "title": string,
        "layout": { "w": number (1-12) },
        // If type is MetricCard:
        "value": string | number,
        "change": number,
        "trend": "up" | "down",
        // If type is DataGrid:
        "headers": string[],
        "rows": Array<Record<string, any>>,
        // If type is InteractiveForm:
        "fields": Array<{ "name": string, "label": string, "type": "text" | "number" | "select", "options"?: string[] }>,
        "submitUrl": string,
        "submitMethod": "POST"
      }>
    }
 
    Respond ONLY with raw JSON. Do not wrap in markdown blocks. Do not add conversational text.
  `;
 
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: [
      { role: 'system', content: systemPrompt },
      { role: 'user', content: prompt }
    ],
    stream: true,
    response_format: { type: 'json_object' }
  });
 
  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      let accumulatedText = '';
 
      for await (const chunk of response) {
        const text = chunk.choices[0]?.delta?.content || '';
        accumulatedText += text;
 
        // Clean up the text stream to ensure we only send valid JSON-structured lines to the client
        const cleanChunk = accumulatedText
          .replace(/```json/g, '')
          .replace(/```/g, '')
          .trim();
 
        controller.enqueue(encoder.encode(`data: ${JSON.stringify({ widgets: parseDraftState(cleanChunk) })}\n\n`));
      }
 
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });
 
  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive',
    },
  });
}
 
// Simple backend helper to isolate the array from partial JSON streams
function parseDraftState(text: string): any[] {
  try {
    const parsed = JSON.parse(text);
    return parsed.widgets || [];
  } catch {
    // Return partial array if possible using a simple bracket matching technique
    const match = text.match(/"widgets"\s*:\s*\[([\s\S]*)/);
    if (match) {
      try {
        return JSON.parse(`[${match[1]}`);
      } catch {
        // Handled downstream by client-side incremental parser
      }
    }
    return [];
  }
}

Complete Blueprint Example

To tie this together, here is a complete page component showcasing how the user triggers UI generation. The user types a command like "Show me our cloud infrastructure cost metrics, a breakdown of servers, and an action form to scale down unused instances."

The system bypasses standard chat bubbles and renders the canvas directly.

tsx
// app/page.tsx
'use client';
 
import React, { useState } from 'react';
import { useGenerativeCanvas } from '../hooks/useGenerativeCanvas';
import { GenerativeCanvas } from '../components/GenerativeCanvas';
 
export default function NeuralExpressiveCanvasPage() {
  const [prompt, setPrompt] = useState('Show me SaaS revenue performance, a breakdown of customer tiers, and a dynamic refund form.');
  const { canvasState, isStreaming, streamCanvasUpdate } = useGenerativeCanvas();
 
  const handleGenerate = (e: React.FormEvent) => {
    e.preventDefault();
    streamCanvasUpdate(prompt);
  };
 
  return (
    <div className="flex flex-col min-h-screen bg-slate-950 text-white">
      {/* Control Panel */}
      <header className="border-b border-slate-900 bg-slate-950/80 backdrop-blur sticky top-0 z-50 p-4">
        <div className="max-w-7xl mx-auto flex flex-col md:flex-row items-center justify-between gap-4">
          <div>
            <h1 className="text-xl font-bold bg-gradient-to-r from-indigo-400 to-purple-400 bg-clip-text text-transparent">
              Neural Expressive Canvas
            </h1>
            <p className="text-xs text-slate-500">State-driven generative UI streaming engine</p>
          </div>
          
          <form onSubmit={handleGenerate} className="flex w-full md:w-auto items-center gap-2 flex-grow max-w-2xl">
            <input
              type="text"
              value={prompt}
              onChange={(e) => setPrompt(e.target.value)}
              className="flex-grow bg-slate-900 border border-slate-800 rounded-xl px-4 py-2.5 text-sm focus:outline-none focus:border-indigo-500 placeholder-slate-600"
              placeholder="Describe the interface you want..."
            />
            <button
              type="submit"
              disabled={isStreaming}
              className="bg-indigo-600 hover:bg-indigo-500 disabled:opacity-50 text-white font-semibold text-sm px-5 py-2.5 rounded-xl transition-all shadow-lg shadow-indigo-600/10 whitespace-nowrap"
            >
              {isStreaming ? 'Streaming UI...' : 'Generate Interface'}
            </button>
          </form>
        </div>
      </header>
 
      {/* Dynamic Canvas Workspace */}
      <main className="flex-grow">
        {canvasState.widgets.length === 0 && !isStreaming ? (
          <div className="flex flex-col items-center justify-center min-h-[60vh] text-slate-500 gap-2">
            <div className="p-4 rounded-full bg-slate-900 border border-slate-800 text-indigo
ShareTweet

Related Posts

Building Generative UI with LLMs and React: Beyond the Chatbox

Move beyond boring text streaming. Learn how to build production-ready, dynamic generative UI using Gemini, React Server Components, and Zod schemas.

→

How to Build an OpenAI Compatible API in TypeScript (And Why You Should)

Avoid vendor lock-in by replicating the chat completions spec. Learn how to build an OpenAI compatible API using TypeScript, Hono, and Zod.

→

Upgrading to React 19.2.7: Fixing Hydration Mismatches and Action State Edge Cases

A deep dive into upgrading to React 19.2.7, exploring the critical bug fixes for hydration mismatches, Server Component ref handoffs, and Action Hook regressions.

→