GPU support check

Edge AI on Web Platforms

LLM

WebLLM

import { CreateMLCEngine } from '@mlc-ai/web-llm';

const engine = await CreateMLCEngine(selectedModel, {
  initProgressCallback: ({ progress }) => {
    console.log(`Initialization progress: ${(progress * 100).toFixed()}%`);
  },
});

const messages = [
  {
    role: 'user',
    content: message.value,
  },
];
const chunks = await engine.chat.completions.create({
  // @ts-ignore
  messages,
  stream: true,
});

reply.value = ''; // Reset the reply before appending new content
for await (const chunk of chunks) {
  reply.value += chunk.choices[0].delta.content || '';
}

Prompt API

Stable Diffusion