WebLLM is a modular, customizable javascript package that directly brings language model chats directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU. We can bring a lot of fun opportunities to build AI assistants for everyone and enable privacy while enjoying GPU acceleration. WebLLM offers a minimalist and modular interface to access the chatbot in the browser. The WebLLM package itself does not come with UI, and is designed in a modular way to hook to any of the UI components. The following code snippet demonstrates a simple example that generates a streaming response on a webpage.

Features

  • WebLLM comes with API support for WebWorker so you can hook the generation process into a separate worker thread so that the compute in the webworker won't disrupt the UI
  • You can find a complete a complete chat app example in examples/simple-chat
  • WebLLM works as a companion project of MLC LLM
  • It reuses the model artifact and builds flow of MLC LLM
  • Build WebLLM Package From Source
  • WebLLM package is a web runtime designed for MLC LLM

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow WebLLM

WebLLM Web Site

You Might Also Like
Get Avast Free Antivirus, our award-winning protection for all Icon
Get Avast Free Antivirus, our award-winning protection for all

Get advanced privacy protection beyond antivirus software

Avast Free Antivirus protects your computer against viruses and malware, and it helps you protect your home network against intruders.
Free Download
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of WebLLM!

Additional Project Details

Programming Language

TypeScript

Related Categories

TypeScript Large Language Models (LLM)

Registered

2023-08-21