vLLM
An example of using vLLM with LWS
Examples
This section contains examples of using LWS with or without specific inference runtime.
TensorRT-LLM
An example of using TensorRT-LLM with LWS
llama.cpp
An example of using llama.cpp with LWS
SGLang
An example of using SGLang with LWS
Saxml
An example of using Saxml with LWS
Last modified March 26, 2025: Address feedback (552ff61)