Skip to content

Add LightRAG demo script with vLLM integration#2582

Merged
danielaskdd merged 2 commits intoHKUDS:mainfrom
vishvaRam:patch-6
Jan 15, 2026
Merged

Add LightRAG demo script with vLLM integration#2582
danielaskdd merged 2 commits intoHKUDS:mainfrom
vishvaRam:patch-6

Conversation

@vishvaRam
Copy link
Contributor

Description

This pull request adds a new example demonstrating how to use LightRAG with vLLM-served models for large language model inference, text embeddings, and reranking.
The example mirrors the structure and conventions of existing LightRAG demos (e.g. Gemini) while showcasing a fully self-hosted, OpenAI-compatible vLLM deployment.

Related Issues

  • N/A (example contribution)

Changes Made

  • Added a new example script: examples/lightrag_vllm_demo.py

  • Demonstrates integration of:

    • vLLM-served LLM via OpenAI-compatible API
    • vLLM-served embedding model
    • Jina-compatible reranker served via vLLM
  • Included clear setup instructions and required environment variables

  • Aligned code structure and formatting with existing LightRAG example demos

  • Provided a realistic hybrid query with reranking enabled

Checklist

  • Changes tested locally
  • Code reviewed
  • Documentation updated (inline script docstring)
  • Unit tests added (not applicable for example script)

Additional Notes

This example is intended to help users deploy LightRAG in self-hosted and on-prem environments using vLLM, without reliance on external proprietary APIs. It follows existing LightRAG patterns to ensure consistency and ease of adoption.

This script demonstrates the usage of LightRAG with vLLM for LLM, embeddings, and reranking. It includes setup instructions, environment variable requirements, and a main function that indexes a book and performs a query.
@danielaskdd danielaskdd merged commit ea05413 into HKUDS:main Jan 15, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants