feat: Enable web search feature without external services #5

[JumpLink]

This PR adds a development setup that enables testing PRs and local changes to services directly, with a focus on completing the Simple Reranker integration and ensuring web search works fully with local services.

Changes

Git Submodules for Development

• Added Git submodules for rag_api, librechat, and agents in dev/ directory
• Configured submodules to track specific PR branches:
  • dev/rag_api: reranker branch (Simple Reranker endpoint PR)
  • dev/librechat: new/feature/simple_reranker branch (Simple Reranker integration PR)
  • dev/agents: simple_reranker branch

Docker Compose Override for Local Builds

• Added docker-compose.librechat.override.yml to enable building services from local submodules
• Configured local builds for librechat and rag_api services
• Override file must be explicitly included in commands (not auto-loaded)

Simple Reranker Configuration

• Added environment variables for Simple Reranker configuration:
  • SIMPLE_RERANKER_MODEL_NAME (default: mixedbread-ai/mxbai-rerank-large-v1)
  • SIMPLE_RERANKER_MODEL_TYPE (default: cross-encoder)
• Updated docker-compose.librechat.yml to pass reranker environment variables to rag_api service

Documentation

• Added dev/README.md with setup instructions for:
  • Initializing submodules
  • Building and starting services with local builds
  • Testing the reranker endpoint
  • Switching between local and published images
• Updated main README.md with reference to development setup

Purpose

1. Test Simple Reranker PRs: Enable testing of the Simple Reranker implementation from rag_api PR and the integration from librechat PR
2. General Development Workflow: Provide an initial setup for testing any changes to services directly without waiting for upstream merges
3. Complete Local Web Search: Complete the previous PR by ensuring web search functionality works fully with local services

Usage

To use local builds:

# Initialize submodules
git submodule update --init --remote

# Since `agents` is an npm package used by LibreChat, build it before starting
cd dev/agents
npm install
npm run build
cd ../..

# Build and start with local builds
docker compose -f docker-compose.librechat.yml -f docker-compose.librechat.override.yml build
docker compose -f docker-compose.librechat.yml -f docker-compose.librechat.override.yml up -d

To use published images (default):

docker compose -f docker-compose.librechat.yml build
docker compose -f docker-compose.librechat.yml up -d

Testing

Test the reranker endpoint:

curl -s http://localhost:8000/rerank \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -d '{
    "query": "I love you",
    "docs": ["I hate you", "I really like you"],
    "k": 5
  }'

Notes

• The override file is not automatically loaded by Docker Compose, so it must be explicitly included when using local builds
• Submodules track specific PR branches and can be updated with git submodule update --remote
• Environment variables for optional reranker parameters are commented out in docker-compose.librechat.yml pending review of the PR implementation

Related

• Initial discussion: Add support for Simple ReRanker or similar locally hosted version danny-avila/LibreChat#9102
• LibreChat PR: feat: Adding code to call the rag_api simple reranker. danny-avila/LibreChat#10574
• RAG API: feat: New Reranking Endpoint danny-avila/rag_api#190
• Agents: Adding code to call the rag_api simple reranker. danny-avila/agents#33

[correctiv-team-digital-bot]

Review of PR #1: Enable Web Search Feature Without External Services

Summary

This PR introduces a development setup to enable testing of PRs and local changes to services directly, with a focus on integrating the Simple Reranker and ensuring web search functionality works fully with local services. The changes include:

1. Git Submodules for Development: Added submodules for rag_api, librechat, and agents to track specific PR branches.
2. Docker Compose Override: Added docker-compose.librechat.override.yml to enable local builds from submodules.
3. Simple Reranker Configuration: Added environment variables for reranker configuration and updated docker-compose.librechat.yml to pass these variables to the rag_api service.
4. Documentation: Added dev/README.md with setup instructions and updated the main README.md.

---

Detailed Analysis

Strengths

1. Modular Development Setup: The use of Git submodules allows for isolated testing of specific PR branches, which is beneficial for development and debugging.
2. Flexibility: The Docker Compose override file provides flexibility to switch between local builds and published images, catering to different development needs.
3. Comprehensive Documentation: The dev/README.md provides clear instructions for setting up and using the development environment, which is crucial for onboarding and troubleshooting.
4. Environment Configuration: The addition of environment variables for the Simple Reranker allows for easy customization and experimentation with different models.

Weaknesses

1. Complexity: The setup introduces complexity with multiple submodules and Docker configurations, which might be overwhelming for new contributors.
2. Maintenance Overhead: Tracking specific PR branches in submodules requires regular updates to ensure compatibility, which could lead to maintenance challenges.
3. Potential Conflicts: The override file might cause conflicts or confusion if not managed properly, especially when switching between local and published images.

Suggestions for Improvement

1. Simplify Setup: Consider providing a script or additional documentation to automate the setup process, reducing the manual steps required.
2. Error Handling: Add error handling and validation in the Docker configurations to provide meaningful feedback if something goes wrong.
3. Testing: Include instructions or scripts for automated testing to ensure the setup works as expected and to catch issues early.

---

Inline Comments

.gitmodules

• Line 1-12: The submodules are well-defined, but consider adding comments to explain the purpose of each submodule and branch.

docker-compose.librechat.override.yml

• Line 3-18: The override file is clear, but it would be helpful to add comments explaining the purpose of each service configuration.

env.example

• Line 42-70: The environment variables for the Simple Reranker are well-documented, but consider adding examples for different use cases (e.g., CPU vs. GPU).

librechat.yaml

• Line 88: The change from rerankerType: \"jina\" to rerankerType: \"simple\" is noted, but it would be helpful to explain the reasoning behind this change in the PR description.

---

Recommendation

The PR is well-structured and addresses the need for a flexible development setup. However, the complexity introduced by the submodules and Docker configurations should be mitigated with additional documentation or automation. Overall, the changes are beneficial for enabling local testing and development.