Feat: Add crawl4ai as an option for local scrape

[relic664]

This is a PR to add crawl4ai as an option for local crawl, to supplement the existing firecrawl option. There's going to be a complementary PR in the LibreChat main repo, which I've drafted here

This PR builds on @lukolszewski original work and adds fit as a default option for crawl4ai's /md endpoint, which uses adaptive filtering to filter the markdown, or returns just the raw markdown (raw for fitStrategy).

I've tested locally and it works fine. I have docker image for testing if anybody is interested (ghcr.io/relic664/librechat:latest).

It's worth nothing that this is a very basic implementation to provide a simple option for a self-hosted scrape option. This implementation doesn't provide options for the scrape beyond fit (filtered markdown) or raw (raw markdown). Given that there's only one self-hosted option for scrape, I thought it was prudent to go ahead and make a MVP PR for crawl4ai before a full featured implementation with all the configuration knobs.

A simple quick start is to set the env var CRAWL4AI_API_URL to your instance, and in your librechat.config:

webSearch:
  crawl4aiApiUrl: "${CRAWL4AI_API_URL}"
  scraperProvider: "crawl4ai"