@@ -8,137 +8,89 @@
-B0Bot is a CyberSecurity News API tailored for automated bots on social media platforms. It is a cutting-edge Flask-based API that grants seamless access to the latest cybersecurity and hacker news. Users can effortlessly retrieve news articles either through specific keywords or without, streamlining the information acquisition process. -Once a user requests our API, it retrieves news data from our knowledge base and feeds it to the LLM. After the LLM processes the data, the API obtains the response and returns it in JSON format. The API is powered by LangChain and a Huggingface endpoint, ensuring that users receive accurate and up-to-date information. +B0Bot is a cybersecurity news intelligence platform built around a three-service architecture: an ingestion service that polls RSS feeds and enriches articles with CVE/severity metadata, an api-service that runs a LangGraph agent pipeline for search, analysis, and a grounded Ask AI chat, and a notification service that sends digest emails to subscribers.
+## Architecture -## App Screenshots - -| Home Page | LLM Page | News Page | News Keywords Page | -| :--------:| :-------:| :---------:| :-----------------:| -|  |  |  |  | - -## Setup -1. Install all necessary packages - -`pip install -r ./requirements.txt` - + -2. Set up your Pinecone database -``` -https://www.pinecone.io/ -``` -Login to Pinecone and create a new index with the name `news-index`. Then, add the Pinecone API key in the `.env` file. +The project has three services, using PostgreSQL (with pgvector for embeddings) and Redis to share data and handle caching, sessions, and job queues: +- **ingestion-service** - polls RSS feeds loaded from the sources table (falls back to a hardcoded list if empty), extracts CVE/severity/affected-system metadata via LLM, computes embeddings, writes to Postgres +- **api-service** - Flask app serving the dashboard, chat, sources, and subscribe pages; runs every `/chat` request through a LangGraph agent pipeline +- **notification-service** - polls Postgres for subscribers due for a digest and sends via SMTP; subscriptions are created directly by api-service, no queue involved +All three run together via Docker Compose, alongside Postgres and Redis. +## Features -3. Set up your HuggingFace account -``` -https://huggingface.co/ -``` +- **Dashboard** - CVE Watchlist, Top News, and a filterable article feed (Newest / Critical / Frequent, by source) +- **Ask AI** - click into any article to ask questions grounded in that specific article's content, powered by a hosted Cohere model +- **Hybrid search** - chat queries combine keyword relevance and vector similarity search over article embeddings +- **Sentiment & trend analysis** - per-article sentiment (DistilBERT) and keyword/trend surfacing across search results +- **Sources management** - view and add RSS sources feeding the ingestion pipeline +- **Subscribe / Unsubscribe** - email digests by interest tag and frequency (daily/weekly), manageable via chat or the subscribe form. Chat-based subscribe can span multiple turns - if the email or interests aren't in the message, it asks as a follow-up instead of failing silently +## Setup -4. Set up your environment variables +1. Clone the repo and set up your environment file: -Copy `.env.example` to `.env` and fill in your keys: -``` +```bash cp .env.example .env ``` -Refer to [`.env.example`](.env.example) for all available keys and where to get them. - - -5. Enrich/Update news data into your database - -Run `./db_update/Update.py` as a worker on a cloud service (e.g. heroku). -Or, run `./db_update/Update.py` manually in local. - - -6. Run the flask app - -`flask --app app.py run` - -> By default, the home page will open. The routes have to be defined manually. - - -7. We have added support for the following routes: -``` -/llama # Loads the Meta-Llama-3-8B-Instruct -/gemma # Loads the Gemma-2b -/mistralai # Loads the Mistral-7B-Instruct-v0.2 -``` - -> [!NOTE] -> The Huggingface token you are using must have access to the LLama3 model listed above. -> You can do so by visiting this [link](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct). +Fill in the values - see [`.env.example`](.env.example) for what each one is for. At minimum you'll need a [HuggingFace token](https://huggingface.co/settings/tokens) - used for the local embedding/sentiment models, and to authenticate HuggingFace's InferenceClient, which is how the app reaches the hosted Cohere model for summaries, intent classification, and Ask AI. +2. Bring up the full stack with Docker Compose: -8. Available url paths +```bash +docker compose up -d ``` -//news{{ item.source }}
- {% if item.url != 'N/A' %} - Read more - {% else %} - No URL provided - {% endif %} -{{ item.source }}
- {% if item.url != 'N/A' %} - Read more - {% else %} - No URL provided - {% endif %} -/raw/news (Latest Docs)/news