Sign inSign up

obeoneorg/rss-watcher

By obeoneorg

β€’Updated 7 months ago

Image
0

1.6K

obeoneorg/rss-watcher repository overview

β πŸ“‘ RSS Watcher

Monitor RSS/Atom feeds and receive notifications via Telegram and SimpleX Chat with advanced filtering

Python 3.11+ Docker Ready Telegram Bot SimpleX Chat MIT License Async Powered


⁠✨ Features

FeatureDescription
πŸ“‘ Multiple FeedsMonitor unlimited RSS/Atom feeds simultaneously
πŸ” Smart FilteringKeywords, categories, authors, and regex patterns with AND/OR logic
πŸ“± Telegram AlertsBeautifully formatted notifications via Telegram bot
πŸ”’ SimpleX ChatPrivacy-first notifications via SimpleX decentralized messenger
πŸ”€ Notifier RoutingRoute each feed to Telegram, SimpleX, or both
πŸ’Ύ PersistenceSQLite storage prevents duplicates across restarts
🐳 Docker ReadyOne-command deployment with docker compose
🎬 Media DownloadAutomatically download videos from feed entries
🌐 Proxy SupportSOCKS4/5 and HTTP proxy for all requests
πŸͺ Auth CookiesPer-feed cookie support for authenticated feeds
βš™οΈ ConfigurableYAML config with environment variable substitution

β πŸš€ Quick Start

⁠Prerequisites
  • Python 3.11+ (local) or Docker (containerized)
  • At least one notification backend:

Pre-built images are available on GitHub Container Registry⁠ and Docker Hub⁠.

# Clone and configure
git clone https://github.com/obeone/rss-watcher.git
cd rss-watcher
cp config.example.yaml config.yaml
cp .env.example .env

# Edit .env with your Telegram credentials
# TELEGRAM_BOT_TOKEN=your_bot_token
# TELEGRAM_CHAT_ID=your_chat_id

# Launch (uses pre-built image)
docker compose up -d

# Or build locally
docker compose up -d --build

Available images:

  • ghcr.io/obeone/rss-watcher:latest (recommended)
  • obeoneorg/rss-watcher:latest (Docker Hub mirror)
⁠🐍 Local Installation
# Clone and setup
git clone https://github.com/obeone/rss-watcher.git
cd rss-watcher

# Create environment with uv
uv venv && source .venv/bin/activate
uv pip install .

# Configure
cp config.example.yaml config.yaml
export TELEGRAM_BOT_TOKEN="your_bot_token"
export TELEGRAM_CHAT_ID="your_chat_id"

# Run
rss-watcher -c config.yaml

β βš™οΈ Configuration

Configuration uses YAML with environment variable substitution (${VAR} or ${VAR:-default}).

β πŸ“‹ Basic Structure
# At least one notifier (telegram or simplex) is required
telegram:
  bot_token: "${TELEGRAM_BOT_TOKEN}"
  chat_id: "${TELEGRAM_CHAT_ID}"
  parse_mode: "HTML"  # or "Markdown"

simplex:  # optional, see SimpleX Chat section below
  websocket_url: "ws://simplex-chat:6225"  # via simplex-ws-proxy
  contacts: ["alice"]

defaults:
  check_interval: 300  # seconds between checks
  request_timeout: 30
  max_retries: 3
  media_dir: "media"  # optional, for video downloads

storage:
  database_path: "data/rss_watcher.db"

feeds:
  - name: "Tech News"
    url: "https://example.com/feed.xml"
    check_interval: 600  # override default
    notifiers: ["telegram"]  # optional: route to specific backends
    filters:
      keywords:
        include: ["python", "rust"]
β πŸ” Filter Options

All filters combine with AND logic (all must pass). Within each filter, include rules use OR logic (any match passes).

β πŸ“ Keywords Filter

Filter by words in title/content:

filters:
  keywords:
    include: ["python", "rust"]  # Match if contains ANY
    exclude: ["spam", "ad"]      # Reject if contains ANY
    case_sensitive: false
⁠🏷️ Categories Filter

Filter by RSS categories/tags:

filters:
  categories:
    include: ["tech", "programming"]
    exclude: ["offtopic"]
    case_sensitive: false
β πŸ‘€ Authors Filter

Filter by author name:

filters:
  authors:
    include: ["john", "jane"]
    exclude: ["bot"]
    case_sensitive: false
β πŸ”€ Regex Filter

Filter using regular expressions:

filters:
  regex:
    title: "^\\[IMPORTANT\\]"      # Match title pattern
    content: "release.*v[0-9]+"    # Match content pattern
⁠🌐 Proxy Configuration

Route all requests through a proxy server:

defaults:
  proxy: "socks5://user:[email protected]:1080"

Supported protocols: socks4://, socks5://, http://

Applies to RSS fetching, Telegram API, and SimpleX Chat connections.

For feeds requiring authentication:

feeds:
  - name: "Private Feed"
    url: "https://example.com/private-feed.xml"
    cookies:
      session_id: "${RSS_SESSION_ID}"
      auth_token: "your-auth-token"
⁠🎬 Media Downloads

Automatically download videos from feed entries:

defaults:
  media_dir: "media"  # Global default

feeds:
  - name: "Video Feed"
    url: "https://example.com/videos.xml"
    media_dir: "videos/special"  # Override per feed
    media_all_entries: true      # Download from all entries, not just filtered

Videos are extracted from HTML <video> tags, RSS enclosures, and Media RSS extensions.

β πŸ”’ SimpleX Chat Notifications

SimpleX Chat⁠ is a privacy-first decentralized messenger. RSS Watcher connects to the simplex-chat CLI via WebSocket to send notifications.

In the Docker Compose setup, a simplex-ws-proxy⁠ sidecar handles the WebSocket connection. The proxy shares the network namespace with simplex-chat (via network_mode), connects to the CLI on localhost:5225, and exposes a multiplexed WebSocket on port 6225. This replaces the previous socat relay and adds features like upstream reconnection, request/response routing, and multi-client support.

⁠Configuration
simplex:
  websocket_url: "ws://simplex-chat:6225"  # Docker Compose default (via proxy)
  contacts:
    - "alice"
    - "bob"
  groups:
    - "rss-alerts"
  connect_timeout: 10   # seconds (default: 10)
  message_timeout: 30   # seconds (default: 30)

At least one contact or group is required. Messages are sent to all listed recipients.

⁠Initial Setup (Required Before First Use)

The simplex-chat CLI needs an interactive setup to create a profile and establish contacts/groups. This must be done once before rss-watcher can send notifications.

Step 1: Start the simplex-chat container

docker compose up -d simplex-chat

Step 2: Open an interactive CLI session

# Stop the background service first (only one process can access the database)
docker compose stop simplex-chat

# Launch interactive CLI with the same volume
docker compose run --rm simplex-chat simplex-chat

# When done, restart the service
docker compose start simplex-chat

Step 3: Set up your profile and contacts

Inside the interactive CLI:

ActionCommand
View your profile/u
Create a contact address/ad
Show your address/sa
Accept a contact request/ac <name>
Create a group/g <group_name>
Invite someone to a group/a <group> <contact>
List contacts/cs
List groups/gs
Send a test message@contact hello or #group hello
Quit/q

Important: The contact and group names in your config.yaml must match exactly the names shown by /cs and /gs in the CLI.

Step 4: Verify the configuration

After restarting the service, rss-watcher tests the SimpleX connection at startup. Check the logs:

docker compose logs rss-watcher | grep -i simplex
⁠Local Setup (Without Docker)

If running simplex-chat locally (without the proxy):

# Download from https://github.com/simplex-chat/simplex-chat/releases
# Start with WebSocket server on port 5225
simplex-chat -p 5225

Then set websocket_url: "ws://localhost:5225" in your config (direct connection, no proxy).

β πŸ”€ Per-Feed Notifier Routing

Route specific feeds to specific notification backends:

feeds:
  - name: "Public News"
    url: "https://example.com/feed.xml"
    # notifiers not set β†’ sends to ALL configured backends (Telegram + SimpleX)

  - name: "Telegram Only"
    url: "https://example.com/telegram-feed.xml"
    notifiers: ["telegram"]

  - name: "SimpleX Only"
    url: "https://example.com/private-feed.xml"
    notifiers: ["simplex"]

Valid values: "telegram", "simplex". Omit or set to null to send to all.


β πŸ“– Usage

⁠Command Line
rss-watcher [-h] [-c CONFIG] [-v]

Options:
  -h, --help            Show help message
  -c, --config CONFIG   Config file path (default: config.yaml)
  -v, --verbose         Enable debug logging
⁠Docker Commands
CommandDescription
docker compose up -dStart all services
docker compose logs -fView live logs
docker compose downStop all services
docker compose up -d --buildRebuild and restart
docker compose stop simplex-chatStop SimpleX service
docker compose run --rm simplex-chat simplex-chatOpen interactive SimpleX CLI
docker compose start simplex-chatRestart SimpleX service after CLI use
β πŸ“¬ Message Format

Telegram notifications use HTML format:

[Feed Name]
πŸ“° Entry Title (linked)
πŸ‘€ by Author Name
🏷️ #tag1 #tag2

Entry summary/content...

SimpleX notifications use markdown-style formatting:

*[Feed Name]*
*Entry Title*
https://example.com/entry-link
_by Author Name_
#tag1 #tag2

Entry summary/content...

β πŸ› οΈ Development

⁠Setup
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
⁠Commands
TaskCommand
Run testspytest
Single test filepytest tests/test_filters.py -v
Lint coderuff check .
Format coderuff format .
⁠Architecture
flowchart TB
    A[config.yaml] --> B[RSSWatcher.start]

    B --> F1[Feed 1]
    B --> F2[Feed 2]
    B --> FN[Feed N]

    F1 & F2 & FN --> P1[Fetch]
    P1 --> P2[Filter]
    P2 --> P3[Dedupe]
    P3 --> P4[Download Media]
    P4 --> P5{Notifier Routing}
    P5 --> T[Telegram]
    P5 --> WS[simplex-ws-proxy :6225]
    WS --> S[SimpleX Chat CLI :5225]

β πŸ“„ License

MIT License - see LICENSE⁠ for details.


Made with ❀️ by Grégoire Compagnon⁠

Tag summary

Content type

Image

Digest

sha256:1453c3944…

Size

55.6 MB

Last updated

7 months ago

docker pull obeoneorg/rss-watcher