Telegram Harvest

Telegram Message Forwarding & Archival System — Forward-to-Group architecture with zero local storage.

Architecture

Messages are forwarded from source channels/groups to archive groups using Telegram's native forward API. No media is downloaded or stored locally — all content remains in Telegram's cloud. The database stores only lightweight metadata (message IDs, timestamps, sender info, text previews) for search and analytics.

Source Channel → [Filter] → [Rate Limiter] → Archive Group
                                                   ↓
                                            PostgreSQL (metadata only)

Tech Stack

Backend: Python 3.11, FastAPI, Telethon (MTProto), SQLAlchemy 2.x, Celery, Redis Frontend: React 18, TypeScript, Vite, TailwindCSS, React Query, Recharts Database: PostgreSQL 15 (metadata), Redis 7 (queue/cache) DevOps: Docker Compose, Nginx

Quick Start

Prerequisites

• Docker and Docker Compose
• Telegram API credentials from my.telegram.org/apps

Setup

1. Clone the repository:

   git clone https://github.com/subkhanibnuaji/telegrambot.git
   cd telegrambot
   

2. Copy environment file and configure:

   cp .env.example .env
   # Edit .env with your settings
   

3. Start all services:

   docker-compose up -d
   

4. Run database migrations:

   docker-compose exec backend alembic upgrade head
   

5. Access the application:

   • Frontend: http://localhost:3000
   • Backend API: http://localhost:8000
   • API docs: http://localhost:8000/docs

Usage

1. Register an account in the web UI
2. Connect your Telegram account (API ID, API Hash, phone number)
3. Add sources — select channels/groups to archive from
4. Add archives — select or create groups to forward messages to
5. Create jobs — start bulk forwarding or continuous monitoring
6. Search & analyze — use the dashboard for analytics and message search

Key Features

• Forward-to-Group: Messages forwarded via Telegram API, no local media storage
• Rate Limiting: Token bucket algorithm (30 msg/s global, 1 msg/s per chat)
• Incremental Forwarding: Tracks last forwarded message per source
• Filters: Keyword, sender, media type, and date range filters
• Real-time Monitoring: Continuous mode listens for new messages
• Analytics Dashboard: Activity charts, media distribution, top senders
• Full-text Search: Search indexed message metadata
• WebSocket Updates: Real-time job progress
• Encrypted Sessions: Telethon session data encrypted at rest

Rate Limits

Respects Telegram's official limits:

• Global: 30 messages/second
• Per chat: 1 message/second
• FloodWait errors handled with exponential backoff

Project Structure

telegrambot/
├── backend/           # FastAPI + Telethon + Celery
│   ├── app/
│   │   ├── api/       # REST API routes
│   │   ├── core/      # Security, rate limiter, Telethon client
│   │   ├── models/    # SQLAlchemy models
│   │   ├── schemas/   # Pydantic schemas
│   │   ├── services/  # Business logic
│   │   └── tasks/     # Celery background tasks
│   └── alembic/       # Database migrations
├── frontend/          # React + TypeScript + Vite
│   └── src/
│       ├── api/       # API client
│       ├── pages/     # Page components
│       ├── hooks/     # Custom hooks
│       └── types/     # TypeScript types
├── nginx/             # Reverse proxy config
└── docker-compose.yml

Security

• Each user must use their own Telegram API credentials
• Session data encrypted with Fernet symmetric encryption
• JWT authentication with refresh token rotation
• Credentials never logged or exposed
• .env file excluded from version control
• Input validation on all API endpoints

License

MIT