KembaliArsip / superapp-automation / bots-automation

superapp-automation

bots-automation

8 file · 202.1 KB

Buka di GitHub Lihat Sumber Halaman

Preview langsung dinonaktifkan untuk static export. Buka repo di GitHub untuk melihat halaman HTML asli.

README

🤖 Bot Automation Hub

A comprehensive suite of automation bots and tools for Discord, Telegram, n8n workflows, and PDF data extraction.

Overview
Architecture
Bot Index
Quick Start
Installation
Deployment
Environment Variables
Project Structure
API Reference
Contributing
Credits
License

Overview

Bot Automation Hub is a unified platform for deploying and managing multiple automation bots and services. It provides:

Discord Bot: Message forwarding and automation with web dashboard
Telegram Bot: Channel/group automation with data harvesting
n8n Workflows: Visual workflow automation platform
PDF Data Extractor: Intelligent PDF data extraction service
Additional Tools: Backup utilities, website monitoring bots
Unified Deployment: Docker, Vercel, Railway, and local deployment support
Centralized Orchestration: Deploy all services together with docker-compose

This hub simplifies managing multiple bot projects with consistent configuration, logging, and deployment strategies.

Architecture

graph TB
    subgraph "Bot Automation Hub"
        subgraph "Discord Ecosystem"
            DB[Discord Backend<br/>FastAPI]
            DF[Discord Frontend<br/>React]
            DB -->|API| DF
        end

        subgraph "Telegram Ecosystem"
            TB[Telegram Backend<br/>FastAPI]
            TF[Telegram Frontend<br/>Vite]
            TC[Celery Worker<br/>Task Processing]
            TB -->|API| TF
            TC -->|Tasks| TB
        end

        subgraph "Automation Stack"
            N8N[n8n<br/>Workflow Engine]
            PDF[PDF Extractor<br/>Flask]
        end

        subgraph "Shared Infrastructure"
            PG[(PostgreSQL<br/>Database)]
            RD[(Redis<br/>Cache)]
        end

        DB --> PG
        TB --> PG
        TC --> RD
        N8N --> PG
        PDF -->|File Storage| FS[File System]

        style DB fill:#4CAF50
        style TB fill:#0088cc
        style N8N fill:#FF6B6B
        style PDF fill:#FFA500
        style PG fill:#336699
        style RD fill:#DC3545
    end

Technology Stack

Layer	Technologies
Backend	FastAPI, SQLAlchemy, Celery, Redis
Frontend	React, Vue, Vite, Radix UI, TailwindCSS
Databases	PostgreSQL, SQLite, Redis
Automation	n8n, Telegram API, Discord.py
Infrastructure	Docker, Docker Compose, Vercel, Railway
Languages	Python 3.9+, Node.js 18+, TypeScript

Bot Index

Name	Description	Tech Stack	Deploy	Status
Discord Bot	Message forwarding & automation with web dashboard	FastAPI, React, PostgreSQL	Docker, Local	✅ Active
Telegram Bot	Channel/group automation & data harvesting	FastAPI, Vite, PostgreSQL, Celery	Docker, Local	✅ Active
n8n Automation	Visual workflow automation engine	n8n, PostgreSQL, Node.js	Docker	✅ Active
PDF Data Extractor	Intelligent PDF parsing & data extraction	Flask, Python, PyPDF	Docker, Local	✅ Active
Telegram Backup Tool	Backup Telegram messages & media	Python, Telethon	Docker, Local	✅ Active
Meetbot	Meeting management & scheduling bot	Node.js, Express	Docker, Vercel	⚠️ In Development
Website Bot	Website monitoring & alert system	Python, BeautifulSoup	Local	⚠️ In Development
Pintar.bi Scraper	Indonesian website data scraper	Python, Selenium	Local	⚠️ Inactive

Quick Start

1. Clone & Setup

# Clone the repository
git clone https://github.com/subkhanibnuaji/bots-automation.git
cd bots-automation

# Create environment file
cp .env.example .env

# Edit configuration
nano .env

2. Deploy with Docker

# All bots together
docker-compose -f docker-compose.hub.yml up -d

# Or specific bot
docker-compose -f discord-bot/docker-compose.yml up -d

# View logs
docker-compose -f docker-compose.hub.yml logs -f

# Stop services
docker-compose -f docker-compose.hub.yml down

3. Deploy with Universal Script

# Make deploy script executable
chmod +x deploy.sh

# Check status
./deploy.sh status

# Deploy specific bot
./deploy.sh discord-bot docker
./deploy.sh telegrambot docker
./deploy.sh n8n docker

# Deploy all Docker bots
./deploy.sh all docker

# Health check
./deploy.sh health

# Stop all
./deploy.sh all stop

4. Access Services

Service	URL	Credentials
Discord Frontend	http://localhost:3001	Configure in app
Telegram Frontend	http://localhost:3002	Configure in app
n8n	http://localhost:5678	Setup on first run
PDF Extractor	http://localhost:5000	No auth required
Discord Backend API	http://localhost:8001	Token-based
Telegram Backend API	http://localhost:8002	Token-based

Installation

Prerequisites

Docker & Docker Compose 20.10+
Node.js 18+ (for local development)
Python 3.9+ (for local Python bots)
4GB RAM minimum (8GB+ recommended)
20GB disk space

Step 1: Clone Repository

git clone https://github.com/subkhanibnuaji/bots-automation.git
cd bots-automation

Step 2: Configure Environment

# Copy example environment file
cp .env.example .env

# Edit with your credentials
nano .env

Step 3: Start Services

Option A: All services together

docker-compose -f docker-compose.hub.yml up -d

Option B: Individual bots

cd discord-bot
docker-compose up -d

# OR
cd telegrambot
docker-compose up -d

# OR
cd n8n-automation
docker-compose up -d

Option C: Local development

# Discord Bot
cd discord-bot/backend
pip install -r requirements.txt
uvicorn app.main:app --reload

# In another terminal
cd discord-bot/frontend
npm install
npm run dev

Step 4: Verify Installation

# Check all services
./deploy.sh status

# Health check
./deploy.sh health

# View logs
docker-compose -f docker-compose.hub.yml logs -f

Deployment

Docker Deployment (Recommended)

Individual Bot

# Build and run single bot
docker-compose -f discord-bot/docker-compose.yml up -d

# View logs
docker-compose -f discord-bot/docker-compose.yml logs -f

# Stop service
docker-compose -f discord-bot/docker-compose.yml down

All Bots Together

# Use unified docker-compose
docker-compose -f docker-compose.hub.yml up -d

# Scale specific service
docker-compose -f docker-compose.hub.yml up -d --scale telegram-celery-worker=2

# View logs
docker-compose -f docker-compose.hub.yml logs -f [service-name]

Vercel Deployment

Supported for: Spam, Discord Bot (Frontend)

# Install Vercel CLI
npm install -g vercel

# Deploy spam bot
cd spam
vercel deploy --prod

# Or use deploy script
../deploy.sh spam vercel

Railway Deployment

Supported for: Telegram Bot, n8n, PDF Extractor

# Link Railway project
railway link

# Deploy
railway up

# View logs
railway logs

Manual Deployment

For custom infrastructure:

# Build images
docker build -t discord-bot:latest ./discord-bot/backend
docker build -t telegram-bot:latest ./telegrambot/backend

# Push to registry
docker tag discord-bot:latest myregistry/discord-bot:latest
docker push myregistry/discord-bot:latest

# Deploy on your infrastructure
# (Kubernetes, AWS ECS, DigitalOcean, etc.)

Environment Variables

Core Configuration

# Environment mode
ENVIRONMENT=production  # or development

# API Configuration
FRONTEND_URL=http://localhost:3000
VITE_API_URL=http://localhost:8000
API_PORT=8000

Discord Bot

# Discord API
DISCORD_BOT_TOKEN=your_discord_token_here
DISCORD_SECRET_KEY=your_secret_key_here

# Frontend
FRONTEND_URL=http://localhost:3000

Telegram Bot

# Telegram API
TELEGRAM_API_ID=your_api_id
TELEGRAM_API_HASH=your_api_hash
TELEGRAM_BOT_TOKEN=your_bot_token

# Authentication
SECRET_KEY=your_secret_key_here
JWT_SECRET_KEY=your_jwt_secret_here

# Database
POSTGRES_USER=telegram
POSTGRES_PASSWORD=secure_password
POSTGRES_DB=telegram_harvest
TELEGRAM_DB=telegram_harvest

# Redis
REDIS_URL=redis://redis:6379/1

n8n Configuration

# n8n Core
N8N_HOST=n8n.localhost
N8N_PROTOCOL=https
N8N_WEBHOOK_URL=https://n8n.localhost/webhook
N8N_ENCRYPTION_KEY=your_encryption_key_here

# Database
POSTGRES_USER=hubadmin
POSTGRES_PASSWORD=hubsecure123
POSTGRES_DB=n8n
N8N_DB=n8n

# Settings
GENERIC_TIMEZONE=UTC
N8N_COMMUNITY_PACKAGES_ENABLED=true

Shared Infrastructure

# PostgreSQL
POSTGRES_USER=hubadmin
POSTGRES_PASSWORD=hubsecure123
POSTGRES_INITDB_ARGS="-c max_connections=200"

# Redis
REDIS_URL=redis://redis:6379/0

Create .env File

# Template
cat > .env << EOF
ENVIRONMENT=development
DISCORD_BOT_TOKEN=your_token_here
TELEGRAM_API_ID=your_id_here
TELEGRAM_API_HASH=your_hash_here
TELEGRAM_BOT_TOKEN=your_token_here
POSTGRES_PASSWORD=secure_password
N8N_ENCRYPTION_KEY=your_key_here
EOF

Project Structure

bots-automation/
├── deploy.sh                          # Universal deployment script
├── docker-compose.hub.yml             # Unified Docker Compose
├── README.md                          # This file
├── .env.example                       # Environment variables template
├── .gitignore
│
├── discord-bot/                       # Discord Message Forwarder
│   ├── backend/                       # FastAPI backend
│   │   ├── app/
│   │   ├── Dockerfile
│   │   └── requirements.txt
│   ├── frontend/                      # React frontend
│   │   ├── src/
│   │   ├── Dockerfile
│   │   └── package.json
│   ├── nginx/                         # Reverse proxy config
│   ├── docker-compose.yml
│   ├── vercel.json
│   └── README.md
│
├── telegrambot/                       # Telegram Automation Bot
│   ├── backend/                       # FastAPI backend
│   │   ├── app/
│   │   ├── Dockerfile
│   │   └── requirements.txt
│   ├── frontend/                      # Vite frontend
│   │   ├── src/
│   │   ├── Dockerfile
│   │   └── package.json
│   ├── nginx/                         # Reverse proxy config
│   ├── docker-compose.yml
│   ├── vercel.json
│   ├── railway.json
│   └── README.md
│
├── n8n-automation/                    # n8n Workflow Engine
│   ├── docker-compose.yml
│   ├── .env.example
│   └── README.md
│
├── pdf-data-extractor/                # PDF Data Extraction
│   ├── app.py                         # Flask application
│   ├── extractor.py                   # PDF extraction logic
│   ├── batch_extract.py               # Batch processing
│   ├── requirements.txt
│   ├── Dockerfile
│   └── README.md
│
├── telegram-backup-tool/              # Telegram Backup Utility
│   ├── main.py
│   ├── requirements.txt
│   └── README.md
│
├── meetbot-app-new/                   # Meeting Bot
│   ├── backend/
│   ├── frontend/
│   ├── docker-compose.yml
│   └── README.md
│
└── pintar.bi.go.id-bot/               # Website Scraper
    ├── scraper.py
    └── README.md

API Reference

Discord Bot API

Base URL: http://localhost:8001

Health Check

GET /health

Response:

{
  "status": "healthy",
  "uptime": "2.5 hours"
}

Get Bot Status

GET /api/status
Authorization: Bearer TOKEN

Forward Message

POST /api/messages/forward
Authorization: Bearer TOKEN
Content-Type: application/json

{
  "source_channel": "channel_id",
  "target_channel": "target_id",
  "message": "content"
}

Telegram Bot API

Base URL: http://localhost:8002

Health Check

GET /health

Get Chats

GET /api/chats
Authorization: Bearer TOKEN

Get Messages

GET /api/chats/{chat_id}/messages
Authorization: Bearer TOKEN

Extract Data

POST /api/extract
Authorization: Bearer TOKEN
Content-Type: application/json

{
  "chat_id": 123456,
  "start_date": "2024-01-01",
  "end_date": "2024-03-20"
}

PDF Extractor API

Base URL: http://localhost:5000

Health Check

GET /health

Extract Text

POST /extract/text
Content-Type: multipart/form-data

file: (binary PDF)

Response:

{
  "text": "extracted text...",
  "pages": 10,
  "confidence": 0.95
}

Extract Tables

POST /extract/tables
Content-Type: multipart/form-data

file: (binary PDF)

Response:

{
  "tables": [
    {
      "page": 1,
      "data": [[...], [...]]
    }
  ]
}

n8n API

Base URL: http://localhost:5678

Documentation: http://localhost:5678/api/

Full API Documentation

Discord Bot: See discord-bot/README.md
Telegram Bot: See telegrambot/README.md
n8n: http://localhost:5678/api/
PDF Extractor: See pdf-data-extractor/README.md

Individual Bot Guides

Discord Bot Setup

cd discord-bot

# Create .env
cp ../.env .

# Build and run
docker-compose up -d

# Access at http://localhost:3001

Configuration:

Get Discord bot token from https://discord.com/developers/applications
Set DISCORD_BOT_TOKEN in .env
Invite bot to server with OAuth2 URL
Configure forwarding rules in web dashboard

Telegram Bot Setup

cd telegrambot

# Create .env
cp ../.env .

# Build and run
docker-compose up -d

# Access at http://localhost:3002

Configuration:

Get API ID/Hash from https://my.telegram.org
Get bot token from @BotFather
Set credentials in .env
Configure channels and automation rules

n8n Setup

cd n8n-automation

# Create .env
cp ../.env .

# Start
docker-compose up -d

# Access at http://localhost:5678

First Run:

Set admin email and password
Create workflows via UI
Configure integrations with external services
Set up webhooks and triggers

PDF Extractor Setup

cd pdf-data-extractor

# Install dependencies
pip install -r requirements.txt

# Run locally
python app.py

# Or with Docker
docker build -t pdf-extractor .
docker run -p 5000:5000 pdf-extractor

Monitoring & Maintenance

View Logs

# All services
docker-compose -f docker-compose.hub.yml logs -f

# Specific service
docker-compose -f docker-compose.hub.yml logs -f telegram-backend

# Last 100 lines
docker-compose -f docker-compose.hub.yml logs --tail=100

Health Checks

# Run health check
./deploy.sh health

# Manual check
curl http://localhost:8001/health   # Discord
curl http://localhost:8002/health   # Telegram
curl http://localhost:5678/healthz  # n8n
curl http://localhost:5000/health   # PDF Extractor

Database Backup

# PostgreSQL backup
docker exec hub_postgres_main pg_dump -U hubadmin hubadmin > backup.sql

# Restore
docker exec -i hub_postgres_main psql -U hubadmin hubadmin < backup.sql

Performance Monitoring

# Check resource usage
docker stats

# View container details
docker inspect hub_telegram_backend

Cleanup

# Remove stopped containers
docker container prune

# Remove unused images
docker image prune

# Remove all volumes (WARNING: data loss)
docker volume prune

Troubleshooting

Connection Issues

# Test service connectivity
docker-compose -f docker-compose.hub.yml exec telegram-backend curl http://postgres-main:5432

# Check network
docker network inspect hub-network

# Restart containers
docker-compose -f docker-compose.hub.yml restart

Database Issues

# Check PostgreSQL
docker logs hub_postgres_main

# Connect to database
docker exec -it hub_postgres_main psql -U hubadmin -d telegram_harvest

# Run migrations
docker-compose -f docker-compose.hub.yml exec telegram-backend alembic upgrade head

Memory Issues

# Check resource limits
docker inspect hub_telegram_backend | grep -A 10 MemoryLimit

# Increase if needed - update docker-compose.yml:
# services:
#   telegram-backend:
#     mem_limit: 2g

Port Conflicts

# Check port usage
lsof -i :8001
lsof -i :5000

# Change ports in docker-compose.yml if needed

Contributing

Contributions are welcome! Please follow these guidelines:

Development Setup

# Clone and setup
git clone https://github.com/subkhanibnuaji/bots-automation.git
cd bots-automation

# Create feature branch
git checkout -b feature/my-feature

# Make changes and commit
git add .
git commit -m "feat: add my feature"

# Push and create PR
git push origin feature/my-feature

Code Standards

Follow PEP 8 for Python code
Use ESLint for JavaScript/TypeScript
Add tests for new features
Update documentation

Reporting Issues

Please include:

Detailed description
Steps to reproduce
Error logs
System information (OS, Docker version)

Credits

Created by: Ibnu Aji Repository: https://github.com/subkhanibnuaji/bots-automation

Technologies Used

License

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Support

Last Updated: 2024-03-20 Status: Active Development

File Utama

.gitignore
README.md
api-docs.html
deploy.sh
docker-compose.hub.yml
health-monitor.html
hub-dashboard.html
index.html

bots-automation

README

🤖 Bot Automation Hub

Table of Contents

Overview

Architecture

Technology Stack

Bot Index

Quick Start

1. Clone & Setup

2. Deploy with Docker

3. Deploy with Universal Script

4. Access Services

Installation

Prerequisites

Step 1: Clone Repository

Step 2: Configure Environment

Step 3: Start Services

Step 4: Verify Installation

Deployment

Docker Deployment (Recommended)

Individual Bot

All Bots Together

Vercel Deployment

Railway Deployment

Manual Deployment

Environment Variables

Core Configuration

Discord Bot

Telegram Bot

n8n Configuration

Shared Infrastructure

Create .env File

Project Structure

API Reference

Discord Bot API

Health Check

Get Bot Status

Forward Message

Telegram Bot API

Health Check

Get Chats

Get Messages

Extract Data

PDF Extractor API

Health Check

Extract Text

Extract Tables

n8n API

Full API Documentation

Individual Bot Guides

Discord Bot Setup

Telegram Bot Setup

n8n Setup

PDF Extractor Setup

Monitoring & Maintenance

View Logs

Health Checks

Database Backup

Performance Monitoring

Cleanup

Troubleshooting

Connection Issues

Database Issues

Memory Issues

Port Conflicts

Contributing

Development Setup

Code Standards

Reporting Issues

Credits

Technologies Used

License

Support

File Utama