Starter Kit
Platform OverviewArchitecture Overview
Platform Overview

Architecture Overview

System architecture — backend, frontend, VPS workers, database, and deployment

Architecture Overview

Stack

LayerTechnology
FrontendReact + Vite + TanStack Router + shadcn/ui (port 5000)
BackendFastAPI + SQLModel + Supabase (port 8000)
AuthClerk (OAuth, JWT, RBAC)
DatabaseSupabase (PostgreSQL)
VPS 176.13.123.120 — 8 vCPU, 32GB RAM, 400GB NVMe
VPS 231.97.43.159 — ChangeDetection + WaterCrawl
ProxyWebshare.io residential (p.webshare.io)
DeploymentRender (backend + frontend), Coolify (VPS services)

VPS 1 Services (76.13.123.120)

PortService
8889JobSpy Worker
8890Playwright Worker
8891ScrapeGraph Worker
8892Scrapy Worker
8893UndetectedChrome Worker
8894BeautifulSoup Worker
8896Crawl4AI Worker
8866PaddleOCR
8000Coolify Dashboard

VPS 2 Services (31.97.43.159)

PortService
5000ChangeDetection.io
5001Screenshot File Server
8000Coolify Dashboard
9005WaterCrawl

Data Flow

User → Frontend (Clerk auth) → Backend API
  ↓
ScraperRouter → Engine → Proxy → VPS Worker → Target Site
  ↓
Normalizer → SynthralDocument → Supabase DB
  ↓
Frontend displays results

Key Backend Paths

  • backend/app/scraping/service.py — Main scraping service
  • backend/app/scraping/engines/ — 17 engine implementations
  • backend/app/scraping/proxy_pool.py — Webshare proxy management
  • backend/app/ocr/service.py — OCR processing
  • backend/app/api/routes/ — All API endpoints
  • backend/app/models.py — Database models

Environment

  • Python venv: venv_final/ (not system Python)
  • PYTHONIOENCODING=utf-8 required (emojis in main.py)
  • PYTHONUNBUFFERED=1 required (log buffering)
  • Backend takes ~15-20s to fully initialize
Was this page helpful?

Last updated today

Built with Documentation.AI