Initializing
Back to Projects
Year2025
DomainFullstack
AccessPrivate Repo
Complexity7.8 / 10
Video DemoArchitecture DocsScreenshots
TypeScriptNode.jsNestJSPostgreSQLRedisBullMQReactNext.jsPrismaSocket.IOMeta APIDockerPM2
FullstackProduction

HERALD — Social Media Automation System

A self-hosted social media automation platform for scheduling Facebook and Instagram posts. Features Web Dashboard, CLI, and Headless modes with Meta Graph API integration.

Supported Platforms0
Operating Modes0
Queue WorkersBullMQ
Content SourcesCSV + Google Sheets
Image ValidationPre-flight Check
NotificationsTelegram Bot

Table of Contents


The Challenge

Managing social media presence across Facebook and Instagram manually is time-consuming and error-prone:

  • Manual posting: Scheduling posts requires constant attention and multiple login sessions
  • Cross-platform duplication: Posting the same content to multiple platforms means duplicating work
  • No offline capability: Relying on cloud tools means losing control when services go down
  • Platform limits: Meta's API has rate limits that need intelligent handling
  • Content management: Managing images, links, and scheduling across multiple accounts is fragmented
  • No analytics: Understanding post performance requires manual data gathering

The requirement: Build a self-hosted social media automation system that can schedule posts to Facebook and Instagram, handle rate limiting, provide a Web Dashboard and CLI, and run autonomously on a VPS — with all data staying on your own server.


Architecture & Solution

HERALD provides three operating modes (Web Dashboard, CLI, Headless) backed by a single NestJS API with PostgreSQL and BullMQ job queues.

Parsing system architecture diagram...

Component Responsibilities

ComponentResponsibility
Web DashboardGlassmorphism UI for post management, analytics visualization
CLI (herald)Terminal commands for power users and scripting
Headless WorkerPM2-managed background process for autonomous execution
NestJS APIREST API, WebSocket, auth, business logic
PostgreSQLPersistent storage for posts, schedules, analytics
BullMQ + RedisJob queue with retry logic, rate limiting per token
Meta Graph APIFacebook and Instagram post execution

Tech Stack

LayerTechnologyRole
BackendNestJS 11Modular API framework with CLI, CRON, WebSockets
LanguageTypeScript 5.xType safety across all layers
ORMPrisma 5Type-safe SQL, auto migrations
DatabasePostgreSQL 16Concurrent write-safe, free
Job QueueBullMQ 5Reliable Redis-backed scheduled jobs
Queue BrokerRedis 7Required by BullMQ, session caching
HTTP ClientAxiosMeta Graph API calls
Scheduler@nestjs/scheduleCRON expression support
Real-timeSocket.IOLive post status updates
DashboardNext.js 14React frontend with SSR
ChartsChart.js + react-chartjs-2Free analytics visualization
Process ManagerPM2Keeps workers alive, auto-restart
Reverse ProxyNginxServes frontend + API + uploads

Key Engineering Decisions

1. PostgreSQL over SQLite

PM2 launches API and Worker as separate processes. Both write simultaneously (status updates, analytics, job tracking). SQLite uses file-level locking and throws SQLITE_BUSY under concurrent writes.

prisma
// schema.prisma
model Post {
  id          String   @id @default(uuid())
  platform    Platform @default(FACEBOOK)
  content     String
  mediaUrls   String[]
  status      PostStatus @default(DRAFT)
  scheduledAt DateTime?
  publishedAt DateTime?
  platformPostId String?
  createdAt   DateTime @default(now())
  updatedAt   DateTime @updatedAt
  
  @@index([status])
  @@index([scheduledAt])
}

2. BullMQ Rate Limiting per Token

Each Meta token has different rate limits. BullMQ handles per-token queue separation:

typescript
// queue/post.queue.ts
@Injectable()
export class PostQueueService {
  async addPublishJob(post: Post, token: MetaToken) {
    const queue = this.queueManager.getQueue(token.id);
    
    await queue.add('publish', {
      postId: post.id,
      tokenId: token.id,
    }, {
      removeOnComplete: true,
      attempts: 3,
      backoff: {
        type: 'exponential',
        delay: 5000,
      },
    });
  }
}

3. Post State Machine with Partial Failure Handling

Cross-platform posts can partially fail. HERALD tracks state with PARTIALLY_PUBLISHED:

code
DRAFT ──(user schedules)──► PENDING
    │                         │
(Outbox Poller picks up)
    │                         │
QUEUED
    │                         │
(Worker executes)
    │                         │
    ┌──────────────┬──────────┴────────────┐
    │              │                         │
PUBLISHED    PARTIALLY_PUBLISHED       FAILED
    │              │                         │
(some platforms      (all failed)
published)
    │              │
    ▼              ▼
  ARCHIVED       ARCHIVED

4. Pre-flight Image URL Validation

Before posting, HERALD validates all media URLs are accessible:

typescript
// services/media-validator.service.ts
async function validateMediaUrls(urls: string[]): Promise<ValidationResult> {
  const results = await Promise.all(
    urls.map(async (url) => {
      try {
        const response = await axios.head(url, { timeout: 5000 });
        if (!response.headers['content-type']?.startsWith('image/')) {
          return { url, valid: false, reason: 'Not an image' };
        }
        return { url, valid: true };
      } catch (error) {
        return { url, valid: false, reason: 'Unreachable' };
      }
    })
  );
  
  const allValid = results.every(r => r.valid);
  return { valid: allValid, results };
}

5. Content Ingestion from CSV and Google Sheets

Bulk import posts from external sources:

typescript
// services/ingestion.service.ts
async function ingestFromCSV(filePath: string): Promise<IngestionResult> {
  const content = fs.readFileSync(filePath, 'utf-8');
  const records = parse(content, { columns: true, skip_empty_lines: true });
  
  const posts = records.map((row, index) => ({
    platform: row.platform?.toUpperCase() || 'FACEBOOK',
    content: row.caption,
    mediaUrls: row.images?.split('|').filter(Boolean) || [],
    scheduledAt: row.scheduled_at ? new Date(row.scheduled_at) : null,
  }));
  
  return this.postService.bulkCreate(posts);
}

Post State Machine

typescript
enum PostStatus {
  DRAFT = 'DRAFT',
  PENDING = 'PENDING',
  QUEUED = 'QUEUED',
  PUBLISHED = 'PUBLISHED',
  PARTIALLY_PUBLISHED = 'PARTIALLY_PUBLISHED',
  FAILED = 'FAILED',
  VALIDATION_FAILED = 'VALIDATION_FAILED',
  ARCHIVED = 'ARCHIVED',
}

State Transitions

FromToTrigger
DRAFTPENDINGUser schedules post
PENDINGQUEUEDOutbox poller picks up
QUEUEDPUBLISHEDWorker executes successfully
QUEUEDPARTIALLY_PUBLISHEDSome platforms succeed, some fail
QUEUEDFAILEDAll platforms fail after retries
QUEUEDVALIDATION_FAILEDMedia URLs unreachable
PUBLISHED/PARTIALLY_PUBLISHEDARCHIVEDUser archives

Meta API Integration

Facebook Page Posts

typescript
// modules/facebook/facebook-post.service.ts
async function publishToPage(post: Post, token: string): Promise<PublishResult> {
  const endpoint = `/${process.env.FB_PAGE_ID}/photos`;
  
  const formData = new FormData();
  formData.append('message', post.content);
  if (post.mediaUrls?.[0]) {
    formData.append('url', post.mediaUrls[0]);
  }
  
  const response = await axios.post(endpoint, formData, {
    headers: { Authorization: `Bearer ${token}` },
  });
  
  return { platformPostId: response.data.id, platform: 'FACEBOOK' };
}

Instagram Business Posts

typescript
// modules/instagram/ig-post.service.ts
async function publishToInstagram(post: Post, token: string): Promise<PublishResult> {
  // 1. Upload media to container
  const container = await axios.post(
    `/${process.env.IG_BUSINESS_ID}/media`,
    {
      image_url: post.mediaUrls[0],
      caption: post.content,
    },
    { headers: { Authorization: `Bearer ${token}` } }
  );
  
  // 2. Publish from container
  const publish = await axios.post(
    `/${process.env.IG_BUSINESS_ID}/media_publish`,
    { creation_id: container.data.id },
    { headers: { Authorization: `Bearer ${token}` } }
  );
  
  return { platformPostId: publish.data.id, platform: 'INSTAGRAM' };
}

Content Ingestion Pipeline

CSV Import

csv
platform,caption,images,scheduled_at
FACEBOOK,"Launch announcement!","https://example.com/img1.jpg|https://example.com/img2.jpg","2026-05-20T10:00:00Z"
INSTAGRAM,"Behind the scenes","https://example.com/bts.jpg",,

Google Sheets Integration

typescript
// modules/ingestion/google-sheets.service.ts
async function fetchFromSheet(sheetId: string): Promise<Post[]> {
  const response = await this.googleClient.spreadsheets.values.get({
    spreadsheetId: sheetId,
    range: 'Posts!A:E',
  });
  
  return this.mapRowsToPosts(response.data.values);
}

Three Operating Modes

1. Web Dashboard — Premium Glassmorphism UI

  • Post CRUD operations
  • Schedule management with calendar view
  • Analytics with Chart.js visualizations
  • Real-time updates via Socket.IO
  • Dark mode with glassmorphism styling

2. CLI — Terminal Commands for Power Users

bash
# Schedule a post
herald post create --platform facebook --content "Hello World" --media "https://..."

# List scheduled posts
herald post list --status pending

# Import from CSV
herald import csv ./posts.csv

# Check queue status
herald queue status

3. Headless — Autonomous Background Process

  • PM2-managed worker process
  • Watches scheduled posts and executes automatically
  • Graceful shutdown handling
  • Automatic retry on failure

Deployment

Quick Start

bash
# Install dependencies
npm install

# Setup database
npx prisma migrate deploy
npx prisma db seed

# Start services
docker compose up -d redis postgres
npm run start:api
npm run start:worker

Infrastructure (Docker Compose)

yaml
services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: herald
      POSTGRES_PASSWORD: password
      POSTGRES_DB: herald_db
    volumes:
      - postgres_data:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data

Production Checklist

bash
# Build for production
npm run build

# Start with PM2
pm2 start ecosystem.config.js
pm2 save

# Access at http://your-vps:3000

Roadmap

Phase 1 — Core Foundation (Complete)

  • NestJS API with Prisma
  • PostgreSQL database
  • Basic post CRUD

Phase 2 — Platform Integration (Complete)

  • Facebook Page posting
  • Instagram Business posting
  • Meta OAuth flow

Phase 3 — Scheduling Engine (Complete)

  • BullMQ job queue
  • CRON-based scheduling
  • Rate limiting per token

Phase 4 — Dashboard (Complete)

  • Next.js dashboard
  • Real-time updates via Socket.IO
  • Analytics charts

Phase 5 — CLI & Headless (Complete)

  • Herald CLI commands
  • PM2 worker process

Phase 6 — Content Ingestion (Complete)

  • CSV import
  • Google Sheets integration
  • Pre-flight image validation

Phase 7 — Enhanced Analytics (In Progress)

  • Advanced metrics
  • Engagement tracking

Phase 8 — Multi-Platform Support (Planned)

  • Twitter/X integration
  • LinkedIn integration

Phase 9 — AI Enhancement (Future)

  • LLM-generated captions
  • Smart posting times

Engineering Proof

Real-world validation, system demonstrations, and interface captures of the execution states.

System Demonstration

Video walkthrough detailing core logic, interactions, and system behaviors in action.

System Captures

Architecture Feedback

Spotted a potential optimization or antipattern? Let me know.

Submit a Technical Suggestion