Table of Contents
- The Challenge
- Architecture & Solution
- Tech Stack
- Key Engineering Decisions
- Post State Machine
- Meta API Integration
- Content Ingestion Pipeline
- Three Operating Modes
- Deployment
- Roadmap
The Challenge
Managing social media presence across Facebook and Instagram manually is time-consuming and error-prone:
- Manual posting: Scheduling posts requires constant attention and multiple login sessions
- Cross-platform duplication: Posting the same content to multiple platforms means duplicating work
- No offline capability: Relying on cloud tools means losing control when services go down
- Platform limits: Meta's API has rate limits that need intelligent handling
- Content management: Managing images, links, and scheduling across multiple accounts is fragmented
- No analytics: Understanding post performance requires manual data gathering
The requirement: Build a self-hosted social media automation system that can schedule posts to Facebook and Instagram, handle rate limiting, provide a Web Dashboard and CLI, and run autonomously on a VPS — with all data staying on your own server.
Architecture & Solution
HERALD provides three operating modes (Web Dashboard, CLI, Headless) backed by a single NestJS API with PostgreSQL and BullMQ job queues.
Component Responsibilities
| Component | Responsibility |
|---|---|
| Web Dashboard | Glassmorphism UI for post management, analytics visualization |
| CLI (herald) | Terminal commands for power users and scripting |
| Headless Worker | PM2-managed background process for autonomous execution |
| NestJS API | REST API, WebSocket, auth, business logic |
| PostgreSQL | Persistent storage for posts, schedules, analytics |
| BullMQ + Redis | Job queue with retry logic, rate limiting per token |
| Meta Graph API | Facebook and Instagram post execution |
Tech Stack
| Layer | Technology | Role |
|---|---|---|
| Backend | NestJS 11 | Modular API framework with CLI, CRON, WebSockets |
| Language | TypeScript 5.x | Type safety across all layers |
| ORM | Prisma 5 | Type-safe SQL, auto migrations |
| Database | PostgreSQL 16 | Concurrent write-safe, free |
| Job Queue | BullMQ 5 | Reliable Redis-backed scheduled jobs |
| Queue Broker | Redis 7 | Required by BullMQ, session caching |
| HTTP Client | Axios | Meta Graph API calls |
| Scheduler | @nestjs/schedule | CRON expression support |
| Real-time | Socket.IO | Live post status updates |
| Dashboard | Next.js 14 | React frontend with SSR |
| Charts | Chart.js + react-chartjs-2 | Free analytics visualization |
| Process Manager | PM2 | Keeps workers alive, auto-restart |
| Reverse Proxy | Nginx | Serves frontend + API + uploads |
Key Engineering Decisions
1. PostgreSQL over SQLite
PM2 launches API and Worker as separate processes. Both write simultaneously (status updates, analytics, job tracking). SQLite uses file-level locking and throws SQLITE_BUSY under concurrent writes.
// schema.prisma
model Post {
id String @id @default(uuid())
platform Platform @default(FACEBOOK)
content String
mediaUrls String[]
status PostStatus @default(DRAFT)
scheduledAt DateTime?
publishedAt DateTime?
platformPostId String?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([status])
@@index([scheduledAt])
}2. BullMQ Rate Limiting per Token
Each Meta token has different rate limits. BullMQ handles per-token queue separation:
// queue/post.queue.ts
@Injectable()
export class PostQueueService {
async addPublishJob(post: Post, token: MetaToken) {
const queue = this.queueManager.getQueue(token.id);
await queue.add('publish', {
postId: post.id,
tokenId: token.id,
}, {
removeOnComplete: true,
attempts: 3,
backoff: {
type: 'exponential',
delay: 5000,
},
});
}
}3. Post State Machine with Partial Failure Handling
Cross-platform posts can partially fail. HERALD tracks state with PARTIALLY_PUBLISHED:
DRAFT ──(user schedules)──► PENDING
│ │
│ (Outbox Poller picks up)
│ │
│ QUEUED
│ │
│ (Worker executes)
│ │
┌──────────────┬──────────┴────────────┐
│ │ │
PUBLISHED PARTIALLY_PUBLISHED FAILED
│ │ │
│ (some platforms (all failed)
│ published)
│ │
▼ ▼
ARCHIVED ARCHIVED4. Pre-flight Image URL Validation
Before posting, HERALD validates all media URLs are accessible:
// services/media-validator.service.ts
async function validateMediaUrls(urls: string[]): Promise<ValidationResult> {
const results = await Promise.all(
urls.map(async (url) => {
try {
const response = await axios.head(url, { timeout: 5000 });
if (!response.headers['content-type']?.startsWith('image/')) {
return { url, valid: false, reason: 'Not an image' };
}
return { url, valid: true };
} catch (error) {
return { url, valid: false, reason: 'Unreachable' };
}
})
);
const allValid = results.every(r => r.valid);
return { valid: allValid, results };
}5. Content Ingestion from CSV and Google Sheets
Bulk import posts from external sources:
// services/ingestion.service.ts
async function ingestFromCSV(filePath: string): Promise<IngestionResult> {
const content = fs.readFileSync(filePath, 'utf-8');
const records = parse(content, { columns: true, skip_empty_lines: true });
const posts = records.map((row, index) => ({
platform: row.platform?.toUpperCase() || 'FACEBOOK',
content: row.caption,
mediaUrls: row.images?.split('|').filter(Boolean) || [],
scheduledAt: row.scheduled_at ? new Date(row.scheduled_at) : null,
}));
return this.postService.bulkCreate(posts);
}Post State Machine
enum PostStatus {
DRAFT = 'DRAFT',
PENDING = 'PENDING',
QUEUED = 'QUEUED',
PUBLISHED = 'PUBLISHED',
PARTIALLY_PUBLISHED = 'PARTIALLY_PUBLISHED',
FAILED = 'FAILED',
VALIDATION_FAILED = 'VALIDATION_FAILED',
ARCHIVED = 'ARCHIVED',
}State Transitions
| From | To | Trigger |
|---|---|---|
| DRAFT | PENDING | User schedules post |
| PENDING | QUEUED | Outbox poller picks up |
| QUEUED | PUBLISHED | Worker executes successfully |
| QUEUED | PARTIALLY_PUBLISHED | Some platforms succeed, some fail |
| QUEUED | FAILED | All platforms fail after retries |
| QUEUED | VALIDATION_FAILED | Media URLs unreachable |
| PUBLISHED/PARTIALLY_PUBLISHED | ARCHIVED | User archives |
Meta API Integration
Facebook Page Posts
// modules/facebook/facebook-post.service.ts
async function publishToPage(post: Post, token: string): Promise<PublishResult> {
const endpoint = `/${process.env.FB_PAGE_ID}/photos`;
const formData = new FormData();
formData.append('message', post.content);
if (post.mediaUrls?.[0]) {
formData.append('url', post.mediaUrls[0]);
}
const response = await axios.post(endpoint, formData, {
headers: { Authorization: `Bearer ${token}` },
});
return { platformPostId: response.data.id, platform: 'FACEBOOK' };
}Instagram Business Posts
// modules/instagram/ig-post.service.ts
async function publishToInstagram(post: Post, token: string): Promise<PublishResult> {
// 1. Upload media to container
const container = await axios.post(
`/${process.env.IG_BUSINESS_ID}/media`,
{
image_url: post.mediaUrls[0],
caption: post.content,
},
{ headers: { Authorization: `Bearer ${token}` } }
);
// 2. Publish from container
const publish = await axios.post(
`/${process.env.IG_BUSINESS_ID}/media_publish`,
{ creation_id: container.data.id },
{ headers: { Authorization: `Bearer ${token}` } }
);
return { platformPostId: publish.data.id, platform: 'INSTAGRAM' };
}Content Ingestion Pipeline
CSV Import
platform,caption,images,scheduled_at
FACEBOOK,"Launch announcement!","https://example.com/img1.jpg|https://example.com/img2.jpg","2026-05-20T10:00:00Z"
INSTAGRAM,"Behind the scenes","https://example.com/bts.jpg",,Google Sheets Integration
// modules/ingestion/google-sheets.service.ts
async function fetchFromSheet(sheetId: string): Promise<Post[]> {
const response = await this.googleClient.spreadsheets.values.get({
spreadsheetId: sheetId,
range: 'Posts!A:E',
});
return this.mapRowsToPosts(response.data.values);
}Three Operating Modes
1. Web Dashboard — Premium Glassmorphism UI
- Post CRUD operations
- Schedule management with calendar view
- Analytics with Chart.js visualizations
- Real-time updates via Socket.IO
- Dark mode with glassmorphism styling
2. CLI — Terminal Commands for Power Users
# Schedule a post
herald post create --platform facebook --content "Hello World" --media "https://..."
# List scheduled posts
herald post list --status pending
# Import from CSV
herald import csv ./posts.csv
# Check queue status
herald queue status3. Headless — Autonomous Background Process
- PM2-managed worker process
- Watches scheduled posts and executes automatically
- Graceful shutdown handling
- Automatic retry on failure
Deployment
Quick Start
# Install dependencies
npm install
# Setup database
npx prisma migrate deploy
npx prisma db seed
# Start services
docker compose up -d redis postgres
npm run start:api
npm run start:workerInfrastructure (Docker Compose)
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: herald
POSTGRES_PASSWORD: password
POSTGRES_DB: herald_db
volumes:
- postgres_data:/var/lib/postgresql/data
redis:
image: redis:7-alpine
command: redis-server --appendonly yes
volumes:
- redis_data:/dataProduction Checklist
# Build for production
npm run build
# Start with PM2
pm2 start ecosystem.config.js
pm2 save
# Access at http://your-vps:3000Roadmap
Phase 1 — Core Foundation (Complete)
- NestJS API with Prisma
- PostgreSQL database
- Basic post CRUD
Phase 2 — Platform Integration (Complete)
- Facebook Page posting
- Instagram Business posting
- Meta OAuth flow
Phase 3 — Scheduling Engine (Complete)
- BullMQ job queue
- CRON-based scheduling
- Rate limiting per token
Phase 4 — Dashboard (Complete)
- Next.js dashboard
- Real-time updates via Socket.IO
- Analytics charts
Phase 5 — CLI & Headless (Complete)
- Herald CLI commands
- PM2 worker process
Phase 6 — Content Ingestion (Complete)
- CSV import
- Google Sheets integration
- Pre-flight image validation
Phase 7 — Enhanced Analytics (In Progress)
- Advanced metrics
- Engagement tracking
Phase 8 — Multi-Platform Support (Planned)
- Twitter/X integration
- LinkedIn integration
Phase 9 — AI Enhancement (Future)
- LLM-generated captions
- Smart posting times
Engineering Proof
Real-world validation, system demonstrations, and interface captures of the execution states.
System Demonstration
Video walkthrough detailing core logic, interactions, and system behaviors in action.
System Captures
Architecture Feedback
Spotted a potential optimization or antipattern? Let me know.
