Initializing
Back to Projects
Year2024
DomainBackend
AccessOpen Source
Complexity0 / 10
PythonNocoDBData MigrationAutomation
BackendArchived

NocoDB Image Converter

A Python utility that migrates image URLs from NocoDB text columns to native attachment fields by downloading and uploading images via NocoDB's storage API.

# NocoDB Image Converter

A standalone Python utility that bridges the gap between external image URLs and NocoDB's native attachment system. This tool enables bulk migration of image references stored as text URLs into proper NocoDB attachment fields.

Problem Statement

NocoDB allows two ways to store images:

  1. Attachment fields - Native file storage with thumbnails, metadata
  2. URL text fields - External links (brittle, no offline access)

Many NocoDB bases were created with URL text columns. This utility converts them:

code
Before: Product table with Image_URL (text) → "https://example.com/product.jpg"
After:  Product table with Product_Images (attachment) → [Inline attachment]

How It Works

Parsing system architecture diagram...

Core Class

python
class NocoDBImageConverter:
    def __init__(self, base_url: str, api_token: str, table_id: str):
        self.base_url = base_url
        self.api_token = api_token
        self.table_id = table_id
        self.headers = {"xc-token": api_token, "Content-Type": "application/json"}

Key Methods

Fetch Records

python
def get_all_records(self, limit: int = 1000) -> List[Dict]:
    """Paginated fetch of all table records"""
    url = f"{self.base_url}/api/v2/tables/{self.table_id}/records"
    all_records = []
    offset = 0
    
    while True:
        response = requests.get(url, headers=self.headers, params={
            'limit': limit, 'offset': offset
        })
        records = response.json().get('list', [])
        
        if not records:
            break
            
        all_records.extend(records)
        offset += limit
        time.sleep(0.5)  # Rate limiting
        
    return all_records

Download Image

python
def download_image(self, image_url: str, filename: str) -> Optional[bytes]:
    """Download image and validate content-type"""
    response = requests.get(image_url, timeout=30)
    response.raise_for_status()
    
    content_type = response.headers.get('content-type', '')
    if not content_type.startswith('image/'):
        return None  # Not a valid image
        
    return response.content

Upload as Attachment

python
def upload_attachment(self, file_content: bytes, filename: str) -> Optional[Dict]:
    """Upload to NocoDB storage endpoint"""
    files = {'file': (filename, file_content, 'image/*')}
    upload_headers = {"xc-token": self.api_token}  # No Content-Type for multipart
    
    url = f"{self.base_url}/api/v2/storage/upload"
    response = requests.post(url, headers=upload_headers, files=files)
    
    return response.json()  # {url, title, mimetype, size}

Update Record

python
def update_record_with_attachment(self, record_id: str, attachment_data: List[Dict], attachment_column: str):
    """PATCH record with attachment JSON"""
    url = f"{self.base_url}/api/v2/tables/{self.table_id}/records"
    
    data = [{
        "Id": record_id,
        attachment_column: attachment_data  # NocoDB attachment format
    }]
    
    response = requests.patch(url, headers=self.headers, json=data)
    response.raise_for_status()
    return True

Usage

python
# Configuration
BASE_URL = "https://db.aatmanova.in"
API_TOKEN = "your_nocodb_api_token"
TABLE_ID = "table_id_from_url"

# Initialize
converter = NocoDBImageConverter(BASE_URL, API_TOKEN, TABLE_ID)

# Run conversion
converter.process_records(
    image_url_column="Image_URL",      # Source: text column with URLs
    attachment_column="Product_Images", # Target: attachment field
    skip_existing=True                   # Skip records that already have attachments
)

Configuration

ParameterSourceDescription
base_urlNocoDB instancee.g., https://db.aatmanova.in
api_tokenNocoDB profileGenerate from Account settings
table_idTable URLExtract from /nc/{table_id}/...
image_url_columnColumn nameExisting text column with URLs
attachment_columnColumn nameTarget attachment column

Output Summary

code
==================================================
CONVERSION SUMMARY
==================================================
Total records processed: 150
✅ Successful updates: 142
⏩ Skipped records: 5
❌ Failed updates: 3
==================================================

Safety Features

  1. skip_existing=True - Won't re-process records that already have attachments
  2. Content-Type validation - Skips non-image URLs gracefully
  3. Rate limiting - 0.5s between fetches, 1s between updates
  4. Error isolation - Single record failure doesn't stop batch

Error Handling

ErrorCauseResolution
404 on downloadInvalid URLMarked as failed, continues
401 on uploadInvalid API tokenCheck token permissions
500 on updateField type mismatchVerify attachment column exists

Use Cases

  • E-commerce catalogs: Migrate product image URLs to native attachments
  • Asset libraries: Consolidate external CDN links to NocoDB storage
  • Data migration: Move from Airtable/Sheet URLs to NocoDB native

Limitations

  • Requires NocoDB API token with table write permissions
  • No directory/file structure support (flat uploads only)
  • No automatic thumbnail regeneration (NocoDB handles this)
  • Max file size depends on NocoDB server configuration

Architecture Feedback

Spotted a potential optimization or antipattern? Let me know.

Submit a Technical Suggestion