---

Recipe Digitizer

A modern web application for digitizing recipes from PDF files and web URLs using OCR and Mammouth AI processing.

🚀 Features

📁 PDF-Based Processing

• PDF upload via a user-friendly web interface
• OCR-based text extraction from scanned recipes (Tesseract)
• PDF conversion via poppler-utils

🔗 URL-Based Processing (IMPROVED!)

• URL input for recipes from websites
• Intelligent extraction using multiple methods:
  • JSON-LD/Schema.org for structured recipe data (best quality)
  • trafilatura for automatic main-content extraction
  • readability-lxml for readability-optimized extraction
  • BeautifulSoup4 as a fallback with recipe-specific parsing
• Clean quantities such as "300 g flour", "200 g sugar"

🤖 AI-Powered Processing

• Mammouth AI-powered text processing for structured formatting
• Intelligent recipe structuring in Markdown
• Consistent formatting of ingredients, steps, and metadata

📄 Output & Export

• Conversion to structured Markdown format
• Download of digitized recipes in Markdown format
• Organized directory structure: PDFs in uploads/pdf/, Markdown in uploads/markdown/
• Custom filenames: Optional independent naming for all generated files

🎨 Modern Web Interface

• Mobile-First Design: Custom CSS design system with variables (colors, radii, shadows, spacing)
• Touch-optimized: Large touch targets (min. 44–48px) for buttons, links, and cards
• Sticky Header: Navigation remains visible while scrolling, active page is highlighted
• Recipe Search: Client-side filtering of the recipe list by name
• Responsive Design: Optimized for desktop, tablet, and smartphone
• Template Inheritance: Jinja2 templates with a clean structure
• Consistent Template Engine: Custom stable Jinja2 integration bypasses framework bugs
• Partial Components: Reusable header/navigation

✏️ Recipe Editor (NEW!)

• Markdown Editor: Edit recipes directly in the web interface (EasyMDE/CodeMirror)
• Toolbar: Bold, italic, headings, lists, undo/redo with large touch buttons
• Live Preview: Toggle between the editor and rendered Markdown display (desktop: side-by-side)
• Automatic Drafts: Changes are saved locally (localStorage) and restored when reopened
• Adjust Quantities: Directly edit quantities in the text (e.g. "300 g flour" → "600 g flour")
• Keyboard Shortcut: Ctrl+S / Cmd+S saves directly from the editor
• Protection Against Data Loss: Warning about unsaved changes when canceling/closing
• Mobile Optimized: 16px editor font (prevents iOS auto-zoom), 44px toolbar buttons, no horizontal overflow

📋 Requirements

Local Hosting

• Python 3.9+
• Tesseract OCR (tesseract-ocr, tesseract-ocr-deu)
• Poppler Utilities (poppler-utils)
• Pip package manager

Docker Hosting

• Docker
• Docker Compose
• (All other dependencies are installed in the container)

🛠️ Installation

🖥️ Local Hosting

1. Install System Dependencies

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install tesseract-ocr tesseract-ocr-deu poppler-utils

macOS (with Homebrew):

brew install tesseract poppler

2. Prepare the Project

# In das Projektverzeichnis wechseln
cd recipe-digitizer

# Virtuelle Umgebung erstellen und aktivieren
python -m venv .venv
source .venv/bin/activate  # Linux/macOS
# oder
.venv\Scripts\activate     # Windows

# Abhängigkeiten installieren
pip install -r requirements.txt

3. Configure Environment Variables

# .env-Datei aus Vorlage erstellen
cp .env.example .env

# .env-Datei bearbeiten und Mammouth AI API-Token eintragen
nano .env  # MAMMOUTH_API_TOKEN=sk-...

4. Start the Application

# Statische Verzeichnisse erstellen
mkdir -p uploads/upload/pdf uploads/markdown static

# Anwendung starten
python app.py

5. Access the Application

Open http://localhost:8000 in your browser

🐳 Docker Hosting

The application is optimized for running in Docker. All required system dependencies are installed in the Docker image.

1. Install Docker (if not already installed)

# Docker und Docker Compose installieren
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
sudo apt-get install docker-compose

2. Configure Environment Variables

# .env-Datei aus Vorlage erstellen
cp .env.example .env

# .env-Datei bearbeiten und Mammouth AI API-Token eintragen
nano .env

Alternatively: Set the environment variable directly:

export MAMMOUTH_API_TOKEN="sk-jzWo1zg3..."

3. Start the Application

# In das Projektverzeichnis wechseln
cd recipe-digitizer

# Docker Container bauen und starten
docker-compose up --build

4. Access the Application

Open http://localhost:8000 in your browser

5. Additional Docker Commands

# Container im Hintergrund starten
docker-compose up -d --build

# Logs anzeigen
docker-compose logs -f

# Container stoppen
docker-compose down

# Image neu bauen
docker-compose build --no-cache

6. Docker Volumes for Development

The docker-compose.yml automatically configures volumes for:

• uploads/ - Persistent file storage
• static/ - Live CSS/JS updates without rebuilding
• templates/ - Live template updates

📖 Usage

1. Select a Method

• Upload PDF: For scanned or PDF recipes
• Process URL: For online recipes from websites
• My Recipes: Overview of all digitized recipes with Markdown rendering

2. PDF Processing

1. Upload PDF: Select a recipe PDF file using the upload button
2. Set Filename (optional): Enter a custom filename in the designated field
3. Mammouth AI Processing (optional): Enable AI processing for the best formatting results
4. Process Recipe: Click the "Process Recipe" button
5. Download: Once processing is complete, the Markdown file can be downloaded

Filename Note: The specified filename is used for both files:

• PDF file: [filename].pdf (saved in uploads/pdf/)
• Markdown file: [filename].md (saved in uploads/markdown/)

3. URL Processing (NEW!)

1. Select URL Method: Click "Process URL"
2. Enter Recipe URL: Enter the full URL (e.g. https://www.chefkoch.de/rezepte/12345/)
3. Set Filename (optional): Enter a custom filename (same field as for PDF processing)
4. AI Option: Optionally enable Mammouth AI processing
5. Process URL: Click the button or press Enter
6. Download: Download the Markdown file after successful processing

Filename Note: Without user input, a filename is automatically generated from the URL

4. Mammouth AI Processing (optional)

• Enable: Checkbox "Use Mammouth AI for formatting"
• Benefits: Structured formatting, consistent quantities, numbered steps
• Required: Valid API token in .env or an environment variable

🔑 Environment Variables

MAMMOUTH_API_TOKEN

• Description: API token for Mammouth AI (required for AI processing)
• Example: sk-jzWo1zg3tONA...
• Obtain from: Mammouth AI Platform
• Location: .env file or system environment variable

File Management

• PDF Storage Location: uploads/pdf/ - All uploaded PDF files
• Markdown Storage Location: uploads/markdown/ - All generated Markdown files
• Custom Filenames: A central input field for all filenames
• Automatic File Extensions: .pdf and .md are added automatically
• Backward Compatibility: Old files in the root directory continue to be supported

Configuration Flow

The application uses a robust configuration system:

1. src/config.py loads the .env file (if it exists)
2. Falls back to the system environment variable
3. Works in Docker (environment variable) and locally (.env)

Note: Hugging Face is no longer supported; the application uses exclusively Mammouth AI for AI processing.

📁 Project Structure

recipe-digitizer/
├── app.py              # Hauptanwendung mit FastAPI + Jinja2
├── requirements.txt    # Python-Abhängigkeiten (Mammouth AI + BeautifulSoup4 + trafilatura + ...)
├── Dockerfile          # Docker-Konfiguration (Python 3.12-slim)
├── docker-compose.yml  # Docker Compose-Konfiguration mit Volumes
├── .env.example        # Vorlage für Umgebungsvariablen
├── .env                # Eigene Umgebungsvariablen (nicht versioniert)
├── tests/              # Vollständige Test-Suite
│   ├── __init__.py
│   ├── README.md        # Detaillierte Test-Dokumentation
│   ├── test_*.py        # Einheitstests (API, Services, Docker)
│   ├── test_api_complete.py  # Kompletter API-Test (neue Features)
│   └── debug_*.py       # Debug-Skripte
├── src/                # Modularer Quellcode
│   ├── config.py       # Zentrale Konfiguration (dotenv + env)
│   ├── api/           # API-Endpunkte
│   │   └── endpoints.py
│   ├── services/      # Geschäftslogik Services
│   │   └── ai_service.py (Mammouth AI Service)
│   └── utils/         # Hilfsfunktionen
│       ├── pdf_extractor.py (OCR & Markdown-Konvertierung)
│       ├── pdf_extractor.py (OCR & Markdown-Konvertierung)
│       ├── url_extractor.py (HTML → Text, Legacy)
│       └── url_extractor_enhanced.py (JSON-LD + trafilatura + readability, VERBESSERT!)
├── templates/          # Jinja2 HTML Templates
│   ├── base.html      # Basis-Template (Viewport, CSS/JS-Einbindung)
│   ├── index.html     # Hauptseite mit PDF/URL Auswahl
│   ├── recipes.html   # Rezept-Übersicht mit Suche & Markdown-Rendering
│   └── partials/      # Teilbare Komponenten
│       └── header.html  # Sticky Navigation mit aktivem Seiten-Highlight
├── static/            # Frontend Assets
│   ├── css/
│   │   └── style.css  # Zentrales Mobile-First Design-System (CSS-Variablen)
│   └── js/
│       └── main.js    # JavaScript Funktionen (PDF + URL)
└── uploads/           # Organisiertes Verzeichnis für Dateien
    ├── pdf/           # Hochgeladene PDF-Dateien
    └── markdown/      # Generierte Markdown-Dateien

Architecture Principles:

• Separation of Concerns: Clear separation between API, services, and utils
• Modularity: Easy replacement/update of individual components
• Reusability: Services can be used independently
• Extensibility: New features can be added easily

🌐 API Endpoints

Overview

All API endpoints begin with /api/ and are cleanly organized in src/api/endpoints.py.

Endpoints

📊 Health & Monitoring

• GET / - Main page (Jinja2 template)
• GET /api/health - Health check ({"status": "healthy"})

📁 PDF Processing

• POST /api/upload - PDF upload (multipart/form-data with optional custom_filename)
• POST /api/process/{filename} - Recipe processing (PDF → Markdown with optional custom_filename in the JSON body)
• GET /api/download/{filename} - File download with optional directory parameter (pdf or markdown)

🔗 URL Processing (NEW!)

• POST /api/process/url - URL processing (HTML → Markdown)

Request Body:

{
  "url": "https://www.chefkoch.de/rezepte/12345/rezeptname",
  "use_ai": true,
  "custom_filename": "mein_rezeptname"  // Optional
}

Response:

{
  "markdown_filename": "mein_rezeptname.md",  // Oder automatisch generiert
  "markdown_path": "uploads/markdown/mein_rezeptname.md",
  "original_url": "https://www.chefkoch.de/rezepte/12345/rezeptname",
  "directory": "markdown"
}

📁 PDF Processing

• POST /api/upload - PDF upload

Request (FormData):

• file: PDF-Datei (erforderlich)
• custom_filename: Benutzerdefinierter Dateiname (optional)

Response:

{
  "filename": "mein_rezept.pdf",  // Oder Originaldateiname
  "saved_path": "uploads/pdf/mein_rezept.pdf",
  "directory": "pdf"
}

📄 PDF Conversion

• POST /api/process/{pdf_filename} - PDF to Markdown conversion

Request Body (JSON):

{
  "use_ai": true,
  "custom_filename": "mein_rezeptname"  // Optional
}

Response:

{
  "markdown_filename": "mein_rezeptname.md",
  "markdown_path": "uploads/markdown/mein_rezeptname.md",
  "pdf_filename": "hochgeladenes_rezept.pdf",
  "directory": "markdown"
}

📥 Download

• GET /api/download/{filename} - File download with optional query parameter

Parameters:

• filename: Name of the file to download
• directory: Optional: pdf or markdown (if omitted, all directories are searched)

Response:

{
  "markdown_filename": "chefkochde_rezepte_12345_rezeptname.md",
  "markdown_path": "uploads/chefkochde_rezepte_12345_rezeptname.md",
  "original_url": "https://www.chefkoch.de/rezepte/12345/rezeptname"
}

📋 Recipe Display & Management

• GET /recipes - Recipe overview page (frontend with all marked recipes)
• GET /api/recipes/list - List of all available Markdown recipes

Response:

{
  "recipes": [
    {
      "filename": "Veganer-Apfelkuchen.md",
      "name": "Veganer-Apfelkuchen",
      "path": "uploads/markdown/Veganer-Apfelkuchen.md",
      "size": 2466,
      "modified": 1748307993
    }
  ],
  "count": 1
}

• GET /api/recipes/{filename} - Content of a specific Markdown file

Response:

{
  "filename": "Veganer-Apfelkuchen.md",
  "name": "Veganer-Apfelkuchen",
  "path": "uploads/markdown/Veganer-Apfelkuchen.md",
  "content": "# Veganer Apfelkuchen\n\n...",
  "size": 2466,
  "modified": 1748307993
}

✏️ Edit Recipe (NEW!)

• PUT /api/recipes - Overwrites the content of a Markdown file

Request Body:

{
  "filename": "Veganer-Apfelkuchen.md",
  "content": "# Veganer Apfelkuchen\n\n## Zutaten\n- 600 g Mehl\n..."
}

Response:

{
  "filename": "Veganer-Apfelkuchen.md",
  "name": "Veganer-Apfelkuchen",
  "path": "uploads/markdown/Veganer-Apfelkuchen.md",
  "size": 2466,
  "modified": 1748307993,
  "message": "Recipe updated successfully"
}

Notes:

• Also works without the .md extension (it is added automatically)
• Security check against directory traversal (.., /, \ → 400)
• 404 if the file does not exist (no new files are created)

🐳 Docker-Specific Details

Docker Image Features

• Base Image: Python 3.12-slim
• System Packages: poppler-utils, tesseract-ocr, tesseract-ocr-deu
• Python Packages: All on the latest stable versions (FastAPI 0.136.3, Jinja2 3.1.6, etc.)
• Workdir: /app
• Port: 8000 exposed
• Stable Template Engine: Custom Jinja2 integration (bypasses Starlette/FastAPI incompatibilities)

Docker Compose Configuration

volumes:
  - ./uploads:/app/uploads           # Uploads persistent (mit Unterverzeichnissen)
  - ./static:/app/static             # CSS/JS live updates
  - ./templates:/app/templates       # Template live updates

environment:
  - MAMMOUTH_API_TOKEN=${MAMMOUTH_API_TOKEN}  # Von Host oder .env

env_file:
  - .env                             # Optional: .env Datei

Docker-Ready Features

• ✅ Robust configuration for Docker environments
• ✅ Persistent file storage via volumes
• ✅ Live updates for frontend assets
• ✅ AI integration correctly configured
• ✅ Stable template rendering with Jinja2 3.1.6
• ✅ All tests pass on the latest package versions

🔧 Development

For Developers:

# Neues Modul importieren
from src.services.ai_service import ai_service
from src.utils.pdf_extractor import extract_text_from_pdf
from src.utils.url_extractor_enhanced import extract_recipe_url

# API-Endpunkte sind in src/api/endpoints.py
# Templates verwenden Jinja2 Vererbung (base.html → index.html)
# URL-Verarbeitung: EnhancedRecipeURLExtractor Klasse in url_extractor_enhanced.py
# - JSON-LD/Schema.org Extraktion für strukturierte Rezept-Daten
# - Fallback auf trafilatura + readability-lxml
# - Recipe-spezifisches BeautifulSoup Parsing als letzte Option

Run Tests

# Navigiere ins Projekt-Verzeichnis
cd recipe-digitizer

# Alle Tests ausführen (mit sys.path Correction)
for test in tests/*.py; do 
  echo "=== Testing $test ==="
  python "$test" 2>&1 | grep -E "(✅|❌|✓|✗|completed|PASSED|erfolgreich)" || echo "Test durchgelaufen"
  echo ""
done

# Spezifische Tests
python tests/test_token.py                    # API Token Test  
python tests/test_api_complete.py             # Vollständiger API-Test (neue Features)
python tests/test_recipe_edit.py              # Rezept-Editiermodus (GET/PUT + Frontend)
python tests/test_docker_complete.py          # Docker-E2E Test
python tests/test_docker_config.py            # Docker-Konfigurations-Test
python tests/test_enhanced_url_workflow.py    # Verbesserte URL-Extraktion

# Wichtig: Alle Tests benötigen sys.path.insert in den ersten Zeilen!
# Siehe tests/README.md für Details

ℹ️ New Features

✅ File Management & Organization

• Separate Directories: PDFs (uploads/pdf/) and Markdown files (uploads/markdown/) are stored separately
• Centralized Filename Assignment: A unified input field for all filenames
• Consistent Naming: Same filename for PDF and Markdown (with corresponding extensions)
• Automatic File Extensions: .pdf and .md are added automatically

✅ Improved UX

• Simplified User Interface: Only one filename input for all operations
• Intelligent Name Generation: If no name is provided, a name is generated automatically
• Backward Compatibility: Old files in the root directory continue to be supported
• API Consistency: Unified parameters for all endpoints

✅ Stable Template Support (v1.4)

• Custom Jinja2 Integration: Bypasses Starlette/FastAPI incompatibilities
• Latest Package Versions: All Python dependencies updated and tested
• Working Template Inheritance: {% extends "base.html" %} correctly implemented
• Test Verification: Full test suite on the latest versions

Current Extensions (v1.2):

• ✅ Recipe Overview Page: Browser-based display of all digitized recipes
• ✅ One-Click Display: Open Markdown recipes in the web browser with a click
• ✅ Markdown Rendering: Formatted display with headings, lists, and formatting
• ✅ Responsive Navigation: Switch between upload and recipe views
• ✅ Recipe List API: Structured access to all available recipes

Possible Future Extensions:

• Batch Processing: Multiple PDFs/URLs at once
• Export Formats: JSON, YAML, XML
• Recipe Recognition: Automatic ingredient recognition
• Image Extraction: Recipe images from websites
• Quality Check: Assess OCR quality
• Caching: Prevent duplicate downloads
• User-Agent Rotation: Bypass blocking
• File Management: File list and deletion function via the web interface

🐛 Troubleshooting

General Problems

1. 401 Unauthorized (AI Service)

# Token prüfen
python -c "import os; from dotenv import load_dotenv; load_dotenv(); print('Token:', os.getenv('MAMMOUTH_API_TOKEN', 'NOT SET'))"

# In Docker:
docker exec -it rezept-digitizer python -c "import os; print('Token:', os.getenv('MAMMOUTH_API_TOKEN', 'NOT SET'))"

2. OCR/PDF Problems

# Tesseract prüfen
tesseract --version
# Poppler prüfen
pdftoppm -v

3. URL Extraction Does Not Work or Has Poor Quality

• Check the internet connection
• The URL must be publicly accessible
• No login-protected pages
• JavaScript-heavy pages may be incomplete
• For better quality: The website should contain structured JSON-LD/Schema.org data
• Tip: Test with python tests/test_enhanced_url_extractor.py for diagnosis
• Improved logs: Check extraction_method in the API response (json-ld, trafilatura, readability, bs4)

4. Template Rendering Error (Starlette/Jinja2)

• Symptom: TypeError: cannot use 'tuple' as a dict key (unhashable type: 'dict')
• Cause: Starlette 1.1.0 + FastAPI 0.136.3 incompatibility
• Solution: App.py uses custom Jinja2 integration
• Check: Make sure current versions from requirements.txt are installed
• Fix: If the problem occurs again: pip install --upgrade -r requirements.txt

3. Saving in the Recipe Editor Does Nothing

• Cause: src/api/endpoints.py is not mounted as a Docker volume, but is part of the image. After backend changes, the container must be rebuilt:

docker-compose up --build

• If the PUT /api/recipes endpoint is missing (404/405), the frontend now displays a corresponding notice directly below the editor buttons
• Editor status messages appear within the recipe view below the buttons (not at the end of the page)

Docker-Specific

1. Port Conflicts

# Prüfe laufende Container
docker ps

# Anderen Port verwenden
# In docker-compose.yml: ports: "8001:8000"

2. File Permission Issues

# Uploads Verzeichnis Rechte anpassen
chmod 777 uploads

# Oder im Dockerfile: USER 1000:1000

3. View Logs

docker-compose logs -f
docker logs rezept-digitizer

📊 Data Flow

PDF Processing:

Frontend → POST /api/upload → PDF speichern
Frontend → POST /api/process/{filename} → PDF Extractor (OCR) → [Optional: AI Service] → Markdown Konverter → Datei speichern
Frontend → GET /api/download/{filename} → File download

URL Processing (NEW!):

Frontend → POST /api/process/url → URL Extractor (HTML Download) → [Optional: AI Service] → Markdown Konverter → Datei speichern
Frontend → GET /api/download/{filename} → File download

🆕 Changes

Version 1.6.1

• Save Bugfix: Error and success messages now appear within the recipe view below the buttons (previously at the end of the page, outside the visible area)
• Stale Backend Detection: On 404/405 for PUT /api/recipes, the frontend displays a rebuild notice (docker-compose up --build), since endpoints.py is built into the Docker image
• Robust Error Handling: Network errors, JSON parsing errors, and HTTP errors are caught and displayed; the editor remains open on errors (no changes lost)
• Mobile Header Bugfix: Recipe title and buttons are deterministically stacked (column layout ≤768px) – prevents the title from collapsing to 0px with long names/system font scaling (letters appearing one below another)

Version 1.6

• Recipe Editing Mode: Edit Markdown recipes directly in the web interface
• EasyMDE Editor: CodeMirror-based editor with toolbar (bold, italic, headings, lists, undo/redo), live preview, and side-by-side mode (desktop)
• Automatic Drafts: Unsaved changes are saved in localStorage and restored when reopened
• Data Loss Protection: Confirmation dialog when canceling/closing with unsaved changes
• Save with Ctrl+S/Cmd+S: Directly from the editor
• Mobile Optimization: 16px editor font (prevents iOS auto-zoom), 44px touch toolbar buttons, no horizontal overflow
• New API Endpoint: PUT /api/recipes for saving edited recipes (with directory traversal protection)
• Status Messages Instead of alert(): Inline status messages in the page layout

Version 1.5

• Mobile-First Redesign: Completely redesigned CSS with a central design system (CSS variables for colors, radii, shadows, spacing)
• Sticky Navigation: Header remains visible while scrolling, active page is highlighted
• Touch-Optimized Controls: Buttons, links, and recipe cards with a min. height of 44–48px
• Recipe Search: New filter field on the overview page
• Bugfix: Fixed horizontal overflow on mobile devices caused by long filenames in recipe cards
• Cleaned-Up Template CSS: No more inline <style> blocks in recipes.html/header.html; everything centralized in static/css/style.css

Version 1.4

• Stable Template Integration: Custom Jinja2 environment bypasses Starlette/FastAPI bugs
• Updated Dependencies: All Python packages updated to the latest stable versions
• Complete Test Verification: Full test suite with all updates
• Jinja2 Template Inheritance: Correctly functioning {% extends "base.html" %}

Version 1.3

• Improved URL Extraction: JSON-LD/Schema.org support + trafilatura + readability
• Clean Quantities: Correct formatting such as "300 g flour" instead of fragmented data
• Intelligent Fallbacks: Multiple extraction methods for maximum compatibility

Version 1.2

• Recipe Overview Page: Browser-based display of all digitized recipes
• One-Click Display: Open Markdown recipes in the web browser with a click
• Markdown Rendering: Formatted display with headings, lists, and formatting
• Responsive Navigation: Switch between upload and recipe views
• Recipe Management API: Structured access to available recipes

Version 1.1

• Organized File Management: Separate directories for PDF and Markdown files
• Custom Filenames: Easy naming for all generated files
• Simplified User Interface: Centralized input fields
• Improved API Consistency: Unified parameters for all endpoints

Version 1.0

• Basic functionality: PDF and URL processing
• Mammouth AI integration
• Docker support

📄 License

MIT License - See the LICENSE file for details.

🙏 Acknowledgments

• Tesseract OCR: Open-source OCR engine
• Mammouth AI: For AI-based recipe structuring
• FastAPI: Modern Python web framework
• Schema.org/JSON-LD: Structured recipe data standard
• trafilatura & readability-lxml: Improved content extraction
• BeautifulSoup4: HTML parsing as a fallback

---