chalvien.eu
GuidesAutomation

n8n Quick Guide (Data Extraction)

A beginner-friendly guide to build n8n workflows for extracting, cleaning, and storing document data.

n8n is a workflow automation tool where you connect nodes like building blocks. For data extraction, it helps you move from "raw files/emails" to "clean structured data" with minimal code.

This quick guide is for junior developers and focuses on one practical goal:

  • Receive a document
  • Extract useful text/fields
  • Normalize the output
  • Save it to a database or send it to an API

1. What You Need

  • Docker Engine + Docker Compose v2 (docker compose version)
  • n8n running locally or on a server
  • A document source (Webhook, Google Drive, S3, or email)
  • One extraction service (for example, Docling, Unstructured, or Apache Tika)
  • A destination (Postgres, internal API, or vector index)

2. Start n8n Locally

npm install n8n -g / npx n8n is deprecated for new installs from n8n 3.0 (launching October 2026).

Use Docker-based setup instead.

Option A: One-line setup (fastest)

curl -fsSL https://get.n8n.io | sh

This creates a local ./n8n folder, writes compose.yml + .env, and starts n8n on:

  • http://localhost:5678

Useful day-2 commands:

# stop
docker compose -f ./n8n/compose.yml down

# start again
docker compose -f ./n8n/compose.yml up -d

# upgrade
curl -fsSL https://get.n8n.io | sh -s -- --upgrade

Option B: Manual Docker Compose (more control)

Use this when you want to customize services, environment variables, and local networking.

mkdir n8n && cd n8n

Create .env:

# Pin versions
N8N_VERSION=3.x.x
N8N_SANDBOX_VERSION=x.y.z

# Required secrets (set your own strong values)
N8N_RUNNERS_AUTH_TOKEN=replace-me
SANDBOX_API_KEYS=replace-me
SANDBOX_API_RUNNER_REGISTRATION_TOKEN=replace-me
SANDBOX_API_RUNNER_API_KEY=replace-me
N8N_SANDBOX_SERVICE_API_KEY=replace-me
SEARXNG_SECRET=replace-me

# Optional AI assistant model key (set later if preferred)
# N8N_INSTANCE_AI_MODEL_API_KEY=sk-...

Start services:

docker compose up -d
docker compose ps

Health checks:

curl -sf http://localhost:5678/healthz
docker compose logs -f n8n

Open http://localhost:5678, complete owner onboarding, then create your workflow.

2.1 Local self-hosting notes (important)

  • Default setup uses SQLite (good for local/dev).
  • For team/production, switch to Postgres.
  • Persist data with volumes; avoid ephemeral containers.
  • Keep secrets in .env; do not commit them.
  • Expose only port 5678 publicly.

2.2 AI Assistant setup on self-hosted n8n

The sandbox services can run by default, but the assistant stays off until you add a model key.

Enable assistant:

  1. Add provider key in your n8n instance AI settings (UI), or in .env:
N8N_INSTANCE_AI_MODEL_API_KEY=sk-...
  1. Restart n8n:
docker compose up -d n8n
  1. Optional web-search provider:
  • Default is bundled SearXNG.
  • If you prefer Brave Search, set:
INSTANCE_AI_BRAVE_SEARCH_API_KEY=BSA-...

2.3 Build a local AI assistant workflow (practical)

After instance-level assistant is enabled, create a user-facing workflow assistant:

  1. Chat Trigger node
  2. AI Agent node
  3. OpenAI Chat Model or Ollama Chat Model
  4. Tool nodes (HTTP Request, Postgres, Call n8n Workflow Tool)
  5. Respond to Webhook or chat output

Starter flow:

Chat Trigger -> AI Agent -> Tool(s) -> Response

Tips:

  • Start with read-only tools first.
  • Add guardrails for tool inputs and allowed domains.
  • Log prompts and outputs for debugging and evaluation.

3. Core Concepts (Beginner Version)

  • Node: one step in your automation
  • Trigger node: starts the workflow
  • Data item: JSON object moving between nodes
  • Expression: dynamic value like {{$json.document_id}}
  • Execution: one workflow run (one event)

Tip: always inspect node output before moving to the next node.

4. Your First Data-Extraction Workflow

Build this pipeline:

  1. Trigger: Webhook (or Drive/S3 trigger)
  2. Route: Switch by file type (PDF, DOCX, EML)
  3. Extract: HTTP Request to extraction API
  4. Normalize: Code node to map into a clean schema
  5. Persist: Postgres (or HTTP API)

Suggested canonical schema:

{
	"document_id": "string",
	"checksum": "string",
	"parser_version": "v1",
	"source": "webhook|drive|s3|email",
	"extracted_text": "string",
	"structured": {},
	"extracted_at": "2026-09-27T12:00:00.000Z"
}

Why this schema helps:

  • Easy reprocessing
  • Easy debugging
  • Stable contract for downstream apps

5. Reliability Basics (Do This Early)

  • Enable retry on HTTP extraction nodes (for transient failures)
  • Use idempotency keys: document_id + checksum
  • Add dead-letter handling for failed runs
  • Track simple metrics: error rate, empty output rate, latency

These 4 steps prevent most production issues in document pipelines.

6. Common Beginner Mistakes

  • Skipping output inspection between nodes
  • Mixing extraction and chunking in the same logic step
  • Not storing parser version (hard to debug output drift)
  • No fallback parser for edge-case files

7. Practical Next Step

After completing this quick guide, follow the deeper extraction playbook in content/docs/guides/automation/02-data-extraction.mdx for:

  • Importable workflow JSON examples
  • Dead-letter companion workflow
  • SQL schema and operations for failure management

8. 10-Minute Lab (Hands-On)

Goal: build and test a tiny extraction workflow in one sitting.

Success criteria:

  • You can trigger the workflow from a webhook request
  • The output matches the canonical schema shape
  • The workflow is stable across repeated test runs

Step A: Create 4 Nodes

  1. Webhook (POST) with path: lab-extract
  2. Set node named Mock Extractor
  3. Code node named Normalize
  4. Respond to Webhook

Connect them in this order:

Webhook -> Mock Extractor -> Normalize -> Respond to Webhook

Step B: Configure Mock Extractor

In Mock Extractor, set these fields:

  • document_id: ={{$json.document_id}}
  • source: ={{$json.source || 'webhook'}}
  • raw_text: Invoice #INV-2026-1001\nTotal: 1450 USD\nVendor: ACME Corp
  • parser_version: lab-v1

Step C: Configure Normalize Code

Paste this code into Normalize:

const j = $json;

return [{
	json: {
		document_id: j.document_id || null,
		checksum: j.checksum || "demo-checksum",
		parser_version: j.parser_version || "lab-v1",
		source: j.source || "webhook",
		extracted_text: j.raw_text || "",
		structured: {
			invoice_number: "INV-2026-1001",
			total_amount: 1450,
			currency: "USD",
			vendor: "ACME Corp"
		},
		extracted_at: new Date().toISOString()
	}
}];

Step D: Test Request

Use this request body in your webhook test:

{
	"document_id": "doc-001",
	"checksum": "sha256-demo-001",
	"source": "webhook"
}

Step E: Expected Output

Your response should include:

  • document_id: doc-001
  • parser_version: lab-v1
  • structured.invoice_number: INV-2026-1001
  • structured.total_amount: 1450
  • extracted_at as a valid ISO timestamp

Step F: Quick Validation Checklist

  • Workflow executes successfully (green check)
  • Normalize output has all required schema fields
  • Missing input fields do not crash the workflow
  • Output is stable across 2-3 repeated requests

If this lab works, replace Mock Extractor with your real HTTP Request extraction node and keep the same normalization step.

9. Mini-Lab: Failure Simulation and Dead-Letter Capture

Goal: intentionally fail a workflow and verify the failure is captured for triage.

Success criteria:

  • A forced failure appears in your dead-letter table
  • You can query unresolved incidents successfully
  • After fixing the issue, new runs complete without creating a new dead-letter row

Step A: Create an Intentional Failure

In your extraction workflow, duplicate your real extraction HTTP Request node and change the URL to an invalid endpoint, for example:

  • https://example.invalid/extract

Run one test execution to force an error.

Step B: Enable a Dead-Letter Workflow

Create a separate workflow with these nodes:

  1. Error Trigger
  2. Code (build a compact failure payload)
  3. Postgres (insert into extraction_dead_letter)

Use the table and SQL patterns from content/docs/guides/automation/02-data-extraction.mdx.

Step C: Failure Payload Example

In the dead-letter Code node, keep at least these fields:

{
	"failed_at": "2026-09-27T12:30:00.000Z",
	"workflow_name": "Document Extraction Pipeline",
	"execution_id": "1234",
	"last_node": "Extract via Docling",
	"message": "Request failed with status code 500"
}

Step D: Verify It Worked

Run this query and confirm the new failure appears:

SELECT id, failed_at, workflow_name, execution_id, last_node, message
FROM extraction_dead_letter
WHERE resolved_at IS NULL
ORDER BY failed_at DESC
LIMIT 20;

Step E: Resolve and Re-test

  1. Mark the incident resolved.
  2. Fix the broken extraction URL.
  3. Re-run the workflow and confirm no new dead-letter entry is created.

This mini-lab teaches the full operational loop: fail fast, capture context, fix safely, and validate recovery.

Astro Theming

This guide documents the theming system behind the `chalvien/ui` system for Astro.

Data Extraction (open source)

Learn how to extract data efficiently using automation tools.

On this page

1. What You Need2. Start n8n LocallyOption A: One-line setup (fastest)Option B: Manual Docker Compose (more control)2.1 Local self-hosting notes (important)2.2 AI Assistant setup on self-hosted n8n2.3 Build a local AI assistant workflow (practical)3. Core Concepts (Beginner Version)4. Your First Data-Extraction Workflow5. Reliability Basics (Do This Early)6. Common Beginner Mistakes7. Practical Next Step8. 10-Minute Lab (Hands-On)Step A: Create 4 NodesStep B: Configure Mock ExtractorStep C: Configure Normalize CodeStep D: Test RequestStep E: Expected OutputStep F: Quick Validation Checklist9. Mini-Lab: Failure Simulation and Dead-Letter CaptureStep A: Create an Intentional FailureStep B: Enable a Dead-Letter WorkflowStep C: Failure Payload ExampleStep D: Verify It WorkedStep E: Resolve and Re-test