Data Extraction (open source)

These guides focus on open-source tools for data extraction, providing alternatives to proprietary solutions.

This guide complements:

When choosing a stack, decide first on your target output quality (layout-aware Markdown/JSON vs plain text), then on your operating constraints (local-only processing, throughput, and compliance requirements). A practical pattern is to combine two layers: a parser optimized for structure (Docling or Unstructured) and a fallback extractor (Apache Tika) for unsupported edge-case files. This reduces failure rates in production and gives you a deterministic path when one parser struggles with malformed documents.

1. For AI and LLM-Ready Document Layouts

If you are moving away from a proprietary tool because rigid templates break and you need clean, structured outputs (like Markdown or JSON) for AI workflows, these tools are the modern standards:

Docling: Developed by IBM, this is an excellent, fast tool built specifically for parsing complex documents (like PDFs) into AI-ready formats. It excels at understanding structural elements like nested tables and figures without needing a visual template layout.

Unstructured: One of the most popular open-source ingestion libraries. It can ingest raw, unstructured documents (PDFs, Word docs, PowerPoints, emails) and cleanly partition them into structured JSON elements.

2. For Email and Mailbox Parsing

If you specifically used a proprietary tool (like Parseur) as an inbox/email parser to turn emails into structured data:

Mailpond: A self-hosted, open-source email automation and parsing platform. It allows you to receive incoming webhooks or IMAP emails, parse them using customizable rules, and send the data downstream via webhooks.

3. For Local, Low-Resource PDF and OCR Extraction

If your primary bottleneck is high volume, strict data privacy, or extracting tables locally:

Light Client: A completely open-source, local-first document processing engine. It runs entirely on your machine using a hybrid framework (combining PDF.js, Tesseract.js, and LibreOffice) to extract structural data without making external cloud API calls.

Apache Tika: A battle-tested enterprise standard. It detects and extracts metadata and text from over a thousand different file types. It can easily be deployed as a local microservice to handle high-throughput file parsing.

4. n8n Integration (Automation Orchestration)

n8n works well as the orchestration layer across all the tools above: trigger on new files (Webhook, IMAP Email, Google Drive, or S3), route by MIME type or filename pattern with Switch/IF, send documents to Docling/Unstructured/Tika over HTTP, normalize outputs in a Code node, then upsert to Postgres, a vector database, or your app API. For reliability, use queue mode, add retry with exponential backoff on extraction nodes, and store a document_id + parser_version + checksum so reprocessing stays idempotent and auditable.

Importable starter workflow (n8n JSON):

{
	"name": "Document Extraction Pipeline",
	"active": false,
	"settings": {
		"executionOrder": "v1"
	},
	"versionId": "00000000-0000-0000-0000-000000000001",
	"nodes": [
		{
			"id": "WebhookTrigger",
			"name": "Webhook Trigger",
			"type": "n8n-nodes-base.webhook",
			"typeVersion": 2,
			"position": [
				-980,
				120
			],
			"parameters": {
				"httpMethod": "POST",
				"path": "doc-ingest",
				"responseMode": "lastNode",
				"options": {}
			},
			"notes": "Receives upload metadata and base64 or URL payload."
		},
		{
			"id": "ClassifyFile",
			"name": "Classify File",
			"type": "n8n-nodes-base.switch",
			"typeVersion": 3,
			"position": [
				-760,
				120
			],
			"parameters": {
				"mode": "rules",
				"rules": {
					"values": [
						{
							"conditions": {
								"options": {
									"caseSensitive": false
								},
								"conditions": [
									{
										"leftValue": "={{$json.mime_type || $json.mimeType || ''}}",
										"rightValue": "pdf|docx|application/",
										"operator": {
											"type": "string",
											"operation": "regex"
										}
									}
								],
								"combinator": "and"
							}
						}
					]
				},
				"fallbackOutput": 1
			},
			"notes": "Output 0 => structured parser path, Output 1 => fallback path."
		},
		{
			"id": "ExtractDocling",
			"name": "Extract via Docling",
			"type": "n8n-nodes-base.httpRequest",
			"typeVersion": 4.2,
			"retryOnFail": true,
			"maxTries": 4,
			"waitBetweenTries": 2000,
			"position": [
				-500,
				40
			],
			"parameters": {
				"method": "POST",
				"url": "={{$env.DOCLING_API_URL}}",
				"sendHeaders": true,
				"headerParameters": {
					"parameters": [
						{
							"name": "Authorization",
							"value": "=Bearer {{$env.DOCLING_API_KEY}}"
						}
					]
				},
				"sendBody": true,
				"specifyBody": "json",
				"jsonBody": "={{ { document_id: $json.document_id, file_url: $json.file_url, file_b64: $json.file_b64, mime_type: $json.mime_type } }}",
				"options": {
					"timeout": 60000
				}
			},
			"notes": "Primary parser for layout-aware extraction."
		},
		{
			"id": "ExtractTika",
			"name": "Fallback via Apache Tika",
			"type": "n8n-nodes-base.httpRequest",
			"typeVersion": 4.2,
			"retryOnFail": true,
			"maxTries": 4,
			"waitBetweenTries": 2000,
			"position": [
				-500,
				220
			],
			"parameters": {
				"method": "PUT",
				"url": "={{$env.TIKA_API_URL}}/tika",
				"sendBody": true,
				"specifyBody": "json",
				"jsonBody": "={{ { file_url: $json.file_url, file_b64: $json.file_b64, mime_type: $json.mime_type } }}",
				"options": {
					"timeout": 60000
				}
			},
			"notes": "Fallback parser for unsupported/edge formats."
		},
		{
			"id": "NormalizeOutput",
			"name": "Normalize Output",
			"type": "n8n-nodes-base.code",
			"typeVersion": 2,
			"position": [
				-220,
				120
			],
			"parameters": {
				"language": "javaScript",
				"jsCode": "const input = $json;\nconst now = new Date().toISOString();\n\nreturn [{\n  json: {\n    document_id: input.document_id || input.id || null,\n    checksum: input.checksum || null,\n    parser_version: input.parser_version || 'v1',\n    source: input.source || 'unknown',\n    extracted_text: input.text || input.content || '',\n    structured: input.structured || input.elements || null,\n    extracted_at: now\n  }\n}];"
			},
			"notes": "Canonical schema before DB/index writes."
		},
		{
			"id": "UpsertMetadata",
			"name": "Upsert Metadata",
			"type": "n8n-nodes-base.postgres",
			"typeVersion": 2.5,
			"position": [
				20,
				120
			],
			"parameters": {
				"operation": "executeQuery",
				"query": "INSERT INTO extracted_docs (document_id, checksum, parser_version, extracted_text, extracted_at) VALUES ('{{$json.document_id}}', '{{$json.checksum}}', '{{$json.parser_version}}', '{{$json.extracted_text}}', '{{$json.extracted_at}}') ON CONFLICT (document_id, checksum) DO UPDATE SET parser_version = EXCLUDED.parser_version, extracted_text = EXCLUDED.extracted_text, extracted_at = EXCLUDED.extracted_at;"
			},
			"notes": "Requires a configured Postgres credential in n8n."
		},
		{
			"id": "IndexForSearch",
			"name": "Index for Search",
			"type": "n8n-nodes-base.httpRequest",
			"typeVersion": 4.2,
			"position": [
				280,
				120
			],
			"parameters": {
				"method": "POST",
				"url": "={{$env.INDEX_API_URL}}/index",
				"sendBody": true,
				"specifyBody": "json",
				"jsonBody": "={{ { document_id: $json.document_id, text: $json.extracted_text, structured: $json.structured } }}"
			},
			"notes": "Send to vector DB/indexing service."
		}
	],
	"connections": {
		"Webhook Trigger": {
			"main": [
				[
					{
						"node": "Classify File",
						"type": "main",
						"index": 0
					}
				]
			]
		},
		"Classify File": {
			"main": [
				[
					{
						"node": "Extract via Docling",
						"type": "main",
						"index": 0
					}
				],
				[
					{
						"node": "Fallback via Apache Tika",
						"type": "main",
						"index": 0
					}
				]
			]
		},
		"Extract via Docling": {
			"main": [
				[
					{
						"node": "Normalize Output",
						"type": "main",
						"index": 0
					}
				]
			]
		},
		"Fallback via Apache Tika": {
			"main": [
				[
					{
						"node": "Normalize Output",
						"type": "main",
						"index": 1
					}
				]
			]
		},
		"Normalize Output": {
			"main": [
				[
					{
						"node": "Upsert Metadata",
						"type": "main",
						"index": 0
					}
				]
			]
		},
		"Upsert Metadata": {
			"main": [
				[
					{
						"node": "Index for Search",
						"type": "main",
						"index": 0
					}
				]
			]
		}
	}
}

Replace placeholders before running: DOCLING_API_URL, DOCLING_API_KEY, TIKA_API_URL, and INDEX_API_URL.

Alternative trigger variant (Google Drive / S3 style):

{
	"name": "Document Extraction Pipeline - Drive Trigger",
	"active": false,
	"nodes": [
		{
			"id": "DriveTrigger",
			"name": "Google Drive Trigger",
			"type": "n8n-nodes-base.googleDriveTrigger",
			"typeVersion": 1,
			"position": [
				-980,
				120
			],
			"parameters": {
				"event": "fileCreated",
				"pollTimes": {
					"item": [
						{
							"mode": "everyMinute"
						}
					]
				}
			},
			"notes": "Swap this for S3 Trigger when your source is object storage."
		},
		{
			"id": "MapDrivePayload",
			"name": "Map Trigger Payload",
			"type": "n8n-nodes-base.code",
			"typeVersion": 2,
			"position": [
				-760,
				120
			],
			"parameters": {
				"language": "javaScript",
				"jsCode": "return [{ json: { document_id: $json.id || $json.fileId, file_url: $json.webViewLink || $json.downloadUrl, mime_type: $json.mimeType || '', source: 'google-drive' } }];"
			}
		}
	],
	"connections": {
		"Google Drive Trigger": {
			"main": [
				[
					{
						"node": "Map Trigger Payload",
						"type": "main",
						"index": 0
					}
				]
			]
		}
	}
}

Dead-letter companion workflow (importable):

{
	"name": "Document Extraction - Dead Letter",
	"active": false,
	"nodes": [
		{
			"id": "ErrorTrigger",
			"name": "Error Trigger",
			"type": "n8n-nodes-base.errorTrigger",
			"typeVersion": 1,
			"position": [
				-780,
				140
			],
			"parameters": {}
		},
		{
			"id": "BuildFailureEnvelope",
			"name": "Build Failure Envelope",
			"type": "n8n-nodes-base.code",
			"typeVersion": 2,
			"position": [
				-520,
				140
			],
			"parameters": {
				"language": "javaScript",
				"jsCode": "const e = $json.execution || {};\nconst err = $json.error || {};\nreturn [{ json: { failed_at: new Date().toISOString(), workflow_name: e.workflowData?.name || 'unknown', execution_id: e.id || null, last_node: e.lastNodeExecuted || null, message: err.message || 'unknown error', stack: err.stack || null, payload: JSON.stringify($json) } }];"
			}
		},
		{
			"id": "PersistDeadLetter",
			"name": "Persist Dead Letter",
			"type": "n8n-nodes-base.postgres",
			"typeVersion": 2.5,
			"position": [
				-220,
				140
			],
			"parameters": {
				"operation": "executeQuery",
				"query": "INSERT INTO extraction_dead_letter (failed_at, workflow_name, execution_id, last_node, message, stack, payload) VALUES ('{{$json.failed_at}}', '{{$json.workflow_name}}', '{{$json.execution_id}}', '{{$json.last_node}}', '{{$json.message}}', '{{$json.stack}}', '{{$json.payload}}');"
			},
			"notes": "Create this table once and add retention cleanup (for example 14-30 days)."
		}
	],
	"connections": {
		"Error Trigger": {
			"main": [
				[
					{
						"node": "Build Failure Envelope",
						"type": "main",
						"index": 0
					}
				]
			]
		},
		"Build Failure Envelope": {
			"main": [
				[
					{
						"node": "Persist Dead Letter",
						"type": "main",
						"index": 0
					}
				]
			]
		}
	}
}

For production, keep the main extraction workflow in queue mode and activate this dead-letter workflow so failed executions are captured automatically for replay and audit.

SQL schema (dead-letter table):

CREATE TABLE IF NOT EXISTS extraction_dead_letter (
	id BIGSERIAL PRIMARY KEY,
	failed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
	workflow_name TEXT,
	execution_id TEXT,
	last_node TEXT,
	message TEXT NOT NULL,
	stack TEXT,
	payload JSONB,
	resolved_at TIMESTAMPTZ
);

CREATE INDEX IF NOT EXISTS idx_extraction_dead_letter_failed_at
	ON extraction_dead_letter (failed_at DESC);

CREATE INDEX IF NOT EXISTS idx_extraction_dead_letter_resolved_at
	ON extraction_dead_letter (resolved_at);

Optional retention cleanup (for example, keep 30 days of resolved rows):

DELETE FROM extraction_dead_letter
WHERE resolved_at IS NOT NULL
	AND resolved_at < NOW() - INTERVAL '30 days';

Mark an incident as resolved:

UPDATE extraction_dead_letter
SET resolved_at = NOW()
WHERE id = 12345
	AND resolved_at IS NULL;

List unresolved incidents (newest first):

SELECT id, failed_at, workflow_name, execution_id, last_node, message
FROM extraction_dead_letter
WHERE resolved_at IS NULL
ORDER BY failed_at DESC
LIMIT 100;

5. Summary Comparison Table

Alternative	Primary Focus	Best For	Output Format
Docling	Document layout & tables	LLM applications & RAG pipelines	Markdown, JSON
Unstructured	Multi-format ingestion	Complex layouts (PDF, DOCX, PPTX)	Structured JSON
Mailpond	Email processing	Direct email-to-webhook parsing	JSON
Light Client	Privacy & local processing	No-cloud local parsing	Markdown, JSON
Apache Tika	Traditional text extraction	High-volume raw text & metadata	Text, XML

6. Practical Notes Before Production

Benchmark with your real document set (not sample PDFs), because extraction quality varies significantly by scanner quality and document layout.
Keep chunking and extraction separate; first extract clean structure, then chunk for embeddings or downstream indexing.
Track parser-level metrics (latency, error rate, empty output rate, table extraction success) to spot silent quality regressions.
Version prompts and post-processing rules if you apply LLM cleanup after extraction, so output changes are explainable over time.