Extract URLs API: The Complete Guide

Need to extract URLs from text? This guide covers everything you need to know about URL extraction via API, including handling different protocols, validation, and domain filtering.

What is URL Extraction?

URL extraction identifies and extracts all URLs and links from unstructured text. This is useful for web scraping, content analysis, and link validation. A robust extractor handles various URL formats including HTTP, HTTPS, and protocol-relative URLs.

Example: "Visit https://example.com or check http://docs.example.org/guide" extracts both URLs.

URL Format Considerations

URLs come in many formats:

• Full URLs: https://example.com/path
• Protocol-relative: //example.com/path
• With query strings: https://example.com?q=search
• With fragments: https://example.com#section

Using the Extract URLs API

TinyFn provides a simple endpoint to extract URLs:

POST https://api.tinyfn.io/v1/text/extract-urls
Headers: X-API-Key: your-api-key
Content-Type: application/json

{
  "text": "Check out https://example.com and https://docs.example.org/guide for more info.",
  "unique_only": true
}

{
  "urls": [
    "https://example.com",
    "https://docs.example.org/guide"
  ],
  "count": 2,
  "domains": ["example.com", "docs.example.org"]
}

Parameters

Code Examples

JavaScript / Node.js

const response = await fetch(
  'https://api.tinyfn.io/v1/text/extract-urls',
  {
    method: 'POST',
    headers: {
      'X-API-Key': 'your-api-key',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      text: 'Visit https://example.com for more info.',
      unique_only: true
    })
  }
);
const result = await response.json();
console.log(result.urls); // ["https://example.com"]

Python

import requests

response = requests.post(
    'https://api.tinyfn.io/v1/text/extract-urls',
    headers={'X-API-Key': 'your-api-key'},
    json={'text': 'Visit https://example.com for more info.'}
)
result = response.json()
print(result['urls'])  # ["https://example.com"]

cURL

curl -X POST "https://api.tinyfn.io/v1/text/extract-urls" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"text": "Visit https://example.com for more info."}'

Common Use Cases

• Web Scraping: Extract links from page content
• Link Validation: Find and verify all links in content
• SEO Analysis: Analyze outbound links in articles
• Content Migration: Identify links for URL rewrites
• Security Scanning: Detect potentially malicious URLs

Best Practices

1. Deduplicate URLs: Remove duplicate links for clean lists
2. Validate URLs: Check that extracted URLs are well-formed
3. Filter by domain: Focus on relevant domains when needed
4. Handle edge cases: URLs with special characters need proper handling

Use via MCP

Your AI agent can call this tool directly via Model Context Protocol — no HTTP code needed. Add TinyFn to Claude Desktop, Cursor, or any MCP client:

{
  "mcpServers": {
    "tinyfn-text": {
      "url": "https://api.tinyfn.io/mcp/text/",
      "headers": {
        "X-API-Key": "your-api-key"
      }
    }
  }
}

See all text analysis tools available via MCP in our Text Analysis MCP Tools for AI Agents guide.