RSS Sources

What is RSS Subscription

RSS (Really Simple Syndication) is a website content distribution format that allows you to automatically receive the latest updates from websites. By subscribing to RSS, you don't need to visit websites frequently - the system automatically collects new content for you.

Use Cases:

• Subscribe to news websites, blogs, podcasts
• Monitor tech communities and forum activities
• Track content updates on specific topics

PlaceholderRSS source workflow diagram - showing RSS URL → content collection → library flow

Basic Configuration

Creating RSS Sources

1. Click "Sources" in the left sidebar
2. Click "New Source" button
3. Select type: RSS - General RSS Subscription
4. Fill in basic configuration:
   • Name: Custom name (e.g., "36Kr Tech News")
   • RSS URL: RSS/Atom feed address
   • Associated Library: Select storage location
   • Schedule: Recommended every 1-6 hours
   • Dedup Strategy: Select KEEP_OLD (recommended, cost-effective)

ℹ️ Tip: Most websites' RSS addresses are at the page footer or navigation bar, usually ending with

/feed

or

/rss

.

Standard RSS URL Examples

https://www.36kr.com/feed

https://sspai.com/feed

https://www.ruanyifeng.com/blog/atom.xml

RSSHub Advanced Features

What is RSSHub

RSSHub is an open-source project that can generate RSS feeds for websites without RSS (like Weibo, Bilibili, Zhihu, GitHub, etc.). OctoReport has built-in RSSHub support with 2000+ routes.

RSSHub Route Search

When creating an RSS source, click "Search RSSHub Routes" button to search all available routes:

1. Enter keywords (e.g., "Weibo", "GitHub")
2. View search results (route path, description, parameters)
3. Click "Use This Route" to auto-fill configuration

💡 Tip: Supports Chinese and English search - searching "bilibili" or "B站" both work.

PlaceholderRSSHub route search interface screenshot - showing search box, result cards, route preview dialog

RSSHub Configuration Examples

Example 1: Subscribe to GitHub Trending

RSSHub Path: /github/trending/daily/javascript
Final URL: https://rsshub.app/github/trending/daily/javascript
Description: Daily JavaScript trending projects

Example 2: Subscribe to Weibo Hot Search

RSSHub Path: /weibo/search/hot
Final URL: https://rsshub.app/weibo/search/hot
Description: Weibo real-time hot search ranking

Cookie Configuration

Why Cookies Are Needed

Some RSSHub routes require login cookies to access content (like Weibo personal pages, Bilibili follow lists, Zhihu columns, etc.).

Configuration Methods

Method 1: Admin Global Cookie (Recommended)

For admins to centrally manage accounts with all users sharing cookies.

Steps:

1. Admin login → Go to

   /admin/api-keys

2. Add API Key, service name select "RSSHub Cookie (Global)"
3. Enter complete Cookie string (from browser dev tools)

Method 2: User Custom Cookie (Optional)

For users with their own accounts needing personalized content.

Steps:

1. Create/edit RSS source
2. Expand "Cookie & User-Agent (Optional)"
3. Enter your Cookie in the Cookie text field
4. Save

⚠️ Note: Cookies usually expire after 7-30 days and need to be retrieved and updated.

Priority: User Cookie > Global Cookie > No Cookie

User-Agent Configuration

Why User-Agent Is Needed

Some websites check request sources and block default crawler requests. Configuring User-Agent can disguise as browser access to improve scraping success rate.

Configuration Methods

Method 1: Global User-Agent (Recommended)

Admin configures "RSSHub User-Agent (Global)" in

/admin/api-keys

Method 2: User Custom User-Agent

In RSS source form, expand "Cookie & User-Agent (Optional)" and enter custom User-Agent.

Preset Options

System provides 8 preset User-Agents:

• Chrome (Windows/macOS)
• Firefox (Windows)
• Safari (macOS)
• Edge (Windows)
• Mobile Chrome (Android)
• Mobile Safari (iOS)
• Curl

Priority: User UA > Global UA > RSSHub Default UA

Custom Request Headers

Use Cases

• Special Auth: Need custom

  Authorization

  header
• Anti-hotlink: Need specific

  Referer

  header
• API Access: Need

  X-API-Key

  header

Configuration Method

In RSS source form, expand "Custom Request Headers (JSON Format)" and enter JSON object:

[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],
,[object Object],
hljs json

Common Templates

{"Authorization": "Bearer xxx"}

{"X-API-Key": "your-api-key"}

{"Referer": "https://example.com"}

{"X-Forwarded-For": "1.2.3.4"}

⚠️ Security: Sensitive headers (like

Authorization

) are encrypted in storage, displayed as

***

in logs.

URL Parameter Configuration

Feature Overview

RSSHub supports 20+ URL parameters for filtering, limiting, and transforming content.

Common Parameters

1. Content Filtering (12 parameters)

filter_title

filterout_title

filter

filterout

filter_time

86400

2. Content Control (4 parameters)

limit

20

mode

fulltext

sorted

true

brief

200

3. Advanced Features (3 parameters)

opencc

s2t

t2s

format

rss

atom

json

Configuration Methods

In RSS source form, expand "Advanced Parameters (Optional)":

Method 1: Common Parameters Form (Recommended)

• Limit article count: Enter number (e.g.,

  20

  )
• Title filter: Enter regex (e.g.,

  AI|GPT

  )
• Content mode: Select "Default" or "Full Text Mode"
• Trad/Simp conversion: Select conversion direction

Method 2: JSON Format (Advanced Users)

[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],
,[object Object],
hljs json

Practical Examples

Scenario 1: Subscribe to Tech Blog, AI Articles Only

[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],
,[object Object],
hljs json

Scenario 2: Subscribe to Traditional Chinese Site, Convert to Simplified

[object Object],
  ,[object Object],[object Object], ,[object Object],[object Object],
  ,[object Object],[object Object], ,[object Object],
,[object Object],
hljs json

RSSHub Instance Authentication

Auth Modes

OctoReport supports 3 RSSHub instance auth modes:

?key=xxx

Authorization: Bearer xxx

Configuration Location

Admin configures in

/admin/rsshub-instances

1. Create/edit RSSHub instance
2. Select auth mode
3. Enter access key/Token (if applicable)

ℹ️ Security: Access keys/tokens stored with AES-256-GCM encryption, shown as

***

in URL logs.

PlaceholderRSSHub instance config interface screenshot - showing auth mode selection, access key input, priority settings

Multi-Instance Failover

How It Works

OctoReport supports configuring multiple RSSHub instances (like official instances, self-hosted instances, mirror sites). When primary instance fails, automatically switches to backup instances.

Configuration Method

Admin configures multiple instances in

/admin/rsshub-instances

1. Add multiple RSSHub instances (different URLs)
2. Set priority (lower number = higher priority)
3. Enable status (whether to participate in failover)

Failover Strategy

1. Requests try by priority from low to high (1 → 2 → 3)
2. If instance returns error (404/500/timeout), automatically switch to next
3. If all instances fail, return last error
4. Automatically log failure reason to task logs

Best Practices

• Configure at least 2 instances (primary + backup)
• Set primary instance priority to 1, backup to 2
• Self-hosted instances are faster, official instances more stable

💡 Tip: RSSHub official provides multiple mirror sites, see full list at rsshub.app.

Best Practices

✅ News Content

Recommended Config:

• Schedule: Every 6 hours
• Dedup strategy: KEEP_OLD (cost-effective)
• Content cleaning: Disabled (original content sufficient)
• URL parameters:

  {"limit": 20}

✅ Social Media Content

Recommended Config:

• Schedule: Every 1-2 hours
• Dedup strategy: UPDATE (get updates)
• Cookie: Global Cookie (requires login)
• URL parameters:

  {"limit": 50, "filter_title": "keywords"}

✅ Traditional Chinese Sites

Recommended Config:

• URL parameters:

  {"limit": 20, "opencc": "t2s"}

• User-Agent: Desktop browser

FAQs

Q1: How to Get RSS URL?

A:

1. Look for RSS icon or link on website
2. Usually at page footer, sidebar or navigation bar
3. For sites without RSS, use RSSHub route search

Q2: What If Cookie Expires?

A:

• Symptom: RSS scraping fails, logs show "not logged in"
• Solution: Re-login to target website, update Cookie config

Q3: How to Test If Config Works?

A:

1. Create test RSS source
2. Configure relevant parameters
3. Click "Run Now" button
4. Check task logs and scraping results

Q4: How to Choose Between Multiple Instances?

A:

• Self-hosted instances: Fast, no limits
• Official instances: Stable and reliable, may have rate limits
• Mirror sites: Distribute traffic, improve availability

Next Steps

• Web & Email Sources - Single page scraping, batch crawling, email monitoring
• Government & News Sources - Tender announcements, Google News
• Configuration Tips - Best practices and optimization tips