Compare commits
No commits in common. "page" and "main" have entirely different histories.
12
.github/ISSUE_TEMPLATE/功能建议.md
vendored
Normal file
@ -0,0 +1,12 @@
|
||||
---
|
||||
name: 功能建议
|
||||
about: 你想要什么高级功能?
|
||||
title: "[功能建议]"
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
你想要什么新功能?请在这里指出!
|
||||
请保留标题中的[功能建议]前缀,在后面直接打出你的问题
|
||||
(编辑后请将该行删掉)
|
19
.github/ISSUE_TEMPLATE/邮箱订阅.md
vendored
Normal file
@ -0,0 +1,19 @@
|
||||
---
|
||||
name: 邮箱订阅
|
||||
about: 请在标题后直接输入您的邮箱,不要带空格及其他符号
|
||||
title: "[邮箱订阅]"
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
* 🤩请在标题后直接输入您的邮箱,不要带空格及其他符号
|
||||
* ⚙️如:
|
||||
```txt
|
||||
[邮箱订阅]01@liushen.fun
|
||||
```
|
||||
* 🤖不要删除前缀,这将作为匹配的依据。
|
||||
* 😶🌫️订阅邮箱后,下次本站更新文章将收到推送
|
||||
* 💻若需要**退订**,可以在issue详情页面右下角删除issue
|
||||
|
||||
**该部分无需删除或编辑,仅作为解释说明,不会进行任何处理**
|
12
.github/ISSUE_TEMPLATE/问题反馈.md
vendored
Normal file
@ -0,0 +1,12 @@
|
||||
---
|
||||
name: 问题反馈
|
||||
about: 你发现了一些问题,想要指出
|
||||
title: "[问题反馈]"
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
你遇到了什么问题?欢迎反馈!
|
||||
请保留标题中的[问题反馈]字样,在后面写上你发现的问题
|
||||
(编辑时请将该部分删掉)
|
36
.github/workflows/deal_subscribe_issue.yml
vendored
Normal file
@ -0,0 +1,36 @@
|
||||
name: Handle Email Issues
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
jobs:
|
||||
handle_email_issues:
|
||||
if: startsWith(github.event.issue.title, '[邮箱订阅]')
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Add subscribed label
|
||||
uses: actions-cool/issues-helper@v3
|
||||
with:
|
||||
actions: 'add-labels'
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
issue-number: ${{ github.event.issue.number }}
|
||||
labels: 'subscribed'
|
||||
|
||||
- name: Comment on issue
|
||||
uses: actions-cool/issues-helper@v3
|
||||
with:
|
||||
actions: 'create-comment'
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
issue-number: ${{ github.event.issue.number }}
|
||||
body: |
|
||||
🤩你好呀! ${{ github.event.issue.user.login }}!你已经成功通过邮件订阅了本站啦!若有新文章将通过邮箱推送给你!谢谢你的订阅!
|
||||
😥如果您实在想要关闭订阅,在右下角直接删除这个issue就好!咕咕咕咕咕咕咕!感谢你的订阅!
|
||||
|
||||
- name: Close issue
|
||||
uses: actions-cool/issues-helper@v3
|
||||
with:
|
||||
actions: 'close-issue'
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
issue-number: ${{ github.event.issue.number }}
|
90
.github/workflows/friend_circle_lite.yml
vendored
Normal file
@ -0,0 +1,90 @@
|
||||
name: Friend Circle Lite
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 */4 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
TZ: Asia/Shanghai
|
||||
|
||||
jobs:
|
||||
friend-circle-lite:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Pull Latest Repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Cache pip packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
|
||||
- name: Install Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
|
||||
- name: Download last_articles.json artifact
|
||||
uses: dawidd6/action-download-artifact@v6
|
||||
with:
|
||||
github_token: ${{secrets.GITHUB_TOKEN}}
|
||||
branch: main
|
||||
name: 'last_articles'
|
||||
path: './rss_subscribe'
|
||||
if_no_artifact_found: warn
|
||||
|
||||
- name: Check RSS feeds
|
||||
env:
|
||||
SMTP_PWD: ${{ secrets.SMTP_PWD }}
|
||||
FCL_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
echo "Checking RSS feeds..."
|
||||
python run.py
|
||||
|
||||
- name: Upload last_articles.json as artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: 'last_articles'
|
||||
path: './rss_subscribe/last_articles.json'
|
||||
retention-days: 90
|
||||
|
||||
- name: git config
|
||||
run: |
|
||||
git config --global user.name 'github-actions[bot]'
|
||||
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
|
||||
|
||||
- name: Commit changes
|
||||
run: |
|
||||
mkdir pages
|
||||
cp -r main ./static/index.html ./static/readme.md ./static/favicon.ico ./static/bg-light.webp ./static/bg-dark.webp all.json errors.json pages/
|
||||
cd pages
|
||||
git init
|
||||
git add .
|
||||
git commit -m "⏱️ $(date +"%Y年%m月%d日-%H时%M分") GitHub Actions定时更新"
|
||||
git push --force https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git HEAD:page
|
||||
|
||||
- name: Delete Workflow Runs
|
||||
uses: Mattraks/delete-workflow-runs@v2
|
||||
with:
|
||||
retain_days: 30
|
||||
keep_minimum_runs: 6
|
||||
|
||||
keepalive-workflow:
|
||||
name: Keepalive Workflow
|
||||
if: ${{ always() }}
|
||||
needs: friend-circle-lite
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
actions: write
|
||||
steps:
|
||||
- uses: liskin/gh-workflow-keepalive@v1
|
15
.gitignore
vendored
Normal file
@ -0,0 +1,15 @@
|
||||
# 忽略 IDE 配置文件
|
||||
.idea/
|
||||
.vscode/
|
||||
|
||||
# 忽略 __pycache__ 文件夹
|
||||
**/__pycache__/
|
||||
|
||||
# 忽略 Jupyter Notebook 文件
|
||||
*.ipynb
|
||||
*.log
|
||||
|
||||
# 忽略数据文件
|
||||
*.json
|
||||
|
||||
*.bat
|
23
LICENSE
Normal file
@ -0,0 +1,23 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 Qinyang Liu
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
LICENSE BY LIUSHEN · WWW.LIUSHEN.FUN
|
66
conf.yaml
Normal file
@ -0,0 +1,66 @@
|
||||
# 爬虫相关配置
|
||||
# 解释:使用request实现友链文章爬取,并放置到根目录的all.json下
|
||||
# enable: 是否启用爬虫
|
||||
# json_url: 请填写对应格式json的地址,仅支持网络地址
|
||||
# article_count: 请填写每个博客需要获取的最大文章数量
|
||||
# marge_result: 是否合并多个json文件,若为true则会合并指定网络地址和本地地址的json文件
|
||||
# enable: 是否启用合并功能,该功能提供与自部署的友链合并功能,可以解决服务器部分国外网站无法访问的问题
|
||||
# marge_json_path: 请填写网络地址的json文件,用于合并,不带空格!!!
|
||||
spider_settings:
|
||||
enable: true
|
||||
json_url: "https://blog.liushen.fun/friend.json"
|
||||
article_count: 5
|
||||
merge_result:
|
||||
enable: false
|
||||
merge_json_url: "https://fc.liushen.fun"
|
||||
|
||||
# 邮箱推送功能配置,暂未实现,等待后续开发
|
||||
# 解释:每天为指定邮箱推送所有友链文章的更新,仅能指定一个
|
||||
# enable: 是否启用邮箱推送功能
|
||||
# to_email: 收件人邮箱地址
|
||||
# subject: 邮件主题
|
||||
# body_template: 邮件正文的 HTML 模板文件
|
||||
email_push:
|
||||
enable: false
|
||||
to_email: recipient@example.com
|
||||
subject: "今天的 RSS 订阅更新"
|
||||
body_template: "rss_template.html"
|
||||
|
||||
# 邮箱issue订阅功能配置
|
||||
# 解释:向在issue中提取的所有邮箱推送您网站中的更新,添加邮箱和删除邮箱均通过添加issue对应格式实现
|
||||
# enable: 是否启用邮箱推送功能
|
||||
# github_username: GitHub 用户名,用于构建issue api地址
|
||||
# github_repo: GitHub 仓库名,用于构建issue api地址
|
||||
# your_blog_url: 你的博客地址
|
||||
# website_info: 你的博客信息
|
||||
# title: 你的博客标题,如果启用了推送,用于生成邮件主题
|
||||
rss_subscribe:
|
||||
enable: true
|
||||
github_username: willow-god
|
||||
github_repo: Friend-Circle-Lite
|
||||
your_blog_url: https://blog.liushen.fun/
|
||||
email_template: "./rss_subscribe/email_template.html"
|
||||
website_info:
|
||||
title: "清羽飞扬"
|
||||
|
||||
# SMTP 配置
|
||||
# 解释:使用其中的相关配置实现上面两种功能,若无推送要求可以不配置,请将以上两个配置置为false
|
||||
# email: 发件人邮箱地址
|
||||
# server: SMTP 服务器地址
|
||||
# port: SMTP 端口号
|
||||
# use_tls: 是否使用 tls 加密
|
||||
smtp:
|
||||
email: 3162475700@qq.com
|
||||
server: smtp.qq.com
|
||||
port: 587
|
||||
use_tls: true
|
||||
|
||||
# 特殊RSS地址指定,可以置空但是不要删除!
|
||||
# 解释:用于指定特殊RSS地址,如B站专栏等不常见RSS地址后缀,可以添加多个
|
||||
# name: 友链名称
|
||||
# url: 指定的RSS地址
|
||||
specific_RSS:
|
||||
- name: "阮一峰"
|
||||
url: "http://feeds.feedburner.com/ruanyifeng"
|
||||
# - name: "無名小栈"
|
||||
# url: "https://blog.imsyy.top/rss.xml"
|
62
deploy.sh
Normal file
@ -0,0 +1,62 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 获取当前脚本所在目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
# 定义日志文件路径
|
||||
# CRON_LOG_FILE="$SCRIPT_DIR/cron_grab.log"
|
||||
# API_LOG_FILE="$SCRIPT_DIR/api_grab.log"
|
||||
|
||||
# # 定义要执行的命令
|
||||
# COMMAND="python3 $SCRIPT_DIR/run.py"
|
||||
|
||||
# # 定义定时任务的执行间隔(例如每四小时一次)
|
||||
# INTERVAL="4"
|
||||
|
||||
# 添加定时任务到 crontab
|
||||
# (crontab -l 2>/dev/null; echo "0 */$INTERVAL * * * $COMMAND >> $CRON_LOG_FILE 2>&1 && echo '运行成功'") | crontab -
|
||||
|
||||
# echo "===================================="
|
||||
# echo "定时爬取 成功设置,时间间隔:4h"
|
||||
# echo "定时任务日志:$CRON_LOG_FILE"
|
||||
# echo "===================================="
|
||||
|
||||
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
# 获取当前脚本所在目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
# 定义 API 服务的启动命令
|
||||
API_COMMAND="python3 $SCRIPT_DIR/server.py"
|
||||
|
||||
echo "===================================="
|
||||
|
||||
# 后台运行服务端,将数据映射到API
|
||||
echo "****正在启动API服务****"
|
||||
nohup $API_COMMAND &>/dev/null &
|
||||
API_PID=$!
|
||||
sleep 5 # 等待API服务启动,可能需要调整等待时间
|
||||
|
||||
echo "API 服务已启动:http://localhost:1223"
|
||||
echo "API 服务进程号:$API_PID"
|
||||
echo "API 服务关闭命令:kill -9 $API_PID"
|
||||
echo "文档地址:https://blog.liushen.fun/posts/4dc716ec/"
|
||||
echo "===================================="
|
||||
|
||||
# 用户选择是否执行爬取
|
||||
read -p "选择操作:0 - 退出, 1 - 执行一次爬取: " USER_CHOICE
|
||||
|
||||
if [ "$USER_CHOICE" -eq 1 ]; then
|
||||
echo "****正在执行一次爬取****"
|
||||
python3 $SCRIPT_DIR/run.py
|
||||
echo "****爬取成功****"
|
||||
else
|
||||
echo "退出选项被选择,掰掰!"
|
||||
|
||||
echo "===================================="
|
||||
echo "定时抓取的部分请自行设置,如果有宝塔等面板可以按照说明直接添加,如果没有宝塔可以查看本脚本上面屏蔽的部分,自行添加到 crontab 中"
|
||||
echo "===================================="
|
||||
|
||||
fi
|
72
errors.json
@ -1,72 +0,0 @@
|
||||
[
|
||||
[
|
||||
"LINUX DO",
|
||||
"https://linux.do/?source=blog_liushen_fun",
|
||||
"https://p.liiiu.cn/i/2024/11/11/67321caaa4447.webp"
|
||||
],
|
||||
[
|
||||
"听风小屋",
|
||||
"https://blog.ifeng.asia/",
|
||||
"https://p.liiiu.cn/i/2024/03/31/6608e2697634c.png"
|
||||
],
|
||||
[
|
||||
"雾林博客",
|
||||
"https://www.baiwulin.com/",
|
||||
"https://p.liiiu.cn/i/2024/08/02/66ac3b75826cb.webp"
|
||||
],
|
||||
[
|
||||
"陌颜Hao",
|
||||
"https://blog.imoyan.top/",
|
||||
"https://p.liiiu.cn/i/2024/08/04/66af3318f1d1c.webp"
|
||||
],
|
||||
[
|
||||
"Dreamaker",
|
||||
"https://dreamakerr.cn/",
|
||||
"https://p.liiiu.cn/i/2024/06/05/66604a6f8dba9.webp"
|
||||
],
|
||||
[
|
||||
"GuKaifeng",
|
||||
"https://gukaifeng.cn/",
|
||||
"https://p.liiiu.cn/i/2024/04/09/6614ef03406cc.png"
|
||||
],
|
||||
[
|
||||
"晚夜",
|
||||
"https://www.iczrx.cn",
|
||||
"https://p.liiiu.cn/i/2024/10/06/6702aa07d5bd8.webp"
|
||||
],
|
||||
[
|
||||
"SCFC",
|
||||
"https://blog.scfc.top/",
|
||||
"https://p.liiiu.cn/i/2025/03/10/67ce83a222bc9.webp"
|
||||
],
|
||||
[
|
||||
"L1nSn0w",
|
||||
"https://linsnow.cn/",
|
||||
"https://p.liiiu.cn/i/2024/10/03/66fd7a9e365a0.webp"
|
||||
],
|
||||
[
|
||||
"webfem",
|
||||
"https://webfem.com",
|
||||
"https://p.liiiu.cn/i/2025/02/05/67a2f35452a3e.webp"
|
||||
],
|
||||
[
|
||||
"技研录",
|
||||
"https://linmohan.fun/",
|
||||
"https://p.liiiu.cn/i/2025/03/16/67d6eb9ceca73.webp"
|
||||
],
|
||||
[
|
||||
"Eily",
|
||||
"https://ngc2237.love",
|
||||
"https://p.liiiu.cn/i/2025/01/19/678c7bf47c438.webp"
|
||||
],
|
||||
[
|
||||
"虫不知喔",
|
||||
"https://blog.ssyc.moe/",
|
||||
"https://p.liiiu.cn/i/2025/05/03/6815fc6873afb.webp"
|
||||
],
|
||||
[
|
||||
"fishcpy",
|
||||
"https://blog.fishcpy.top/",
|
||||
"https://p.liiiu.cn/i/2025/04/16/67ff9e8d5175d.webp"
|
||||
]
|
||||
]
|
0
friend_circle_lite/__init__.py
Normal file
14
friend_circle_lite/get_conf.py
Normal file
@ -0,0 +1,14 @@
|
||||
import yaml
|
||||
|
||||
def load_config(config_file):
|
||||
"""
|
||||
加载配置文件。
|
||||
|
||||
参数:
|
||||
config_file (str): 配置文件的路径。
|
||||
|
||||
返回:
|
||||
dict: 加载的配置数据。
|
||||
"""
|
||||
with open(config_file, 'r', encoding='utf-8') as file:
|
||||
return yaml.safe_load(file)
|
450
friend_circle_lite/get_info.py
Normal file
@ -0,0 +1,450 @@
|
||||
import logging
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import re
|
||||
from typing import Any
|
||||
from urllib.parse import urljoin, urlparse
|
||||
from dateutil import parser
|
||||
import requests
|
||||
import feedparser
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
# 标准化的请求头
|
||||
HEADERS_JSON = {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/123.0.0.0 Safari/537.36 "
|
||||
"(Friend-Circle-Lite/1.0; +https://github.com/willow-god/Friend-Circle-Lite)"
|
||||
),
|
||||
"X-Friend-Circle": "1.0"
|
||||
}
|
||||
|
||||
HEADERS_XML = {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/123.0.0.0 Safari/537.36 "
|
||||
"(Friend-Circle-Lite/1.0; +https://github.com/willow-god/Friend-Circle-Lite)"
|
||||
),
|
||||
"Accept": "application/rss+xml, application/xml;q=0.9, */*;q=0.8",
|
||||
"Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8",
|
||||
"Accept-Encoding": "gzip, deflate, br",
|
||||
"Connection": "keep-alive",
|
||||
"X-Friend-Circle": "1.0"
|
||||
}
|
||||
|
||||
timeout = (10, 15) # 连接超时和读取超时,防止requests接受时间过长
|
||||
|
||||
def format_published_time(time_str):
|
||||
"""
|
||||
格式化发布时间为统一格式 YYYY-MM-DD HH:MM
|
||||
|
||||
参数:
|
||||
time_str (str): 输入的时间字符串,可能是多种格式。
|
||||
|
||||
返回:
|
||||
str: 格式化后的时间字符串,若解析失败返回空字符串。
|
||||
"""
|
||||
# 尝试自动解析输入时间字符串
|
||||
try:
|
||||
parsed_time = parser.parse(time_str, fuzzy=True)
|
||||
except (ValueError, parser.ParserError):
|
||||
# 定义支持的时间格式
|
||||
time_formats = [
|
||||
'%a, %d %b %Y %H:%M:%S %z', # Mon, 11 Mar 2024 14:08:32 +0000
|
||||
'%a, %d %b %Y %H:%M:%S GMT', # Wed, 19 Jun 2024 09:43:53 GMT
|
||||
'%Y-%m-%dT%H:%M:%S%z', # 2024-03-11T14:08:32+00:00
|
||||
'%Y-%m-%dT%H:%M:%SZ', # 2024-03-11T14:08:32Z
|
||||
'%Y-%m-%d %H:%M:%S', # 2024-03-11 14:08:32
|
||||
'%Y-%m-%d' # 2024-03-11
|
||||
]
|
||||
for fmt in time_formats:
|
||||
try:
|
||||
parsed_time = datetime.strptime(time_str, fmt)
|
||||
break
|
||||
except ValueError:
|
||||
continue
|
||||
else:
|
||||
logging.warning(f"无法解析时间字符串:{time_str}")
|
||||
return ''
|
||||
|
||||
# 处理时区转换
|
||||
if parsed_time.tzinfo is None:
|
||||
parsed_time = parsed_time.replace(tzinfo=timezone.utc)
|
||||
shanghai_time = parsed_time.astimezone(timezone(timedelta(hours=8)))
|
||||
return shanghai_time.strftime('%Y-%m-%d %H:%M')
|
||||
|
||||
|
||||
|
||||
def check_feed(blog_url, session):
|
||||
"""
|
||||
检查博客的 RSS 或 Atom 订阅链接。
|
||||
|
||||
此函数接受一个博客地址,尝试在其后拼接 '/atom.xml', '/rss2.xml' 和 '/feed',并检查这些链接是否可访问。
|
||||
Atom 优先,如果都不能访问,则返回 ['none', 源地址]。
|
||||
|
||||
参数:
|
||||
blog_url (str): 博客的基础 URL。
|
||||
session (requests.Session): 用于请求的会话对象。
|
||||
|
||||
返回:
|
||||
list: 包含类型和拼接后的链接的列表。如果 atom 链接可访问,则返回 ['atom', atom_url];
|
||||
如果 rss2 链接可访问,则返回 ['rss2', rss_url];
|
||||
如果 feed 链接可访问,则返回 ['feed', feed_url];
|
||||
如果都不可访问,则返回 ['none', blog_url]。
|
||||
"""
|
||||
|
||||
possible_feeds = [
|
||||
('atom', '/atom.xml'),
|
||||
('rss', '/rss.xml'), # 2024-07-26 添加 /rss.xml内容的支持
|
||||
('rss2', '/rss2.xml'),
|
||||
('rss3', '/rss.php'), # 2024-12-07 添加 /rss.php内容的支持
|
||||
('feed', '/feed'),
|
||||
('feed2', '/feed.xml'), # 2024-07-26 添加 /feed.xml内容的支持
|
||||
('feed3', '/feed/'),
|
||||
('index', '/index.xml') # 2024-07-25 添加 /index.xml内容的支持
|
||||
]
|
||||
|
||||
for feed_type, path in possible_feeds:
|
||||
feed_url = blog_url.rstrip('/') + path
|
||||
try:
|
||||
response = session.get(feed_url, headers=HEADERS_XML, timeout=timeout)
|
||||
if response.status_code == 200:
|
||||
return [feed_type, feed_url]
|
||||
except requests.RequestException:
|
||||
continue
|
||||
logging.warning(f"无法找到 {blog_url} 的订阅链接")
|
||||
return ['none', blog_url]
|
||||
|
||||
|
||||
def parse_feed(url, session, count=5, blog_url=''):
|
||||
"""
|
||||
解析 Atom 或 RSS2 feed 并返回包含网站名称、作者、原链接和每篇文章详细内容的字典。
|
||||
|
||||
此函数接受一个 feed 的地址(atom.xml 或 rss2.xml),解析其中的数据,并返回一个字典结构,
|
||||
其中包括网站名称、作者、原链接和每篇文章的详细内容。
|
||||
|
||||
参数:
|
||||
url (str): Atom 或 RSS2 feed 的 URL。
|
||||
session (requests.Session): 用于请求的会话对象。
|
||||
count (int): 获取文章数的最大数。如果小于则全部获取,如果文章数大于则只取前 count 篇文章。
|
||||
|
||||
返回:
|
||||
dict: 包含网站名称、作者、原链接和每篇文章详细内容的字典。
|
||||
"""
|
||||
try:
|
||||
response = session.get(url, headers=HEADERS_XML, timeout=timeout)
|
||||
response.encoding = response.apparent_encoding or 'utf-8'
|
||||
feed = feedparser.parse(response.text)
|
||||
|
||||
result = {
|
||||
'website_name': feed.feed.title if 'title' in feed.feed else '', # type: ignore
|
||||
'author': feed.feed.author if 'author' in feed.feed else '', # type: ignore
|
||||
'link': feed.feed.link if 'link' in feed.feed else '', # type: ignore
|
||||
'articles': []
|
||||
}
|
||||
|
||||
for _ , entry in enumerate(feed.entries):
|
||||
|
||||
if 'published' in entry:
|
||||
published = format_published_time(entry.published)
|
||||
elif 'updated' in entry:
|
||||
published = format_published_time(entry.updated)
|
||||
# 输出警告信息
|
||||
logging.warning(f"文章 {entry.title} 未包含发布时间,已使用更新时间 {published}")
|
||||
else:
|
||||
published = ''
|
||||
logging.warning(f"文章 {entry.title} 未包含任何时间信息, 请检查原文, 设置为默认时间")
|
||||
|
||||
# 处理链接中可能存在的错误,比如ip或localhost
|
||||
article_link = replace_non_domain(entry.link, blog_url) if 'link' in entry else '' # type: ignore
|
||||
|
||||
article = {
|
||||
'title': entry.title if 'title' in entry else '',
|
||||
'author': result['author'],
|
||||
'link': article_link,
|
||||
'published': published,
|
||||
'summary': entry.summary if 'summary' in entry else '',
|
||||
'content': entry.content[0].value if 'content' in entry and entry.content else entry.description if 'description' in entry else ''
|
||||
}
|
||||
result['articles'].append(article)
|
||||
|
||||
# 对文章按时间排序,并只取前 count 篇文章
|
||||
result['articles'] = sorted(result['articles'], key=lambda x: datetime.strptime(x['published'], '%Y-%m-%d %H:%M'), reverse=True)
|
||||
if count < len(result['articles']):
|
||||
result['articles'] = result['articles'][:count]
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
logging.error(f"无法解析FEED地址:{url} ,请自行排查原因!")
|
||||
return {
|
||||
'website_name': '',
|
||||
'author': '',
|
||||
'link': '',
|
||||
'articles': []
|
||||
}
|
||||
|
||||
def replace_non_domain(link: str, blog_url: str) -> str:
|
||||
"""
|
||||
暂未实现
|
||||
检测并替换字符串中的非正常域名部分(如 IP 地址或 localhost),替换为 blog_url。
|
||||
替换后强制使用 https,且考虑 blog_url 尾部是否有斜杠。
|
||||
|
||||
:param link: 原始地址字符串
|
||||
:param blog_url: 替换为的博客地址
|
||||
:return: 替换后的地址字符串
|
||||
"""
|
||||
|
||||
# 提取link中的路径部分,无需协议和域名
|
||||
# path = re.sub(r'^https?://[^/]+', '', link)
|
||||
# print(path)
|
||||
|
||||
try:
|
||||
parsed = urlparse(link)
|
||||
if 'localhost' in parsed.netloc or re.match(r'^\d{1,3}(\.\d{1,3}){3}$', parsed.netloc): # IP地址或localhost
|
||||
# 提取 path + query
|
||||
path = parsed.path or '/'
|
||||
if parsed.query:
|
||||
path += '?' + parsed.query
|
||||
return urljoin(blog_url.rstrip('/') + '/', path.lstrip('/'))
|
||||
else:
|
||||
return link # 合法域名则返回原链接
|
||||
except Exception as e:
|
||||
logging.warning(f"替换链接时出错:{link}, error: {e}")
|
||||
return link
|
||||
|
||||
def process_friend(friend, session, count, specific_RSS=[]):
|
||||
"""
|
||||
处理单个朋友的博客信息。
|
||||
|
||||
参数:
|
||||
friend (list): 包含朋友信息的列表 [name, blog_url, avatar]。
|
||||
session (requests.Session): 用于请求的会话对象。
|
||||
count (int): 获取每个博客的最大文章数。
|
||||
specific_RSS (list): 包含特定 RSS 源的字典列表 [{name, url}]
|
||||
|
||||
返回:
|
||||
dict: 包含朋友博客信息的字典。
|
||||
"""
|
||||
name, blog_url, avatar = friend
|
||||
|
||||
# 如果 specific_RSS 中有对应的 name,则直接返回 feed_url
|
||||
if specific_RSS is None:
|
||||
specific_RSS = []
|
||||
rss_feed = next((rss['url'] for rss in specific_RSS if rss['name'] == name), None)
|
||||
if rss_feed:
|
||||
feed_url = rss_feed
|
||||
feed_type = 'specific'
|
||||
logging.info(f"“{name}”的博客“ {blog_url} ”为特定RSS源“ {feed_url} ”")
|
||||
else:
|
||||
feed_type, feed_url = check_feed(blog_url, session)
|
||||
logging.info(f"“{name}”的博客“ {blog_url} ”的feed类型为“{feed_type}”, feed地址为“ {feed_url} ”")
|
||||
|
||||
if feed_type != 'none':
|
||||
feed_info = parse_feed(feed_url, session, count, blog_url)
|
||||
articles = [
|
||||
{
|
||||
'title': article['title'],
|
||||
'created': article['published'],
|
||||
'link': article['link'],
|
||||
'author': name,
|
||||
'avatar': avatar
|
||||
}
|
||||
for article in feed_info['articles']
|
||||
]
|
||||
|
||||
for article in articles:
|
||||
logging.info(f"{name} 发布了新文章:{article['title']},时间:{article['created']},链接:{article['link']}")
|
||||
|
||||
return {
|
||||
'name': name,
|
||||
'status': 'active',
|
||||
'articles': articles
|
||||
}
|
||||
else:
|
||||
logging.warning(f"{name} 的博客 {blog_url} 无法访问")
|
||||
return {
|
||||
'name': name,
|
||||
'status': 'error',
|
||||
'articles': []
|
||||
}
|
||||
|
||||
def fetch_and_process_data(json_url, specific_RSS=[], count=5):
|
||||
"""
|
||||
读取 JSON 数据并处理订阅信息,返回统计数据和文章信息。
|
||||
|
||||
参数:
|
||||
json_url (str): 包含朋友信息的 JSON 文件的 URL。
|
||||
count (int): 获取每个博客的最大文章数。
|
||||
specific_RSS (list): 包含特定 RSS 源的字典列表 [{name, url}]
|
||||
|
||||
返回:
|
||||
dict: 包含统计数据和文章信息的字典。
|
||||
"""
|
||||
session = requests.Session()
|
||||
|
||||
try:
|
||||
response = session.get(json_url, headers=HEADERS_JSON, timeout=timeout)
|
||||
friends_data = response.json()
|
||||
except Exception as e:
|
||||
logging.error(f"无法获取链接:{json_url} :{e}", exc_info=True)
|
||||
return None
|
||||
|
||||
total_friends = len(friends_data['friends'])
|
||||
active_friends = 0
|
||||
error_friends = 0
|
||||
total_articles = 0
|
||||
article_data = []
|
||||
error_friends_info = []
|
||||
|
||||
with ThreadPoolExecutor(max_workers=10) as executor:
|
||||
future_to_friend = {
|
||||
executor.submit(process_friend, friend, session, count, specific_RSS): friend
|
||||
for friend in friends_data['friends']
|
||||
}
|
||||
|
||||
for future in as_completed(future_to_friend):
|
||||
friend = future_to_friend[future]
|
||||
try:
|
||||
result = future.result()
|
||||
if result['status'] == 'active':
|
||||
active_friends += 1
|
||||
article_data.extend(result['articles'])
|
||||
total_articles += len(result['articles'])
|
||||
else:
|
||||
error_friends += 1
|
||||
error_friends_info.append(friend)
|
||||
except Exception as e:
|
||||
logging.error(f"处理 {friend} 时发生错误: {e}", exc_info=True)
|
||||
error_friends += 1
|
||||
error_friends_info.append(friend)
|
||||
|
||||
result = {
|
||||
'statistical_data': {
|
||||
'friends_num': total_friends,
|
||||
'active_num': active_friends,
|
||||
'error_num': error_friends,
|
||||
'article_num': total_articles,
|
||||
'last_updated_time': datetime.now().strftime('%Y-%m-%d %H:%M:%S')
|
||||
},
|
||||
'article_data': article_data
|
||||
}
|
||||
|
||||
logging.info(f"数据处理完成,总共有 {total_friends} 位朋友,其中 {active_friends} 位博客可访问,{error_friends} 位博客无法访问")
|
||||
|
||||
return result, error_friends_info
|
||||
|
||||
def sort_articles_by_time(data):
|
||||
"""
|
||||
对文章数据按时间排序
|
||||
|
||||
参数:
|
||||
data (dict): 包含文章信息的字典
|
||||
|
||||
返回:
|
||||
dict: 按时间排序后的文章信息字典
|
||||
"""
|
||||
# 先确保每个元素存在时间
|
||||
for article in data['article_data']:
|
||||
if article['created'] == '' or article['created'] == None:
|
||||
article['created'] = '2024-01-01 00:00'
|
||||
# 输出警告信息
|
||||
logging.warning(f"文章 {article['title']} 未包含时间信息,已设置为默认时间 2024-01-01 00:00")
|
||||
|
||||
if 'article_data' in data:
|
||||
sorted_articles = sorted(
|
||||
data['article_data'],
|
||||
key=lambda x: datetime.strptime(x['created'], '%Y-%m-%d %H:%M'),
|
||||
reverse=True
|
||||
)
|
||||
data['article_data'] = sorted_articles
|
||||
return data
|
||||
|
||||
def marge_data_from_json_url(data, marge_json_url):
|
||||
"""
|
||||
从另一个 JSON 文件中获取数据并合并到原数据中。
|
||||
|
||||
参数:
|
||||
data (dict): 包含文章信息的字典
|
||||
marge_json_url (str): 包含另一个文章信息的 JSON 文件的 URL。
|
||||
|
||||
返回:
|
||||
dict: 合并后的文章信息字典,已去重处理
|
||||
"""
|
||||
try:
|
||||
response = requests.get(marge_json_url, headers=HEADERS_JSON, timeout=timeout)
|
||||
marge_data = response.json()
|
||||
except Exception as e:
|
||||
logging.error(f"无法获取链接:{marge_json_url},出现的问题为:{e}", exc_info=True)
|
||||
return data
|
||||
|
||||
if 'article_data' in marge_data:
|
||||
logging.info(f"开始合并数据,原数据共有 {len(data['article_data'])} 篇文章,第三方数据共有 {len(marge_data['article_data'])} 篇文章")
|
||||
data['article_data'].extend(marge_data['article_data'])
|
||||
data['article_data'] = list({v['link']:v for v in data['article_data']}.values())
|
||||
logging.info(f"合并数据完成,现在共有 {len(data['article_data'])} 篇文章")
|
||||
return data
|
||||
|
||||
import requests
|
||||
|
||||
def marge_errors_from_json_url(errors, marge_json_url):
|
||||
"""
|
||||
从另一个网络 JSON 文件中获取错误信息并遍历,删除在errors中,
|
||||
不存在于marge_errors中的友链信息。
|
||||
|
||||
参数:
|
||||
errors (list): 包含错误信息的列表
|
||||
marge_json_url (str): 包含另一个错误信息的 JSON 文件的 URL。
|
||||
|
||||
返回:
|
||||
list: 合并后的错误信息列表
|
||||
"""
|
||||
try:
|
||||
response = requests.get(marge_json_url, timeout=10) # 设置请求超时时间
|
||||
marge_errors = response.json()
|
||||
except Exception as e:
|
||||
logging.error(f"无法获取链接:{marge_json_url},出现的问题为:{e}", exc_info=True)
|
||||
return errors
|
||||
|
||||
# 提取 marge_errors 中的 URL
|
||||
marge_urls = {item[1] for item in marge_errors}
|
||||
|
||||
# 使用过滤器保留 errors 中在 marge_errors 中出现的 URL
|
||||
filtered_errors = [error for error in errors if error[1] in marge_urls]
|
||||
|
||||
logging.info(f"合并错误信息完成,合并后共有 {len(filtered_errors)} 位朋友")
|
||||
return filtered_errors
|
||||
|
||||
def deal_with_large_data(result):
|
||||
"""
|
||||
处理文章数据,保留前150篇及其作者在后续文章中的出现。
|
||||
|
||||
参数:
|
||||
result (dict): 包含统计数据和文章数据的字典。
|
||||
|
||||
返回:
|
||||
dict: 处理后的数据,只包含需要的文章。
|
||||
"""
|
||||
result = sort_articles_by_time(result)
|
||||
article_data = result.get("article_data", [])
|
||||
|
||||
# 检查文章数量是否大于 150
|
||||
max_articles = 150
|
||||
if len(article_data) > max_articles:
|
||||
logging.info("数据量较大,开始进行处理...")
|
||||
# 获取前 max_articles 篇文章的作者集合
|
||||
top_authors = {article["author"] for article in article_data[:max_articles]}
|
||||
|
||||
# 从第 {max_articles + 1} 篇开始过滤,只保留前 max_articles 篇出现过的作者的文章
|
||||
filtered_articles = article_data[:max_articles] + [
|
||||
article for article in article_data[max_articles:]
|
||||
if article["author"] in top_authors
|
||||
]
|
||||
|
||||
# 更新结果中的 article_data
|
||||
result["article_data"] = filtered_articles
|
||||
# 更新结果中的统计数据
|
||||
result["statistical_data"]["article_num"] = len(filtered_articles)
|
||||
logging.info(f"数据处理完成,保留 {len(filtered_articles)} 篇文章")
|
||||
|
||||
return result
|
0
push_rss_update/__init__.py
Normal file
80
push_rss_update/send_email.py
Normal file
@ -0,0 +1,80 @@
|
||||
import logging
|
||||
import smtplib
|
||||
from email.mime.multipart import MIMEMultipart
|
||||
from email.mime.text import MIMEText
|
||||
from jinja2 import Environment, FileSystemLoader
|
||||
import os
|
||||
|
||||
def email_sender(
|
||||
target_email,
|
||||
sender_email,
|
||||
smtp_server,
|
||||
port,
|
||||
password,
|
||||
subject,
|
||||
body,
|
||||
template_path=None,
|
||||
template_data=None,
|
||||
use_tls=True,
|
||||
):
|
||||
"""
|
||||
发送电子邮件。
|
||||
|
||||
参数:
|
||||
target_email (str): 目标邮箱地址。
|
||||
sender_email (str): 发信邮箱地址。
|
||||
smtp_server (str): SMTP 服务地址。
|
||||
port (int): SMTP 服务端口。
|
||||
password (str): SMTP 服务密码。
|
||||
subject (str): 邮件主题。
|
||||
body (str): 邮件内容。
|
||||
template_path (str): HTML 模板文件路径。默认为 None。
|
||||
template_data (dict): 渲染模板的数据。默认为 None。
|
||||
use_tls (bool): 是否使用 TLS 加密。默认为 True。
|
||||
"""
|
||||
# 创建 MIME 对象
|
||||
msg = MIMEMultipart()
|
||||
msg['From'] = sender_email
|
||||
msg['To'] = target_email
|
||||
msg['Subject'] = subject
|
||||
|
||||
if template_path and template_data:
|
||||
# 使用 Jinja2 渲染 HTML 模板
|
||||
env = Environment(loader=FileSystemLoader(os.path.dirname(template_path)))
|
||||
template = env.get_template(os.path.basename(template_path))
|
||||
html_content = template.render(template_data)
|
||||
msg.attach(MIMEText(html_content, 'html'))
|
||||
else:
|
||||
# 添加纯文本邮件内容
|
||||
msg.attach(MIMEText(body, 'plain'))
|
||||
|
||||
# 连接到 SMTP 服务器并发送邮件
|
||||
try:
|
||||
with smtplib.SMTP(smtp_server, port) as server:
|
||||
if use_tls:
|
||||
server.starttls() # 启动安全模式
|
||||
server.login(sender_email, password)
|
||||
server.sendmail(sender_email, target_email, msg.as_string())
|
||||
print(f'邮件已发送到 {target_email}')
|
||||
except Exception as e:
|
||||
logging.error(f'邮件发送失败,目标地址: {target_email},错误信息: {e}')
|
||||
|
||||
def send_emails(emails, sender_email, smtp_server, port, password, subject, body, template_path=None, template_data=None, use_tls=True):
|
||||
"""
|
||||
循环发送邮件给指定的多个邮箱。
|
||||
|
||||
参数:
|
||||
emails (list): 包含目标邮箱地址的列表。
|
||||
sender_email (str): 发信邮箱地址。
|
||||
smtp_server (str): SMTP 服务地址。
|
||||
port (int): SMTP 服务端口。
|
||||
password (str): SMTP 服务密码。
|
||||
subject (str): 邮件主题。
|
||||
body (str): 邮件内容。
|
||||
template_path (str): HTML 模板文件路径。默认为 None。
|
||||
template_data (dict): 渲染模板的数据。默认为 None。
|
||||
use_tls (bool): 是否使用 TLS 加密。默认为 True。
|
||||
"""
|
||||
for email in emails:
|
||||
logging.info(f'正在发送邮件到 {email},邮件内容: {subject}')
|
||||
email_sender(email, sender_email, smtp_server, port, password, subject, body, template_path, template_data, use_tls)
|
418
readme.md
@ -1,16 +1,336 @@
|
||||
# RSS 订阅前端页面
|
||||
<div align="center">
|
||||
<img src="./static/favicon.ico" width="200" alt="fclite">
|
||||
|
||||
这是一个用于展示 RSS 订阅内容的简单 HTML 页面。该前端页面用于渲染从后端获取的 RSS 订阅数据。本分支仅包含用于展示的静态资源(HTML、CSS、JS)。
|
||||
[前端展示](https://fc.liushen.fun) | [详细文档](https://blog.liushen.fun/posts/4dc716ec/)
|
||||
|
||||
## 功能
|
||||
# Friend-Circle-Lite
|
||||
|
||||
- **展示 RSS 订阅内容**:可以显示 RSS 订阅文章的标题、描述和发布时间。
|
||||
- **简洁设计**:简单直观的用户界面,适用于浏览和查看 RSS 内容。
|
||||
- **响应式布局**:适配不同设备的浏览体验。
|
||||
</div>
|
||||
|
||||
## 部署到网站
|
||||
友链朋友圈简单版,实现了[友链朋友圈](https://github.com/Rock-Candy-Tea/hexo-circle-of-friends)的基本功能,能够定时爬取rss文章并输出有序内容,为了较好的兼容性,输入格式与友链朋友圈的json格式一致,为了轻量化,暂不支持从友链页面自动爬取,下面会附带`hexo-theme-butterfly`主题的解决方案,其他主题可以类比。
|
||||
|
||||
如果你已经正确托管本分支到静态托管平台,你可以通过以下几个步骤将数据渲染到你的前端页面:
|
||||
## 开发进度
|
||||
|
||||
### 2024-10-29
|
||||
|
||||
* 完善github数据获取,从环境变量中直接获取,配置文件仅用于自部署
|
||||
* 限制文章数量,防止因为数量过大导致的api文件加载缓慢,仅保留150左右文章([#23](https://github.com/willow-god/Friend-Circle-Lite/pull/23))
|
||||
* 修改action中写错的github_token拼写
|
||||
|
||||
### 2024-10-07
|
||||
|
||||
* 添加随机文章刷新按钮
|
||||
* 完善邮件通知模板自定义程度
|
||||
|
||||
### 2024-09-28
|
||||
|
||||
* 更新自部署的api地址,统一为all.json,提高js兼容性
|
||||
* 美化展示页面UI(@JLinMr),添加背景图片
|
||||
* 优化作者卡片弹窗动效(@JLinMr)
|
||||
|
||||
<details>
|
||||
<summary>查看更多</summary>
|
||||
|
||||
<h3>2024-09-22</h3>
|
||||
|
||||
* 修复 #18 提出的,由于rss倒序导致限制抓取错误的问题,改为先全部获取后,按照时间排序,再选择性获取
|
||||
|
||||
<h3>2024-09-05</h3>
|
||||
|
||||
* 更新部署方式,将静态文件放到page分支下,主分支不放数据文件
|
||||
* 前后端分离,部署方式不变但更加直观方便
|
||||
|
||||
<h3>2024-09-03</h3>
|
||||
|
||||
* 添加特定RSS选项,用于指定部分友链特殊RSS地址
|
||||
* 更新文档,添加特定RSS选项配置部分
|
||||
|
||||
<h3>2024-08-28</h3>
|
||||
|
||||
* 日常维护,修复issue中提出的时间为空导致错误的情况,使用更新时间代替
|
||||
|
||||
<h3>2024-08-11</h3>
|
||||
|
||||
* 添加服务器部署的情况下,合并github结果的选项
|
||||
* 由于复杂性,决定将服务和定时抓取分开,使用面板自带进行配置,防止小白无法配置
|
||||
* 修改文档,添加自部署部分
|
||||
|
||||
<h3>2024-08-03</h3>
|
||||
|
||||
* 将自部署分离为API服务和定时爬取
|
||||
* 尝试更加系统的启动脚本
|
||||
* 删除server.py中的爬取内容,使用定时任务crontab实现
|
||||
|
||||
<h3>2024-07-28</h3>
|
||||
|
||||
* 自部署添加跨域请求
|
||||
* 修复内存占用异常问题
|
||||
* 将html资源分开存放,实现更加美观的页面
|
||||
|
||||
<h3>2024-07-26</h3>
|
||||
|
||||
* 自部署添加跨域请求
|
||||
* 添加`/rss.xml`,`/feed/`,`feed.xml`接口的爬取,提高兼容性
|
||||
* 修复PJAX下会多次出现模态框的问题,并且切换页面不消失
|
||||
* 修复模态框宽度问题,添加日历图标以更加美观
|
||||
|
||||
<h3>2024-07-25</h3>
|
||||
|
||||
* 自部署正在开发中,仅供测试
|
||||
* 添加`/errors.json`,用于获取丢失友链数据,提高自定义程度
|
||||
* 添加`/index.xml`接口的爬取,提高兼容性
|
||||
</details>
|
||||
|
||||
|
||||
|
||||
## 展示页面
|
||||
|
||||
* [清羽飞扬の友链朋友圈](https://blog.liushen.fun/fcircle/)
|
||||
|
||||
* [❖星港◎Star☆ 的友链朋友圈](https://blog.starsharbor.com/fcircle/)
|
||||
|
||||
* [梦爱吃鱼的友链朋友圈](https://blog.bsgun.cn/fcircle/)
|
||||
|
||||
* 欢迎在issue中[提交](https://github.com/willow-god/Friend-Circle-Lite/issues/20)以展示你独特的设计!
|
||||
|
||||
## 项目介绍
|
||||
|
||||
- **爬取文章**: 爬取所有友链的文章,结果放置在根目录的all.json文件中,方便读取并部署到前端。
|
||||
- **邮箱推送更新(对作者推送所有友链更新)**: 作者可以通过邮箱订阅所有rss的更新(未来开发)。
|
||||
- **issue邮箱订阅(对访客实时推送最新文章邮件)**: 基于`GitHub issue`的博客更新邮件订阅功能,游客可以通过简单的提交`issue`进行邮箱订阅站点更新,删除对应`issue`即可取消订阅。
|
||||
- **文件分离**: 将前后端分离,前端文件放在page分支,后端文件放在主分支
|
||||
|
||||
## 特点介绍
|
||||
|
||||
* **轻量化**:对比原版友链朋友圈的功能,该友圈功能简洁,去掉了设置和fastAPI的臃肿,仅保留关键内容。
|
||||
* **无数据库**:因为内容较少,我采用`json`直接存储文章信息,减少数据库操作,提升`action`运行效率。
|
||||
* **部署简单**:原版友链朋友圈由于功能多,导致部署较为麻烦,本方案仅需简单的部署action即可使用,vercel仅用于部署前端静态页面和实时获取最新内容。
|
||||
* **文件占用**:对比原版`4MB`的`bundle.js`文件大小,本项目仅需要`5.50KB`的`fclite.min.js`文件即可轻量的展示到前端。
|
||||
|
||||
## 功能概览
|
||||
|
||||
* 文章爬取
|
||||
* 暗色适配
|
||||
* 显示作者所有文章
|
||||
* 获取丢失友链数据
|
||||
* 随机钓鱼
|
||||
* 邮箱推送
|
||||
* 美观邮箱模板
|
||||
* 自部署(2024-08-11添加)
|
||||
* 前端单开分支(2024-09-05添加) @CCKNBC
|
||||
|
||||
## action部署使用方法
|
||||
|
||||
### 前置工作
|
||||
|
||||
1. **Fork 本仓库:**
|
||||
点击页面右上角的 Fork 按钮,将本仓库复制到你自己的`GitHub`账号下,仅复刻main分支即可。
|
||||

|
||||
|
||||
2. **配置 Secrets:**
|
||||
在你 Fork 的仓库中,依次进入 `Settings` -> `Secrets` -> `New repository secret`,添加以下 Secrets:
|
||||
- `SMTP_PWD`(可选): SMTP 服务器的密码,用于发送电子邮件,如果你不需要,可以不进行配置。
|
||||
|
||||

|
||||
|
||||
3. **配置action权限:**
|
||||
|
||||
在设置中,点击`action`,拉到最下面,勾选`Read and write permissions`选项并保存,确保action有读写权限。
|
||||
|
||||
4. **启用 GitHub Actions:**
|
||||
GitHub Actions 已经配置好在仓库的 `.github/workflows/*.yml` 文件中,当到一定时间时将自动执行,也可以手动运行。
|
||||
其中,每个action功能如下:
|
||||
|
||||
- `friend_circle_lite.yml`实现核心功能,爬取并发送邮箱,需要在Action中启用;
|
||||
- `deal_subscribe_issue.yml`处理固定格式的issue,打上固定标签,评论,并关闭issue;
|
||||
|
||||
5. **设置issue格式:**
|
||||
这个我已经设置好了,你只需要检查issue部分是否有对应格式即可,可以自行修改对应参数以进行自定义。
|
||||
|
||||
### 配置选项
|
||||
|
||||
1. 如果需要修改爬虫设置或邮件模板等配置,需要修改仓库中的 `config.yaml` 文件:
|
||||
|
||||
- **爬虫相关配置**
|
||||
使用 `requests` 库实现友链文章的爬取,并将结果存储到根目录下的 `all.json` 文件中。
|
||||
|
||||
```yaml
|
||||
spider_settings:
|
||||
enable: true
|
||||
json_url: "https://blog.qyliu.top/friend.json"
|
||||
article_count: 5
|
||||
merge_result:
|
||||
enable: true
|
||||
merge_json_url: "https://fc.liushen.fun"
|
||||
```
|
||||
|
||||
`enable`:开启或关闭,默认开启;
|
||||
|
||||
`json_url`:友链朋友圈通用爬取格式第一种(下方有配置方法);
|
||||
|
||||
`article_count`:每个作者留存文章个数。
|
||||
|
||||
`marge_result`:是否合并多个json文件,若为true则会合并指定网络地址和本地地址的json文件并去重
|
||||
|
||||
- `enable`:是否启用合并功能,该功能提供与自部署的友链合并功能,可以解决服务器部分国外网站,服务器无法访问的问题
|
||||
|
||||
- `marge_json_path`:请填写网络地址的json文件,用于合并,不带空格!!!
|
||||
|
||||
- **邮箱推送功能配置**
|
||||
暂未实现,预留用于将每天的友链文章更新推送给指定邮箱。
|
||||
|
||||
```yaml
|
||||
email_push:
|
||||
enable: false
|
||||
to_email: recipient@example.com
|
||||
subject: "今天的 RSS 订阅更新"
|
||||
body_template: "rss_template.html"
|
||||
```
|
||||
|
||||
**暂未实现**:该部分暂未实现,由于感觉用处不大,保留接口后期酌情更新。
|
||||
|
||||
- **邮箱 issue 订阅功能配置**
|
||||
通过 GitHub issue 实现向提取的所有邮箱推送博客更新的功能。
|
||||
|
||||
```yaml
|
||||
rss_subscribe:
|
||||
enable: true
|
||||
github_username: willow-god
|
||||
github_repo: Friend-Circle-Lite
|
||||
your_blog_url: https://blog.qyliu.top/
|
||||
```
|
||||
|
||||
`enable`:开启或关闭,默认开启,如果没有配置请关闭。
|
||||
|
||||
`github_username`:github用户名,用来拼接github api地址
|
||||
|
||||
`github_repo`:仓库名称,作用同上。
|
||||
|
||||
`your_blog_url`:用来定时检测是否有最新文章,请确保你的网站可以被FCLite抓取到
|
||||
|
||||
- **SMTP 配置**
|
||||
使用配置中的相关信息实现邮件发送功能。
|
||||
|
||||
```yaml
|
||||
smtp:
|
||||
email: 3162475700@qq.com
|
||||
server: smtp.qq.com
|
||||
port: 587
|
||||
use_tls: true
|
||||
```
|
||||
|
||||
`email`:发件人邮箱地址
|
||||
|
||||
`server`:`SMTP` 服务器地址
|
||||
|
||||
`port`:`SMTP` 端口号
|
||||
|
||||
`use_tls`:是否使用 `tls` 加密
|
||||
|
||||
这部分配置较为复杂,请自行学习使用。
|
||||
|
||||
- **特定 RSS 配置**
|
||||
|
||||
用于指定特定友链特殊RSS,样例如下:
|
||||
|
||||
```yaml
|
||||
specific_RSS:
|
||||
- name: "Redish101"
|
||||
url: "https://reblog.redish101.top/api/feed"
|
||||
# - name: "無名小栈"
|
||||
# url: "https://blog.imsyy.top/rss.xml"
|
||||
```
|
||||
|
||||
`name`:友链名称,需要严格匹配
|
||||
|
||||
`url`:该友链对应RSS地址
|
||||
|
||||
可以添加多个,如果不需要也可以置空。
|
||||
|
||||
2. **贡献与定制:**
|
||||
欢迎对仓库进行贡献或根据需要进行定制。
|
||||
|
||||
**如果你配置正常,那么等action运行一次(可以手动运行)应该就可以在page分支看到结果了,检查一下,如果结果无误,可以继续看下一步**
|
||||
|
||||
### 友圈json生成
|
||||
|
||||
**注意,以下可能仅适用于hexo-theme-butterfly或部分类butterfly主题,如果你是其他主题,可以自行适配,理论上只要存在友链数据文件都可以整理为该类型,甚至可以自行整理为对应json格式后放到 `/source` 目录下即可,格式可以参考:`https://blog.qyliu.top/friend.json` **
|
||||
|
||||
1. 将以下文件放置到博客根目录:
|
||||
|
||||
```javascript
|
||||
const YML = require('yamljs')
|
||||
const fs = require('fs')
|
||||
|
||||
let friends = [],
|
||||
data_f = YML.parse(fs.readFileSync('source/_data/link.yml').toString().replace(/(?<=rss:)\s*\n/g, ' ""\n'));
|
||||
|
||||
data_f.forEach((entry, index) => {
|
||||
let lastIndex = 2;
|
||||
if (index < lastIndex) {
|
||||
const filteredLinkList = entry.link_list.filter(linkItem => !blacklist.includes(linkItem.name));
|
||||
friends = friends.concat(filteredLinkList);
|
||||
}
|
||||
});
|
||||
|
||||
// 根据规定的格式构建 JSON 数据
|
||||
const friendData = {
|
||||
friends: friends.map(item => {
|
||||
return [item.name, item.link, item.avatar];
|
||||
})
|
||||
};
|
||||
|
||||
// 将 JSON 对象转换为字符串
|
||||
const friendJSON = JSON.stringify(friendData, null, 2);
|
||||
|
||||
// 写入 friend.json 文件
|
||||
fs.writeFileSync('./source/friend.json', friendJSON);
|
||||
|
||||
console.log('friend.json 文件已生成。');
|
||||
```
|
||||
|
||||
2. 在根目录下运行:
|
||||
|
||||
```bash
|
||||
node link.js
|
||||
```
|
||||
|
||||
你将会在source文件中发现文件`friend.json`,即为对应格式文件,下面正常hexo三件套即可放置到网站根目录。
|
||||
|
||||
3. (可选)添加运行命令到脚本中方便执行,在根目录下创建:
|
||||
|
||||
```bash
|
||||
@echo off
|
||||
E:
|
||||
cd E:\Programming\HTML_Language\willow-God\blog
|
||||
node link.js && hexo g && hexo algolia && hexo d
|
||||
```
|
||||
|
||||
地址改成自己的,上传时仅需双击即可完成。
|
||||
|
||||
如果是github action,可以在hexo g脚本前添加即可完整构建,注意需要安装yaml包才可解析yml文件。
|
||||
|
||||
## 部署静态网站
|
||||
|
||||
首先,将该项目部署到vercel,部署到vercel等平台的目的主要是检测仓库变动并实时更新数据,及时获取all.json文件内容。任意平台均可,但是注意,部署的分支为page分支。
|
||||
|
||||
1. vercel 部署完成后,检查对应页面,如果页面中没有数据,且 `/all.json` 路径无法访问可能是部署到main分支了,可以通过 `setting-git-Production Branch` ,填写为page并重新进行部署即可
|
||||
|
||||

|
||||
|
||||
2. zeabur 可以在部署时直接选择分支:
|
||||
|
||||

|
||||
|
||||
3. CloudFlare Page 也可以在构建时即选择对应的分支,这里不再细讲。
|
||||
|
||||

|
||||
|
||||
部署完成后,你将获得一个地址,如果是通过vercel部署的,建议自行绑定域名。
|
||||
|
||||
检查 `https://example.com/all.json` 是否有数据,如果有,则部署成功。
|
||||
|
||||
## 部署到你的页面
|
||||
|
||||
在前端页面的md文件中写入:
|
||||
|
||||
@ -33,3 +353,85 @@
|
||||
```
|
||||
|
||||
其中第一个地址填入你自己的地址即可,**注意**尾部带`/`,不要遗漏。
|
||||
|
||||
然后你就可以在前端页面看到我们的结果了。效果图如上展示网站,其中两个文件你可以自行修改,在同目录下我也提供了未压缩版本,有基础的可以很便捷的进行修改。
|
||||
|
||||
## 自部署使用方法
|
||||
|
||||
如果你有一台境内服务器,你也可以通过以下操作将其部署到你的服务器上,操作如下:
|
||||
|
||||
### 前置工作
|
||||
|
||||
确保你的服务器有定时任务 `crontab` 功能包,一般是linux自带,如果你没有宝塔等可以管理定时任务的面板工具,可能需要你自行了解定时工具并导入,本教程提供了简单的介绍。
|
||||
|
||||
首先克隆仓库并进入对应路径:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/willow-god/Friend-Circle-Lite.git
|
||||
cd Friend-Circle-Lite
|
||||
```
|
||||
|
||||
由于不存在issue,所以不支持邮箱推送(主要是懒得分类写了,要不然还得从secret中获取密码的功能剥离QAQ),请将除第一部分抓取以外的功能均设置为false
|
||||
|
||||
下载服务相关包,其中 `requirements-server.txt` 是部署API服务所用包, `requirements.txt` 是抓取服务所用包,请均下载一遍。
|
||||
|
||||
```bash
|
||||
pip install -r ./requirements.txt
|
||||
pip install -r ./server/requirements-server.txt
|
||||
```
|
||||
|
||||
### 部署API服务
|
||||
|
||||
如果环境配置完毕,你可以进入目录路径后直接运行`deploy.sh`脚本启动API服务:
|
||||
|
||||
```bash
|
||||
chmod +x ./deploy.sh
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
其中的注释应该是较为详细的,如果部署成功你可以使用以下命令进行测试,如果获取到了首页html内容则成功:
|
||||
|
||||
```bash
|
||||
curl 127.0.0.1:1223
|
||||
```
|
||||
|
||||
这个端口号可以修改,在server.py最后一行修改数字即可,如果你想删除该API服务,可以使用ps找到对应进程并使用Kill命令杀死进程:
|
||||
|
||||
```bash
|
||||
ps aux | grep python
|
||||
kill -9 [这里填写上面查询结果中对应的进程号]
|
||||
```
|
||||
|
||||
### 合并github数据
|
||||
|
||||
你是不是以为github数据没用了?并不是!因为有很多站长是使用的GitHub page等服务部署的,这种服务可能无法被你的服务器抓取,此时你就需要合并两个的爬取数据。修改第一个配置中的以下部分:
|
||||
|
||||
```yaml
|
||||
merge_result:
|
||||
enable: true
|
||||
merge_json_url: "https://fc.liushen.fun"
|
||||
```
|
||||
其中地址项不要添加最后的斜杠,这样就会在本地爬取结束后合并远程的数据,以做到更高的准确率!
|
||||
|
||||
### 定时抓取文章
|
||||
|
||||
由于原生的crontab可能较为复杂,如果有兴趣可以查看./deploy.sh文件中,屏蔽掉的部分,这里我不会细讲,这里我主要讲解宝塔面板添加定时任务,这样可以最大程度减少内存占用,其他面板服务类似:
|
||||
|
||||

|
||||
|
||||
点击宝塔右侧的定时任务后,点击添加,按照上图配置,并在命令中输入:
|
||||
|
||||
```bash
|
||||
cd /www/wwwroot/Friend-Circle-Lite
|
||||
python3 run.py
|
||||
```
|
||||
|
||||
具体地址可以按照自己的需要进行修改,这样我们就可以做到定时修改文件内容了!然后请求api就是从本地文件中返回所有内容的过程,和爬取是分开的,所以并不影响!
|
||||
|
||||
## 问题与贡献
|
||||
|
||||
如果遇到任何问题或有建议,请[提交一个 issue](https://github.com/willow-god/Friend-Circle-Lite/issues)。欢迎贡献代码!
|
||||
|
||||
## Star增长曲线
|
||||
|
||||
[](https://star-history.com/#willow-god/Friend-Circle-Lite&Timeline)
|
6
requirements.txt
Normal file
@ -0,0 +1,6 @@
|
||||
datetime
|
||||
python-dateutil==2.9.0.post0
|
||||
requests
|
||||
feedparser==6.0.11
|
||||
PyYAML==6.0.1
|
||||
jinja2==3.1.2
|
0
rss_subscribe/__init__.py
Normal file
135
rss_subscribe/email_template.html
Normal file
@ -0,0 +1,135 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>最新文章通知</title>
|
||||
<style>
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
background-color: #f4f4f4;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
.container {
|
||||
background-color: #ffffff;
|
||||
margin: 50px auto;
|
||||
padding: 40px;
|
||||
border-radius: 10px;
|
||||
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
|
||||
width: 80%;
|
||||
max-width: 600px;
|
||||
}
|
||||
.header {
|
||||
margin-top: 30px;
|
||||
text-align: center;
|
||||
padding-bottom: 20px;
|
||||
}
|
||||
.header h1 {
|
||||
margin: 0;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
.content {
|
||||
font-size: 16px;
|
||||
line-height: 1.6;
|
||||
}
|
||||
.content p {
|
||||
margin: 10px 0;
|
||||
}
|
||||
|
||||
.content .title {
|
||||
display: inline-block;
|
||||
max-width: 100%;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
.content p strong {
|
||||
display: inline-block;
|
||||
max-width: 100px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
vertical-align: middle;
|
||||
}
|
||||
.content .summary {
|
||||
display: -webkit-box;
|
||||
-webkit-box-orient: vertical;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
word-wrap: break-word;
|
||||
word-break: break-all;
|
||||
}
|
||||
.content .published {
|
||||
display: inline-block;
|
||||
max-width: 100%;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
vertical-align: middle;
|
||||
}
|
||||
.button {
|
||||
display: block;
|
||||
width: 200px;
|
||||
max-width: 100%;
|
||||
margin: 20px auto;
|
||||
padding: 10px 20px;
|
||||
text-align: center;
|
||||
background-color: #007bff;
|
||||
color: #ffffff;
|
||||
text-decoration: none;
|
||||
border-radius: 5px;
|
||||
}
|
||||
.button:hover {
|
||||
background-color: #0056b3;
|
||||
}
|
||||
@media (max-width: 300px) {
|
||||
.button {
|
||||
width: auto;
|
||||
}
|
||||
}
|
||||
.footer {
|
||||
text-align: center;
|
||||
margin-top: 20px;
|
||||
font-size: 18px;
|
||||
color: #777777;
|
||||
}
|
||||
.unsubscribe {
|
||||
text-align: center;
|
||||
margin-top: 60px;
|
||||
font-size: 12px;
|
||||
color: #777777;
|
||||
}
|
||||
.unsubscribe a {
|
||||
color: #777777;
|
||||
text-decoration: none;
|
||||
}
|
||||
.unsubscribe a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h1>{{ website_title }}の最新文章</h1>
|
||||
</div>
|
||||
<div class="content">
|
||||
<p><strong>文章标题:</strong> <span class="title">{{ title }}</span></p>
|
||||
<p><strong>文章内容:</strong> <span class="summary">{{ summary }}</span></p>
|
||||
<p><strong>发布时间:</strong> <span class="published">{{ published }}</span></p>
|
||||
</div>
|
||||
<a href="{{ link }}" class="button">阅读更多</a>
|
||||
<div class="footer">
|
||||
<p>感谢您的订阅!</p>
|
||||
</div>
|
||||
<div class="unsubscribe">
|
||||
<p><a href="{{ github_issue_url }}">取消订阅</a></p>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
105
rss_subscribe/push_article_update.py
Normal file
@ -0,0 +1,105 @@
|
||||
import logging
|
||||
import requests
|
||||
import re
|
||||
from friend_circle_lite.get_info import check_feed, parse_feed
|
||||
import json
|
||||
import os
|
||||
|
||||
# 标准化的请求头
|
||||
HEADERS_JSON = {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/123.0.0.0 Safari/537.36 "
|
||||
"(Friend-Circle-Lite/1.0; +https://github.com/willow-god/Friend-Circle-Lite)"
|
||||
),
|
||||
"X-Friend-Circle": "1.0"
|
||||
}
|
||||
|
||||
|
||||
def extract_emails_from_issues(api_url):
|
||||
"""
|
||||
从GitHub issues API中提取以[e-mail]开头的title中的邮箱地址。
|
||||
|
||||
参数:
|
||||
api_url (str): GitHub issues API的URL。
|
||||
|
||||
返回:
|
||||
dict: 包含所有提取的邮箱地址的字典。
|
||||
{
|
||||
"emails": [
|
||||
"3162475700@qq.com"
|
||||
]
|
||||
}
|
||||
"""
|
||||
try:
|
||||
response = requests.get(api_url, headers=HEADERS_JSON, timeout=10)
|
||||
response.raise_for_status()
|
||||
issues = response.json()
|
||||
except Exception as e:
|
||||
logging.error(f"无法获取 GitHub issues 数据,错误信息: {e}")
|
||||
return None
|
||||
|
||||
email_pattern = re.compile(r'^\[邮箱订阅\](.+)$')
|
||||
emails = []
|
||||
|
||||
for issue in issues:
|
||||
title = issue.get("title", "")
|
||||
match = email_pattern.match(title)
|
||||
if match:
|
||||
email = match.group(1).strip()
|
||||
emails.append(email)
|
||||
|
||||
return {"emails": emails}
|
||||
|
||||
def get_latest_articles_from_link(url, count=5, last_articles_path="./rss_subscribe/last_articles.json"):
|
||||
"""
|
||||
从指定链接获取最新的文章数据并与本地存储的上次的文章数据进行对比。
|
||||
|
||||
参数:
|
||||
url (str): 用于获取文章数据的链接。
|
||||
count (int): 获取文章数的最大数。如果小于则全部获取,如果文章数大于则只取前 count 篇文章。
|
||||
|
||||
返回:
|
||||
list: 更新的文章列表,如果没有更新的文章则返回 None。
|
||||
"""
|
||||
# 本地存储上次文章数据的文件
|
||||
local_file = last_articles_path
|
||||
|
||||
# 检查和解析 feed
|
||||
session = requests.Session()
|
||||
feed_type, feed_url = check_feed(url, session)
|
||||
if feed_type == 'none':
|
||||
logging.error(f"无法获取 {url} 的文章数据")
|
||||
return None
|
||||
|
||||
# 获取最新的文章数据
|
||||
latest_data = parse_feed(feed_url, session ,count)
|
||||
latest_articles = latest_data['articles']
|
||||
|
||||
# 读取本地存储的上次的文章数据
|
||||
if os.path.exists(local_file):
|
||||
with open(local_file, 'r', encoding='utf-8') as file:
|
||||
last_data = json.load(file)
|
||||
else:
|
||||
last_data = {'articles': []}
|
||||
|
||||
last_articles = last_data['articles']
|
||||
|
||||
# 找到更新的文章
|
||||
updated_articles = []
|
||||
last_titles = {article['link'] for article in last_articles}
|
||||
|
||||
for article in latest_articles:
|
||||
if article['link'] not in last_titles:
|
||||
updated_articles.append(article)
|
||||
|
||||
logging.info(f"从 {url} 获取到 {len(latest_articles)} 篇文章,其中 {len(updated_articles)} 篇为新文章")
|
||||
|
||||
# 更新本地存储的文章数据
|
||||
with open(local_file, 'w', encoding='utf-8') as file:
|
||||
json.dump({'articles': latest_articles}, file, ensure_ascii=False, indent=4)
|
||||
|
||||
# 如果有更新的文章,返回这些文章,否则返回 None
|
||||
return updated_articles if updated_articles else None
|
||||
|
163
run.py
Normal file
@ -0,0 +1,163 @@
|
||||
import logging
|
||||
import json
|
||||
import sys
|
||||
import os
|
||||
|
||||
from friend_circle_lite.get_info import (
|
||||
fetch_and_process_data,
|
||||
marge_data_from_json_url,
|
||||
marge_errors_from_json_url,
|
||||
deal_with_large_data
|
||||
)
|
||||
from friend_circle_lite.get_conf import load_config
|
||||
from rss_subscribe.push_article_update import (
|
||||
get_latest_articles_from_link,
|
||||
extract_emails_from_issues
|
||||
)
|
||||
from push_rss_update.send_email import send_emails
|
||||
|
||||
# ========== 日志设置 ==========
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='😋 %(levelname)s: %(message)s'
|
||||
)
|
||||
|
||||
# ========== 加载配置 ==========
|
||||
config = load_config("./conf.yaml")
|
||||
|
||||
# ========== 爬虫模块 ==========
|
||||
if config["spider_settings"]["enable"]:
|
||||
logging.info("✅ 爬虫已启用")
|
||||
|
||||
json_url = config['spider_settings']['json_url']
|
||||
article_count = config['spider_settings']['article_count']
|
||||
specific_rss = config['specific_RSS']
|
||||
|
||||
logging.info(f"📥 正在从 {json_url} 获取数据,每个博客获取 {article_count} 篇文章")
|
||||
result, lost_friends = fetch_and_process_data(
|
||||
json_url=json_url,
|
||||
specific_RSS=specific_rss,
|
||||
count=article_count
|
||||
) # type: ignore
|
||||
|
||||
if config["spider_settings"]["merge_result"]["enable"]:
|
||||
merge_url = config['spider_settings']["merge_result"]['merge_json_url']
|
||||
logging.info(f"🔀 合并功能开启,从 {merge_url} 获取外部数据")
|
||||
|
||||
result = marge_data_from_json_url(result, f"{merge_url}/all.json")
|
||||
lost_friends = marge_errors_from_json_url(lost_friends, f"{merge_url}/errors.json")
|
||||
|
||||
article_count = len(result.get("article_data", []))
|
||||
logging.info(f"📦 数据获取完毕,共有 {article_count} 位好友的动态,正在处理数据")
|
||||
|
||||
result = deal_with_large_data(result)
|
||||
|
||||
with open("all.json", "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False, indent=2)
|
||||
|
||||
with open("errors.json", "w", encoding="utf-8") as f:
|
||||
json.dump(lost_friends, f, ensure_ascii=False, indent=2)
|
||||
|
||||
# ========== 推送准备 ==========
|
||||
if config["email_push"]["enable"] or config["rss_subscribe"]["enable"]:
|
||||
logging.info("📨 推送功能已启用,正在准备中...")
|
||||
|
||||
smtp_conf = config["smtp"]
|
||||
sender_email = smtp_conf["email"]
|
||||
server = smtp_conf["server"]
|
||||
port = smtp_conf["port"]
|
||||
use_tls = smtp_conf["use_tls"]
|
||||
password = os.getenv("SMTP_PWD")
|
||||
|
||||
logging.info(f"📡 SMTP 服务器:{server}:{port}")
|
||||
if not password:
|
||||
logging.error("❌ 环境变量 SMTP_PWD 未设置,无法发送邮件")
|
||||
sys.exit(1)
|
||||
else:
|
||||
logging.info(f"🔐 密码(部分):{password[:3]}*****")
|
||||
|
||||
# ========== 邮件推送(待实现)==========
|
||||
if config["email_push"]["enable"]:
|
||||
logging.info("📧 邮件推送已启用")
|
||||
logging.info("⚠️ 抱歉,目前尚未实现邮件推送功能")
|
||||
|
||||
# ========== RSS 订阅推送 ==========
|
||||
if config["rss_subscribe"]["enable"]:
|
||||
logging.info("📰 RSS 订阅推送已启用")
|
||||
|
||||
smtp_conf = config["smtp"]
|
||||
sender_email = smtp_conf["email"]
|
||||
server = smtp_conf["server"]
|
||||
port = smtp_conf["port"]
|
||||
use_tls = smtp_conf["use_tls"]
|
||||
password = os.getenv("SMTP_PWD")
|
||||
|
||||
# 获取 GitHub 仓库信息
|
||||
fcl_repo = os.getenv('FCL_REPO')
|
||||
if fcl_repo:
|
||||
github_username, github_repo = fcl_repo.split('/')
|
||||
else:
|
||||
github_username = str(config["rss_subscribe"]["github_username"]).strip()
|
||||
github_repo = str(config["rss_subscribe"]["github_repo"]).strip()
|
||||
|
||||
logging.info(f"👤 GitHub 用户名:{github_username}")
|
||||
logging.info(f"📁 GitHub 仓库:{github_repo}")
|
||||
|
||||
your_blog_url = config["rss_subscribe"]["your_blog_url"]
|
||||
email_template = config["rss_subscribe"]["email_template"]
|
||||
website_title = config["rss_subscribe"]["website_info"]["title"]
|
||||
|
||||
latest_articles = get_latest_articles_from_link(
|
||||
url=your_blog_url,
|
||||
count=5,
|
||||
last_articles_path="./rss_subscribe/last_articles.json"
|
||||
)
|
||||
|
||||
if not latest_articles:
|
||||
logging.info("📭 无新文章,无需推送")
|
||||
else:
|
||||
logging.info(f"🆕 获取到的最新文章:{latest_articles}")
|
||||
|
||||
github_api_url = (
|
||||
f"https://api.github.com/repos/{github_username}/{github_repo}/issues"
|
||||
f"?state=closed&label=subscribed&per_page=200"
|
||||
)
|
||||
logging.info(f"🔎 正在从 GitHub 获取订阅邮箱:{github_api_url}")
|
||||
email_list = extract_emails_from_issues(github_api_url)
|
||||
|
||||
if not email_list:
|
||||
logging.info("⚠️ 无订阅邮箱,请检查格式或是否有订阅者")
|
||||
sys.exit(0)
|
||||
|
||||
logging.info(f"📬 获取到邮箱列表:{email_list}")
|
||||
|
||||
for article in latest_articles:
|
||||
template_data = {
|
||||
"title": article["title"],
|
||||
"summary": article["summary"],
|
||||
"published": article["published"],
|
||||
"link": article["link"],
|
||||
"website_title": website_title,
|
||||
"github_issue_url": (
|
||||
f"https://github.com/{github_username}/{github_repo}"
|
||||
"/issues?q=is%3Aissue+is%3Aclosed"
|
||||
),
|
||||
}
|
||||
|
||||
send_emails(
|
||||
emails=email_list["emails"],
|
||||
sender_email=sender_email,
|
||||
smtp_server=server,
|
||||
port=port,
|
||||
password=password,
|
||||
subject=f"{website_title} の最新文章:{article['title']}",
|
||||
body=(
|
||||
f"📄 文章标题:{article['title']}\n"
|
||||
f"🔗 链接:{article['link']}\n"
|
||||
f"📝 简介:{article['summary']}\n"
|
||||
f"🕒 发布时间:{article['published']}"
|
||||
),
|
||||
template_path=email_template,
|
||||
template_data=template_data,
|
||||
use_tls=use_tls
|
||||
)
|
95
server.py
Normal file
@ -0,0 +1,95 @@
|
||||
from fastapi import FastAPI
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
|
||||
from starlette.middleware.cors import CORSMiddleware
|
||||
import json
|
||||
import random
|
||||
|
||||
from friend_circle_lite.get_info import fetch_and_process_data, sort_articles_by_time
|
||||
from friend_circle_lite.get_conf import load_config
|
||||
|
||||
app = FastAPI()
|
||||
|
||||
# 设置静态文件目录
|
||||
app.mount("/static", StaticFiles(directory="static"), name="static")
|
||||
app.mount("/main", StaticFiles(directory="main"), name="main")
|
||||
|
||||
# 添加 CORS 中间件
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
# 返回图标图片
|
||||
@app.get("/favicon.ico", response_class=HTMLResponse)
|
||||
async def favicon():
|
||||
return FileResponse('static/favicon.ico')
|
||||
|
||||
# 返回背景图片
|
||||
@app.get("/bg-light.webp", response_class=HTMLResponse)
|
||||
async def bg_light():
|
||||
return FileResponse('static/bg-light.webp')
|
||||
|
||||
# 返回背景图片
|
||||
@app.get("/bg-dark.webp", response_class=HTMLResponse)
|
||||
async def bg_dark():
|
||||
return FileResponse('static/bg-dark.webp')
|
||||
|
||||
# 返回资源文件
|
||||
# 返回 CSS 文件
|
||||
@app.get("/fclite.css", response_class=HTMLResponse)
|
||||
async def get_fclite_css():
|
||||
return FileResponse('./main/fclite.css')
|
||||
|
||||
# 返回 JS 文件
|
||||
@app.get("/fclite.js", response_class=HTMLResponse)
|
||||
async def get_fclite_js():
|
||||
return FileResponse('./main/fclite.js')
|
||||
|
||||
@app.get("/", response_class=HTMLResponse)
|
||||
async def root():
|
||||
return FileResponse('./static/index.html')
|
||||
|
||||
@app.get('/all.json')
|
||||
async def get_all_articles():
|
||||
try:
|
||||
with open('./all.json', 'r', encoding='utf-8') as f:
|
||||
articles_data = json.load(f)
|
||||
return JSONResponse(content=articles_data)
|
||||
except FileNotFoundError:
|
||||
return JSONResponse(content={"error": "File not found"}, status_code=404)
|
||||
except json.JSONDecodeError:
|
||||
return JSONResponse(content={"error": "Failed to decode JSON"}, status_code=500)
|
||||
|
||||
@app.get('/errors.json')
|
||||
async def get_error_friends():
|
||||
try:
|
||||
with open('./errors.json', 'r', encoding='utf-8') as f:
|
||||
errors_data = json.load(f)
|
||||
return JSONResponse(content=errors_data)
|
||||
except FileNotFoundError:
|
||||
return JSONResponse(content={"error": "File not found"}, status_code=404)
|
||||
except json.JSONDecodeError:
|
||||
return JSONResponse(content={"error": "Failed to decode JSON"}, status_code=500)
|
||||
|
||||
@app.get('/random')
|
||||
async def get_random_article():
|
||||
try:
|
||||
with open('./all.json', 'r', encoding='utf-8') as f:
|
||||
articles_data = json.load(f)
|
||||
if articles_data.get("article_data"):
|
||||
random_article = random.choice(articles_data["article_data"])
|
||||
return JSONResponse(content=random_article)
|
||||
else:
|
||||
return JSONResponse(content={"error": "No articles available"}, status_code=404)
|
||||
except FileNotFoundError:
|
||||
return JSONResponse(content={"error": "File not found"}, status_code=404)
|
||||
except json.JSONDecodeError:
|
||||
return JSONResponse(content={"error": "Failed to decode JSON"}, status_code=500)
|
||||
|
||||
if __name__ == '__main__':
|
||||
# 启动FastAPI应用
|
||||
import uvicorn
|
||||
uvicorn.run(app, host='0.0.0.0', port=1223)
|
65
server/deploy-home.html
Normal file
@ -0,0 +1,65 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<meta name="description" content="Friend Circle Lite - 服务已运行">
|
||||
<title>轻量友链朋友圈</title>
|
||||
<link rel="icon" href="https://i.p-i.vip/30/20240803-66adf2c2e4931.webp" type="image/x-icon">
|
||||
<style>
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
height: 100vh;
|
||||
margin: 0;
|
||||
background-color: white;
|
||||
}
|
||||
.container {
|
||||
text-align: center;
|
||||
}
|
||||
.avatar {
|
||||
width: 150px;
|
||||
height: 150px;
|
||||
border-radius: 50%;
|
||||
}
|
||||
p {
|
||||
color: #666;
|
||||
margin-bottom: 30px;
|
||||
}
|
||||
.button-container {
|
||||
margin-top: 20px;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
gap: 10px;
|
||||
}
|
||||
.button {
|
||||
background-color: white;
|
||||
border: 2px solid #007BFF;
|
||||
color: #007BFF;
|
||||
padding: 10px 20px;
|
||||
border-radius: 25px;
|
||||
text-decoration: none;
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
display: inline-block;
|
||||
transition: background-color 0.3s, color 0.3s;
|
||||
}
|
||||
.button:hover {
|
||||
background-color: #007BFF;
|
||||
color: white;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<img src="https://i.p-i.vip/30/20240803-66adf2c2e4931.webp" alt="Avatar" class="avatar">
|
||||
<p>Friend-Circle-Lite<br>服务已运行</p>
|
||||
<div class="button-container">
|
||||
<a href="https://blog.liushen.fun/posts/4dc716ec/" class="button">查看文档</a>
|
||||
<a href="/all" class="button">测试接口</a>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
2
server/requirements-server.txt
Normal file
@ -0,0 +1,2 @@
|
||||
fastapi
|
||||
uvicorn
|
BIN
static/1.png
Normal file
After Width: | Height: | Size: 36 KiB |
BIN
static/baota.png
Normal file
After Width: | Height: | Size: 108 KiB |
Before Width: | Height: | Size: 280 KiB After Width: | Height: | Size: 280 KiB |
Before Width: | Height: | Size: 105 KiB After Width: | Height: | Size: 105 KiB |
BIN
static/cloudflare.png
Normal file
After Width: | Height: | Size: 68 KiB |
Before Width: | Height: | Size: 202 KiB After Width: | Height: | Size: 202 KiB |
BIN
static/fork.png
Normal file
After Width: | Height: | Size: 139 KiB |
35
static/readme.md
Normal file
@ -0,0 +1,35 @@
|
||||
# RSS 订阅前端页面
|
||||
|
||||
这是一个用于展示 RSS 订阅内容的简单 HTML 页面。该前端页面用于渲染从后端获取的 RSS 订阅数据。本分支仅包含用于展示的静态资源(HTML、CSS、JS)。
|
||||
|
||||
## 功能
|
||||
|
||||
- **展示 RSS 订阅内容**:可以显示 RSS 订阅文章的标题、描述和发布时间。
|
||||
- **简洁设计**:简单直观的用户界面,适用于浏览和查看 RSS 内容。
|
||||
- **响应式布局**:适配不同设备的浏览体验。
|
||||
|
||||
## 部署到网站
|
||||
|
||||
如果你已经正确托管本分支到静态托管平台,你可以通过以下几个步骤将数据渲染到你的前端页面:
|
||||
|
||||
在前端页面的md文件中写入:
|
||||
|
||||
```html
|
||||
<div id="friend-circle-lite-root"></div>
|
||||
<script>
|
||||
if (typeof UserConfig === 'undefined') {
|
||||
var UserConfig = {
|
||||
// 填写你的fc Lite地址
|
||||
private_api_url: 'https://fc.liushen.fun/',
|
||||
// 点击加载更多时,一次最多加载几篇文章,默认20
|
||||
page_turning_number: 20,
|
||||
// 头像加载失败时,默认头像地址
|
||||
error_img: 'https://i.p-i.vip/30/20240815-66bced9226a36.webp',
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="https://fastly.jsdelivr.net/gh/willow-god/Friend-Circle-Lite/main/fclite.min.css">
|
||||
<script src="https://fastly.jsdelivr.net/gh/willow-god/Friend-Circle-Lite/main/fclite.min.js"></script>
|
||||
```
|
||||
|
||||
其中第一个地址填入你自己的地址即可,**注意**尾部带`/`,不要遗漏。
|
BIN
static/vercel.png
Normal file
After Width: | Height: | Size: 373 KiB |
BIN
static/zeabur.png
Normal file
After Width: | Height: | Size: 264 KiB |
9
vercel.json
Normal file
@ -0,0 +1,9 @@
|
||||
{
|
||||
"git": {
|
||||
"branch": "page",
|
||||
"deploymentEnabled": {
|
||||
"main": false
|
||||
}
|
||||
}
|
||||
}
|
||||
|