35 Commits

Author SHA1 Message Date
starry
61f09192bb Update README.md 2025-06-27 09:06:44 +08:00
starry
d876809086 完善一些小细节 2025-06-27 08:50:04 +08:00
user123456
fe9156f878 Merge commit 'refs/pull/origin/28' 2025-06-21 00:30:51 +08:00
starry
35651e214f proxy字段修复 2025-06-21 00:15:27 +08:00
user123456
d373e0104d 获取更多镜像tag 2025-06-20 23:44:13 +08:00
starry
207a03a511 Merge pull request #25 from beck-8/me/op_proxy
优化代理配置
2025-06-19 23:00:44 +08:00
beck-8
5bd32cd6c1 go fmt . 2025-06-19 22:53:20 +08:00
beck-8
8c127a795b op http client proxy 2025-06-19 22:52:51 +08:00
user123456
2567652a7d 更新配置说明 2025-06-18 22:26:19 +08:00
user123456
c023e6a9c4 清理冗余written字段 2025-06-18 22:05:28 +08:00
user123456
44c6e4cd7b 修复双重写入 2025-06-18 21:29:56 +08:00
user123456
c22bd0637a 更新默认配置 2025-06-18 20:49:45 +08:00
user123456
a94b476726 移除冗余的限流智能判断逻辑 2025-06-18 20:44:26 +08:00
user123456
4c6751b862 限流改为全局应用 2025-06-18 19:44:32 +08:00
user123456
acc63d7b68 删除热重载 2025-06-18 19:14:13 +08:00
starry
d0b1ea8582 LF 2025-06-18 17:08:14 +08:00
starry
c607061dae LF 2025-06-18 17:07:43 +08:00
starry
143de7b254 Normalize all line endings to LF 2025-06-18 17:03:29 +08:00
user123456
51ace73b78 优化离线镜像的防抖以及日志 2025-06-18 16:04:53 +08:00
user123456
fa9e9210ab 默认为原始压缩层 2025-06-18 15:14:33 +08:00
user123456
f308410920 修复函数调用点传递 2025-06-18 15:00:41 +08:00
user123456
252dc319c6 优化离线包体积 2025-06-18 14:55:35 +08:00
user123456
29ceeef45b IPv6日志适配 2025-06-17 18:49:34 +08:00
user123456
182dced403 修复ipv6标准化的潜在BUG 2025-06-17 18:38:48 +08:00
user123456
aea36939a3 增加支持走代理 2025-06-17 18:18:17 +08:00
starry
4240c1452a Update README.md 2025-06-16 00:51:06 +08:00
starry
212c8e529d Update README.md 2025-06-15 16:18:54 +08:00
starry
3fd630159b Update config.toml 2025-06-14 14:11:14 +08:00
starry
17d827f50b Update README.md 2025-06-14 14:10:31 +08:00
starry
7dcbc839c6 Update README.md 2025-06-14 14:10:07 +08:00
starry
45ffebc820 Update README.md 2025-06-14 14:08:08 +08:00
starry
3027b1f218 Update README.md 2025-06-13 18:31:13 +08:00
starry
3d2c419ebe Update README.md 2025-06-13 18:30:14 +08:00
starry
b529fbfdd2 Update README.md 2025-06-13 18:29:38 +08:00
user123456
737c1dbf46 io.Copy 2025-06-13 17:58:13 +08:00
20 changed files with 2990 additions and 2701 deletions

1
.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
* text=auto eol=lf

4
.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
.idea
.vscode
.DS_Store
hubproxy*

136
README.md
View File

@@ -2,7 +2,7 @@
🚀 **Docker 和 GitHub 加速代理服务器** 🚀 **Docker 和 GitHub 加速代理服务器**
一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速等功能。 一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速、下载离线镜像、在线搜索 Docker 镜像等功能。
## ✨ 特性 ## ✨ 特性
@@ -14,7 +14,8 @@
- 🚫 **仓库审计** - 强大的自定义黑名单白名单同时审计镜像仓库和GitHub仓库 - 🚫 **仓库审计** - 强大的自定义黑名单白名单同时审计镜像仓库和GitHub仓库
- 🔍 **镜像搜索** - 在线搜索 Docker 镜像 - 🔍 **镜像搜索** - 在线搜索 Docker 镜像
-**轻量高效** - 基于 Go 语言,单二进制文件运行,资源占用低,优雅的内存清理机制。 -**轻量高效** - 基于 Go 语言,单二进制文件运行,资源占用低,优雅的内存清理机制。
- 🔧 **配置热重载** - 统一配置管理,部分配置项支持热重载,无需重启服务 - 🔧 **统一配置** - 统一配置管理
## 🚀 快速开始 ## 🚀 快速开始
@@ -29,13 +30,13 @@ docker run -d \
### 一键安装 ### 一键脚本安装
```bash ```bash
curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-service.sh | sudo bash curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-service.sh | sudo bash
``` ```
直接下载二进制文件执行`./hubproxy`使用无需配置文件即可启动内置默认配置支持所有功能。初始内存占用约18M二进制文件大小约12M 也可以直接下载二进制文件执行`./hubproxy`使用无需配置文件即可启动内置默认配置支持所有功能。初始内存占用约18M二进制文件大小约12M
这个命令会: 这个命令会:
- 🔍 自动检测系统架构AMD64/ARM64 - 🔍 自动检测系统架构AMD64/ARM64
@@ -54,10 +55,10 @@ curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-serv
docker pull nginx docker pull nginx
# 使用加速 # 使用加速
docker pull demo.52013120.xyz/nginx docker pull yourdomain.com/nginx
# ghcr加速 # ghcr加速
docker pull demo.52013120.xyz/ghcr.io/sky22333/hubproxy docker pull yourdomain.com/ghcr.io/sky22333/hubproxy
# 符合Docker Registry API v2标准的仓库都支持 # 符合Docker Registry API v2标准的仓库都支持
``` ```
@@ -70,16 +71,134 @@ https://github.com/user/repo/releases/download/v1.0.0/file.tar.gz
# 加速链接 # 加速链接
https://yourdomain.com/https://github.com/user/repo/releases/download/v1.0.0/file.tar.gz https://yourdomain.com/https://github.com/user/repo/releases/download/v1.0.0/file.tar.gz
# 加速下载仓库
git clone https://yourdomain.com/https://github.com/sky22333/hubproxy.git
``` ```
## ⚙️ 配置
<details>
<summary>config.toml配置说明</summary>
## ⚙️ 提示 此配置是默认配置
主配置文件位于 `/opt/hubproxy/config.toml` ```
[server]
host = "0.0.0.0"
# 监听端口
port = 5000
# Github文件大小限制字节默认2GB
fileSize = 2147483648
[rateLimit]
# 每个IP每小时允许的请求数(注意Docker镜像会有多个层会消耗多个次数)
requestLimit = 500
# 限流周期(小时)
periodHours = 1.0
[security]
# IP白名单支持单个IP或IP段
# 白名单中的IP不受限流限制
whiteList = [
"127.0.0.1",
"172.17.0.0/16",
"192.168.1.0/24"
]
# IP黑名单支持单个IP或IP段
# 黑名单中的IP将被直接拒绝访问
blackList = [
"192.168.100.1",
"192.168.100.0/24"
]
[proxy]
# 代理服务白名单支持GitHub仓库和Docker镜像支持通配符
# 只允许访问白名单中的仓库/镜像,为空时不限制
whiteList = []
# 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
# 禁止访问黑名单中的仓库/镜像
blackList = [
"baduser/malicious-repo",
"*/malicious-repo",
"baduser/*"
]
# 代理配置,支持有用户名/密码认证和无认证模式
# 无认证: socks5://127.0.0.1:1080
# 有认证: socks5://username:password@127.0.0.1:1080
# HTTP 代理示例
# http://username:password@127.0.0.1:7890
# SOCKS5 代理示例
# socks5://username:password@127.0.0.1:1080
# SOCKS5H 代理示例
# socks5h://username:password@127.0.0.1:1080
# 留空不使用代理
proxy = ""
[download]
# 批量下载离线镜像数量限制
maxImages = 10
# Registry映射配置支持多种镜像仓库上游
[registries]
# GitHub Container Registry
[registries."ghcr.io"]
upstream = "ghcr.io"
authHost = "ghcr.io/token"
authType = "github"
enabled = true
# Google Container Registry
[registries."gcr.io"]
upstream = "gcr.io"
authHost = "gcr.io/v2/token"
authType = "google"
enabled = true
# Quay.io Container Registry
[registries."quay.io"]
upstream = "quay.io"
authHost = "quay.io/v2/auth"
authType = "quay"
enabled = true
# Kubernetes Container Registry
[registries."registry.k8s.io"]
upstream = "registry.k8s.io"
authHost = "registry.k8s.io"
authType = "anonymous"
enabled = true
[tokenCache]
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
enabled = true
# 默认缓存时间(分钟)
defaultTTL = "20m"
```
</details>
容器内的配置文件位于 `/root/config.toml`
脚本部署配置文件位于 `/opt/hubproxy/config.toml`
为了IP限流能够正常运行反向代理需要传递IP头用来获取访客真实IP以caddy为例 为了IP限流能够正常运行反向代理需要传递IP头用来获取访客真实IP以caddy为例
``` ```
example.com {
reverse_proxy {
to 127.0.0.1:5000
header_up X-Real-IP {remote}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Proto {scheme}
}
}
```
cloudflare CDN
```
example.com { example.com {
reverse_proxy 127.0.0.1:5000 { reverse_proxy 127.0.0.1:5000 {
header_up X-Forwarded-For {http.request.header.CF-Connecting-IP} header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
@@ -91,6 +210,7 @@ example.com {
``` ```
## ⚠️ 免责声明 ## ⚠️ 免责声明
- 本程序仅供学习交流使用,请勿用于非法用途 - 本程序仅供学习交流使用,请勿用于非法用途

View File

@@ -1,220 +1,212 @@
package main package main
import ( import (
"strings" "strings"
"sync" "sync"
) )
// ResourceType 资源类型 // ResourceType 资源类型
type ResourceType string type ResourceType string
const ( const (
ResourceTypeGitHub ResourceType = "github" ResourceTypeGitHub ResourceType = "github"
ResourceTypeDocker ResourceType = "docker" ResourceTypeDocker ResourceType = "docker"
) )
// AccessController 统一访问控制器 // AccessController 统一访问控制器
type AccessController struct { type AccessController struct {
mu sync.RWMutex mu sync.RWMutex
} }
// DockerImageInfo Docker镜像信息 // DockerImageInfo Docker镜像信息
type DockerImageInfo struct { type DockerImageInfo struct {
Namespace string Namespace string
Repository string Repository string
Tag string Tag string
FullName string FullName string
} }
// 全局访问控制器实例 // 全局访问控制器实例
var GlobalAccessController = &AccessController{} var GlobalAccessController = &AccessController{}
// ParseDockerImage 解析Docker镜像名称 // ParseDockerImage 解析Docker镜像名称
func (ac *AccessController) ParseDockerImage(image string) DockerImageInfo { func (ac *AccessController) ParseDockerImage(image string) DockerImageInfo {
image = strings.TrimPrefix(image, "docker://") image = strings.TrimPrefix(image, "docker://")
var tag string var tag string
if idx := strings.LastIndex(image, ":"); idx != -1 { if idx := strings.LastIndex(image, ":"); idx != -1 {
part := image[idx+1:] part := image[idx+1:]
if !strings.Contains(part, "/") { if !strings.Contains(part, "/") {
tag = part tag = part
image = image[:idx] image = image[:idx]
} }
} }
if tag == "" { if tag == "" {
tag = "latest" tag = "latest"
} }
var namespace, repository string var namespace, repository string
if strings.Contains(image, "/") { if strings.Contains(image, "/") {
parts := strings.Split(image, "/") parts := strings.Split(image, "/")
if len(parts) >= 2 { if len(parts) >= 2 {
if strings.Contains(parts[0], ".") { if strings.Contains(parts[0], ".") {
if len(parts) >= 3 { if len(parts) >= 3 {
namespace = parts[1] namespace = parts[1]
repository = parts[2] repository = parts[2]
} else { } else {
namespace = "library" namespace = "library"
repository = parts[1] repository = parts[1]
} }
} else { } else {
namespace = parts[0] namespace = parts[0]
repository = parts[1] repository = parts[1]
} }
} }
} else { } else {
namespace = "library" namespace = "library"
repository = image repository = image
} }
fullName := namespace + "/" + repository fullName := namespace + "/" + repository
return DockerImageInfo{ return DockerImageInfo{
Namespace: namespace, Namespace: namespace,
Repository: repository, Repository: repository,
Tag: tag, Tag: tag,
FullName: fullName, FullName: fullName,
} }
} }
// CheckDockerAccess 检查Docker镜像访问权限 // CheckDockerAccess 检查Docker镜像访问权限
func (ac *AccessController) CheckDockerAccess(image string) (allowed bool, reason string) { func (ac *AccessController) CheckDockerAccess(image string) (allowed bool, reason string) {
cfg := GetConfig() cfg := GetConfig()
// 解析镜像名称 // 解析镜像名称
imageInfo := ac.ParseDockerImage(image) imageInfo := ac.ParseDockerImage(image)
// 检查白名单(如果配置了白名单,则只允许白名单中的镜像) // 检查白名单(如果配置了白名单,则只允许白名单中的镜像)
if len(cfg.Proxy.WhiteList) > 0 { if len(cfg.Access.WhiteList) > 0 {
if !ac.matchImageInList(imageInfo, cfg.Proxy.WhiteList) { if !ac.matchImageInList(imageInfo, cfg.Access.WhiteList) {
return false, "不在Docker镜像白名单内" return false, "不在Docker镜像白名单内"
} }
} }
// 检查黑名单 // 检查黑名单
if len(cfg.Proxy.BlackList) > 0 { if len(cfg.Access.BlackList) > 0 {
if ac.matchImageInList(imageInfo, cfg.Proxy.BlackList) { if ac.matchImageInList(imageInfo, cfg.Access.BlackList) {
return false, "Docker镜像在黑名单内" return false, "Docker镜像在黑名单内"
} }
} }
return true, "" return true, ""
} }
// CheckGitHubAccess 检查GitHub仓库访问权限 // CheckGitHubAccess 检查GitHub仓库访问权限
func (ac *AccessController) CheckGitHubAccess(matches []string) (allowed bool, reason string) { func (ac *AccessController) CheckGitHubAccess(matches []string) (allowed bool, reason string) {
if len(matches) < 2 { if len(matches) < 2 {
return false, "无效的GitHub仓库格式" return false, "无效的GitHub仓库格式"
} }
cfg := GetConfig() cfg := GetConfig()
// 检查白名单 // 检查白名单
if len(cfg.Proxy.WhiteList) > 0 && !ac.checkList(matches, cfg.Proxy.WhiteList) { if len(cfg.Access.WhiteList) > 0 && !ac.checkList(matches, cfg.Access.WhiteList) {
return false, "不在GitHub仓库白名单内" return false, "不在GitHub仓库白名单内"
} }
// 检查黑名单 // 检查黑名单
if len(cfg.Proxy.BlackList) > 0 && ac.checkList(matches, cfg.Proxy.BlackList) { if len(cfg.Access.BlackList) > 0 && ac.checkList(matches, cfg.Access.BlackList) {
return false, "GitHub仓库在黑名单内" return false, "GitHub仓库在黑名单内"
} }
return true, "" return true, ""
} }
// matchImageInList 检查Docker镜像是否在指定列表中 // matchImageInList 检查Docker镜像是否在指定列表中
func (ac *AccessController) matchImageInList(imageInfo DockerImageInfo, list []string) bool { func (ac *AccessController) matchImageInList(imageInfo DockerImageInfo, list []string) bool {
fullName := strings.ToLower(imageInfo.FullName) fullName := strings.ToLower(imageInfo.FullName)
namespace := strings.ToLower(imageInfo.Namespace) namespace := strings.ToLower(imageInfo.Namespace)
for _, item := range list { for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item)) item = strings.ToLower(strings.TrimSpace(item))
if item == "" { if item == "" {
continue continue
} }
if fullName == item { if fullName == item {
return true return true
} }
if item == namespace || item == namespace+"/*" { if item == namespace || item == namespace+"/*" {
return true return true
} }
if strings.HasSuffix(item, "*") { if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*") prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullName, prefix) { if strings.HasPrefix(fullName, prefix) {
return true return true
} }
} }
if strings.HasPrefix(item, "*/") { if strings.HasPrefix(item, "*/") {
repoPattern := strings.TrimPrefix(item, "*/") repoPattern := strings.TrimPrefix(item, "*/")
if strings.HasSuffix(repoPattern, "*") { if strings.HasSuffix(repoPattern, "*") {
repoPrefix := strings.TrimSuffix(repoPattern, "*") repoPrefix := strings.TrimSuffix(repoPattern, "*")
if strings.HasPrefix(imageInfo.Repository, repoPrefix) { if strings.HasPrefix(imageInfo.Repository, repoPrefix) {
return true return true
} }
} else { } else {
if strings.ToLower(imageInfo.Repository) == repoPattern { if strings.ToLower(imageInfo.Repository) == repoPattern {
return true return true
} }
} }
} }
if strings.HasPrefix(fullName, item+"/") { if strings.HasPrefix(fullName, item+"/") {
return true return true
} }
} }
return false return false
} }
// checkList GitHub仓库检查逻辑 // checkList GitHub仓库检查逻辑
func (ac *AccessController) checkList(matches, list []string) bool { func (ac *AccessController) checkList(matches, list []string) bool {
if len(matches) < 2 { if len(matches) < 2 {
return false return false
} }
username := strings.ToLower(strings.TrimSpace(matches[0])) username := strings.ToLower(strings.TrimSpace(matches[0]))
repoName := strings.ToLower(strings.TrimSpace(strings.TrimSuffix(matches[1], ".git"))) repoName := strings.ToLower(strings.TrimSpace(strings.TrimSuffix(matches[1], ".git")))
fullRepo := username + "/" + repoName fullRepo := username + "/" + repoName
for _, item := range list { for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item)) item = strings.ToLower(strings.TrimSpace(item))
if item == "" { if item == "" {
continue continue
} }
// 支持多种匹配模式 // 支持多种匹配模式
if fullRepo == item { if fullRepo == item {
return true return true
} }
// 用户级匹配 // 用户级匹配
if item == username || item == username+"/*" { if item == username || item == username+"/*" {
return true return true
} }
// 前缀匹配(支持通配符) // 前缀匹配(支持通配符)
if strings.HasSuffix(item, "*") { if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*") prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullRepo, prefix) { if strings.HasPrefix(fullRepo, prefix) {
return true return true
} }
} }
// 子仓库匹配(防止 user/repo 匹配到 user/repo-fork // 子仓库匹配(防止 user/repo 匹配到 user/repo-fork
if strings.HasPrefix(fullRepo, item+"/") { if strings.HasPrefix(fullRepo, item+"/") {
return true return true
} }
} }
return false return false
} }
// Reload 热重载访问控制规则
func (ac *AccessController) Reload() {
ac.mu.Lock()
defer ac.mu.Unlock()
// 访问控制器本身不缓存配置
}

View File

@@ -1,367 +1,276 @@
package main package main
import ( import (
"fmt" "fmt"
"os" "os"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
"time" "time"
"github.com/pelletier/go-toml/v2" "github.com/pelletier/go-toml/v2"
"github.com/spf13/viper" )
"github.com/fsnotify/fsnotify"
) // RegistryMapping Registry映射配置
type RegistryMapping struct {
// RegistryMapping Registry映射配置 Upstream string `toml:"upstream"` // 上游Registry地址
type RegistryMapping struct { AuthHost string `toml:"authHost"` // 认证服务器地址
Upstream string `toml:"upstream"` // 上游Registry地址 AuthType string `toml:"authType"` // 认证类型: docker/github/google/basic
AuthHost string `toml:"authHost"` // 认证服务器地址 Enabled bool `toml:"enabled"` // 是否启用
AuthType string `toml:"authType"` // 认证类型: docker/github/google/basic }
Enabled bool `toml:"enabled"` // 是否启用
} // AppConfig 应用配置结构体
type AppConfig struct {
// AppConfig 应用配置结构体 Server struct {
type AppConfig struct { Host string `toml:"host"` // 监听地址
Server struct { Port int `toml:"port"` // 监听端口
Host string `toml:"host"` // 监听地址 FileSize int64 `toml:"fileSize"` // 文件大小限制(字节)
Port int `toml:"port"` // 监听端口 } `toml:"server"`
FileSize int64 `toml:"fileSize"` // 文件大小限制(字节)
} `toml:"server"` RateLimit struct {
RequestLimit int `toml:"requestLimit"` // 每小时请求限制
RateLimit struct { PeriodHours float64 `toml:"periodHours"` // 限制周期(小时)
RequestLimit int `toml:"requestLimit"` // 每小时请求限制 } `toml:"rateLimit"`
PeriodHours float64 `toml:"periodHours"` // 限制周期(小时)
} `toml:"rateLimit"` Security struct {
WhiteList []string `toml:"whiteList"` // 白名单IP/CIDR列表
Security struct { BlackList []string `toml:"blackList"` // 黑名单IP/CIDR列表
WhiteList []string `toml:"whiteList"` // 白名单IP/CIDR列表 } `toml:"security"`
BlackList []string `toml:"blackList"` // 黑名单IP/CIDR列表
} `toml:"security"` Access struct {
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别)
Proxy struct { BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别)
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别) Proxy string `toml:"proxy"` // 代理地址: 支持 http/https/socks5/socks5h
BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别) } `toml:"access"`
} `toml:"proxy"`
Download struct {
Download struct { MaxImages int `toml:"maxImages"` // 单次下载最大镜像数量限制
MaxImages int `toml:"maxImages"` // 单次下载最大镜像数量限制 } `toml:"download"`
} `toml:"download"`
Registries map[string]RegistryMapping `toml:"registries"`
Registries map[string]RegistryMapping `toml:"registries"`
TokenCache struct {
TokenCache struct { Enabled bool `toml:"enabled"` // 是否启用token缓存
Enabled bool `toml:"enabled"` // 是否启用token缓存 DefaultTTL string `toml:"defaultTTL"` // 默认缓存时间
DefaultTTL string `toml:"defaultTTL"` // 默认缓存时间 } `toml:"tokenCache"`
} `toml:"tokenCache"` }
}
var (
var ( appConfig *AppConfig
appConfig *AppConfig appConfigLock sync.RWMutex
appConfigLock sync.RWMutex
isViperEnabled bool cachedConfig *AppConfig
viperInstance *viper.Viper configCacheTime time.Time
configCacheTTL = 5 * time.Second
cachedConfig *AppConfig configCacheMutex sync.RWMutex
configCacheTime time.Time )
configCacheTTL = 5 * time.Second
configCacheMutex sync.RWMutex // todo:Refactoring is needed
) // DefaultConfig 返回默认配置
func DefaultConfig() *AppConfig {
// DefaultConfig 返回默认配置 return &AppConfig{
func DefaultConfig() *AppConfig { Server: struct {
return &AppConfig{ Host string `toml:"host"`
Server: struct { Port int `toml:"port"`
Host string `toml:"host"` FileSize int64 `toml:"fileSize"`
Port int `toml:"port"` }{
FileSize int64 `toml:"fileSize"` Host: "0.0.0.0",
}{ Port: 5000,
Host: "0.0.0.0", FileSize: 2 * 1024 * 1024 * 1024, // 2GB
Port: 5000, },
FileSize: 2 * 1024 * 1024 * 1024, // 2GB RateLimit: struct {
}, RequestLimit int `toml:"requestLimit"`
RateLimit: struct { PeriodHours float64 `toml:"periodHours"`
RequestLimit int `toml:"requestLimit"` }{
PeriodHours float64 `toml:"periodHours"` RequestLimit: 20,
}{ PeriodHours: 1.0,
RequestLimit: 20, },
PeriodHours: 1.0, Security: struct {
}, WhiteList []string `toml:"whiteList"`
Security: struct { BlackList []string `toml:"blackList"`
WhiteList []string `toml:"whiteList"` }{
BlackList []string `toml:"blackList"` WhiteList: []string{},
}{ BlackList: []string{},
WhiteList: []string{}, },
BlackList: []string{}, Access: struct {
}, WhiteList []string `toml:"whiteList"`
Proxy: struct { BlackList []string `toml:"blackList"`
WhiteList []string `toml:"whiteList"` Proxy string `toml:"proxy"`
BlackList []string `toml:"blackList"` }{
}{ WhiteList: []string{},
WhiteList: []string{}, BlackList: []string{},
BlackList: []string{}, Proxy: "", // 默认不使用代理
}, },
Download: struct { Download: struct {
MaxImages int `toml:"maxImages"` MaxImages int `toml:"maxImages"`
}{ }{
MaxImages: 10, // 默认值最多同时下载10个镜像 MaxImages: 10, // 默认值最多同时下载10个镜像
}, },
Registries: map[string]RegistryMapping{ Registries: map[string]RegistryMapping{
"ghcr.io": { "ghcr.io": {
Upstream: "ghcr.io", Upstream: "ghcr.io",
AuthHost: "ghcr.io/token", AuthHost: "ghcr.io/token",
AuthType: "github", AuthType: "github",
Enabled: true, Enabled: true,
}, },
"gcr.io": { "gcr.io": {
Upstream: "gcr.io", Upstream: "gcr.io",
AuthHost: "gcr.io/v2/token", AuthHost: "gcr.io/v2/token",
AuthType: "google", AuthType: "google",
Enabled: true, Enabled: true,
}, },
"quay.io": { "quay.io": {
Upstream: "quay.io", Upstream: "quay.io",
AuthHost: "quay.io/v2/auth", AuthHost: "quay.io/v2/auth",
AuthType: "quay", AuthType: "quay",
Enabled: true, Enabled: true,
}, },
"registry.k8s.io": { "registry.k8s.io": {
Upstream: "registry.k8s.io", Upstream: "registry.k8s.io",
AuthHost: "registry.k8s.io", AuthHost: "registry.k8s.io",
AuthType: "anonymous", AuthType: "anonymous",
Enabled: true, Enabled: true,
}, },
}, },
TokenCache: struct { TokenCache: struct {
Enabled bool `toml:"enabled"` Enabled bool `toml:"enabled"`
DefaultTTL string `toml:"defaultTTL"` DefaultTTL string `toml:"defaultTTL"`
}{ }{
Enabled: true, // docker认证的匿名Token缓存配置用于提升性能 Enabled: true, // docker认证的匿名Token缓存配置用于提升性能
DefaultTTL: "20m", DefaultTTL: "20m",
}, },
} }
} }
// GetConfig 安全地获取配置副本 // GetConfig 安全地获取配置副本
func GetConfig() *AppConfig { func GetConfig() *AppConfig {
configCacheMutex.RLock() configCacheMutex.RLock()
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL { if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
config := cachedConfig config := cachedConfig
configCacheMutex.RUnlock() configCacheMutex.RUnlock()
return config return config
} }
configCacheMutex.RUnlock() configCacheMutex.RUnlock()
// 缓存过期,重新生成配置 // 缓存过期,重新生成配置
configCacheMutex.Lock() configCacheMutex.Lock()
defer configCacheMutex.Unlock() defer configCacheMutex.Unlock()
// 双重检查,防止重复生成 // 双重检查,防止重复生成
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL { if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
return cachedConfig return cachedConfig
} }
appConfigLock.RLock() appConfigLock.RLock()
if appConfig == nil { if appConfig == nil {
appConfigLock.RUnlock() appConfigLock.RUnlock()
defaultCfg := DefaultConfig() defaultCfg := DefaultConfig()
cachedConfig = defaultCfg cachedConfig = defaultCfg
configCacheTime = time.Now() configCacheTime = time.Now()
return defaultCfg return defaultCfg
} }
// 生成新的配置深拷贝 // 生成新的配置深拷贝
configCopy := *appConfig configCopy := *appConfig
configCopy.Security.WhiteList = append([]string(nil), appConfig.Security.WhiteList...) configCopy.Security.WhiteList = append([]string(nil), appConfig.Security.WhiteList...)
configCopy.Security.BlackList = append([]string(nil), appConfig.Security.BlackList...) configCopy.Security.BlackList = append([]string(nil), appConfig.Security.BlackList...)
configCopy.Proxy.WhiteList = append([]string(nil), appConfig.Proxy.WhiteList...) configCopy.Access.WhiteList = append([]string(nil), appConfig.Access.WhiteList...)
configCopy.Proxy.BlackList = append([]string(nil), appConfig.Proxy.BlackList...) configCopy.Access.BlackList = append([]string(nil), appConfig.Access.BlackList...)
appConfigLock.RUnlock() appConfigLock.RUnlock()
cachedConfig = &configCopy cachedConfig = &configCopy
configCacheTime = time.Now() configCacheTime = time.Now()
return cachedConfig return cachedConfig
} }
// setConfig 安全地设置配置 // setConfig 安全地设置配置
func setConfig(cfg *AppConfig) { func setConfig(cfg *AppConfig) {
appConfigLock.Lock() appConfigLock.Lock()
defer appConfigLock.Unlock() defer appConfigLock.Unlock()
appConfig = cfg appConfig = cfg
configCacheMutex.Lock() configCacheMutex.Lock()
cachedConfig = nil cachedConfig = nil
configCacheMutex.Unlock() configCacheMutex.Unlock()
} }
// LoadConfig 加载配置文件 // LoadConfig 加载配置文件
func LoadConfig() error { func LoadConfig() error {
// 首先使用默认配置 // 首先使用默认配置
cfg := DefaultConfig() cfg := DefaultConfig()
// 尝试加载TOML配置文件 // 尝试加载TOML配置文件
if data, err := os.ReadFile("config.toml"); err == nil { if data, err := os.ReadFile("config.toml"); err == nil {
if err := toml.Unmarshal(data, cfg); err != nil { if err := toml.Unmarshal(data, cfg); err != nil {
return fmt.Errorf("解析配置文件失败: %v", err) return fmt.Errorf("解析配置文件失败: %v", err)
} }
} else { } else {
fmt.Println("未找到config.toml使用默认配置") fmt.Println("未找到config.toml使用默认配置")
} }
// 从环境变量覆盖配置 // 从环境变量覆盖配置
overrideFromEnv(cfg) overrideFromEnv(cfg)
// 设置配置 // 设置配置
setConfig(cfg) setConfig(cfg)
if !isViperEnabled { return nil
go enableViperHotReload() }
}
// overrideFromEnv 从环境变量覆盖配置
return nil func overrideFromEnv(cfg *AppConfig) {
} // 服务器配置
if val := os.Getenv("SERVER_HOST"); val != "" {
func enableViperHotReload() { cfg.Server.Host = val
if isViperEnabled { }
return if val := os.Getenv("SERVER_PORT"); val != "" {
} if port, err := strconv.Atoi(val); err == nil && port > 0 {
cfg.Server.Port = port
// 创建Viper实例 }
viperInstance = viper.New() }
if val := os.Getenv("MAX_FILE_SIZE"); val != "" {
// 配置Viper if size, err := strconv.ParseInt(val, 10, 64); err == nil && size > 0 {
viperInstance.SetConfigName("config") cfg.Server.FileSize = size
viperInstance.SetConfigType("toml") }
viperInstance.AddConfigPath(".") }
// 读取配置文件 // 限流配置
if err := viperInstance.ReadInConfig(); err != nil { if val := os.Getenv("RATE_LIMIT"); val != "" {
fmt.Printf("读取配置失败,继续使用当前配置: %v\n", err) if limit, err := strconv.Atoi(val); err == nil && limit > 0 {
return cfg.RateLimit.RequestLimit = limit
} }
}
isViperEnabled = true if val := os.Getenv("RATE_PERIOD_HOURS"); val != "" {
if period, err := strconv.ParseFloat(val, 64); err == nil && period > 0 {
viperInstance.WatchConfig() cfg.RateLimit.PeriodHours = period
viperInstance.OnConfigChange(func(e fsnotify.Event) { }
fmt.Printf("检测到配置文件变化: %s\n", e.Name) }
hotReloadWithViper()
}) // IP限制配置
} if val := os.Getenv("IP_WHITELIST"); val != "" {
cfg.Security.WhiteList = append(cfg.Security.WhiteList, strings.Split(val, ",")...)
func hotReloadWithViper() { }
start := time.Now() if val := os.Getenv("IP_BLACKLIST"); val != "" {
fmt.Println("🔄 自动热重载...") cfg.Security.BlackList = append(cfg.Security.BlackList, strings.Split(val, ",")...)
}
// 创建新配置
cfg := DefaultConfig() // 下载限制配置
if val := os.Getenv("MAX_IMAGES"); val != "" {
// 使用Viper解析配置到结构体 if maxImages, err := strconv.Atoi(val); err == nil && maxImages > 0 {
if err := viperInstance.Unmarshal(cfg); err != nil { cfg.Download.MaxImages = maxImages
fmt.Printf("❌ 配置解析失败: %v\n", err) }
return }
} }
overrideFromEnv(cfg) // CreateDefaultConfigFile 创建默认配置文件
func CreateDefaultConfigFile() error {
setConfig(cfg) cfg := DefaultConfig()
// 异步更新受影响的组件 data, err := toml.Marshal(cfg)
go func() { if err != nil {
updateAffectedComponents() return fmt.Errorf("序列化默认配置失败: %v", err)
fmt.Printf("✅ Viper配置热重载完成耗时: %v\n", time.Since(start)) }
}()
} return os.WriteFile("config.toml", data, 0644)
}
func updateAffectedComponents() {
// 重新初始化限流器
if globalLimiter != nil {
fmt.Println("📡 重新初始化限流器...")
initLimiter()
}
// 重新加载访问控制
fmt.Println("🔒 重新加载访问控制规则...")
if GlobalAccessController != nil {
GlobalAccessController.Reload()
}
fmt.Println("🌐 更新Registry配置映射...")
reloadRegistryConfig()
// 其他需要重新初始化的组件可以在这里添加
fmt.Println("🔧 组件更新完成")
}
func reloadRegistryConfig() {
cfg := GetConfig()
enabledCount := 0
// 统计启用的Registry数量
for _, mapping := range cfg.Registries {
if mapping.Enabled {
enabledCount++
}
}
fmt.Printf("🌐 Registry配置已更新: %d个启用\n", enabledCount)
}
// overrideFromEnv 从环境变量覆盖配置
func overrideFromEnv(cfg *AppConfig) {
// 服务器配置
if val := os.Getenv("SERVER_HOST"); val != "" {
cfg.Server.Host = val
}
if val := os.Getenv("SERVER_PORT"); val != "" {
if port, err := strconv.Atoi(val); err == nil && port > 0 {
cfg.Server.Port = port
}
}
if val := os.Getenv("MAX_FILE_SIZE"); val != "" {
if size, err := strconv.ParseInt(val, 10, 64); err == nil && size > 0 {
cfg.Server.FileSize = size
}
}
// 限流配置
if val := os.Getenv("RATE_LIMIT"); val != "" {
if limit, err := strconv.Atoi(val); err == nil && limit > 0 {
cfg.RateLimit.RequestLimit = limit
}
}
if val := os.Getenv("RATE_PERIOD_HOURS"); val != "" {
if period, err := strconv.ParseFloat(val, 64); err == nil && period > 0 {
cfg.RateLimit.PeriodHours = period
}
}
// IP限制配置
if val := os.Getenv("IP_WHITELIST"); val != "" {
cfg.Security.WhiteList = append(cfg.Security.WhiteList, strings.Split(val, ",")...)
}
if val := os.Getenv("IP_BLACKLIST"); val != "" {
cfg.Security.BlackList = append(cfg.Security.BlackList, strings.Split(val, ",")...)
}
// 下载限制配置
if val := os.Getenv("MAX_IMAGES"); val != "" {
if maxImages, err := strconv.Atoi(val); err == nil && maxImages > 0 {
cfg.Download.MaxImages = maxImages
}
}
}
// CreateDefaultConfigFile 创建默认配置文件
func CreateDefaultConfigFile() error {
cfg := DefaultConfig()
data, err := toml.Marshal(cfg)
if err != nil {
return fmt.Errorf("序列化默认配置失败: %v", err)
}
return os.WriteFile("config.toml", data, 0644)
}

View File

@@ -1,89 +1,94 @@
[server] [server]
# 监听地址,默认监听所有接口 host = "0.0.0.0"
host = "0.0.0.0" # 监听端口
# 监听端口 port = 5000
port = 5000 # Github文件大小限制字节默认2GB
# 文件大小限制字节默认2GB fileSize = 2147483648
fileSize = 2147483648
[rateLimit]
[rateLimit] # 每个IP每小时允许的请求数(注意Docker镜像会有多个层会消耗多个次数)
# 每个IP每小时允许的请求数(Docker镜像每个层为一个请求) requestLimit = 500
requestLimit = 200 # 限流周期(小时)
# 限流周期(小时) periodHours = 1.0
periodHours = 1.0
[security]
[security] # IP白名单支持单个IP或IP段
# IP白名单支持单个IP或CIDR格式 # 白名单中的IP不受限流限制
# 白名单中的IP不受限流限制 whiteList = [
whiteList = [ "127.0.0.1",
"127.0.0.1", "172.17.0.0/16",
"192.168.1.0/24" "192.168.1.0/24"
] ]
# IP黑名单支持单个IP或CIDR格式 # IP黑名单支持单个IP或IP段
# 黑名单中的IP将被直接拒绝访问 # 黑名单中的IP将被直接拒绝访问
blackList = [ blackList = [
"192.168.100.1" "192.168.100.1",
] "192.168.100.0/24"
]
[proxy]
# 代理服务白名单支持GitHub仓库和Docker镜像支持通配符 [access]
# 只允许访问白名单中的仓库/镜像,为空时不限制 # 代理服务白名单支持GitHub仓库和Docker镜像支持通配符
whiteList = [] # 只允许访问白名单中的仓库/镜像,为空时不限制
whiteList = []
# 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
# 禁止访问黑名单中的仓库/镜像 # 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
blackList = [ # 禁止访问黑名单中的仓库/镜像
"baduser/malicious-repo", blackList = [
"*/malicious-repo", "baduser/malicious-repo",
"baduser/*" "*/malicious-repo",
] "baduser/*"
]
[download]
# 单次并发下载离线镜像数量限制 # 代理配置,支持有用户名/密码认证和无认证模式
maxImages = 10 # 无认证: socks5://127.0.0.1:1080
# 有认证: socks5://username:password@127.0.0.1:1080
# Registry映射配置支持多种Container Registry # HTTP 代理示例
[registries] # http://username:password@127.0.0.1:7890
# SOCKS5 代理示例
# GitHub Container Registry # socks5://username:password@127.0.0.1:1080
[registries."ghcr.io"] # SOCKS5H 代理示例
upstream = "ghcr.io" # socks5h://username:password@127.0.0.1:1080
authHost = "ghcr.io/token" # 留空不使用代理
authType = "github" proxy = ""
enabled = true
[download]
# Google Container Registry # 批量下载离线镜像数量限制
[registries."gcr.io"] maxImages = 10
upstream = "gcr.io"
authHost = "gcr.io/v2/token" # Registry映射配置支持多种镜像仓库上游
authType = "google" [registries]
enabled = true
# GitHub Container Registry
# Quay.io Container Registry [registries."ghcr.io"]
[registries."quay.io"] upstream = "ghcr.io"
upstream = "quay.io" authHost = "ghcr.io/token"
authHost = "quay.io/v2/auth" authType = "github"
authType = "quay" enabled = true
enabled = true
# Google Container Registry
# Kubernetes Container Registry [registries."gcr.io"]
[registries."registry.k8s.io"] upstream = "gcr.io"
upstream = "registry.k8s.io" authHost = "gcr.io/v2/token"
authHost = "registry.k8s.io" authType = "google"
authType = "anonymous" enabled = true
enabled = true
# Quay.io Container Registry
# 私有Registry示例默认禁用 [registries."quay.io"]
# [registries."harbor.company.com"] upstream = "quay.io"
# upstream = "harbor.company.com" authHost = "quay.io/v2/auth"
# authHost = "harbor.company.com/service/token" authType = "quay"
# authType = "basic" enabled = true
# enabled = false
# Kubernetes Container Registry
# 缓存配置Docker临时Token和Manifest统一管理显著提升性能 [registries."registry.k8s.io"]
[tokenCache] upstream = "registry.k8s.io"
# 是否启用缓存(同时控制Token和Manifest缓存) authHost = "registry.k8s.io"
enabled = true authType = "anonymous"
# 默认缓存时间 enabled = true
defaultTTL = "20m"
[tokenCache]
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
enabled = true
# 默认缓存时间(分钟)
defaultTTL = "20m"

File diff suppressed because it is too large Load Diff

View File

@@ -3,11 +3,9 @@ module hubproxy
go 1.24.0 go 1.24.0
require ( require (
github.com/fsnotify/fsnotify v1.8.0
github.com/gin-gonic/gin v1.10.0 github.com/gin-gonic/gin v1.10.0
github.com/google/go-containerregistry v0.20.5 github.com/google/go-containerregistry v0.20.5
github.com/pelletier/go-toml/v2 v2.2.3 github.com/pelletier/go-toml/v2 v2.2.3
github.com/spf13/viper v1.20.1
golang.org/x/time v0.11.0 golang.org/x/time v0.11.0
) )
@@ -25,11 +23,11 @@ require (
github.com/go-playground/locales v0.14.1 // indirect github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/goccy/go-json v0.10.2 // indirect github.com/goccy/go-json v0.10.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.0 // indirect github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect github.com/mitchellh/go-homedir v1.1.0 // indirect
@@ -38,18 +36,11 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
github.com/sagikazarmark/locafero v0.7.0 // indirect github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect github.com/sirupsen/logrus v1.9.3 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.12.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect github.com/ugorji/go/codec v1.2.12 // indirect
github.com/vbatts/tar-split v0.12.1 // indirect github.com/vbatts/tar-split v0.12.1 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/arch v0.8.0 // indirect golang.org/x/arch v0.8.0 // indirect
golang.org/x/crypto v0.32.0 // indirect golang.org/x/crypto v0.32.0 // indirect
golang.org/x/net v0.33.0 // indirect golang.org/x/net v0.33.0 // indirect
@@ -57,5 +48,6 @@ require (
golang.org/x/sys v0.33.0 // indirect golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.21.0 // indirect golang.org/x/text v0.21.0 // indirect
google.golang.org/protobuf v1.36.3 // indirect google.golang.org/protobuf v1.36.3 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

View File

@@ -8,6 +8,7 @@ github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY= github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8= github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU= github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -17,10 +18,6 @@ github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBi
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8= github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8=
github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo= github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0= github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk= github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
@@ -35,8 +32,6 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8= github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM= github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU= github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -52,8 +47,11 @@ github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa02
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM= github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M= github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
@@ -77,22 +75,11 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8= github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/sagikazarmark/locafero v0.7.0 h1:5MqpDsTGNDhY8sGp0Aowyf0qKsPrhewaLSsFaodPcyo=
github.com/sagikazarmark/locafero v0.7.0/go.mod h1:2za3Cg5rMaTMoG/2Ulr9AwtFaIppKXTRYnozin4aB5k=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spf13/afero v1.12.0 h1:UcOPyRBYczmFn6yvphxkn9ZEOY65cpwGKb5mL36mrqs=
github.com/spf13/afero v1.12.0/go.mod h1:ZTlWwG4/ahT8W7T0WQ5uYmjI9duaLQGy3Q2OAl4sk/4=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.20.1 h1:ZMi+z/lvLyPSCoNtFCpqjy0S4kPbirhpTMwl8BkW9X4=
github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqjJvu4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -101,20 +88,14 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI= github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE= github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo= github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA= github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc= golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys= golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
@@ -136,8 +117,10 @@ golang.org/x/time v0.11.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU= google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -1,59 +1,68 @@
package main package main
import ( import (
"net" "net"
"net/http" "net/http"
"time" "os"
) "time"
)
var (
// 全局HTTP客户端 - 用于代理请求(长超时) var (
globalHTTPClient *http.Client // 全局HTTP客户端 - 用于代理请求(长超时)
// 搜索HTTP客户端 - 用于API请求短超时 globalHTTPClient *http.Client
searchHTTPClient *http.Client // 搜索HTTP客户端 - 用于API请求短超时
) searchHTTPClient *http.Client
)
// initHTTPClients 初始化HTTP客户端
func initHTTPClients() { // initHTTPClients 初始化HTTP客户端
// 代理客户端配置 - 适用于大文件传输 func initHTTPClients() {
globalHTTPClient = &http.Client{ cfg := GetConfig()
Transport: &http.Transport{
DialContext: (&net.Dialer{ if p := cfg.Access.Proxy; p != "" {
Timeout: 30 * time.Second, os.Setenv("HTTP_PROXY", p)
KeepAlive: 30 * time.Second, os.Setenv("HTTPS_PROXY", p)
}).DialContext, }
MaxIdleConns: 1000, // 代理客户端配置 - 适用于大文件传输
MaxIdleConnsPerHost: 1000, globalHTTPClient = &http.Client{
IdleConnTimeout: 90 * time.Second, Transport: &http.Transport{
TLSHandshakeTimeout: 10 * time.Second, Proxy: http.ProxyFromEnvironment,
ExpectContinueTimeout: 1 * time.Second, DialContext: (&net.Dialer{
ResponseHeaderTimeout: 300 * time.Second, Timeout: 30 * time.Second,
}, KeepAlive: 30 * time.Second,
} }).DialContext,
MaxIdleConns: 1000,
// 搜索客户端配置 - 适用于API调用 MaxIdleConnsPerHost: 1000,
searchHTTPClient = &http.Client{ IdleConnTimeout: 90 * time.Second,
Timeout: 10 * time.Second, TLSHandshakeTimeout: 10 * time.Second,
Transport: &http.Transport{ ExpectContinueTimeout: 1 * time.Second,
DialContext: (&net.Dialer{ ResponseHeaderTimeout: 300 * time.Second,
Timeout: 5 * time.Second, },
KeepAlive: 30 * time.Second, }
}).DialContext,
MaxIdleConns: 100, // 搜索客户端配置 - 适用于API调用
MaxIdleConnsPerHost: 10, searchHTTPClient = &http.Client{
IdleConnTimeout: 90 * time.Second, Timeout: 10 * time.Second,
TLSHandshakeTimeout: 5 * time.Second, Transport: &http.Transport{
DisableCompression: false, Proxy: http.ProxyFromEnvironment,
}, DialContext: (&net.Dialer{
} Timeout: 5 * time.Second,
} KeepAlive: 30 * time.Second,
}).DialContext,
// GetGlobalHTTPClient 获取全局HTTP客户端用于代理 MaxIdleConns: 100,
func GetGlobalHTTPClient() *http.Client { MaxIdleConnsPerHost: 10,
return globalHTTPClient IdleConnTimeout: 90 * time.Second,
} TLSHandshakeTimeout: 5 * time.Second,
DisableCompression: false,
// GetSearchHTTPClient 获取搜索HTTP客户端用于API调用 },
func GetSearchHTTPClient() *http.Client { }
return searchHTTPClient }
}
// GetGlobalHTTPClient 获取全局HTTP客户端用于代理
func GetGlobalHTTPClient() *http.Client {
return globalHTTPClient
}
// GetSearchHTTPClient 获取搜索HTTP客户端用于API调用
func GetSearchHTTPClient() *http.Client {
return searchHTTPClient
}

View File

@@ -33,16 +33,18 @@ type DebounceEntry struct {
// DownloadDebouncer 下载防抖器 // DownloadDebouncer 下载防抖器
type DownloadDebouncer struct { type DownloadDebouncer struct {
mu sync.RWMutex mu sync.RWMutex
entries map[string]*DebounceEntry entries map[string]*DebounceEntry
window time.Duration window time.Duration
lastCleanup time.Time
} }
// NewDownloadDebouncer 创建下载防抖器 // NewDownloadDebouncer 创建下载防抖器
func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer { func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
return &DownloadDebouncer{ return &DownloadDebouncer{
entries: make(map[string]*DebounceEntry), entries: make(map[string]*DebounceEntry),
window: window, window: window,
lastCleanup: time.Now(),
} }
} }
@@ -50,27 +52,28 @@ func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
func (d *DownloadDebouncer) ShouldAllow(userID, contentKey string) bool { func (d *DownloadDebouncer) ShouldAllow(userID, contentKey string) bool {
d.mu.Lock() d.mu.Lock()
defer d.mu.Unlock() defer d.mu.Unlock()
key := userID + ":" + contentKey key := userID + ":" + contentKey
now := time.Now() now := time.Now()
if entry, exists := d.entries[key]; exists { if entry, exists := d.entries[key]; exists {
if now.Sub(entry.LastRequest) < d.window { if now.Sub(entry.LastRequest) < d.window {
return false // 在防抖窗口内,拒绝请求 return false // 在防抖窗口内,拒绝请求
} }
} }
// 更新或创建条目 // 更新或创建条目
d.entries[key] = &DebounceEntry{ d.entries[key] = &DebounceEntry{
LastRequest: now, LastRequest: now,
UserID: userID, UserID: userID,
} }
// 清理过期条目(简单策略每100次请求清理一次) // 清理过期条目(每5分钟清理一次)
if len(d.entries)%100 == 0 { if time.Since(d.lastCleanup) > 5*time.Minute {
d.cleanup(now) d.cleanup(now)
d.lastCleanup = now
} }
return true return true
} }
@@ -89,10 +92,10 @@ func generateContentFingerprint(images []string, platform string) string {
sortedImages := make([]string, len(images)) sortedImages := make([]string, len(images))
copy(sortedImages, images) copy(sortedImages, images)
sort.Strings(sortedImages) sort.Strings(sortedImages)
// 组合内容:镜像列表 + 平台信息 // 组合内容:镜像列表 + 平台信息
content := strings.Join(sortedImages, "|") + ":" + platform content := strings.Join(sortedImages, "|") + ":" + platform
// 生成MD5哈希 // 生成MD5哈希
hash := md5.Sum([]byte(content)) hash := md5.Sum([]byte(content))
return hex.EncodeToString(hash[:]) return hex.EncodeToString(hash[:])
@@ -104,14 +107,14 @@ func getUserID(c *gin.Context) string {
if sessionID, err := c.Cookie("session_id"); err == nil && sessionID != "" { if sessionID, err := c.Cookie("session_id"); err == nil && sessionID != "" {
return "session:" + sessionID return "session:" + sessionID
} }
// 备用方案IP + User-Agent组合 // 备用方案IP + User-Agent组合
ip := c.ClientIP() ip := c.ClientIP()
userAgent := c.GetHeader("User-Agent") userAgent := c.GetHeader("User-Agent")
if userAgent == "" { if userAgent == "" {
userAgent = "unknown" userAgent = "unknown"
} }
// 生成简短标识 // 生成简短标识
combined := ip + ":" + userAgent combined := ip + ":" + userAgent
hash := md5.Sum([]byte(combined)) hash := md5.Sum([]byte(combined))
@@ -128,8 +131,8 @@ var (
func initDebouncer() { func initDebouncer() {
// 单个镜像5秒防抖窗口 // 单个镜像5秒防抖窗口
singleImageDebouncer = NewDownloadDebouncer(5 * time.Second) singleImageDebouncer = NewDownloadDebouncer(5 * time.Second)
// 批量镜像:30秒防抖窗口(影响更大,需要更长保护) // 批量镜像:60秒防抖窗口
batchImageDebouncer = NewDownloadDebouncer(30 * time.Second) batchImageDebouncer = NewDownloadDebouncer(60 * time.Second)
} }
// ImageStreamer 镜像流式下载器 // ImageStreamer 镜像流式下载器
@@ -171,14 +174,15 @@ func NewImageStreamer(config *ImageStreamerConfig) *ImageStreamer {
// StreamOptions 下载选项 // StreamOptions 下载选项
type StreamOptions struct { type StreamOptions struct {
Platform string Platform string
Compression bool Compression bool
UseCompressedLayers bool // 是否保存原始压缩层,默认开启
} }
// StreamImageToWriter 流式下载镜像到Writer // StreamImageToWriter 流式下载镜像到Writer
func (is *ImageStreamer) StreamImageToWriter(ctx context.Context, imageRef string, writer io.Writer, options *StreamOptions) error { func (is *ImageStreamer) StreamImageToWriter(ctx context.Context, imageRef string, writer io.Writer, options *StreamOptions) error {
if options == nil { if options == nil {
options = &StreamOptions{} options = &StreamOptions{UseCompressedLayers: true}
} }
ref, err := name.ParseReference(imageRef) ref, err := name.ParseReference(imageRef)
@@ -218,13 +222,13 @@ func (is *ImageStreamer) getImageDescriptorWithPlatform(ref name.Reference, opti
// StreamImageToGin 流式响应到Gin // StreamImageToGin 流式响应到Gin
func (is *ImageStreamer) StreamImageToGin(ctx context.Context, imageRef string, c *gin.Context, options *StreamOptions) error { func (is *ImageStreamer) StreamImageToGin(ctx context.Context, imageRef string, c *gin.Context, options *StreamOptions) error {
if options == nil { if options == nil {
options = &StreamOptions{} options = &StreamOptions{UseCompressedLayers: true}
} }
filename := strings.ReplaceAll(imageRef, "/", "_") + ".tar" filename := strings.ReplaceAll(imageRef, "/", "_") + ".tar"
c.Header("Content-Type", "application/octet-stream") c.Header("Content-Type", "application/octet-stream")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename)) c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename))
if options.Compression { if options.Compression {
c.Header("Content-Encoding", "gzip") c.Header("Content-Encoding", "gzip")
} }
@@ -277,32 +281,32 @@ func (is *ImageStreamer) streamImageLayers(ctx context.Context, img v1.Image, wr
log.Printf("镜像包含 %d 层", len(layers)) log.Printf("镜像包含 %d 层", len(layers))
return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef) return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef, options)
} }
// streamDockerFormat 生成Docker格式 // streamDockerFormat 生成Docker格式
func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string) error { func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, options *StreamOptions) error {
return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil) return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil, options)
} }
// streamDockerFormatWithReturn 生成Docker格式并返回manifest和repositories信息 // streamDockerFormatWithReturn 生成Docker格式并返回manifest和repositories信息
func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string) error { func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string, options *StreamOptions) error {
configDigest, err := img.ConfigName() configDigest, err := img.ConfigName()
if err != nil { if err != nil {
return err return err
} }
configData, err := json.Marshal(configFile) configData, err := json.Marshal(configFile)
if err != nil { if err != nil {
return err return err
} }
configHeader := &tar.Header{ configHeader := &tar.Header{
Name: configDigest.String() + ".json", Name: configDigest.String() + ".json",
Size: int64(len(configData)), Size: int64(len(configData)),
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(configHeader); err != nil { if err := tarWriter.WriteHeader(configHeader); err != nil {
return err return err
} }
@@ -331,17 +335,29 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
Typeflag: tar.TypeDir, Typeflag: tar.TypeDir,
Mode: 0755, Mode: 0755,
} }
if err := tarWriter.WriteHeader(layerHeader); err != nil { if err := tarWriter.WriteHeader(layerHeader); err != nil {
return err return err
} }
uncompressedSize, err := partial.UncompressedSize(layer) var layerSize int64
if err != nil { var layerReader io.ReadCloser
return err
// 根据配置选择使用压缩层或未压缩层
if options != nil && options.UseCompressedLayers {
layerSize, err = layer.Size()
if err != nil {
return err
}
layerReader, err = layer.Compressed()
} else {
layerSize, err = partial.UncompressedSize(layer)
if err != nil {
return err
}
layerReader, err = layer.Uncompressed()
} }
layerReader, err := layer.Uncompressed()
if err != nil { if err != nil {
return err return err
} }
@@ -349,10 +365,10 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
layerTarHeader := &tar.Header{ layerTarHeader := &tar.Header{
Name: layerDir + "/layer.tar", Name: layerDir + "/layer.tar",
Size: uncompressedSize, Size: layerSize,
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(layerTarHeader); err != nil { if err := tarWriter.WriteHeader(layerTarHeader); err != nil {
return err return err
} }
@@ -369,12 +385,11 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
log.Printf("已处理层 %d/%d", i+1, len(layers)) log.Printf("已处理层 %d/%d", i+1, len(layers))
} }
// 构建单个镜像的manifest信息 // 构建单个镜像的manifest信息
singleManifest := map[string]interface{}{ singleManifest := map[string]interface{}{
"Config": configDigest.String() + ".json", "Config": configDigest.String() + ".json",
"RepoTags": []string{imageRef}, "RepoTags": []string{imageRef},
"Layers": func() []string { "Layers": func() []string {
var layers []string var layers []string
for _, digest := range layerDigests { for _, digest := range layerDigests {
layers = append(layers, digest+"/layer.tar") layers = append(layers, digest+"/layer.tar")
@@ -401,22 +416,22 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
// 单镜像下载直接写入manifest.json // 单镜像下载直接写入manifest.json
manifest := []map[string]interface{}{singleManifest} manifest := []map[string]interface{}{singleManifest}
manifestData, err := json.Marshal(manifest) manifestData, err := json.Marshal(manifest)
if err != nil { if err != nil {
return err return err
} }
manifestHeader := &tar.Header{ manifestHeader := &tar.Header{
Name: "manifest.json", Name: "manifest.json",
Size: int64(len(manifestData)), Size: int64(len(manifestData)),
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(manifestHeader); err != nil { if err := tarWriter.WriteHeader(manifestHeader); err != nil {
return err return err
} }
if _, err := tarWriter.Write(manifestData); err != nil { if _, err := tarWriter.Write(manifestData); err != nil {
return err return err
} }
@@ -426,22 +441,46 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
if err != nil { if err != nil {
return err return err
} }
repositoriesHeader := &tar.Header{ repositoriesHeader := &tar.Header{
Name: "repositories", Name: "repositories",
Size: int64(len(repositoriesData)), Size: int64(len(repositoriesData)),
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(repositoriesHeader); err != nil { if err := tarWriter.WriteHeader(repositoriesHeader); err != nil {
return err return err
} }
_, err = tarWriter.Write(repositoriesData) _, err = tarWriter.Write(repositoriesData)
return err return err
} }
// streamSingleImageForBatch 为批量下载流式处理单个镜像 // processImageForBatch 处理镜像的公共逻辑,用于批量下载
func (is *ImageStreamer) processImageForBatch(ctx context.Context, img v1.Image, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
var manifest map[string]interface{}
var repositories map[string]map[string]string
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories, options)
if err != nil {
return nil, nil, err
}
return manifest, repositories, nil
}
func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) { func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
ref, err := name.ParseReference(imageRef) ref, err := name.ParseReference(imageRef)
if err != nil { if err != nil {
@@ -455,84 +494,30 @@ func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWrite
return nil, nil, fmt.Errorf("获取镜像描述失败: %w", err) return nil, nil, fmt.Errorf("获取镜像描述失败: %w", err)
} }
var manifest map[string]interface{} var img v1.Image
var repositories map[string]map[string]string
switch desc.MediaType { switch desc.MediaType {
case types.OCIImageIndex, types.DockerManifestList: case types.OCIImageIndex, types.DockerManifestList:
// 处理多架构镜像 // 处理多架构镜像
img, err := is.selectPlatformImage(desc, options) img, err = is.selectPlatformImage(desc, options)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("选择平台镜像失败: %w", err) return nil, nil, fmt.Errorf("选择平台镜像失败: %w", err)
} }
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
case types.OCIManifestSchema1, types.DockerManifestSchema2: case types.OCIManifestSchema1, types.DockerManifestSchema2:
img, err := desc.Image() img, err = desc.Image()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("获取镜像失败: %w", err) return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
} }
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
default: default:
img, err := desc.Image() img, err = desc.Image()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("获取镜像失败: %w", err) return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
} }
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
} }
return manifest, repositories, nil return is.processImageForBatch(ctx, img, tarWriter, imageRef, options)
} }
// selectPlatformImage 从多架构镜像中选择合适的平台镜像 // selectPlatformImage 从多架构镜像中选择合适的平台镜像
func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *StreamOptions) (v1.Image, error) { func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *StreamOptions) (v1.Image, error) {
index, err := desc.ImageIndex() index, err := desc.ImageIndex()
@@ -551,7 +536,7 @@ func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *S
if m.Platform == nil { if m.Platform == nil {
continue continue
} }
if options.Platform != "" { if options.Platform != "" {
platformParts := strings.Split(options.Platform, "/") platformParts := strings.Split(options.Platform, "/")
if len(platformParts) >= 2 { if len(platformParts) >= 2 {
@@ -561,10 +546,10 @@ func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *S
if len(platformParts) >= 3 { if len(platformParts) >= 3 {
targetVariant = platformParts[2] targetVariant = platformParts[2]
} }
if m.Platform.OS == targetOS && if m.Platform.OS == targetOS &&
m.Platform.Architecture == targetArch && m.Platform.Architecture == targetArch &&
m.Platform.Variant == targetVariant { m.Platform.Variant == targetVariant {
selectedDesc = &m selectedDesc = &m
break break
} }
@@ -596,7 +581,6 @@ var globalImageStreamer *ImageStreamer
// initImageStreamer 初始化镜像下载器 // initImageStreamer 初始化镜像下载器
func initImageStreamer() { func initImageStreamer() {
globalImageStreamer = NewImageStreamer(nil) globalImageStreamer = NewImageStreamer(nil)
// 镜像下载器初始化完成
} }
// formatPlatformText 格式化平台文本 // formatPlatformText 格式化平台文本
@@ -611,9 +595,9 @@ func formatPlatformText(platform string) string {
func initImageTarRoutes(router *gin.Engine) { func initImageTarRoutes(router *gin.Engine) {
imageAPI := router.Group("/api/image") imageAPI := router.Group("/api/image")
{ {
imageAPI.GET("/download/:image", RateLimitMiddleware(globalLimiter), handleDirectImageDownload) imageAPI.GET("/download/:image", handleDirectImageDownload)
imageAPI.GET("/info/:image", RateLimitMiddleware(globalLimiter), handleImageInfo) imageAPI.GET("/info/:image", handleImageInfo)
imageAPI.POST("/batch", RateLimitMiddleware(globalLimiter), handleSimpleBatchDownload) imageAPI.POST("/batch", handleSimpleBatchDownload)
} }
} }
@@ -628,6 +612,7 @@ func handleDirectImageDownload(c *gin.Context) {
imageRef := strings.ReplaceAll(imageParam, "_", "/") imageRef := strings.ReplaceAll(imageParam, "_", "/")
platform := c.Query("platform") platform := c.Query("platform")
tag := c.DefaultQuery("tag", "") tag := c.DefaultQuery("tag", "")
useCompressed := c.DefaultQuery("compressed", "true") == "true"
if tag != "" && !strings.Contains(imageRef, ":") && !strings.Contains(imageRef, "@") { if tag != "" && !strings.Contains(imageRef, ":") && !strings.Contains(imageRef, "@") {
imageRef = imageRef + ":" + tag imageRef = imageRef + ":" + tag
@@ -643,18 +628,19 @@ func handleDirectImageDownload(c *gin.Context) {
// 防抖检查 // 防抖检查
userID := getUserID(c) userID := getUserID(c)
contentKey := generateContentFingerprint([]string{imageRef}, platform) contentKey := generateContentFingerprint([]string{imageRef}, platform)
if !singleImageDebouncer.ShouldAllow(userID, contentKey) { if !singleImageDebouncer.ShouldAllow(userID, contentKey) {
c.JSON(http.StatusTooManyRequests, gin.H{ c.JSON(http.StatusTooManyRequests, gin.H{
"error": "请求过于频繁,请稍后再试", "error": "请求过于频繁,请稍后再试",
"retry_after": 5, "retry_after": 5,
}) })
return return
} }
options := &StreamOptions{ options := &StreamOptions{
Platform: platform, Platform: platform,
Compression: false, Compression: false,
UseCompressedLayers: useCompressed,
} }
ctx := c.Request.Context() ctx := c.Request.Context()
@@ -670,8 +656,9 @@ func handleDirectImageDownload(c *gin.Context) {
// handleSimpleBatchDownload 处理批量下载 // handleSimpleBatchDownload 处理批量下载
func handleSimpleBatchDownload(c *gin.Context) { func handleSimpleBatchDownload(c *gin.Context) {
var req struct { var req struct {
Images []string `json:"images" binding:"required"` Images []string `json:"images" binding:"required"`
Platform string `json:"platform"` Platform string `json:"platform"`
UseCompressedLayers *bool `json:"useCompressedLayers"`
} }
if err := c.ShouldBindJSON(&req); err != nil { if err := c.ShouldBindJSON(&req); err != nil {
@@ -701,25 +688,31 @@ func handleSimpleBatchDownload(c *gin.Context) {
// 批量下载防抖检查 // 批量下载防抖检查
userID := getUserID(c) userID := getUserID(c)
contentKey := generateContentFingerprint(req.Images, req.Platform) contentKey := generateContentFingerprint(req.Images, req.Platform)
if !batchImageDebouncer.ShouldAllow(userID, contentKey) { if !batchImageDebouncer.ShouldAllow(userID, contentKey) {
c.JSON(http.StatusTooManyRequests, gin.H{ c.JSON(http.StatusTooManyRequests, gin.H{
"error": "批量下载请求过于频繁,请稍后再试", "error": "批量下载请求过于频繁,请稍后再试",
"retry_after": 30, "retry_after": 60,
}) })
return return
} }
useCompressed := true // 默认启用原始压缩层
if req.UseCompressedLayers != nil {
useCompressed = *req.UseCompressedLayers
}
options := &StreamOptions{ options := &StreamOptions{
Platform: req.Platform, Platform: req.Platform,
Compression: false, Compression: false,
UseCompressedLayers: useCompressed,
} }
ctx := c.Request.Context() ctx := c.Request.Context()
log.Printf("批量下载 %d 个镜像 (平台: %s)", len(req.Images), formatPlatformText(req.Platform)) log.Printf("批量下载 %d 个镜像 (平台: %s)", len(req.Images), formatPlatformText(req.Platform))
filename := fmt.Sprintf("batch_%d_images.tar", len(req.Images)) filename := fmt.Sprintf("batch_%d_images.tar", len(req.Images))
c.Header("Content-Type", "application/octet-stream") c.Header("Content-Type", "application/octet-stream")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename)) c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename))
@@ -792,7 +785,7 @@ func handleImageInfo(c *gin.Context) {
// StreamMultipleImages 批量下载多个镜像 // StreamMultipleImages 批量下载多个镜像
func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []string, writer io.Writer, options *StreamOptions) error { func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []string, writer io.Writer, options *StreamOptions) error {
if options == nil { if options == nil {
options = &StreamOptions{} options = &StreamOptions{UseCompressedLayers: true}
} }
var finalWriter io.Writer = writer var finalWriter io.Writer = writer
@@ -817,12 +810,12 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
} }
log.Printf("处理镜像 %d/%d: %s", i+1, len(imageRefs), imageRef) log.Printf("处理镜像 %d/%d: %s", i+1, len(imageRefs), imageRef)
// 防止单个镜像处理时间过长 // 防止单个镜像处理时间过长
timeoutCtx, cancel := context.WithTimeout(ctx, 15*time.Minute) timeoutCtx, cancel := context.WithTimeout(ctx, 15*time.Minute)
manifest, repositories, err := is.streamSingleImageForBatch(timeoutCtx, tarWriter, imageRef, options) manifest, repositories, err := is.streamSingleImageForBatch(timeoutCtx, tarWriter, imageRef, options)
cancel() cancel()
if err != nil { if err != nil {
log.Printf("下载镜像 %s 失败: %v", imageRef, err) log.Printf("下载镜像 %s 失败: %v", imageRef, err)
return fmt.Errorf("下载镜像 %s 失败: %w", imageRef, err) return fmt.Errorf("下载镜像 %s 失败: %w", imageRef, err)
@@ -851,17 +844,17 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
if err != nil { if err != nil {
return fmt.Errorf("序列化manifest失败: %w", err) return fmt.Errorf("序列化manifest失败: %w", err)
} }
manifestHeader := &tar.Header{ manifestHeader := &tar.Header{
Name: "manifest.json", Name: "manifest.json",
Size: int64(len(manifestData)), Size: int64(len(manifestData)),
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(manifestHeader); err != nil { if err := tarWriter.WriteHeader(manifestHeader); err != nil {
return fmt.Errorf("写入manifest header失败: %w", err) return fmt.Errorf("写入manifest header失败: %w", err)
} }
if _, err := tarWriter.Write(manifestData); err != nil { if _, err := tarWriter.Write(manifestData); err != nil {
return fmt.Errorf("写入manifest数据失败: %w", err) return fmt.Errorf("写入manifest数据失败: %w", err)
} }
@@ -871,21 +864,21 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
if err != nil { if err != nil {
return fmt.Errorf("序列化repositories失败: %w", err) return fmt.Errorf("序列化repositories失败: %w", err)
} }
repositoriesHeader := &tar.Header{ repositoriesHeader := &tar.Header{
Name: "repositories", Name: "repositories",
Size: int64(len(repositoriesData)), Size: int64(len(repositoriesData)),
Mode: 0644, Mode: 0644,
} }
if err := tarWriter.WriteHeader(repositoriesHeader); err != nil { if err := tarWriter.WriteHeader(repositoriesHeader); err != nil {
return fmt.Errorf("写入repositories header失败: %w", err) return fmt.Errorf("写入repositories header失败: %w", err)
} }
if _, err := tarWriter.Write(repositoriesData); err != nil { if _, err := tarWriter.Write(repositoriesData); err != nil {
return fmt.Errorf("写入repositories数据失败: %w", err) return fmt.Errorf("写入repositories数据失败: %w", err)
} }
log.Printf("批量下载完成,共处理 %d 个镜像", len(imageRefs)) log.Printf("批量下载完成,共处理 %d 个镜像", len(imageRefs))
return nil return nil
} }

View File

@@ -1,381 +1,379 @@
package main package main
import ( import (
"embed" "embed"
"fmt" "fmt"
"io" "io"
"log" "log"
"net/http" "net/http"
"regexp" "regexp"
"strconv" "strconv"
"strings" "strings"
"time" "time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
) )
//go:embed public/* //go:embed public/*
var staticFiles embed.FS var staticFiles embed.FS
// 服务嵌入的静态文件 // 服务嵌入的静态文件
func serveEmbedFile(c *gin.Context, filename string) { func serveEmbedFile(c *gin.Context, filename string) {
data, err := staticFiles.ReadFile(filename) data, err := staticFiles.ReadFile(filename)
if err != nil { if err != nil {
c.Status(404) c.Status(404)
return return
} }
contentType := "text/html; charset=utf-8" contentType := "text/html; charset=utf-8"
if strings.HasSuffix(filename, ".ico") { if strings.HasSuffix(filename, ".ico") {
contentType = "image/x-icon" contentType = "image/x-icon"
} }
c.Data(200, contentType, data) c.Data(200, contentType, data)
} }
var ( var (
exps = []*regexp.Regexp{ exps = []*regexp.Regexp{
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:releases|archive)/.*$`), regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:releases|archive)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:blob|raw)/.*$`), regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:blob|raw)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:info|git-).*$`), regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:info|git-).*$`),
regexp.MustCompile(`^(?:https?://)?raw\.github(?:usercontent|)\.com/([^/]+)/([^/]+)/.+?/.+$`), regexp.MustCompile(`^(?:https?://)?raw\.github(?:usercontent|)\.com/([^/]+)/([^/]+)/.+?/.+$`),
regexp.MustCompile(`^(?:https?://)?gist\.github(?:usercontent|)\.com/([^/]+)/.+?/.+`), regexp.MustCompile(`^(?:https?://)?gist\.github(?:usercontent|)\.com/([^/]+)/.+?/.+`),
regexp.MustCompile(`^(?:https?://)?api\.github\.com/repos/([^/]+)/([^/]+)/.*`), regexp.MustCompile(`^(?:https?://)?api\.github\.com/repos/([^/]+)/([^/]+)/.*`),
regexp.MustCompile(`^(?:https?://)?huggingface\.co(?:/spaces)?/([^/]+)/(.+)$`), regexp.MustCompile(`^(?:https?://)?huggingface\.co(?:/spaces)?/([^/]+)/(.+)$`),
regexp.MustCompile(`^(?:https?://)?cdn-lfs\.hf\.co(?:/spaces)?/([^/]+)/([^/]+)(?:/(.*))?$`), regexp.MustCompile(`^(?:https?://)?cdn-lfs\.hf\.co(?:/spaces)?/([^/]+)/([^/]+)(?:/(.*))?$`),
regexp.MustCompile(`^(?:https?://)?download\.docker\.com/([^/]+)/.*\.(tgz|zip)$`), regexp.MustCompile(`^(?:https?://)?download\.docker\.com/([^/]+)/.*\.(tgz|zip)$`),
regexp.MustCompile(`^(?:https?://)?(github|opengraph)\.githubassets\.com/([^/]+)/.+?$`), regexp.MustCompile(`^(?:https?://)?(github|opengraph)\.githubassets\.com/([^/]+)/.+?$`),
} }
globalLimiter *IPRateLimiter globalLimiter *IPRateLimiter
// 服务启动时间 // 服务启动时间
serviceStartTime = time.Now() serviceStartTime = time.Now()
) )
func main() { func main() {
// 加载配置 // 加载配置
if err := LoadConfig(); err != nil { if err := LoadConfig(); err != nil {
fmt.Printf("配置加载失败: %v\n", err) fmt.Printf("配置加载失败: %v\n", err)
return return
} }
// 初始化HTTP客户端 // 初始化HTTP客户端
initHTTPClients() initHTTPClients()
// 初始化限流器 // 初始化限流器
initLimiter() initLimiter()
// 初始化Docker流式代理 // 初始化Docker流式代理
initDockerProxy() initDockerProxy()
// 初始化镜像流式下载器 // 初始化镜像流式下载器
initImageStreamer() initImageStreamer()
// 初始化防抖器 // 初始化防抖器
initDebouncer() initDebouncer()
gin.SetMode(gin.ReleaseMode) gin.SetMode(gin.ReleaseMode)
router := gin.Default() router := gin.Default()
// 全局Panic恢复保护 // 全局Panic恢复保护
router.Use(gin.CustomRecovery(func(c *gin.Context, recovered interface{}) { router.Use(gin.CustomRecovery(func(c *gin.Context, recovered interface{}) {
log.Printf("🚨 Panic recovered: %v", recovered) log.Printf("🚨 Panic recovered: %v", recovered)
c.JSON(http.StatusInternalServerError, gin.H{ c.JSON(http.StatusInternalServerError, gin.H{
"error": "Internal server error", "error": "Internal server error",
"code": "INTERNAL_ERROR", "code": "INTERNAL_ERROR",
}) })
})) }))
// 初始化监控端点 // 全局限流中间件 - 应用到所有路由
initHealthRoutes(router) router.Use(RateLimitMiddleware(globalLimiter))
// 初始化镜像tar下载路由 // 初始化监控端点
initImageTarRoutes(router) initHealthRoutes(router)
// 静态文件路由 // 初始化镜像tar下载路由
router.GET("/", func(c *gin.Context) { initImageTarRoutes(router)
serveEmbedFile(c, "public/index.html")
}) // 静态文件路由
router.GET("/public/*filepath", func(c *gin.Context) { router.GET("/", func(c *gin.Context) {
filepath := strings.TrimPrefix(c.Param("filepath"), "/") serveEmbedFile(c, "public/index.html")
serveEmbedFile(c, "public/"+filepath) })
}) router.GET("/public/*filepath", func(c *gin.Context) {
filepath := strings.TrimPrefix(c.Param("filepath"), "/")
router.GET("/images.html", func(c *gin.Context) { serveEmbedFile(c, "public/"+filepath)
serveEmbedFile(c, "public/images.html") })
})
router.GET("/search.html", func(c *gin.Context) { router.GET("/images.html", func(c *gin.Context) {
serveEmbedFile(c, "public/search.html") serveEmbedFile(c, "public/images.html")
}) })
router.GET("/favicon.ico", func(c *gin.Context) { router.GET("/search.html", func(c *gin.Context) {
serveEmbedFile(c, "public/favicon.ico") serveEmbedFile(c, "public/search.html")
}) })
router.GET("/favicon.ico", func(c *gin.Context) {
// 注册dockerhub搜索路由 serveEmbedFile(c, "public/favicon.ico")
RegisterSearchRoute(router) })
// 注册Docker认证路由(/token* // 注册dockerhub搜索路由
router.Any("/token", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin) RegisterSearchRoute(router)
router.Any("/token/*path", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin)
// 注册Docker认证路由/token*
// 注册Docker Registry代理路由 router.Any("/token", ProxyDockerAuthGin)
router.Any("/v2/*path", RateLimitMiddleware(globalLimiter), ProxyDockerRegistryGin) router.Any("/token/*path", ProxyDockerAuthGin)
// 注册Docker Registry代理路由
// 注册NoRoute处理器 router.Any("/v2/*path", ProxyDockerRegistryGin)
router.NoRoute(RateLimitMiddleware(globalLimiter), handler)
// 注册NoRoute处理器
cfg := GetConfig() router.NoRoute(handler)
fmt.Printf("🚀 HubProxy 启动成功\n")
fmt.Printf("📡 监听地址: %s:%d\n", cfg.Server.Host, cfg.Server.Port) cfg := GetConfig()
fmt.Printf("⚡ 限流配置: %d请求/%g小时\n", cfg.RateLimit.RequestLimit, cfg.RateLimit.PeriodHours) fmt.Printf("🚀 HubProxy 启动成功\n")
fmt.Printf("🔗 项目地址: https://github.com/sky22333/hubproxy\n") fmt.Printf("📡 监听地址: %s:%d\n", cfg.Server.Host, cfg.Server.Port)
fmt.Printf("⚡ 限流配置: %d请求/%g小时\n", cfg.RateLimit.RequestLimit, cfg.RateLimit.PeriodHours)
err := router.Run(fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port)) fmt.Printf("🔗 项目地址: https://github.com/sky22333/hubproxy\n")
if err != nil {
fmt.Printf("启动服务失败: %v\n", err) err := router.Run(fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port))
} if err != nil {
} fmt.Printf("启动服务失败: %v\n", err)
}
func handler(c *gin.Context) { }
rawPath := strings.TrimPrefix(c.Request.URL.RequestURI(), "/")
func handler(c *gin.Context) {
for strings.HasPrefix(rawPath, "/") { rawPath := strings.TrimPrefix(c.Request.URL.RequestURI(), "/")
rawPath = strings.TrimPrefix(rawPath, "/")
} for strings.HasPrefix(rawPath, "/") {
rawPath = strings.TrimPrefix(rawPath, "/")
if !strings.HasPrefix(rawPath, "http") { }
c.String(http.StatusForbidden, "无效输入")
return if !strings.HasPrefix(rawPath, "http") {
} c.String(http.StatusForbidden, "无效输入")
return
matches := checkURL(rawPath) }
if matches != nil {
// GitHub仓库访问控制检查 matches := checkURL(rawPath)
if allowed, reason := GlobalAccessController.CheckGitHubAccess(matches); !allowed { if matches != nil {
// 构建仓库名用于日志 // GitHub仓库访问控制检查
var repoPath string if allowed, reason := GlobalAccessController.CheckGitHubAccess(matches); !allowed {
if len(matches) >= 2 { // 构建仓库名用于日志
username := matches[0] var repoPath string
repoName := strings.TrimSuffix(matches[1], ".git") if len(matches) >= 2 {
repoPath = username + "/" + repoName username := matches[0]
} repoName := strings.TrimSuffix(matches[1], ".git")
fmt.Printf("GitHub仓库 %s 访问被拒绝: %s\n", repoPath, reason) repoPath = username + "/" + repoName
c.String(http.StatusForbidden, reason) }
return fmt.Printf("GitHub仓库 %s 访问被拒绝: %s\n", repoPath, reason)
} c.String(http.StatusForbidden, reason)
} else { return
c.String(http.StatusForbidden, "无效输入") }
return } else {
} c.String(http.StatusForbidden, "无效输入")
return
if exps[1].MatchString(rawPath) { }
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
} if exps[1].MatchString(rawPath) {
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
proxy(c, rawPath) }
}
proxyRequest(c, rawPath)
}
func proxy(c *gin.Context, u string) {
proxyWithRedirect(c, u, 0) func proxyRequest(c *gin.Context, u string) {
} proxyWithRedirect(c, u, 0)
}
func proxyWithRedirect(c *gin.Context, u string, redirectCount int) { func proxyWithRedirect(c *gin.Context, u string, redirectCount int) {
// 限制最大重定向次数,防止无限递归 // 限制最大重定向次数,防止无限递归
const maxRedirects = 20 const maxRedirects = 20
if redirectCount > maxRedirects { if redirectCount > maxRedirects {
c.String(http.StatusLoopDetected, "重定向次数过多,可能存在循环重定向") c.String(http.StatusLoopDetected, "重定向次数过多,可能存在循环重定向")
return return
} }
req, err := http.NewRequest(c.Request.Method, u, c.Request.Body) req, err := http.NewRequest(c.Request.Method, u, c.Request.Body)
if err != nil { if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err)) c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return return
} }
for key, values := range c.Request.Header { for key, values := range c.Request.Header {
for _, value := range values { for _, value := range values {
req.Header.Add(key, value) req.Header.Add(key, value)
} }
} }
req.Header.Del("Host") req.Header.Del("Host")
resp, err := GetGlobalHTTPClient().Do(req) resp, err := GetGlobalHTTPClient().Do(req)
if err != nil { if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err)) c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return return
} }
defer func() { defer func() {
if err := resp.Body.Close(); err != nil { if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭响应体失败: %v\n", err) fmt.Printf("关闭响应体失败: %v\n", err)
} }
}() }()
// 检查文件大小限制 // 检查文件大小限制
cfg := GetConfig() cfg := GetConfig()
if contentLength := resp.Header.Get("Content-Length"); contentLength != "" { if contentLength := resp.Header.Get("Content-Length"); contentLength != "" {
if size, err := strconv.ParseInt(contentLength, 10, 64); err == nil && size > cfg.Server.FileSize { if size, err := strconv.ParseInt(contentLength, 10, 64); err == nil && size > cfg.Server.FileSize {
c.String(http.StatusRequestEntityTooLarge, c.String(http.StatusRequestEntityTooLarge,
fmt.Sprintf("文件过大,限制大小: %d MB", cfg.Server.FileSize/(1024*1024))) fmt.Sprintf("文件过大,限制大小: %d MB", cfg.Server.FileSize/(1024*1024)))
return return
} }
} }
// 清理安全相关的头 // 清理安全相关的头
resp.Header.Del("Content-Security-Policy") resp.Header.Del("Content-Security-Policy")
resp.Header.Del("Referrer-Policy") resp.Header.Del("Referrer-Policy")
resp.Header.Del("Strict-Transport-Security") resp.Header.Del("Strict-Transport-Security")
// 获取真实域名 // 获取真实域名
realHost := c.Request.Header.Get("X-Forwarded-Host") realHost := c.Request.Header.Get("X-Forwarded-Host")
if realHost == "" { if realHost == "" {
realHost = c.Request.Host realHost = c.Request.Host
} }
// 如果域名中没有协议前缀添加https:// // 如果域名中没有协议前缀添加https://
if !strings.HasPrefix(realHost, "http://") && !strings.HasPrefix(realHost, "https://") { if !strings.HasPrefix(realHost, "http://") && !strings.HasPrefix(realHost, "https://") {
realHost = "https://" + realHost realHost = "https://" + realHost
} }
if strings.HasSuffix(strings.ToLower(u), ".sh") { if strings.HasSuffix(strings.ToLower(u), ".sh") {
isGzipCompressed := resp.Header.Get("Content-Encoding") == "gzip" isGzipCompressed := resp.Header.Get("Content-Encoding") == "gzip"
processedBody, processedSize, err := ProcessSmart(resp.Body, isGzipCompressed, realHost) processedBody, processedSize, err := ProcessSmart(resp.Body, isGzipCompressed, realHost)
if err != nil { if err != nil {
fmt.Printf("智能处理失败,回退到直接代理: %v\n", err) fmt.Printf("智能处理失败,回退到直接代理: %v\n", err)
processedBody = resp.Body processedBody = resp.Body
processedSize = 0 processedSize = 0
} }
// 智能设置响应头 // 智能设置响应头
if processedSize > 0 { if processedSize > 0 {
resp.Header.Del("Content-Length") resp.Header.Del("Content-Length")
resp.Header.Del("Content-Encoding") resp.Header.Del("Content-Encoding")
resp.Header.Set("Transfer-Encoding", "chunked") resp.Header.Set("Transfer-Encoding", "chunked")
} }
// 复制其他响应头 // 复制其他响应头
for key, values := range resp.Header { for key, values := range resp.Header {
for _, value := range values { for _, value := range values {
c.Header(key, value) c.Header(key, value)
} }
} }
if location := resp.Header.Get("Location"); location != "" { if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil { if checkURL(location) != nil {
c.Header("Location", "/"+location) c.Header("Location", "/"+location)
} else { } else {
proxyWithRedirect(c, location, redirectCount+1) proxyWithRedirect(c, location, redirectCount+1)
return return
} }
} }
c.Status(resp.StatusCode) c.Status(resp.StatusCode)
// 输出处理后的内容 // 输出处理后的内容
if _, err := io.Copy(c.Writer, processedBody); err != nil { if _, err := io.Copy(c.Writer, processedBody); err != nil {
return return
} }
} else { } else {
for key, values := range resp.Header { for key, values := range resp.Header {
for _, value := range values { for _, value := range values {
c.Header(key, value) c.Header(key, value)
} }
} }
// 处理重定向 // 处理重定向
if location := resp.Header.Get("Location"); location != "" { if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil { if checkURL(location) != nil {
c.Header("Location", "/"+location) c.Header("Location", "/"+location)
} else { } else {
proxyWithRedirect(c, location, redirectCount+1) proxyWithRedirect(c, location, redirectCount+1)
return return
} }
} }
c.Status(resp.StatusCode) c.Status(resp.StatusCode)
// 直接流式转发 // 直接流式转发
if _, err := io.Copy(c.Writer, resp.Body); err != nil { io.Copy(c.Writer, resp.Body)
fmt.Printf("直接代理失败: %v\n", err) }
} }
}
} func checkURL(u string) []string {
for _, exp := range exps {
func checkURL(u string) []string { if matches := exp.FindStringSubmatch(u); matches != nil {
for _, exp := range exps { return matches[1:]
if matches := exp.FindStringSubmatch(u); matches != nil { }
return matches[1:] }
} return nil
} }
return nil
} // 初始化健康监控路由
func initHealthRoutes(router *gin.Engine) {
// 初始化健康监控路由 // 健康检查端点
func initHealthRoutes(router *gin.Engine) { router.GET("/health", func(c *gin.Context) {
// 健康检查端点 c.JSON(http.StatusOK, gin.H{
router.GET("/health", func(c *gin.Context) { "status": "healthy",
c.JSON(http.StatusOK, gin.H{ "timestamp": time.Now().Unix(),
"status": "healthy", "uptime": time.Since(serviceStartTime).Seconds(),
"timestamp": time.Now().Unix(), "service": "hubproxy",
"uptime": time.Since(serviceStartTime).Seconds(), })
"service": "hubproxy", })
})
}) // 就绪检查端点
router.GET("/ready", func(c *gin.Context) {
// 就绪检查端点 checks := make(map[string]string)
router.GET("/ready", func(c *gin.Context) { allReady := true
checks := make(map[string]string)
allReady := true if GetConfig() != nil {
checks["config"] = "ok"
if GetConfig() != nil { } else {
checks["config"] = "ok" checks["config"] = "failed"
} else { allReady = false
checks["config"] = "failed" }
allReady = false
} // 检查全局缓存状态
if globalCache != nil {
// 检查全局缓存状态 checks["cache"] = "ok"
if globalCache != nil { } else {
checks["cache"] = "ok" checks["cache"] = "failed"
} else { allReady = false
checks["cache"] = "failed" }
allReady = false
} // 检查限流器状态
if globalLimiter != nil {
// 检查限流器状态 checks["ratelimiter"] = "ok"
if globalLimiter != nil { } else {
checks["ratelimiter"] = "ok" checks["ratelimiter"] = "failed"
} else { allReady = false
checks["ratelimiter"] = "failed" }
allReady = false
} // 检查镜像下载器状态
if globalImageStreamer != nil {
// 检查镜像下载器状态 checks["imagestreamer"] = "ok"
if globalImageStreamer != nil { } else {
checks["imagestreamer"] = "ok" checks["imagestreamer"] = "failed"
} else { allReady = false
checks["imagestreamer"] = "failed" }
allReady = false
} // 检查HTTP客户端状态
if GetGlobalHTTPClient() != nil {
// 检查HTTP客户端状态 checks["httpclient"] = "ok"
if GetGlobalHTTPClient() != nil { } else {
checks["httpclient"] = "ok" checks["httpclient"] = "failed"
} else { allReady = false
checks["httpclient"] = "failed" }
allReady = false
} status := http.StatusOK
if !allReady {
status := http.StatusOK status = http.StatusServiceUnavailable
if !allReady { }
status = http.StatusServiceUnavailable
} c.JSON(status, gin.H{
"ready": allReady,
c.JSON(status, gin.H{ "checks": checks,
"ready": allReady, "timestamp": time.Now().Unix(),
"checks": checks, "uptime": time.Since(serviceStartTime).Seconds(),
"timestamp": time.Now().Unix(), })
"uptime": time.Since(serviceStartTime).Seconds(), })
}) }
})
}

View File

@@ -1,95 +1,95 @@
package main package main
import ( import (
"bytes" "bytes"
"compress/gzip" "compress/gzip"
"fmt" "fmt"
"io" "io"
"regexp" "regexp"
"strings" "strings"
) )
// GitHub URL正则表达式 // GitHub URL正则表达式
var githubRegex = regexp.MustCompile(`https?://(?:github\.com|raw\.githubusercontent\.com|raw\.github\.com|gist\.githubusercontent\.com|gist\.github\.com|api\.github\.com)[^\s'"]+`) var githubRegex = regexp.MustCompile(`https?://(?:github\.com|raw\.githubusercontent\.com|raw\.github\.com|gist\.githubusercontent\.com|gist\.github\.com|api\.github\.com)[^\s'"]+`)
// ProcessSmart Shell脚本智能处理函数 // ProcessSmart Shell脚本智能处理函数
func ProcessSmart(input io.ReadCloser, isCompressed bool, host string) (io.Reader, int64, error) { func ProcessSmart(input io.ReadCloser, isCompressed bool, host string) (io.Reader, int64, error) {
defer input.Close() defer input.Close()
content, err := readShellContent(input, isCompressed) content, err := readShellContent(input, isCompressed)
if err != nil { if err != nil {
return nil, 0, fmt.Errorf("内容读取失败: %v", err) return nil, 0, fmt.Errorf("内容读取失败: %v", err)
} }
if len(content) == 0 { if len(content) == 0 {
return strings.NewReader(""), 0, nil return strings.NewReader(""), 0, nil
} }
if len(content) > 10*1024*1024 { if len(content) > 10*1024*1024 {
return strings.NewReader(content), int64(len(content)), nil return strings.NewReader(content), int64(len(content)), nil
} }
if !strings.Contains(content, "github.com") && !strings.Contains(content, "githubusercontent.com") { if !strings.Contains(content, "github.com") && !strings.Contains(content, "githubusercontent.com") {
return strings.NewReader(content), int64(len(content)), nil return strings.NewReader(content), int64(len(content)), nil
} }
processed := processGitHubURLs(content, host) processed := processGitHubURLs(content, host)
return strings.NewReader(processed), int64(len(processed)), nil return strings.NewReader(processed), int64(len(processed)), nil
} }
func readShellContent(input io.ReadCloser, isCompressed bool) (string, error) { func readShellContent(input io.ReadCloser, isCompressed bool) (string, error) {
var reader io.Reader = input var reader io.Reader = input
// 处理gzip压缩 // 处理gzip压缩
if isCompressed { if isCompressed {
peek := make([]byte, 2) peek := make([]byte, 2)
n, err := input.Read(peek) n, err := input.Read(peek)
if err != nil && err != io.EOF { if err != nil && err != io.EOF {
return "", fmt.Errorf("读取数据失败: %v", err) return "", fmt.Errorf("读取数据失败: %v", err)
} }
if n >= 2 && peek[0] == 0x1f && peek[1] == 0x8b { if n >= 2 && peek[0] == 0x1f && peek[1] == 0x8b {
combinedReader := io.MultiReader(bytes.NewReader(peek[:n]), input) combinedReader := io.MultiReader(bytes.NewReader(peek[:n]), input)
gzReader, err := gzip.NewReader(combinedReader) gzReader, err := gzip.NewReader(combinedReader)
if err != nil { if err != nil {
return "", fmt.Errorf("gzip解压失败: %v", err) return "", fmt.Errorf("gzip解压失败: %v", err)
} }
defer gzReader.Close() defer gzReader.Close()
reader = gzReader reader = gzReader
} else { } else {
reader = io.MultiReader(bytes.NewReader(peek[:n]), input) reader = io.MultiReader(bytes.NewReader(peek[:n]), input)
} }
} }
data, err := io.ReadAll(reader) data, err := io.ReadAll(reader)
if err != nil { if err != nil {
return "", fmt.Errorf("读取内容失败: %v", err) return "", fmt.Errorf("读取内容失败: %v", err)
} }
return string(data), nil return string(data), nil
} }
func processGitHubURLs(content, host string) string { func processGitHubURLs(content, host string) string {
return githubRegex.ReplaceAllStringFunc(content, func(url string) string { return githubRegex.ReplaceAllStringFunc(content, func(url string) string {
return transformURL(url, host) return transformURL(url, host)
}) })
} }
// transformURL URL转换函数 // transformURL URL转换函数
func transformURL(url, host string) string { func transformURL(url, host string) string {
if strings.Contains(url, host) { if strings.Contains(url, host) {
return url return url
} }
if strings.HasPrefix(url, "http://") { if strings.HasPrefix(url, "http://") {
url = "https" + url[4:] url = "https" + url[4:]
} else if !strings.HasPrefix(url, "https://") && !strings.HasPrefix(url, "//") { } else if !strings.HasPrefix(url, "https://") && !strings.HasPrefix(url, "//") {
url = "https://" + url url = "https://" + url
} }
cleanHost := strings.TrimPrefix(host, "https://") cleanHost := strings.TrimPrefix(host, "https://")
cleanHost = strings.TrimPrefix(cleanHost, "http://") cleanHost = strings.TrimPrefix(cleanHost, "http://")
cleanHost = strings.TrimSuffix(cleanHost, "/") cleanHost = strings.TrimSuffix(cleanHost, "/")
return cleanHost + "/" + url return cleanHost + "/" + url
} }

View File

@@ -399,6 +399,67 @@
100% { transform: rotate(360deg); } 100% { transform: rotate(360deg); }
} }
/* 切换开关样式 */
.switch-container {
display: flex;
align-items: center;
gap: 0.75rem;
margin-bottom: 1.5rem;
}
.switch {
position: relative;
display: inline-block;
width: 50px;
height: 24px;
}
.switch input {
opacity: 0;
width: 0;
height: 0;
}
.slider {
position: absolute;
cursor: pointer;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: var(--muted);
transition: 0.2s;
border-radius: 24px;
border: 1px solid var(--border);
}
.slider:before {
position: absolute;
content: "";
height: 18px;
width: 18px;
left: 2px;
bottom: 2px;
background-color: white;
transition: 0.2s;
border-radius: 50%;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
input:checked + .slider {
background-color: var(--primary);
}
input:checked + .slider:before {
transform: translateX(26px);
}
.switch-label {
font-weight: 500;
color: var(--foreground);
cursor: pointer;
}
.hidden { .hidden {
display: none; display: none;
} }
@@ -520,7 +581,7 @@
</div> </div>
<div class="feature"> <div class="feature">
<span class="feature-icon">💾</span> <span class="feature-icon">💾</span>
<span>无需打包</span> <span>无需等待</span>
</div> </div>
<div class="feature"> <div class="feature">
<span class="feature-icon">🏗️</span> <span class="feature-icon">🏗️</span>
@@ -559,6 +620,14 @@
</div> </div>
</div> </div>
<div class="switch-container">
<label class="switch">
<input type="checkbox" id="compressedToggle" checked>
<span class="slider"></span>
</label>
<label for="compressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
</div>
<button type="submit" class="btn btn-primary btn-full" id="downloadBtn"> <button type="submit" class="btn btn-primary btn-full" id="downloadBtn">
<span id="downloadText">立即下载</span> <span id="downloadText">立即下载</span>
<span id="downloadLoading" class="loading hidden"></span> <span id="downloadLoading" class="loading hidden"></span>
@@ -573,7 +642,7 @@
<form id="batchForm"> <form id="batchForm">
<div class="form-group"> <div class="form-group">
<label class="form-label" for="imagesTextarea">镜像列表,每行一个,会将多个镜像自动合并,符合官方标准,完全兼容docker load</label> <label class="form-label" for="imagesTextarea">镜像列表每行一个会将多个镜像自动合并符合官方标准兼容docker load</label>
<textarea <textarea
id="imagesTextarea" id="imagesTextarea"
class="textarea" class="textarea"
@@ -595,6 +664,14 @@
</div> </div>
</div> </div>
<div class="switch-container">
<label class="switch">
<input type="checkbox" id="batchCompressedToggle" checked>
<span class="slider"></span>
</label>
<label for="batchCompressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
</div>
<button type="submit" class="btn btn-primary btn-full" id="batchDownloadBtn"> <button type="submit" class="btn btn-primary btn-full" id="batchDownloadBtn">
<span id="batchDownloadText">开始下载</span> <span id="batchDownloadText">开始下载</span>
<span id="batchDownloadLoading" class="loading hidden"></span> <span id="batchDownloadLoading" class="loading hidden"></span>
@@ -651,12 +728,18 @@
} }
} }
function buildDownloadUrl(imageName, platform = '') { function buildDownloadUrl(imageName, platform = '', useCompressed = true) {
const encodedImage = imageName.replace(/\//g, '_'); const encodedImage = imageName.replace(/\//g, '_');
let url = `/api/image/download/${encodedImage}`; let url = `/api/image/download/${encodedImage}`;
const params = new URLSearchParams();
if (platform && platform.trim()) { if (platform && platform.trim()) {
url += `?platform=${encodeURIComponent(platform.trim())}`; params.append('platform', platform.trim());
}
params.append('compressed', useCompressed.toString());
if (params.toString()) {
url += '?' + params.toString();
} }
return url; return url;
@@ -672,11 +755,12 @@
} }
const platform = document.getElementById('platformInput').value.trim(); const platform = document.getElementById('platformInput').value.trim();
const useCompressed = document.getElementById('compressedToggle').checked;
hideStatus('singleStatus'); hideStatus('singleStatus');
setButtonLoading('downloadBtn', 'downloadText', 'downloadLoading', true); setButtonLoading('downloadBtn', 'downloadText', 'downloadLoading', true);
const downloadUrl = buildDownloadUrl(imageName, platform); const downloadUrl = buildDownloadUrl(imageName, platform, useCompressed);
const link = document.createElement('a'); const link = document.createElement('a');
link.href = downloadUrl; link.href = downloadUrl;
@@ -711,9 +795,11 @@
} }
const platform = document.getElementById('batchPlatformInput').value.trim(); const platform = document.getElementById('batchPlatformInput').value.trim();
const useCompressed = document.getElementById('batchCompressedToggle').checked;
const options = { const options = {
images: images images: images,
useCompressedLayers: useCompressed
}; };
if (platform) { if (platform) {

View File

@@ -609,10 +609,10 @@
<div class="card"> <div class="card">
<div class="card-header"> <div class="card-header">
<h2 class="card-title"> <h2 class="card-title">
⚡ 快速生成加速链接 ⚡ 快速转换加速链接
</h2> </h2>
<p class="card-description"> <p class="card-description">
输入GitHub文件或仓库链接自动转换加速链接可以直接在Github域名前面加上本站域名使用。 输入GitHub文件链接自动转换加速链接可以直接在Github文件链接前加上本站域名使用。
</p> </p>
</div> </div>
@@ -622,7 +622,7 @@
type="text" type="text"
class="input" class="input"
id="githubLinkInput" id="githubLinkInput"
placeholder="请输入GitHub链接例如https://github.com/user/repo/releases/download/..." placeholder="请输入GitHub文件链接例如https://github.com/user/repo/releases/download/..."
> >
<button class="button button-primary" id="formatButton"> <button class="button button-primary" id="formatButton">
获取加速链接 获取加速链接
@@ -653,12 +653,12 @@
🐳 Docker 镜像加速 🐳 Docker 镜像加速
</h3> </h3>
<p class="card-description"> <p class="card-description">
支持多种Registry,在镜像名前添加本站域名即可加速下载。 支持多种镜像仓库,在镜像名前添加本站域名即可加速下载。
</p> </p>
</div> </div>
<button class="docker-button" id="dockerButton"> <button class="docker-button" id="dockerButton">
查看 Docker 镜像加速配置 查看 Docker 镜像加速使用说明
</button> </button>
</div> </div>
</div> </div>
@@ -669,23 +669,23 @@
<button class="close-button" id="closeModal">&times;</button> <button class="close-button" id="closeModal">&times;</button>
<div class="modal-header"> <div class="modal-header">
<h2 class="modal-title">Docker 镜像加速</h2> <h2 class="modal-title">Docker 镜像加速</h2>
<p>支持多种Registry,在镜像名前添加本站域名即可加速下载。</p> <p>支持多种镜像仓库,在镜像名前添加本站域名即可加速下载。</p>
</div> </div>
<div class="domain-examples"> <div class="domain-examples">
<strong>Docker Hub 官方镜像:</strong> <strong>Docker 官方镜像:</strong>
docker pull <span class="domain-base"></span>/nginx docker pull <span class="domain-base"></span>/nginx
<strong>Docker Hub 第三方镜像:</strong> <strong>Docker 镜像:</strong>
docker pull <span class="domain-base"></span>/user/image docker pull <span class="domain-base"></span>/user/image
<strong>GitHub Container Registry</strong> <strong>ghcr.io 镜像</strong>
docker pull <span class="domain-base"></span>/ghcr.io/user/image docker pull <span class="domain-base"></span>/ghcr.io/user/image
<strong>Quay.io Registry</strong> <strong>Quay.io 镜像</strong>
docker pull <span class="domain-base"></span>/quay.io/org/image docker pull <span class="domain-base"></span>/quay.io/org/image
<strong>Kubernetes Registry</strong> <strong>K8s 镜像</strong>
docker pull <span class="domain-base"></span>/registry.k8s.io/pause:3.8 docker pull <span class="domain-base"></span>/registry.k8s.io/pause:3.8
</div> </div>
</div> </div>

View File

@@ -778,7 +778,12 @@
</div> </div>
</div> </div>
<div class="tag-list" id="tagList"></div> <div class="tag-list" id="tagList">
<div class="pagination" id="tagPagination" style="display: none;">
<button id="tagPrevPage" disabled>上一页</button>
<button id="tagNextPage" disabled>下一页</button>
</div>
</div>
</div> </div>
<div id="toast"></div> <div id="toast"></div>
@@ -853,6 +858,10 @@
let totalPages = 1; let totalPages = 1;
let currentQuery = ''; let currentQuery = '';
let currentRepo = null; let currentRepo = null;
// 标签分页相关变量
let currentTagPage = 1;
let totalTagPages = 1;
document.getElementById('searchButton').addEventListener('click', () => { document.getElementById('searchButton').addEventListener('click', () => {
currentPage = 1; currentPage = 1;
@@ -884,6 +893,21 @@
showSearchResults(); showSearchResults();
}); });
// 使用事件委托处理分页按钮点击避免DOM重建导致事件丢失
document.addEventListener('click', (e) => {
if (e.target.id === 'tagPrevPage') {
if (currentTagPage > 1) {
currentTagPage--;
loadTagPage();
}
} else if (e.target.id === 'tagNextPage') {
if (currentTagPage < totalTagPages) {
currentTagPage++;
loadTagPage();
}
}
});
function showLoading() { function showLoading() {
document.querySelector('.loading').style.display = 'block'; document.querySelector('.loading').style.display = 'block';
} }
@@ -901,71 +925,135 @@
}, 3000); }, 3000);
} }
function updatePagination() { // 统一分页更新函数(支持搜索和标签分页)
const prevButton = document.getElementById('prevPage'); function updatePagination(config = {}) {
const nextButton = document.getElementById('nextPage'); const {
currentPage: page = currentPage,
totalPages: total = totalPages,
prefix = ''
} = config;
const prevButtonId = prefix ? `${prefix}PrevPage` : 'prevPage';
const nextButtonId = prefix ? `${prefix}NextPage` : 'nextPage';
const paginationId = prefix ? `${prefix}Pagination` : '.pagination';
const prevButton = document.getElementById(prevButtonId);
const nextButton = document.getElementById(nextButtonId);
const paginationDiv = prefix ? document.getElementById(paginationId) : document.querySelector(paginationId);
prevButton.disabled = currentPage <= 1; if (!prevButton || !nextButton || !paginationDiv) {
nextButton.disabled = currentPage >= totalPages; return; // 静默处理,避免控制台警告
}
// 更新按钮状态
prevButton.disabled = page <= 1;
nextButton.disabled = page >= total;
// 更新或创建页面信息
const pageInfoId = prefix ? `${prefix}PageInfo` : 'pageInfo';
let pageInfo = document.getElementById(pageInfoId);
const paginationDiv = document.querySelector('.pagination');
let pageInfo = document.getElementById('pageInfo');
if (!pageInfo) { if (!pageInfo) {
const container = document.createElement('div'); pageInfo = createPageInfo(pageInfoId, prefix, total);
container.id = 'pageInfo'; paginationDiv.insertBefore(pageInfo, nextButton);
container.style.margin = '0 10px';
container.style.display = 'flex';
container.style.alignItems = 'center';
container.style.gap = '10px';
const pageText = document.createElement('span');
pageText.id = 'pageText';
const jumpInput = document.createElement('input');
jumpInput.type = 'number';
jumpInput.min = '1';
jumpInput.id = 'jumpPage';
jumpInput.style.width = '60px';
jumpInput.style.padding = '4px';
jumpInput.style.borderRadius = '4px';
jumpInput.style.border = '1px solid var(--border)';
jumpInput.style.backgroundColor = 'var(--input)';
jumpInput.style.color = 'var(--foreground)';
const jumpButton = document.createElement('button');
jumpButton.textContent = '跳转';
jumpButton.className = 'btn search-button';
jumpButton.style.padding = '4px 8px';
jumpButton.onclick = () => {
const page = parseInt(jumpInput.value);
if (page && page >= 1 && page <= totalPages) {
currentPage = page;
performSearch();
} else {
showToast('请输入有效的页码');
}
};
container.appendChild(pageText);
container.appendChild(jumpInput);
container.appendChild(jumpButton);
paginationDiv.insertBefore(container, nextButton);
pageInfo = container;
} }
const pageText = document.getElementById('pageText'); updatePageInfo(pageInfo, page, total, prefix);
pageText.textContent = `${currentPage} / ${totalPages || 1} 页 共 ${totalPages || 1}`; paginationDiv.style.display = total > 1 ? 'flex' : 'none';
const jumpInput = document.getElementById('jumpPage');
if (jumpInput) {
jumpInput.max = totalPages;
jumpInput.value = currentPage;
}
paginationDiv.style.display = totalPages > 1 ? 'flex' : 'none';
} }
// 创建页面信息元素
function createPageInfo(pageInfoId, prefix, total) {
const container = document.createElement('div');
container.id = pageInfoId;
container.style.cssText = 'margin: 0 10px; display: flex; align-items: center; gap: 10px;';
const pageText = document.createElement('span');
pageText.id = prefix ? `${prefix}PageText` : 'pageText';
const jumpInput = document.createElement('input');
jumpInput.type = 'number';
jumpInput.min = '1';
jumpInput.max = prefix === 'tag' ? total : Math.min(total, 100); // 搜索页面限制100页
jumpInput.id = prefix ? `${prefix}JumpPage` : 'jumpPage';
jumpInput.style.cssText = 'width: 60px; padding: 4px; border-radius: 4px; border: 1px solid var(--border); background-color: var(--input); color: var(--foreground);';
const jumpButton = document.createElement('button');
jumpButton.textContent = '跳转';
jumpButton.className = 'btn search-button';
jumpButton.style.padding = '4px 8px';
jumpButton.onclick = () => handlePageJump(jumpInput, prefix, total);
container.append(pageText, jumpInput, jumpButton);
return container;
}
// 更新页面信息显示
function updatePageInfo(pageInfo, page, total, prefix) {
const pageText = pageInfo.querySelector('span');
const jumpInput = pageInfo.querySelector('input');
// 标签分页显示策略:根据是否确定总页数显示不同格式
const isTagPagination = prefix === 'tag';
const maxDisplayPages = isTagPagination ? total : Math.min(total, 100);
const pageTextContent = isTagPagination
? `${page}` + (total > page ? ` (至少 ${total} 页)` : ` (共 ${total} 页)`)
: `${page} / ${maxDisplayPages} 页 共 ${maxDisplayPages}` + (total > 100 ? ' (最多100页)' : '');
pageText.textContent = pageTextContent;
jumpInput.max = maxDisplayPages;
jumpInput.value = page;
}
// 处理页面跳转
function handlePageJump(jumpInput, prefix, total) {
const inputPage = parseInt(jumpInput.value);
const maxPage = prefix === 'tag' ? total : Math.min(total, 100);
if (!inputPage || inputPage < 1 || inputPage > maxPage) {
const limitText = prefix === 'tag' ? '页码' : '页码 (最多100页)';
showToast(`请输入有效的${limitText}`);
return;
}
if (prefix === 'tag') {
currentTagPage = inputPage;
loadTagPage();
} else {
currentPage = inputPage;
performSearch();
}
}
// 统一仓库信息处理
function parseRepositoryInfo(repo) {
const namespace = repo.namespace || (repo.is_official ? 'library' : '');
let name = repo.name || repo.repo_name || '';
// 清理名称,确保不包含命名空间前缀
if (name.includes('/')) {
const parts = name.split('/');
name = parts[parts.length - 1];
}
const cleanName = name.replace(/^library\//, '');
const fullRepoName = repo.is_official ? cleanName : `${namespace}/${cleanName}`;
return {
namespace,
name,
cleanName,
fullRepoName
};
}
// 分页更新函数
const updateSearchPagination = () => updatePagination();
const updateTagPagination = () => updatePagination({
currentPage: currentTagPage,
totalPages: totalTagPages,
prefix: 'tag'
});
function showSearchResults() { function showSearchResults() {
document.querySelector('.search-results').style.display = 'block'; document.querySelector('.search-results').style.display = 'block';
document.querySelector('.tag-list').style.display = 'none'; document.querySelector('.tag-list').style.display = 'none';
@@ -1006,7 +1094,7 @@
throw new Error(data.error || '搜索请求失败'); throw new Error(data.error || '搜索请求失败');
} }
totalPages = Math.ceil(data.count / 25); totalPages = Math.min(Math.ceil(data.count / 25), 100);
updatePagination(); updatePagination();
displayResults(data.results, targetRepo); displayResults(data.results, targetRepo);
@@ -1108,23 +1196,58 @@
}); });
} }
// 内存管理
async function loadTags(namespace, name) { async function loadTags(namespace, name) {
currentTagPage = 1;
await loadTagPage(namespace, name);
}
async function loadTagPage(namespace = null, name = null) {
showLoading(); showLoading();
try { try {
if (!namespace || !name) { // 如果传入了新的namespace和name更新currentRepo
if (namespace && name) {
// 清理旧数据,防止内存泄露
cleanupOldTagData();
}
// 获取当前仓库信息
const repoInfo = parseRepositoryInfo(currentRepo);
const currentNamespace = namespace || repoInfo.namespace;
const currentName = name || repoInfo.name;
// 调试日志
console.log(`loadTagPage: namespace=${currentNamespace}, name=${currentName}, page=${currentTagPage}`);
if (!currentNamespace || !currentName) {
showToast('命名空间和镜像名称不能为空'); showToast('命名空间和镜像名称不能为空');
return; return;
} }
const response = await fetch(`/tags/${encodeURIComponent(namespace)}/${encodeURIComponent(name)}`); const response = await fetch(`/tags/${encodeURIComponent(currentNamespace)}/${encodeURIComponent(currentName)}?page=${currentTagPage}&page_size=100`);
if (!response.ok) { if (!response.ok) {
const errorText = await response.text(); const errorText = await response.text();
throw new Error(errorText || '获取标签信息失败'); throw new Error(errorText || '获取标签信息失败');
} }
const data = await response.json(); const data = await response.json();
displayTags(data);
showTagList(); // 改进的总页数计算:使用更准确的分页策略
if (data.has_more) {
// 如果还有更多页面,至少有当前页+1页但可能更多
totalTagPages = Math.max(currentTagPage + 1, totalTagPages);
} else {
// 如果没有更多页面,当前页就是最后一页
totalTagPages = currentTagPage;
}
displayTags(data.tags, data.has_more);
updateTagPagination();
if (namespace && name) {
showTagList();
}
} catch (error) { } catch (error) {
console.error('加载标签错误:', error); console.error('加载标签错误:', error);
showToast(error.message || '获取标签信息失败,请稍后重试'); showToast(error.message || '获取标签信息失败,请稍后重试');
@@ -1133,12 +1256,24 @@
} }
} }
function displayTags(tags) { function cleanupOldTagData() {
// 清理全局变量,释放内存
if (window.currentPageTags) {
window.currentPageTags.length = 0;
window.currentPageTags = null;
}
// 清理DOM缓存
const tagsContainer = document.getElementById('tagsContainer');
if (tagsContainer) {
tagsContainer.innerHTML = '';
}
}
function displayTags(tags, hasMore = false) {
const tagList = document.getElementById('tagList'); const tagList = document.getElementById('tagList');
const namespace = currentRepo.namespace || (currentRepo.is_official ? 'library' : ''); const repoInfo = parseRepositoryInfo(currentRepo);
const name = currentRepo.name || currentRepo.repo_name || ''; const { fullRepoName } = repoInfo;
const cleanName = name.replace(/^library\//, '');
const fullRepoName = currentRepo.is_official ? cleanName : `${namespace}/${cleanName}`;
let header = ` let header = `
<div class="tag-header"> <div class="tag-header">
@@ -1165,22 +1300,60 @@
<button class="tag-search-clear" onclick="clearTagSearch()">×</button> <button class="tag-search-clear" onclick="clearTagSearch()">×</button>
</div> </div>
<div id="tagsContainer"></div> <div id="tagsContainer"></div>
<div class="pagination" id="tagPagination" style="display: none;">
<button id="tagPrevPage" disabled>上一页</button>
<button id="tagNextPage" disabled>下一页</button>
</div>
`; `;
tagList.innerHTML = header; tagList.innerHTML = header;
window.allTags = tags; // 存储当前页标签数据
window.currentPageTags = tags;
renderFilteredTags(tags); renderFilteredTags(tags);
} }
function renderFilteredTags(filteredTags) { function renderFilteredTags(filteredTags) {
const tagsContainer = document.getElementById('tagsContainer'); const tagsContainer = document.getElementById('tagsContainer');
const namespace = currentRepo.namespace || (currentRepo.is_official ? 'library' : ''); const repoInfo = parseRepositoryInfo(currentRepo);
const name = currentRepo.name || currentRepo.repo_name || ''; const { fullRepoName } = repoInfo;
const cleanName = name.replace(/^library\//, '');
const fullRepoName = currentRepo.is_official ? cleanName : `${namespace}/${cleanName}`;
let tagsHtml = filteredTags.map(tag => { if (filteredTags.length === 0) {
tagsContainer.innerHTML = '<div class="text-center" style="padding: 20px;">未找到匹配的标签</div>';
return;
}
// 渐进式渲染:分批处理大数据集
const BATCH_SIZE = 50;
if (filteredTags.length <= BATCH_SIZE) {
// 小数据集:直接渲染
renderTagsBatch(filteredTags, fullRepoName, tagsContainer, true);
} else {
// 大数据集:分批渲染
tagsContainer.innerHTML = ''; // 清空容器
let currentBatch = 0;
function renderNextBatch() {
const start = currentBatch * BATCH_SIZE;
const end = Math.min(start + BATCH_SIZE, filteredTags.length);
const batch = filteredTags.slice(start, end);
renderTagsBatch(batch, fullRepoName, tagsContainer, false);
currentBatch++;
if (end < filteredTags.length) {
// 使用requestAnimationFrame确保UI响应性
requestAnimationFrame(renderNextBatch);
}
}
renderNextBatch();
}
}
function renderTagsBatch(tags, fullRepoName, container, replaceContent = false) {
const tagsHtml = tags.map(tag => {
const vulnIndicators = Object.entries(tag.vulnerabilities || {}) const vulnIndicators = Object.entries(tag.vulnerabilities || {})
.map(([level, count]) => count > 0 ? `<span class="vulnerability-dot vulnerability-${level.toLowerCase()}" title="${level}: ${count}"></span>` : '') .map(([level, count]) => count > 0 ? `<span class="vulnerability-dot vulnerability-${level.toLowerCase()}" title="${level}: ${count}"></span>` : '')
.join(''); .join('');
@@ -1212,23 +1385,23 @@
`; `;
}).join(''); }).join('');
if (filteredTags.length === 0) { if (replaceContent) {
tagsHtml = '<div class="text-center" style="padding: 20px;">未找到匹配的标签</div>'; container.innerHTML = tagsHtml;
} else {
container.insertAdjacentHTML('beforeend', tagsHtml);
} }
tagsContainer.innerHTML = tagsHtml;
} }
function filterTags(searchText) { function filterTags(searchText) {
if (!window.allTags) return; if (!window.currentPageTags) return;
const searchLower = searchText.toLowerCase(); const searchLower = searchText.toLowerCase();
let filteredTags; let filteredTags;
if (!searchText) { if (!searchText) {
filteredTags = window.allTags; filteredTags = window.currentPageTags;
} else { } else {
const scoredTags = window.allTags.map(tag => { const scoredTags = window.currentPageTags.map(tag => {
const name = tag.name.toLowerCase(); const name = tag.name.toLowerCase();
let score = 0; let score = 0;
@@ -1263,6 +1436,8 @@
} }
} }
function copyToClipboard(text) { function copyToClipboard(text) {
navigator.clipboard.writeText(text).then(() => { navigator.clipboard.writeText(text).then(() => {
showToast('已复制到剪贴板'); showToast('已复制到剪贴板');

View File

@@ -1,299 +1,301 @@
package main package main
import ( import (
"fmt" "fmt"
"net" "net"
"strings" "strings"
"sync" "sync"
"time" "time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"golang.org/x/time/rate" "golang.org/x/time/rate"
) )
const ( const (
// 清理间隔 // 清理间隔
CleanupInterval = 10 * time.Minute CleanupInterval = 10 * time.Minute
MaxIPCacheSize = 10000 MaxIPCacheSize = 10000
) )
// IPRateLimiter IP限流器结构体 // IPRateLimiter IP限流器结构体
type IPRateLimiter struct { type IPRateLimiter struct {
ips map[string]*rateLimiterEntry // IP到限流器的映射 ips map[string]*rateLimiterEntry // IP到限流器的映射
mu *sync.RWMutex // 读写锁,保证并发安全 mu *sync.RWMutex // 读写锁,保证并发安全
r rate.Limit // 速率限制(每秒允许的请求数) r rate.Limit // 速率限制(每秒允许的请求数)
b int // 令牌桶容量(突发请求数) b int // 令牌桶容量(突发请求数)
whitelist []*net.IPNet // 白名单IP段 whitelist []*net.IPNet // 白名单IP段
blacklist []*net.IPNet // 黑名单IP段 blacklist []*net.IPNet // 黑名单IP段
} }
// rateLimiterEntry 限流器条目 // rateLimiterEntry 限流器条目
type rateLimiterEntry struct { type rateLimiterEntry struct {
limiter *rate.Limiter limiter *rate.Limiter
lastAccess time.Time lastAccess time.Time
} }
// initGlobalLimiter 初始化全局限流器 // initGlobalLimiter 初始化全局限流器
func initGlobalLimiter() *IPRateLimiter { func initGlobalLimiter() *IPRateLimiter {
cfg := GetConfig() cfg := GetConfig()
whitelist := make([]*net.IPNet, 0, len(cfg.Security.WhiteList)) whitelist := make([]*net.IPNet, 0, len(cfg.Security.WhiteList))
for _, item := range cfg.Security.WhiteList { for _, item := range cfg.Security.WhiteList {
if item = strings.TrimSpace(item); item != "" { if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") { if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式 item = item + "/32" // 单个IP转为CIDR格式
} }
_, ipnet, err := net.ParseCIDR(item) _, ipnet, err := net.ParseCIDR(item)
if err == nil { if err == nil {
whitelist = append(whitelist, ipnet) whitelist = append(whitelist, ipnet)
} else { } else {
fmt.Printf("警告: 无效的白名单IP格式: %s\n", item) fmt.Printf("警告: 无效的白名单IP格式: %s\n", item)
} }
} }
} }
// 解析黑名单IP段 // 解析黑名单IP段
blacklist := make([]*net.IPNet, 0, len(cfg.Security.BlackList)) blacklist := make([]*net.IPNet, 0, len(cfg.Security.BlackList))
for _, item := range cfg.Security.BlackList { for _, item := range cfg.Security.BlackList {
if item = strings.TrimSpace(item); item != "" { if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") { if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式 item = item + "/32" // 单个IP转为CIDR格式
} }
_, ipnet, err := net.ParseCIDR(item) _, ipnet, err := net.ParseCIDR(item)
if err == nil { if err == nil {
blacklist = append(blacklist, ipnet) blacklist = append(blacklist, ipnet)
} else { } else {
fmt.Printf("警告: 无效的黑名单IP格式: %s\n", item) fmt.Printf("警告: 无效的黑名单IP格式: %s\n", item)
} }
} }
} }
// 计算速率:将 "每N小时X个请求" 转换为 "每秒Y个请求" // 计算速率:将 "每N小时X个请求" 转换为 "每秒Y个请求"
ratePerSecond := rate.Limit(float64(cfg.RateLimit.RequestLimit) / (cfg.RateLimit.PeriodHours * 3600)) ratePerSecond := rate.Limit(float64(cfg.RateLimit.RequestLimit) / (cfg.RateLimit.PeriodHours * 3600))
burstSize := cfg.RateLimit.RequestLimit burstSize := cfg.RateLimit.RequestLimit
if burstSize < 1 { if burstSize < 1 {
burstSize = 1 burstSize = 1
} }
limiter := &IPRateLimiter{ limiter := &IPRateLimiter{
ips: make(map[string]*rateLimiterEntry), ips: make(map[string]*rateLimiterEntry),
mu: &sync.RWMutex{}, mu: &sync.RWMutex{},
r: ratePerSecond, r: ratePerSecond,
b: burstSize, b: burstSize,
whitelist: whitelist, whitelist: whitelist,
blacklist: blacklist, blacklist: blacklist,
} }
// 启动定期清理goroutine // 启动定期清理goroutine
go limiter.cleanupRoutine() go limiter.cleanupRoutine()
return limiter return limiter
} }
// initLimiter 初始化限流器 // initLimiter 初始化限流器
func initLimiter() { func initLimiter() {
globalLimiter = initGlobalLimiter() globalLimiter = initGlobalLimiter()
} }
// cleanupRoutine 定期清理过期的限流器 // cleanupRoutine 定期清理过期的限流器
func (i *IPRateLimiter) cleanupRoutine() { func (i *IPRateLimiter) cleanupRoutine() {
ticker := time.NewTicker(CleanupInterval) ticker := time.NewTicker(CleanupInterval)
defer ticker.Stop() defer ticker.Stop()
for range ticker.C { for range ticker.C {
now := time.Now() now := time.Now()
expired := make([]string, 0) expired := make([]string, 0)
// 查找过期的条目 // 查找过期的条目
i.mu.RLock() i.mu.RLock()
for ip, entry := range i.ips { for ip, entry := range i.ips {
// 如果最后访问时间超过1小时认为过期 // 如果最后访问时间超过1小时认为过期
if now.Sub(entry.lastAccess) > 1*time.Hour { if now.Sub(entry.lastAccess) > 1*time.Hour {
expired = append(expired, ip) expired = append(expired, ip)
} }
} }
i.mu.RUnlock() i.mu.RUnlock()
// 如果有过期条目或者缓存过大,进行清理 // 如果有过期条目或者缓存过大,进行清理
if len(expired) > 0 || len(i.ips) > MaxIPCacheSize { if len(expired) > 0 || len(i.ips) > MaxIPCacheSize {
i.mu.Lock() i.mu.Lock()
// 删除过期条目 // 删除过期条目
for _, ip := range expired { for _, ip := range expired {
delete(i.ips, ip) delete(i.ips, ip)
} }
// 如果缓存仍然过大,全部清理 // 如果缓存仍然过大,全部清理
if len(i.ips) > MaxIPCacheSize { if len(i.ips) > MaxIPCacheSize {
i.ips = make(map[string]*rateLimiterEntry) i.ips = make(map[string]*rateLimiterEntry)
} }
i.mu.Unlock() i.mu.Unlock()
} }
} }
} }
// extractIPFromAddress 从地址中提取纯IP,去除端口号 // extractIPFromAddress 从地址中提取纯IP
func extractIPFromAddress(address string) string { func extractIPFromAddress(address string) string {
// 处理IPv6地址 [::1]:8080 格式 if host, _, err := net.SplitHostPort(address); err == nil {
if strings.HasPrefix(address, "[") { return host
if endIndex := strings.Index(address, "]"); endIndex != -1 { }
return address[1:endIndex] return address
} }
}
// normalizeIPForRateLimit 标准化IP地址用于限流IPv4保持不变IPv6标准化为/64网段
// 处理IPv4地址 192.168.1.1:8080 格式 func normalizeIPForRateLimit(ipStr string) string {
if lastColon := strings.LastIndex(address, ":"); lastColon != -1 { ip := net.ParseIP(ipStr)
return address[:lastColon] if ip == nil {
} return ipStr // 解析失败,返回原值
}
return address
} if ip.To4() != nil {
return ipStr // IPv4保持不变
// isIPInCIDRList 检查IP是否在CIDR列表中 }
func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
// 先提取纯IP地址 // IPv6标准化为 /64 网段
cleanIP := extractIPFromAddress(ip) ipv6 := ip.To16()
parsedIP := net.ParseIP(cleanIP) for i := 8; i < 16; i++ {
if parsedIP == nil { ipv6[i] = 0 // 清零后64位
return false }
} return ipv6.String() + "/64"
}
for _, cidr := range cidrList {
if cidr.Contains(parsedIP) { // isIPInCIDRList 检查IP是否在CIDR列表中
return true func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
} // 先提取纯IP地址
} cleanIP := extractIPFromAddress(ip)
return false parsedIP := net.ParseIP(cleanIP)
} if parsedIP == nil {
return false
// GetLimiter 获取指定IP的限流器同时返回是否允许访问 }
func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
// 提取纯IP地址 for _, cidr := range cidrList {
cleanIP := extractIPFromAddress(ip) if cidr.Contains(parsedIP) {
return true
// 检查是否在黑名单中 }
if isIPInCIDRList(cleanIP, i.blacklist) { }
return nil, false return false
} }
// 检查是否在白名单中 // GetLimiter 获取指定IP的限流器同时返回是否允许访问
if isIPInCIDRList(cleanIP, i.whitelist) { func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
return rate.NewLimiter(rate.Inf, i.b), true // 提取纯IP地址
} cleanIP := extractIPFromAddress(ip)
now := time.Now() // 检查是否在黑名单中
if isIPInCIDRList(cleanIP, i.blacklist) {
i.mu.RLock() return nil, false
entry, exists := i.ips[cleanIP] }
i.mu.RUnlock()
// 检查是否在白名单中
if exists { if isIPInCIDRList(cleanIP, i.whitelist) {
i.mu.Lock() return rate.NewLimiter(rate.Inf, i.b), true
if entry, stillExists := i.ips[cleanIP]; stillExists { }
entry.lastAccess = now
i.mu.Unlock() // 标准化IP用于限流IPv4保持不变IPv6标准化为/64网段
return entry.limiter, true normalizedIP := normalizeIPForRateLimit(cleanIP)
}
i.mu.Unlock() now := time.Now()
}
i.mu.RLock()
i.mu.Lock() entry, exists := i.ips[normalizedIP]
if entry, exists := i.ips[cleanIP]; exists { i.mu.RUnlock()
entry.lastAccess = now
i.mu.Unlock() if exists {
return entry.limiter, true i.mu.Lock()
} if entry, stillExists := i.ips[normalizedIP]; stillExists {
entry.lastAccess = now
entry = &rateLimiterEntry{ i.mu.Unlock()
limiter: rate.NewLimiter(i.r, i.b), return entry.limiter, true
lastAccess: now, }
} i.mu.Unlock()
i.ips[cleanIP] = entry }
i.mu.Unlock()
i.mu.Lock()
return entry.limiter, true if entry, exists := i.ips[normalizedIP]; exists {
} entry.lastAccess = now
i.mu.Unlock()
// RateLimitMiddleware 速率限制中间件 return entry.limiter, true
func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc { }
return func(c *gin.Context) {
// 获取客户端真实IP entry = &rateLimiterEntry{
var ip string limiter: rate.NewLimiter(i.r, i.b),
lastAccess: now,
// 优先尝试从请求头获取真实IP }
if forwarded := c.GetHeader("X-Forwarded-For"); forwarded != "" { i.ips[normalizedIP] = entry
// X-Forwarded-For可能包含多个IP取第一个 i.mu.Unlock()
ips := strings.Split(forwarded, ",")
ip = strings.TrimSpace(ips[0]) return entry.limiter, true
} else if realIP := c.GetHeader("X-Real-IP"); realIP != "" { }
// 如果有X-Real-IP头
ip = realIP // RateLimitMiddleware 速率限制中间件
} else if remoteIP := c.GetHeader("X-Original-Forwarded-For"); remoteIP != "" { func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
// 某些代理可能使用此头 return func(c *gin.Context) {
ips := strings.Split(remoteIP, ",") // 静态文件豁免:跳过限流检查
ip = strings.TrimSpace(ips[0]) path := c.Request.URL.Path
} else { if path == "/" || path == "/favicon.ico" || path == "/images.html" || path == "/search.html" ||
// 回退到ClientIP方法 strings.HasPrefix(path, "/public/") {
ip = c.ClientIP() c.Next()
} return
}
// 提取纯IP地址去除端口号
cleanIP := extractIPFromAddress(ip) // 获取客户端真实IP
var ip string
// 日志记录请求IP和头信息
fmt.Printf("请求IP: %s (去除端口后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n", // 优先尝试从请求头获取真实IP
ip, if forwarded := c.GetHeader("X-Forwarded-For"); forwarded != "" {
cleanIP, // X-Forwarded-For可能包含多个IP取第一个
c.GetHeader("X-Forwarded-For"), ips := strings.Split(forwarded, ",")
c.GetHeader("X-Real-IP")) ip = strings.TrimSpace(ips[0])
} else if realIP := c.GetHeader("X-Real-IP"); realIP != "" {
// 获取限流器并检查是否允许访问 // 如果有X-Real-IP头
ipLimiter, allowed := limiter.GetLimiter(cleanIP) ip = realIP
} else if remoteIP := c.GetHeader("X-Original-Forwarded-For"); remoteIP != "" {
// 如果IP在黑名单中 // 某些代理可能使用此头
if !allowed { ips := strings.Split(remoteIP, ",")
c.JSON(403, gin.H{ ip = strings.TrimSpace(ips[0])
"error": "您已被限制访问", } else {
}) // 回退到ClientIP方法
c.Abort() ip = c.ClientIP()
return }
}
// 提取纯IP地址去除可能存在的端口
// 智能限流判断:检查是否应该跳过限流计数 cleanIP := extractIPFromAddress(ip)
shouldSkip := smartLimiter.ShouldSkipRateLimit(cleanIP, c.Request.URL.Path)
// 日志记录请求IP和头信息
// 只有在不跳过的情况下才检查限流 normalizedIP := normalizeIPForRateLimit(cleanIP)
if !shouldSkip && !ipLimiter.Allow() { if cleanIP != normalizedIP {
c.JSON(429, gin.H{ fmt.Printf("请求IP: %s (提纯后: %s, 限流段: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
"error": "请求频率过快,暂时限制访问", ip, cleanIP, normalizedIP,
}) c.GetHeader("X-Forwarded-For"),
c.Abort() c.GetHeader("X-Real-IP"))
return } else {
} fmt.Printf("请求IP: %s (提纯后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
ip, cleanIP,
c.Next() c.GetHeader("X-Forwarded-For"),
} c.GetHeader("X-Real-IP"))
} }
// ApplyRateLimit 应用限流到特定路由 // 获取限流器并检查是否允许访问
func ApplyRateLimit(router *gin.Engine, path string, method string, handler gin.HandlerFunc) { ipLimiter, allowed := limiter.GetLimiter(cleanIP)
// 使用全局限流器
limiter := globalLimiter // 如果IP在黑名单中
if limiter == nil { if !allowed {
limiter = initGlobalLimiter() c.JSON(403, gin.H{
} "error": "您已被限制访问",
})
// 根据HTTP方法应用限流 c.Abort()
switch method { return
case "GET": }
router.GET(path, RateLimitMiddleware(limiter), handler)
case "POST": // 检查限流
router.POST(path, RateLimitMiddleware(limiter), handler) if !ipLimiter.Allow() {
case "PUT": c.JSON(429, gin.H{
router.PUT(path, RateLimitMiddleware(limiter), handler) "error": "请求频率过快,暂时限制访问",
case "DELETE": })
router.DELETE(path, RateLimitMiddleware(limiter), handler) c.Abort()
default: return
router.Any(path, RateLimitMiddleware(limiter), handler) }
}
} c.Next()
}
}

View File

@@ -66,14 +66,21 @@ type Image struct {
Size int64 `json:"size"` Size int64 `json:"size"`
} }
// TagPageResult 分页标签结果
type TagPageResult struct {
Tags []TagInfo `json:"tags"`
HasMore bool `json:"has_more"`
}
type cacheEntry struct { type cacheEntry struct {
data interface{} data interface{}
timestamp time.Time expiresAt time.Time // 存储过期时间
} }
const ( const (
maxCacheSize = 1000 // 最大缓存条目数 maxCacheSize = 1000 // 最大缓存条目数
cacheTTL = 30 * time.Minute maxPaginationCache = 200 // 分页缓存最大条目数
cacheTTL = 30 * time.Minute
) )
type Cache struct { type Cache struct {
@@ -98,7 +105,8 @@ func (c *Cache) Get(key string) (interface{}, bool) {
return nil, false return nil, false
} }
if time.Since(entry.timestamp) > cacheTTL { // 比较过期时间
if time.Now().After(entry.expiresAt) {
c.mu.Lock() c.mu.Lock()
delete(c.data, key) delete(c.data, key)
c.mu.Unlock() c.mu.Unlock()
@@ -109,40 +117,36 @@ func (c *Cache) Get(key string) (interface{}, bool) {
} }
func (c *Cache) Set(key string, data interface{}) { func (c *Cache) Set(key string, data interface{}) {
c.SetWithTTL(key, data, cacheTTL)
}
func (c *Cache) SetWithTTL(key string, data interface{}, ttl time.Duration) {
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
now := time.Now() // 惰性清理:仅在容量超限时清理过期项
for k, v := range c.data {
if now.Sub(v.timestamp) > cacheTTL {
delete(c.data, k)
}
}
if len(c.data) >= c.maxSize { if len(c.data) >= c.maxSize {
toDelete := len(c.data) / 4 c.cleanupExpiredLocked()
for k := range c.data {
if toDelete <= 0 {
break
}
delete(c.data, k)
toDelete--
}
} }
// 计算过期时间
c.data[key] = cacheEntry{ c.data[key] = cacheEntry{
data: data, data: data,
timestamp: now, expiresAt: time.Now().Add(ttl),
} }
} }
func (c *Cache) Cleanup() { func (c *Cache) Cleanup() {
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
c.cleanupExpiredLocked()
}
// cleanupExpiredLocked 清理过期缓存(需要已持有锁)
func (c *Cache) cleanupExpiredLocked() {
now := time.Now() now := time.Now()
for key, entry := range c.data { for key, entry := range c.data {
if now.Sub(entry.timestamp) > cacheTTL { if now.After(entry.expiresAt) {
delete(c.data, key) delete(c.data, key)
} }
} }
@@ -152,6 +156,8 @@ func (c *Cache) Cleanup() {
func init() { func init() {
go func() { go func() {
ticker := time.NewTicker(5 * time.Minute) ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop() // 确保ticker资源释放
for range ticker.C { for range ticker.C {
searchCache.Cleanup() searchCache.Cleanup()
} }
@@ -214,8 +220,43 @@ func filterSearchResults(results []Repository, query string) []Repository {
return filtered return filtered
} }
// normalizeRepository 统一规范化仓库信息(消除重复逻辑)
func normalizeRepository(repo *Repository) {
if repo.IsOfficial {
repo.Namespace = "library"
if !strings.Contains(repo.Name, "/") {
repo.Name = "library/" + repo.Name
}
} else {
// 处理用户仓库设置命名空间但保持Name为纯仓库名
if repo.Namespace == "" && repo.RepoOwner != "" {
repo.Namespace = repo.RepoOwner
}
// 如果Name包含斜杠提取纯仓库名
if strings.Contains(repo.Name, "/") {
parts := strings.Split(repo.Name, "/")
if len(parts) > 1 {
if repo.Namespace == "" {
repo.Namespace = parts[0]
}
repo.Name = parts[len(parts)-1] // 取最后部分作为仓库名
}
}
}
}
// searchDockerHub 搜索镜像 // searchDockerHub 搜索镜像
func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*SearchResult, error) { func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*SearchResult, error) {
return searchDockerHubWithDepth(ctx, query, page, pageSize, 0)
}
// searchDockerHubWithDepth 搜索镜像(带递归深度控制)
func searchDockerHubWithDepth(ctx context.Context, query string, page, pageSize int, depth int) (*SearchResult, error) {
// 防止无限递归最多允许1次递归调用
if depth > 1 {
return nil, fmt.Errorf("搜索请求过于复杂,请尝试更具体的关键词")
}
cacheKey := fmt.Sprintf("search:%s:%d:%d", query, page, pageSize) cacheKey := fmt.Sprintf("search:%s:%d:%d", query, page, pageSize)
// 尝试从缓存获取 // 尝试从缓存获取
@@ -264,11 +305,7 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
if err != nil { if err != nil {
return nil, fmt.Errorf("请求Docker Hub API失败: %v", err) return nil, fmt.Errorf("请求Docker Hub API失败: %v", err)
} }
defer func() { defer safeCloseResponseBody(resp.Body, "搜索响应体")
if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭搜索响应体失败: %v\n", err)
}
}()
body, err := io.ReadAll(resp.Body) body, err := io.ReadAll(resp.Body)
if err != nil { if err != nil {
@@ -281,8 +318,8 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
return nil, fmt.Errorf("请求过于频繁,请稍后重试") return nil, fmt.Errorf("请求过于频繁,请稍后重试")
case http.StatusNotFound: case http.StatusNotFound:
if isUserRepo && namespace != "" { if isUserRepo && namespace != "" {
// 如果用户仓库搜索失败,尝试普通搜索 // 如果用户仓库搜索失败,尝试普通搜索(递归调用)
return searchDockerHub(ctx, repoName, page, pageSize) return searchDockerHubWithDepth(ctx, repoName, page, pageSize, depth+1)
} }
return nil, fmt.Errorf("未找到相关镜像") return nil, fmt.Errorf("未找到相关镜像")
case http.StatusBadGateway, http.StatusServiceUnavailable: case http.StatusBadGateway, http.StatusServiceUnavailable:
@@ -318,18 +355,16 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
for _, repo := range userRepos.Results { for _, repo := range userRepos.Results {
// 如果指定了仓库名,只保留匹配的结果 // 如果指定了仓库名,只保留匹配的结果
if repoName == "" || strings.Contains(strings.ToLower(repo.Name), strings.ToLower(repoName)) { if repoName == "" || strings.Contains(strings.ToLower(repo.Name), strings.ToLower(repoName)) {
// 确保设置正确的命名空间和名称 // 设置命名空间并使用统一的规范化函数
repo.Namespace = namespace repo.Namespace = namespace
if !strings.Contains(repo.Name, "/") { normalizeRepository(&repo)
repo.Name = fmt.Sprintf("%s/%s", namespace, repo.Name)
}
result.Results = append(result.Results, repo) result.Results = append(result.Results, repo)
} }
} }
// 如果没有找到结果,尝试普通搜索 // 如果没有找到结果,尝试普通搜索(递归调用)
if len(result.Results) == 0 { if len(result.Results) == 0 {
return searchDockerHub(ctx, repoName, page, pageSize) return searchDockerHubWithDepth(ctx, repoName, page, pageSize, depth+1)
} }
result.Count = len(result.Results) result.Count = len(result.Results)
@@ -340,23 +375,9 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
return nil, fmt.Errorf("解析响应失败: %v", err) return nil, fmt.Errorf("解析响应失败: %v", err)
} }
// 处理搜索结果 // 处理搜索结果:使用统一的规范化函数
for i := range result.Results { for i := range result.Results {
if result.Results[i].IsOfficial { normalizeRepository(&result.Results[i])
if !strings.Contains(result.Results[i].Name, "/") {
result.Results[i].Name = "library/" + result.Results[i].Name
}
result.Results[i].Namespace = "library"
} else {
parts := strings.Split(result.Results[i].Name, "/")
if len(parts) > 1 {
result.Results[i].Namespace = parts[0]
result.Results[i].Name = parts[1]
} else if result.Results[i].RepoOwner != "" {
result.Results[i].Namespace = result.Results[i].RepoOwner
result.Results[i].Name = fmt.Sprintf("%s/%s", result.Results[i].RepoOwner, result.Results[i].Name)
}
}
} }
// 如果是用户/仓库搜索,过滤结果 // 如果是用户/仓库搜索,过滤结果
@@ -394,61 +415,150 @@ func isRetryableError(err error) bool {
return false return false
} }
// getRepositoryTags 获取仓库标签信息 // getRepositoryTags 获取仓库标签信息(支持分页)
func getRepositoryTags(ctx context.Context, namespace, name string) ([]TagInfo, error) { func getRepositoryTags(ctx context.Context, namespace, name string, page, pageSize int) ([]TagInfo, bool, error) {
if namespace == "" || name == "" { if namespace == "" || name == "" {
return nil, fmt.Errorf("无效输入:命名空间和名称不能为空") return nil, false, fmt.Errorf("无效输入:命名空间和名称不能为空")
} }
cacheKey := fmt.Sprintf("tags:%s:%s", namespace, name) // 默认参数
if page <= 0 {
page = 1
}
if pageSize <= 0 || pageSize > 100 {
pageSize = 100
}
// 分页缓存key
cacheKey := fmt.Sprintf("tags:%s:%s:page_%d", namespace, name, page)
if cached, ok := searchCache.Get(cacheKey); ok { if cached, ok := searchCache.Get(cacheKey); ok {
return cached.([]TagInfo), nil result := cached.(TagPageResult)
return result.Tags, result.HasMore, nil
} }
// 构建API URL // 构建API URL
baseURL := fmt.Sprintf("https://registry.hub.docker.com/v2/repositories/%s/%s/tags", namespace, name) baseURL := fmt.Sprintf("https://registry.hub.docker.com/v2/repositories/%s/%s/tags", namespace, name)
params := url.Values{} params := url.Values{}
params.Set("page_size", "100") params.Set("page", fmt.Sprintf("%d", page))
params.Set("page_size", fmt.Sprintf("%d", pageSize))
params.Set("ordering", "last_updated") params.Set("ordering", "last_updated")
fullURL := baseURL + "?" + params.Encode() fullURL := baseURL + "?" + params.Encode()
// 使用统一的搜索HTTP客户端 // 获取当前页数据
resp, err := GetSearchHTTPClient().Get(fullURL) pageResult, err := fetchTagPage(ctx, fullURL, 3)
if err != nil { if err != nil {
return nil, fmt.Errorf("发送请求失败: %v", err) return nil, false, fmt.Errorf("获取标签失败: %v", err)
} }
defer func() {
if err := resp.Body.Close(); err != nil { hasMore := pageResult.Next != ""
fmt.Printf("关闭搜索响应体失败: %v\n", err)
// 缓存结果(分页缓存时间较短)
result := TagPageResult{Tags: pageResult.Results, HasMore: hasMore}
searchCache.SetWithTTL(cacheKey, result, 30*time.Minute)
return pageResult.Results, hasMore, nil
}
// fetchTagPage 获取单页标签数据,带重试机制
func fetchTagPage(ctx context.Context, url string, maxRetries int) (*struct {
Count int `json:"count"`
Next string `json:"next"`
Previous string `json:"previous"`
Results []TagInfo `json:"results"`
}, error) {
var lastErr error
for retry := 0; retry < maxRetries; retry++ {
if retry > 0 {
// 重试前等待一段时间
time.Sleep(time.Duration(retry) * 500 * time.Millisecond)
} }
}()
// 读取响应体 resp, err := GetSearchHTTPClient().Get(url)
body, err := io.ReadAll(resp.Body) if err != nil {
if err != nil { lastErr = err
return nil, fmt.Errorf("读取响应失败: %v", err) if isRetryableError(err) && retry < maxRetries-1 {
} continue
}
return nil, fmt.Errorf("发送请求失败: %v", err)
}
// 检查响应状态码 // 读取响应体立即关闭避免defer在循环中累积
if resp.StatusCode != http.StatusOK { body, err := func() ([]byte, error) {
return nil, fmt.Errorf("请求失败: 状态码=%d, 响应=%s", resp.StatusCode, string(body)) defer safeCloseResponseBody(resp.Body, "标签响应体")
} return io.ReadAll(resp.Body)
}()
if err != nil {
lastErr = err
if retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("读取响应失败: %v", err)
}
// 解析响应 // 检查响应状态码
var result struct { if resp.StatusCode != http.StatusOK {
Count int `json:"count"` lastErr = fmt.Errorf("状态码=%d, 响应=%s", resp.StatusCode, string(body))
Next string `json:"next"` // 4xx错误通常不需要重试
Previous string `json:"previous"` if resp.StatusCode >= 400 && resp.StatusCode < 500 && resp.StatusCode != 429 {
Results []TagInfo `json:"results"` return nil, fmt.Errorf("请求失败: %v", lastErr)
} }
if err := json.Unmarshal(body, &result); err != nil { if retry < maxRetries-1 {
return nil, fmt.Errorf("解析响应失败: %v", err) continue
} }
return nil, fmt.Errorf("请求失败: %v", lastErr)
}
// 缓存结果 // 解析响应
searchCache.Set(cacheKey, result.Results) var result struct {
return result.Results, nil Count int `json:"count"`
Next string `json:"next"`
Previous string `json:"previous"`
Results []TagInfo `json:"results"`
}
if err := json.Unmarshal(body, &result); err != nil {
lastErr = err
if retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("解析响应失败: %v", err)
}
return &result, nil
}
return nil, lastErr
}
// parsePaginationParams 解析分页参数
func parsePaginationParams(c *gin.Context, defaultPageSize int) (page, pageSize int) {
page = 1
pageSize = defaultPageSize
if p := c.Query("page"); p != "" {
fmt.Sscanf(p, "%d", &page)
}
if ps := c.Query("page_size"); ps != "" {
fmt.Sscanf(ps, "%d", &pageSize)
}
return page, pageSize
}
// safeCloseResponseBody 安全关闭HTTP响应体统一资源管理
func safeCloseResponseBody(body io.ReadCloser, context string) {
if body != nil {
if err := body.Close(); err != nil {
fmt.Printf("关闭%s失败: %v\n", context, err)
}
}
}
// sendErrorResponse 统一错误响应处理
func sendErrorResponse(c *gin.Context, message string) {
c.JSON(http.StatusBadRequest, gin.H{"error": message})
} }
// RegisterSearchRoute 注册搜索相关路由 // RegisterSearchRoute 注册搜索相关路由
@@ -457,22 +567,15 @@ func RegisterSearchRoute(r *gin.Engine) {
r.GET("/search", func(c *gin.Context) { r.GET("/search", func(c *gin.Context) {
query := c.Query("q") query := c.Query("q")
if query == "" { if query == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "搜索关键词不能为空"}) sendErrorResponse(c, "搜索关键词不能为空")
return return
} }
page := 1 page, pageSize := parsePaginationParams(c, 25)
pageSize := 25
if p := c.Query("page"); p != "" {
fmt.Sscanf(p, "%d", &page)
}
if ps := c.Query("page_size"); ps != "" {
fmt.Sscanf(ps, "%d", &pageSize)
}
result, err := searchDockerHub(c.Request.Context(), query, page, pageSize) result, err := searchDockerHub(c.Request.Context(), query, page, pageSize)
if err != nil { if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) sendErrorResponse(c, err.Error())
return return
} }
@@ -485,16 +588,27 @@ func RegisterSearchRoute(r *gin.Engine) {
name := c.Param("name") name := c.Param("name")
if namespace == "" || name == "" { if namespace == "" || name == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "命名空间和名称不能为空"}) sendErrorResponse(c, "命名空间和名称不能为空")
return return
} }
tags, err := getRepositoryTags(c.Request.Context(), namespace, name) page, pageSize := parsePaginationParams(c, 100)
tags, hasMore, err := getRepositoryTags(c.Request.Context(), namespace, name, page, pageSize)
if err != nil { if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) sendErrorResponse(c, err.Error())
return return
} }
c.JSON(http.StatusOK, tags) if c.Query("page") != "" || c.Query("page_size") != "" {
c.JSON(http.StatusOK, gin.H{
"tags": tags,
"has_more": hasMore,
"page": page,
"page_size": pageSize,
})
} else {
c.JSON(http.StatusOK, tags)
}
}) })
} }

View File

@@ -1,108 +0,0 @@
package main
import (
"strings"
"sync"
"time"
)
// SmartRateLimit 智能限流会话管理
type SmartRateLimit struct {
sessions sync.Map
}
// PullSession Docker拉取会话
type PullSession struct {
LastManifestTime time.Time
RequestCount int
}
// 全局智能限流实例
var smartLimiter = &SmartRateLimit{}
const (
// manifest请求后的活跃窗口时间
activeWindowDuration = 3 * time.Minute
// 活跃窗口内最大免费blob请求数(防止滥用)
maxFreeBlobRequests = 100
sessionCleanupInterval = 10 * time.Minute
sessionExpireTime = 30 * time.Minute
)
func init() {
go smartLimiter.cleanupSessions()
}
// ShouldSkipRateLimit 判断是否应该跳过限流计数
func (s *SmartRateLimit) ShouldSkipRateLimit(ip, path string) bool {
requestType, _ := parseRequestInfo(path)
if requestType != "manifests" && requestType != "blobs" {
return false
}
sessionKey := ip
sessionInterface, _ := s.sessions.LoadOrStore(sessionKey, &PullSession{})
session := sessionInterface.(*PullSession)
now := time.Now()
if requestType == "manifests" {
session.LastManifestTime = now
session.RequestCount = 0
return false
}
if requestType == "blobs" {
if !session.LastManifestTime.IsZero() &&
now.Sub(session.LastManifestTime) <= activeWindowDuration {
session.RequestCount++
if session.RequestCount <= maxFreeBlobRequests {
return true
}
}
}
return false
}
func parseRequestInfo(path string) (requestType, imageRef string) {
path = strings.TrimPrefix(path, "/v2/")
if idx := strings.Index(path, "/manifests/"); idx != -1 {
return "manifests", path[:idx]
}
if idx := strings.Index(path, "/blobs/"); idx != -1 {
return "blobs", path[:idx]
}
if idx := strings.Index(path, "/tags/"); idx != -1 {
return "tags", path[:idx]
}
return "unknown", ""
}
// cleanupSessions 定期清理过期会话,防止内存泄露
func (s *SmartRateLimit) cleanupSessions() {
ticker := time.NewTicker(sessionCleanupInterval)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expiredKeys := make([]string, 0)
s.sessions.Range(func(key, value interface{}) bool {
session := value.(*PullSession)
if !session.LastManifestTime.IsZero() &&
now.Sub(session.LastManifestTime) > sessionExpireTime {
expiredKeys = append(expiredKeys, key.(string))
}
return true
})
for _, key := range expiredKeys {
s.sessions.Delete(key)
}
}
}

View File

@@ -13,10 +13,10 @@ import (
// CachedItem 通用缓存项支持Token和Manifest // CachedItem 通用缓存项支持Token和Manifest
type CachedItem struct { type CachedItem struct {
Data []byte // 缓存数据(token字符串或manifest字节) Data []byte // 缓存数据(token字符串或manifest字节)
ContentType string // 内容类型 ContentType string // 内容类型
Headers map[string]string // 额外的响应头 Headers map[string]string // 额外的响应头
ExpiresAt time.Time // 过期时间 ExpiresAt time.Time // 过期时间
} }
// UniversalCache 通用缓存支持Token和Manifest // UniversalCache 通用缓存支持Token和Manifest
@@ -71,14 +71,6 @@ func buildManifestCacheKey(imageRef, reference string) string {
return buildCacheKey("manifest", key) return buildCacheKey("manifest", key)
} }
func buildManifestCacheKeyWithPlatform(imageRef, reference, platform string) string {
if platform == "" {
platform = "default"
}
key := fmt.Sprintf("%s:%s@%s", imageRef, reference, platform)
return buildCacheKey("manifest", key)
}
func getManifestTTL(reference string) time.Duration { func getManifestTTL(reference string) time.Duration {
cfg := GetConfig() cfg := GetConfig()
defaultTTL := 30 * time.Minute defaultTTL := 30 * time.Minute
@@ -87,19 +79,18 @@ func getManifestTTL(reference string) time.Duration {
defaultTTL = parsed defaultTTL = parsed
} }
} }
if strings.HasPrefix(reference, "sha256:") { if strings.HasPrefix(reference, "sha256:") {
return 24 * time.Hour return 24 * time.Hour
} }
// mutable tag的智能判断 // mutable tag的智能判断
if reference == "latest" || reference == "main" || reference == "master" || if reference == "latest" || reference == "main" || reference == "master" ||
reference == "dev" || reference == "develop" { reference == "dev" || reference == "develop" {
// 热门可变标签: 短期缓存 // 热门可变标签: 短期缓存
return 10 * time.Minute return 10 * time.Minute
} }
// 普通tag: 中等缓存时间
return defaultTTL return defaultTTL
} }
@@ -108,17 +99,17 @@ func extractTTLFromResponse(responseBody []byte) time.Duration {
var tokenResp struct { var tokenResp struct {
ExpiresIn int `json:"expires_in"` ExpiresIn int `json:"expires_in"`
} }
// 默认30分钟TTL确保稳定性 // 默认30分钟TTL确保稳定性
defaultTTL := 30 * time.Minute defaultTTL := 30 * time.Minute
if json.Unmarshal(responseBody, &tokenResp) == nil && tokenResp.ExpiresIn > 0 { if json.Unmarshal(responseBody, &tokenResp) == nil && tokenResp.ExpiresIn > 0 {
safeTTL := time.Duration(tokenResp.ExpiresIn-300) * time.Second safeTTL := time.Duration(tokenResp.ExpiresIn-300) * time.Second
if safeTTL > 5*time.Minute { if safeTTL > 5*time.Minute {
return safeTTL return safeTTL
} }
} }
return defaultTTL return defaultTTL
} }
@@ -131,12 +122,12 @@ func writeCachedResponse(c *gin.Context, item *CachedItem) {
if item.ContentType != "" { if item.ContentType != "" {
c.Header("Content-Type", item.ContentType) c.Header("Content-Type", item.ContentType)
} }
// 设置额外的响应头 // 设置额外的响应头
for key, value := range item.Headers { for key, value := range item.Headers {
c.Header(key, value) c.Header(key, value)
} }
// 返回数据 // 返回数据
c.Data(200, item.ContentType, item.Data) c.Data(200, item.ContentType, item.Data)
} }
@@ -150,4 +141,28 @@ func isCacheEnabled() bool {
// isTokenCacheEnabled 检查token缓存是否启用(向后兼容) // isTokenCacheEnabled 检查token缓存是否启用(向后兼容)
func isTokenCacheEnabled() bool { func isTokenCacheEnabled() bool {
return isCacheEnabled() return isCacheEnabled()
} }
// 定期清理过期缓存,防止内存泄漏
func init() {
go func() {
ticker := time.NewTicker(20 * time.Minute)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expiredKeys := make([]string, 0)
globalCache.cache.Range(func(key, value interface{}) bool {
if cached := value.(*CachedItem); now.After(cached.ExpiresAt) {
expiredKeys = append(expiredKeys, key.(string))
}
return true
})
for _, key := range expiredKeys {
globalCache.cache.Delete(key)
}
}
}()
}