Compare commits
30 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2567652a7d | ||
|
|
c023e6a9c4 | ||
|
|
44c6e4cd7b | ||
|
|
c22bd0637a | ||
|
|
a94b476726 | ||
|
|
4c6751b862 | ||
|
|
acc63d7b68 | ||
|
|
d0b1ea8582 | ||
|
|
c607061dae | ||
|
|
143de7b254 | ||
|
|
51ace73b78 | ||
|
|
fa9e9210ab | ||
|
|
f308410920 | ||
|
|
252dc319c6 | ||
|
|
29ceeef45b | ||
|
|
182dced403 | ||
|
|
aea36939a3 | ||
|
|
4240c1452a | ||
|
|
212c8e529d | ||
|
|
3fd630159b | ||
|
|
17d827f50b | ||
|
|
7dcbc839c6 | ||
|
|
45ffebc820 | ||
|
|
3027b1f218 | ||
|
|
3d2c419ebe | ||
|
|
b529fbfdd2 | ||
|
|
737c1dbf46 | ||
|
|
a67ef6c52c | ||
|
|
0adf11099e | ||
|
|
dbb9432eb0 |
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
* text=auto eol=lf
|
||||||
21
LICENSE
Normal file
21
LICENSE
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2025 sky22333
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
113
README.md
113
README.md
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
🚀 **Docker 和 GitHub 加速代理服务器**
|
🚀 **Docker 和 GitHub 加速代理服务器**
|
||||||
|
|
||||||
一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速等功能。
|
一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速、下载离线镜像、在线搜索 Docker 镜像等功能。
|
||||||
|
|
||||||
## ✨ 特性
|
## ✨ 特性
|
||||||
|
|
||||||
@@ -14,7 +14,8 @@
|
|||||||
- 🚫 **仓库审计** - 强大的自定义黑名单,白名单,同时审计镜像仓库,和GitHub仓库
|
- 🚫 **仓库审计** - 强大的自定义黑名单,白名单,同时审计镜像仓库,和GitHub仓库
|
||||||
- 🔍 **镜像搜索** - 在线搜索 Docker 镜像
|
- 🔍 **镜像搜索** - 在线搜索 Docker 镜像
|
||||||
- ⚡ **轻量高效** - 基于 Go 语言,单二进制文件运行,资源占用低,优雅的内存清理机制。
|
- ⚡ **轻量高效** - 基于 Go 语言,单二进制文件运行,资源占用低,优雅的内存清理机制。
|
||||||
- 🔧 **配置热重载** - 统一配置管理,部分配置项支持热重载,无需重启服务
|
- 🔧 **统一配置** - 统一配置管理
|
||||||
|
|
||||||
|
|
||||||
## 🚀 快速开始
|
## 🚀 快速开始
|
||||||
|
|
||||||
@@ -29,12 +30,14 @@ docker run -d \
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 一键安装
|
### 一键脚本安装
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-service.sh | sudo bash
|
curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-service.sh | sudo bash
|
||||||
```
|
```
|
||||||
|
|
||||||
|
也可以直接下载二进制文件执行`./hubproxy`使用,无需配置文件即可启动,内置默认配置,支持所有功能。初始内存占用约18M,二进制文件大小约12M
|
||||||
|
|
||||||
这个命令会:
|
这个命令会:
|
||||||
- 🔍 自动检测系统架构(AMD64/ARM64)
|
- 🔍 自动检测系统架构(AMD64/ARM64)
|
||||||
- 📥 从 GitHub Releases 下载最新版本
|
- 📥 从 GitHub Releases 下载最新版本
|
||||||
@@ -55,7 +58,9 @@ docker pull nginx
|
|||||||
docker pull yourdomain.com/nginx
|
docker pull yourdomain.com/nginx
|
||||||
|
|
||||||
# ghcr加速
|
# ghcr加速
|
||||||
docker pull yourdomain.com/ghcr.io/user/images
|
docker pull yourdomain.com/ghcr.io/sky22333/hubproxy
|
||||||
|
|
||||||
|
# 符合Docker Registry API v2标准的仓库都支持
|
||||||
```
|
```
|
||||||
|
|
||||||
### GitHub 文件加速
|
### GitHub 文件加速
|
||||||
@@ -70,9 +75,11 @@ https://yourdomain.com/https://github.com/user/repo/releases/download/v1.0.0/fil
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
## ⚙️ 提示
|
## ⚙️ 配置
|
||||||
|
|
||||||
主配置文件位于 `/opt/hubproxy/config.toml`:
|
容器内的配置文件位于 `/root/config.toml`
|
||||||
|
|
||||||
|
脚本部署配置文件位于 `/opt/hubproxy/config.toml`
|
||||||
|
|
||||||
为了IP限流能够正常运行,反向代理需要传递IP头用来获取访客真实IP,以caddy为例:
|
为了IP限流能够正常运行,反向代理需要传递IP头用来获取访客真实IP,以caddy为例:
|
||||||
```
|
```
|
||||||
@@ -86,6 +93,100 @@ example.com {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### `config.toml`配置示例
|
||||||
|
|
||||||
|
不配置也可以,程序内置有默认值
|
||||||
|
```
|
||||||
|
[server]
|
||||||
|
host = "0.0.0.0"
|
||||||
|
# 监听端口
|
||||||
|
port = 5000
|
||||||
|
# Github文件大小限制(字节),默认2GB
|
||||||
|
fileSize = 2147483648
|
||||||
|
|
||||||
|
[rateLimit]
|
||||||
|
# 每个IP每小时允许的请求数(注意Docker镜像会有多个层,会消耗多个次数)
|
||||||
|
requestLimit = 500
|
||||||
|
# 限流周期(小时)
|
||||||
|
periodHours = 1.0
|
||||||
|
|
||||||
|
[security]
|
||||||
|
# IP白名单,支持单个IP或IP段
|
||||||
|
# 白名单中的IP不受限流限制
|
||||||
|
whiteList = [
|
||||||
|
"127.0.0.1",
|
||||||
|
"192.168.1.0/24"
|
||||||
|
]
|
||||||
|
|
||||||
|
# IP黑名单,支持单个IP或IP段
|
||||||
|
# 黑名单中的IP将被直接拒绝访问
|
||||||
|
blackList = [
|
||||||
|
"192.168.100.1",
|
||||||
|
"192.168.100.0/24"
|
||||||
|
]
|
||||||
|
|
||||||
|
[proxy]
|
||||||
|
# 代理服务白名单(支持GitHub仓库和Docker镜像,支持通配符)
|
||||||
|
# 只允许访问白名单中的仓库/镜像,为空时不限制
|
||||||
|
whiteList = []
|
||||||
|
|
||||||
|
# 代理服务黑名单(支持GitHub仓库和Docker镜像,支持通配符)
|
||||||
|
# 禁止访问黑名单中的仓库/镜像
|
||||||
|
blackList = [
|
||||||
|
"baduser/malicious-repo",
|
||||||
|
"*/malicious-repo",
|
||||||
|
"baduser/*"
|
||||||
|
]
|
||||||
|
|
||||||
|
# SOCKS5代理配置,支持有用户名/密码认证和无认证模式
|
||||||
|
# 无认证: socks5://127.0.0.1:1080
|
||||||
|
# 有认证: socks5://username:password@127.0.0.1:1080
|
||||||
|
# 留空不使用代理
|
||||||
|
socks5 = ""
|
||||||
|
|
||||||
|
[download]
|
||||||
|
# 批量下载离线镜像数量限制
|
||||||
|
maxImages = 10
|
||||||
|
|
||||||
|
# Registry映射配置,支持多种镜像仓库上游
|
||||||
|
[registries]
|
||||||
|
|
||||||
|
# GitHub Container Registry
|
||||||
|
[registries."ghcr.io"]
|
||||||
|
upstream = "ghcr.io"
|
||||||
|
authHost = "ghcr.io/token"
|
||||||
|
authType = "github"
|
||||||
|
enabled = true
|
||||||
|
|
||||||
|
# Google Container Registry
|
||||||
|
[registries."gcr.io"]
|
||||||
|
upstream = "gcr.io"
|
||||||
|
authHost = "gcr.io/v2/token"
|
||||||
|
authType = "google"
|
||||||
|
enabled = true
|
||||||
|
|
||||||
|
# Quay.io Container Registry
|
||||||
|
[registries."quay.io"]
|
||||||
|
upstream = "quay.io"
|
||||||
|
authHost = "quay.io/v2/auth"
|
||||||
|
authType = "quay"
|
||||||
|
enabled = true
|
||||||
|
|
||||||
|
# Kubernetes Container Registry
|
||||||
|
[registries."registry.k8s.io"]
|
||||||
|
upstream = "registry.k8s.io"
|
||||||
|
authHost = "registry.k8s.io"
|
||||||
|
authType = "anonymous"
|
||||||
|
enabled = true
|
||||||
|
|
||||||
|
[tokenCache]
|
||||||
|
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
|
||||||
|
enabled = true
|
||||||
|
# 默认缓存时间(分钟)
|
||||||
|
defaultTTL = "20m"
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## ⚠️ 免责声明
|
## ⚠️ 免责声明
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
services:
|
services:
|
||||||
ghproxy:
|
hubproxy:
|
||||||
build: .
|
build: .
|
||||||
restart: always
|
restart: always
|
||||||
ports:
|
ports:
|
||||||
|
|||||||
@@ -211,10 +211,4 @@ func (ac *AccessController) checkList(matches, list []string) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reload 热重载访问控制规则
|
|
||||||
func (ac *AccessController) Reload() {
|
|
||||||
ac.mu.Lock()
|
|
||||||
defer ac.mu.Unlock()
|
|
||||||
|
|
||||||
// 访问控制器本身不缓存配置
|
|
||||||
}
|
|
||||||
@@ -9,8 +9,6 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/pelletier/go-toml/v2"
|
"github.com/pelletier/go-toml/v2"
|
||||||
"github.com/spf13/viper"
|
|
||||||
"github.com/fsnotify/fsnotify"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// RegistryMapping Registry映射配置
|
// RegistryMapping Registry映射配置
|
||||||
@@ -42,6 +40,7 @@ type AppConfig struct {
|
|||||||
Proxy struct {
|
Proxy struct {
|
||||||
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别)
|
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别)
|
||||||
BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别)
|
BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别)
|
||||||
|
Socks5 string `toml:"socks5"` // SOCKS5代理地址: socks5://[user:pass@]host:port
|
||||||
} `toml:"proxy"`
|
} `toml:"proxy"`
|
||||||
|
|
||||||
Download struct {
|
Download struct {
|
||||||
@@ -59,8 +58,6 @@ type AppConfig struct {
|
|||||||
var (
|
var (
|
||||||
appConfig *AppConfig
|
appConfig *AppConfig
|
||||||
appConfigLock sync.RWMutex
|
appConfigLock sync.RWMutex
|
||||||
isViperEnabled bool
|
|
||||||
viperInstance *viper.Viper
|
|
||||||
|
|
||||||
cachedConfig *AppConfig
|
cachedConfig *AppConfig
|
||||||
configCacheTime time.Time
|
configCacheTime time.Time
|
||||||
@@ -97,9 +94,11 @@ func DefaultConfig() *AppConfig {
|
|||||||
Proxy: struct {
|
Proxy: struct {
|
||||||
WhiteList []string `toml:"whiteList"`
|
WhiteList []string `toml:"whiteList"`
|
||||||
BlackList []string `toml:"blackList"`
|
BlackList []string `toml:"blackList"`
|
||||||
|
Socks5 string `toml:"socks5"`
|
||||||
}{
|
}{
|
||||||
WhiteList: []string{},
|
WhiteList: []string{},
|
||||||
BlackList: []string{},
|
BlackList: []string{},
|
||||||
|
Socks5: "", // 默认不使用代理
|
||||||
},
|
},
|
||||||
Download: struct {
|
Download: struct {
|
||||||
MaxImages int `toml:"maxImages"`
|
MaxImages int `toml:"maxImages"`
|
||||||
@@ -215,100 +214,9 @@ func LoadConfig() error {
|
|||||||
// 设置配置
|
// 设置配置
|
||||||
setConfig(cfg)
|
setConfig(cfg)
|
||||||
|
|
||||||
if !isViperEnabled {
|
|
||||||
go enableViperHotReload()
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func enableViperHotReload() {
|
|
||||||
if isViperEnabled {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// 创建Viper实例
|
|
||||||
viperInstance = viper.New()
|
|
||||||
|
|
||||||
// 配置Viper
|
|
||||||
viperInstance.SetConfigName("config")
|
|
||||||
viperInstance.SetConfigType("toml")
|
|
||||||
viperInstance.AddConfigPath(".")
|
|
||||||
|
|
||||||
// 读取配置文件
|
|
||||||
if err := viperInstance.ReadInConfig(); err != nil {
|
|
||||||
fmt.Printf("读取配置失败,继续使用当前配置: %v\n", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
isViperEnabled = true
|
|
||||||
|
|
||||||
viperInstance.WatchConfig()
|
|
||||||
viperInstance.OnConfigChange(func(e fsnotify.Event) {
|
|
||||||
fmt.Printf("检测到配置文件变化: %s\n", e.Name)
|
|
||||||
hotReloadWithViper()
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func hotReloadWithViper() {
|
|
||||||
start := time.Now()
|
|
||||||
fmt.Println("🔄 自动热重载...")
|
|
||||||
|
|
||||||
// 创建新配置
|
|
||||||
cfg := DefaultConfig()
|
|
||||||
|
|
||||||
// 使用Viper解析配置到结构体
|
|
||||||
if err := viperInstance.Unmarshal(cfg); err != nil {
|
|
||||||
fmt.Printf("❌ 配置解析失败: %v\n", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
overrideFromEnv(cfg)
|
|
||||||
|
|
||||||
setConfig(cfg)
|
|
||||||
|
|
||||||
// 异步更新受影响的组件
|
|
||||||
go func() {
|
|
||||||
updateAffectedComponents()
|
|
||||||
fmt.Printf("✅ Viper配置热重载完成,耗时: %v\n", time.Since(start))
|
|
||||||
}()
|
|
||||||
}
|
|
||||||
|
|
||||||
func updateAffectedComponents() {
|
|
||||||
// 重新初始化限流器
|
|
||||||
if globalLimiter != nil {
|
|
||||||
fmt.Println("📡 重新初始化限流器...")
|
|
||||||
initLimiter()
|
|
||||||
}
|
|
||||||
|
|
||||||
// 重新加载访问控制
|
|
||||||
fmt.Println("🔒 重新加载访问控制规则...")
|
|
||||||
if GlobalAccessController != nil {
|
|
||||||
GlobalAccessController.Reload()
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println("🌐 更新Registry配置映射...")
|
|
||||||
reloadRegistryConfig()
|
|
||||||
|
|
||||||
// 其他需要重新初始化的组件可以在这里添加
|
|
||||||
fmt.Println("🔧 组件更新完成")
|
|
||||||
}
|
|
||||||
|
|
||||||
func reloadRegistryConfig() {
|
|
||||||
cfg := GetConfig()
|
|
||||||
enabledCount := 0
|
|
||||||
|
|
||||||
// 统计启用的Registry数量
|
|
||||||
for _, mapping := range cfg.Registries {
|
|
||||||
if mapping.Enabled {
|
|
||||||
enabledCount++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("🌐 Registry配置已更新: %d个启用\n", enabledCount)
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
// overrideFromEnv 从环境变量覆盖配置
|
// overrideFromEnv 从环境变量覆盖配置
|
||||||
func overrideFromEnv(cfg *AppConfig) {
|
func overrideFromEnv(cfg *AppConfig) {
|
||||||
// 服务器配置
|
// 服务器配置
|
||||||
|
|||||||
@@ -1,29 +1,29 @@
|
|||||||
[server]
|
[server]
|
||||||
# 监听地址,默认监听所有接口
|
|
||||||
host = "0.0.0.0"
|
host = "0.0.0.0"
|
||||||
# 监听端口
|
# 监听端口
|
||||||
port = 5000
|
port = 5000
|
||||||
# 文件大小限制(字节),默认2GB
|
# Github文件大小限制(字节),默认2GB
|
||||||
fileSize = 2147483648
|
fileSize = 2147483648
|
||||||
|
|
||||||
[rateLimit]
|
[rateLimit]
|
||||||
# 每个IP每小时允许的请求数(Docker镜像每个层为一个请求)
|
# 每个IP每小时允许的请求数(注意Docker镜像会有多个层,会消耗多个次数)
|
||||||
requestLimit = 200
|
requestLimit = 500
|
||||||
# 限流周期(小时)
|
# 限流周期(小时)
|
||||||
periodHours = 1.0
|
periodHours = 1.0
|
||||||
|
|
||||||
[security]
|
[security]
|
||||||
# IP白名单,支持单个IP或CIDR格式
|
# IP白名单,支持单个IP或IP段
|
||||||
# 白名单中的IP不受限流限制
|
# 白名单中的IP不受限流限制
|
||||||
whiteList = [
|
whiteList = [
|
||||||
"127.0.0.1",
|
"127.0.0.1",
|
||||||
"192.168.1.0/24"
|
"192.168.1.0/24"
|
||||||
]
|
]
|
||||||
|
|
||||||
# IP黑名单,支持单个IP或CIDR格式
|
# IP黑名单,支持单个IP或IP段
|
||||||
# 黑名单中的IP将被直接拒绝访问
|
# 黑名单中的IP将被直接拒绝访问
|
||||||
blackList = [
|
blackList = [
|
||||||
"192.168.100.1"
|
"192.168.100.1",
|
||||||
|
"192.168.100.0/24"
|
||||||
]
|
]
|
||||||
|
|
||||||
[proxy]
|
[proxy]
|
||||||
@@ -39,11 +39,17 @@ blackList = [
|
|||||||
"baduser/*"
|
"baduser/*"
|
||||||
]
|
]
|
||||||
|
|
||||||
|
# SOCKS5代理配置,支持有用户名/密码认证和无认证模式
|
||||||
|
# 无认证: socks5://127.0.0.1:1080
|
||||||
|
# 有认证: socks5://username:password@127.0.0.1:1080
|
||||||
|
# 留空不使用代理
|
||||||
|
socks5 = ""
|
||||||
|
|
||||||
[download]
|
[download]
|
||||||
# 单次并发下载离线镜像数量限制
|
# 批量下载离线镜像数量限制
|
||||||
maxImages = 10
|
maxImages = 10
|
||||||
|
|
||||||
# Registry映射配置,支持多种Container Registry
|
# Registry映射配置,支持多种镜像仓库上游
|
||||||
[registries]
|
[registries]
|
||||||
|
|
||||||
# GitHub Container Registry
|
# GitHub Container Registry
|
||||||
@@ -74,16 +80,8 @@ authHost = "registry.k8s.io"
|
|||||||
authType = "anonymous"
|
authType = "anonymous"
|
||||||
enabled = true
|
enabled = true
|
||||||
|
|
||||||
# 私有Registry示例(默认禁用)
|
|
||||||
# [registries."harbor.company.com"]
|
|
||||||
# upstream = "harbor.company.com"
|
|
||||||
# authHost = "harbor.company.com/service/token"
|
|
||||||
# authType = "basic"
|
|
||||||
# enabled = false
|
|
||||||
|
|
||||||
# 缓存配置:Docker临时Token和Manifest统一管理,显著提升性能
|
|
||||||
[tokenCache]
|
[tokenCache]
|
||||||
# 是否启用缓存(同时控制Token和Manifest缓存)
|
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
|
||||||
enabled = true
|
enabled = true
|
||||||
# 默认缓存时间
|
# 默认缓存时间(分钟)
|
||||||
defaultTTL = "20m"
|
defaultTTL = "20m"
|
||||||
|
|||||||
@@ -72,6 +72,7 @@ func initDockerProxy() {
|
|||||||
options := []remote.Option{
|
options := []remote.Option{
|
||||||
remote.WithAuth(authn.Anonymous),
|
remote.WithAuth(authn.Anonymous),
|
||||||
remote.WithUserAgent("hubproxy/go-containerregistry"),
|
remote.WithUserAgent("hubproxy/go-containerregistry"),
|
||||||
|
remote.WithTransport(GetGlobalHTTPClient().Transport),
|
||||||
}
|
}
|
||||||
|
|
||||||
dockerProxy = &DockerProxy{
|
dockerProxy = &DockerProxy{
|
||||||
@@ -363,11 +364,9 @@ func proxyDockerAuthWithCache(c *gin.Context) {
|
|||||||
globalCache.SetToken(cacheKey, string(recorder.body), ttl)
|
globalCache.SetToken(cacheKey, string(recorder.body), ttl)
|
||||||
}
|
}
|
||||||
|
|
||||||
// 6. 写入实际响应(如果还没写入)
|
// 6. 写入实际响应
|
||||||
if !recorder.written {
|
c.Writer = recorder.ResponseWriter
|
||||||
c.Writer = recorder.ResponseWriter
|
c.Data(recorder.statusCode, "application/json", recorder.body)
|
||||||
c.Data(recorder.statusCode, "application/json", recorder.body)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResponseRecorder HTTP响应记录器
|
// ResponseRecorder HTTP响应记录器
|
||||||
@@ -375,7 +374,6 @@ type ResponseRecorder struct {
|
|||||||
gin.ResponseWriter
|
gin.ResponseWriter
|
||||||
statusCode int
|
statusCode int
|
||||||
body []byte
|
body []byte
|
||||||
written bool
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *ResponseRecorder) WriteHeader(code int) {
|
func (r *ResponseRecorder) WriteHeader(code int) {
|
||||||
@@ -384,8 +382,7 @@ func (r *ResponseRecorder) WriteHeader(code int) {
|
|||||||
|
|
||||||
func (r *ResponseRecorder) Write(data []byte) (int, error) {
|
func (r *ResponseRecorder) Write(data []byte) (int, error) {
|
||||||
r.body = append(r.body, data...)
|
r.body = append(r.body, data...)
|
||||||
r.written = true
|
return len(data), nil
|
||||||
return r.ResponseWriter.Write(data)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func proxyDockerAuthOriginal(c *gin.Context) {
|
func proxyDockerAuthOriginal(c *gin.Context) {
|
||||||
@@ -407,9 +404,10 @@ func proxyDockerAuthOriginal(c *gin.Context) {
|
|||||||
authURL += "?" + c.Request.URL.RawQuery
|
authURL += "?" + c.Request.URL.RawQuery
|
||||||
}
|
}
|
||||||
|
|
||||||
// 创建HTTP客户端
|
// 创建HTTP客户端,复用全局传输配置(包含代理设置)
|
||||||
client := &http.Client{
|
client := &http.Client{
|
||||||
Timeout: 30 * time.Second,
|
Timeout: 30 * time.Second,
|
||||||
|
Transport: GetGlobalHTTPClient().Transport,
|
||||||
}
|
}
|
||||||
|
|
||||||
// 创建请求
|
// 创建请求
|
||||||
@@ -664,6 +662,7 @@ func createUpstreamOptions(mapping RegistryMapping) []remote.Option {
|
|||||||
options := []remote.Option{
|
options := []remote.Option{
|
||||||
remote.WithAuth(authn.Anonymous),
|
remote.WithAuth(authn.Anonymous),
|
||||||
remote.WithUserAgent("hubproxy/go-containerregistry"),
|
remote.WithUserAgent("hubproxy/go-containerregistry"),
|
||||||
|
remote.WithTransport(GetGlobalHTTPClient().Transport),
|
||||||
}
|
}
|
||||||
|
|
||||||
// 根据Registry类型添加特定的认证选项(方便后续扩展)
|
// 根据Registry类型添加特定的认证选项(方便后续扩展)
|
||||||
|
|||||||
16
src/go.mod
16
src/go.mod
@@ -3,11 +3,10 @@ module hubproxy
|
|||||||
go 1.24.0
|
go 1.24.0
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/fsnotify/fsnotify v1.8.0
|
|
||||||
github.com/gin-gonic/gin v1.10.0
|
github.com/gin-gonic/gin v1.10.0
|
||||||
github.com/google/go-containerregistry v0.20.5
|
github.com/google/go-containerregistry v0.20.5
|
||||||
github.com/pelletier/go-toml/v2 v2.2.3
|
github.com/pelletier/go-toml/v2 v2.2.3
|
||||||
github.com/spf13/viper v1.20.1
|
golang.org/x/net v0.33.0
|
||||||
golang.org/x/time v0.11.0
|
golang.org/x/time v0.11.0
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -25,11 +24,11 @@ require (
|
|||||||
github.com/go-playground/locales v0.14.1 // indirect
|
github.com/go-playground/locales v0.14.1 // indirect
|
||||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||||
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
||||||
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
|
|
||||||
github.com/goccy/go-json v0.10.2 // indirect
|
github.com/goccy/go-json v0.10.2 // indirect
|
||||||
github.com/json-iterator/go v1.1.12 // indirect
|
github.com/json-iterator/go v1.1.12 // indirect
|
||||||
github.com/klauspost/compress v1.18.0 // indirect
|
github.com/klauspost/compress v1.18.0 // indirect
|
||||||
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
|
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
|
||||||
|
github.com/kr/pretty v0.3.0 // indirect
|
||||||
github.com/leodido/go-urn v1.4.0 // indirect
|
github.com/leodido/go-urn v1.4.0 // indirect
|
||||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||||
github.com/mitchellh/go-homedir v1.1.0 // indirect
|
github.com/mitchellh/go-homedir v1.1.0 // indirect
|
||||||
@@ -38,24 +37,17 @@ require (
|
|||||||
github.com/opencontainers/go-digest v1.0.0 // indirect
|
github.com/opencontainers/go-digest v1.0.0 // indirect
|
||||||
github.com/opencontainers/image-spec v1.1.1 // indirect
|
github.com/opencontainers/image-spec v1.1.1 // indirect
|
||||||
github.com/pkg/errors v0.9.1 // indirect
|
github.com/pkg/errors v0.9.1 // indirect
|
||||||
github.com/sagikazarmark/locafero v0.7.0 // indirect
|
github.com/rogpeppe/go-internal v1.9.0 // indirect
|
||||||
github.com/sirupsen/logrus v1.9.3 // indirect
|
github.com/sirupsen/logrus v1.9.3 // indirect
|
||||||
github.com/sourcegraph/conc v0.3.0 // indirect
|
|
||||||
github.com/spf13/afero v1.12.0 // indirect
|
|
||||||
github.com/spf13/cast v1.7.1 // indirect
|
|
||||||
github.com/spf13/pflag v1.0.6 // indirect
|
|
||||||
github.com/subosito/gotenv v1.6.0 // indirect
|
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||||
github.com/ugorji/go/codec v1.2.12 // indirect
|
github.com/ugorji/go/codec v1.2.12 // indirect
|
||||||
github.com/vbatts/tar-split v0.12.1 // indirect
|
github.com/vbatts/tar-split v0.12.1 // indirect
|
||||||
go.uber.org/atomic v1.9.0 // indirect
|
|
||||||
go.uber.org/multierr v1.9.0 // indirect
|
|
||||||
golang.org/x/arch v0.8.0 // indirect
|
golang.org/x/arch v0.8.0 // indirect
|
||||||
golang.org/x/crypto v0.32.0 // indirect
|
golang.org/x/crypto v0.32.0 // indirect
|
||||||
golang.org/x/net v0.33.0 // indirect
|
|
||||||
golang.org/x/sync v0.14.0 // indirect
|
golang.org/x/sync v0.14.0 // indirect
|
||||||
golang.org/x/sys v0.33.0 // indirect
|
golang.org/x/sys v0.33.0 // indirect
|
||||||
golang.org/x/text v0.21.0 // indirect
|
golang.org/x/text v0.21.0 // indirect
|
||||||
google.golang.org/protobuf v1.36.3 // indirect
|
google.golang.org/protobuf v1.36.3 // indirect
|
||||||
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
|
||||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
)
|
)
|
||||||
|
|||||||
39
src/go.sum
39
src/go.sum
@@ -8,6 +8,7 @@ github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
|
|||||||
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
|
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
|
||||||
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
|
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
|
||||||
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
|
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
|
||||||
|
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
|
||||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
@@ -17,10 +18,6 @@ github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBi
|
|||||||
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
|
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
|
||||||
github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8=
|
github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8=
|
||||||
github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
|
github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
|
||||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
|
||||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
|
||||||
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
|
|
||||||
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||||
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
|
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
|
||||||
@@ -35,8 +32,6 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
|
|||||||
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||||
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
||||||
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||||
github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
|
|
||||||
github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
|
||||||
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
||||||
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
||||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
@@ -52,8 +47,11 @@ github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa02
|
|||||||
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
|
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
|
||||||
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
|
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
|
||||||
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
|
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
|
||||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
|
||||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
|
||||||
|
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
|
||||||
|
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||||
|
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||||
@@ -77,22 +75,11 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
|||||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
|
||||||
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
||||||
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
||||||
github.com/sagikazarmark/locafero v0.7.0 h1:5MqpDsTGNDhY8sGp0Aowyf0qKsPrhewaLSsFaodPcyo=
|
|
||||||
github.com/sagikazarmark/locafero v0.7.0/go.mod h1:2za3Cg5rMaTMoG/2Ulr9AwtFaIppKXTRYnozin4aB5k=
|
|
||||||
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
|
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
|
||||||
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
||||||
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
|
|
||||||
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
|
|
||||||
github.com/spf13/afero v1.12.0 h1:UcOPyRBYczmFn6yvphxkn9ZEOY65cpwGKb5mL36mrqs=
|
|
||||||
github.com/spf13/afero v1.12.0/go.mod h1:ZTlWwG4/ahT8W7T0WQ5uYmjI9duaLQGy3Q2OAl4sk/4=
|
|
||||||
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
|
|
||||||
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
|
|
||||||
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
|
|
||||||
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
|
||||||
github.com/spf13/viper v1.20.1 h1:ZMi+z/lvLyPSCoNtFCpqjy0S4kPbirhpTMwl8BkW9X4=
|
|
||||||
github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqjJvu4=
|
|
||||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||||
@@ -101,20 +88,14 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
|
|||||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||||
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
|
|
||||||
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
|
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||||
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
|
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
|
||||||
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
||||||
github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
|
github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
|
||||||
github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
|
github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
|
||||||
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
|
|
||||||
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
|
|
||||||
go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
|
|
||||||
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
|
|
||||||
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
||||||
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
|
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
|
||||||
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
|
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
|
||||||
@@ -136,8 +117,10 @@ golang.org/x/time v0.11.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
|
|||||||
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=
|
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=
|
||||||
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
|
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
|
||||||
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
|
||||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
@@ -1,9 +1,14 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"log"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/url"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"golang.org/x/net/proxy"
|
||||||
)
|
)
|
||||||
|
|
||||||
var (
|
var (
|
||||||
@@ -15,13 +20,65 @@ var (
|
|||||||
|
|
||||||
// initHTTPClients 初始化HTTP客户端
|
// initHTTPClients 初始化HTTP客户端
|
||||||
func initHTTPClients() {
|
func initHTTPClients() {
|
||||||
|
cfg := GetConfig()
|
||||||
|
|
||||||
|
// 创建DialContext函数,支持SOCKS5代理
|
||||||
|
createDialContext := func(timeout time.Duration) func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||||
|
if cfg.Proxy.Socks5 == "" {
|
||||||
|
// 没有配置代理,使用直连
|
||||||
|
dialer := &net.Dialer{
|
||||||
|
Timeout: timeout,
|
||||||
|
KeepAlive: 30 * time.Second,
|
||||||
|
}
|
||||||
|
return dialer.DialContext
|
||||||
|
}
|
||||||
|
|
||||||
|
// 解析SOCKS5代理URL
|
||||||
|
proxyURL, err := url.Parse(cfg.Proxy.Socks5)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("SOCKS5代理配置错误,使用直连: %v", err)
|
||||||
|
dialer := &net.Dialer{
|
||||||
|
Timeout: timeout,
|
||||||
|
KeepAlive: 30 * time.Second,
|
||||||
|
}
|
||||||
|
return dialer.DialContext
|
||||||
|
}
|
||||||
|
|
||||||
|
// 创建基础dialer
|
||||||
|
baseDialer := &net.Dialer{
|
||||||
|
Timeout: timeout,
|
||||||
|
KeepAlive: 30 * time.Second,
|
||||||
|
}
|
||||||
|
|
||||||
|
// 创建SOCKS5代理dialer
|
||||||
|
var auth *proxy.Auth
|
||||||
|
if proxyURL.User != nil {
|
||||||
|
if password, ok := proxyURL.User.Password(); ok {
|
||||||
|
auth = &proxy.Auth{
|
||||||
|
User: proxyURL.User.Username(),
|
||||||
|
Password: password,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
socks5Dialer, err := proxy.SOCKS5("tcp", proxyURL.Host, auth, baseDialer)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("创建SOCKS5代理失败,使用直连: %v", err)
|
||||||
|
return baseDialer.DialContext
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Printf("使用SOCKS5代理: %s", proxyURL.Host)
|
||||||
|
|
||||||
|
// 返回带上下文的dial函数
|
||||||
|
return func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||||
|
return socks5Dialer.Dial(network, addr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// 代理客户端配置 - 适用于大文件传输
|
// 代理客户端配置 - 适用于大文件传输
|
||||||
globalHTTPClient = &http.Client{
|
globalHTTPClient = &http.Client{
|
||||||
Transport: &http.Transport{
|
Transport: &http.Transport{
|
||||||
DialContext: (&net.Dialer{
|
DialContext: createDialContext(30 * time.Second),
|
||||||
Timeout: 30 * time.Second,
|
|
||||||
KeepAlive: 30 * time.Second,
|
|
||||||
}).DialContext,
|
|
||||||
MaxIdleConns: 1000,
|
MaxIdleConns: 1000,
|
||||||
MaxIdleConnsPerHost: 1000,
|
MaxIdleConnsPerHost: 1000,
|
||||||
IdleConnTimeout: 90 * time.Second,
|
IdleConnTimeout: 90 * time.Second,
|
||||||
@@ -35,10 +92,7 @@ func initHTTPClients() {
|
|||||||
searchHTTPClient = &http.Client{
|
searchHTTPClient = &http.Client{
|
||||||
Timeout: 10 * time.Second,
|
Timeout: 10 * time.Second,
|
||||||
Transport: &http.Transport{
|
Transport: &http.Transport{
|
||||||
DialContext: (&net.Dialer{
|
DialContext: createDialContext(5 * time.Second),
|
||||||
Timeout: 5 * time.Second,
|
|
||||||
KeepAlive: 30 * time.Second,
|
|
||||||
}).DialContext,
|
|
||||||
MaxIdleConns: 100,
|
MaxIdleConns: 100,
|
||||||
MaxIdleConnsPerHost: 10,
|
MaxIdleConnsPerHost: 10,
|
||||||
IdleConnTimeout: 90 * time.Second,
|
IdleConnTimeout: 90 * time.Second,
|
||||||
|
|||||||
229
src/imagetar.go
229
src/imagetar.go
@@ -33,16 +33,18 @@ type DebounceEntry struct {
|
|||||||
|
|
||||||
// DownloadDebouncer 下载防抖器
|
// DownloadDebouncer 下载防抖器
|
||||||
type DownloadDebouncer struct {
|
type DownloadDebouncer struct {
|
||||||
mu sync.RWMutex
|
mu sync.RWMutex
|
||||||
entries map[string]*DebounceEntry
|
entries map[string]*DebounceEntry
|
||||||
window time.Duration
|
window time.Duration
|
||||||
|
lastCleanup time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewDownloadDebouncer 创建下载防抖器
|
// NewDownloadDebouncer 创建下载防抖器
|
||||||
func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
|
func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
|
||||||
return &DownloadDebouncer{
|
return &DownloadDebouncer{
|
||||||
entries: make(map[string]*DebounceEntry),
|
entries: make(map[string]*DebounceEntry),
|
||||||
window: window,
|
window: window,
|
||||||
|
lastCleanup: time.Now(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -66,9 +68,10 @@ func (d *DownloadDebouncer) ShouldAllow(userID, contentKey string) bool {
|
|||||||
UserID: userID,
|
UserID: userID,
|
||||||
}
|
}
|
||||||
|
|
||||||
// 清理过期条目(简单策略:每100次请求清理一次)
|
// 清理过期条目(每5分钟清理一次)
|
||||||
if len(d.entries)%100 == 0 {
|
if time.Since(d.lastCleanup) > 5*time.Minute {
|
||||||
d.cleanup(now)
|
d.cleanup(now)
|
||||||
|
d.lastCleanup = now
|
||||||
}
|
}
|
||||||
|
|
||||||
return true
|
return true
|
||||||
@@ -128,8 +131,8 @@ var (
|
|||||||
func initDebouncer() {
|
func initDebouncer() {
|
||||||
// 单个镜像:5秒防抖窗口
|
// 单个镜像:5秒防抖窗口
|
||||||
singleImageDebouncer = NewDownloadDebouncer(5 * time.Second)
|
singleImageDebouncer = NewDownloadDebouncer(5 * time.Second)
|
||||||
// 批量镜像:30秒防抖窗口(影响更大,需要更长保护)
|
// 批量镜像:60秒防抖窗口
|
||||||
batchImageDebouncer = NewDownloadDebouncer(30 * time.Second)
|
batchImageDebouncer = NewDownloadDebouncer(60 * time.Second)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ImageStreamer 镜像流式下载器
|
// ImageStreamer 镜像流式下载器
|
||||||
@@ -171,14 +174,15 @@ func NewImageStreamer(config *ImageStreamerConfig) *ImageStreamer {
|
|||||||
|
|
||||||
// StreamOptions 下载选项
|
// StreamOptions 下载选项
|
||||||
type StreamOptions struct {
|
type StreamOptions struct {
|
||||||
Platform string
|
Platform string
|
||||||
Compression bool
|
Compression bool
|
||||||
|
UseCompressedLayers bool // 是否保存原始压缩层,默认开启
|
||||||
}
|
}
|
||||||
|
|
||||||
// StreamImageToWriter 流式下载镜像到Writer
|
// StreamImageToWriter 流式下载镜像到Writer
|
||||||
func (is *ImageStreamer) StreamImageToWriter(ctx context.Context, imageRef string, writer io.Writer, options *StreamOptions) error {
|
func (is *ImageStreamer) StreamImageToWriter(ctx context.Context, imageRef string, writer io.Writer, options *StreamOptions) error {
|
||||||
if options == nil {
|
if options == nil {
|
||||||
options = &StreamOptions{}
|
options = &StreamOptions{UseCompressedLayers: true}
|
||||||
}
|
}
|
||||||
|
|
||||||
ref, err := name.ParseReference(imageRef)
|
ref, err := name.ParseReference(imageRef)
|
||||||
@@ -211,57 +215,14 @@ func (is *ImageStreamer) getImageDescriptor(ref name.Reference, options []remote
|
|||||||
|
|
||||||
// getImageDescriptorWithPlatform 获取指定平台的镜像描述符
|
// getImageDescriptorWithPlatform 获取指定平台的镜像描述符
|
||||||
func (is *ImageStreamer) getImageDescriptorWithPlatform(ref name.Reference, options []remote.Option, platform string) (*remote.Descriptor, error) {
|
func (is *ImageStreamer) getImageDescriptorWithPlatform(ref name.Reference, options []remote.Option, platform string) (*remote.Descriptor, error) {
|
||||||
if isCacheEnabled() {
|
// 直接从网络获取完整的descriptor,确保对象完整性
|
||||||
var reference string
|
return remote.Get(ref, options...)
|
||||||
if tagged, ok := ref.(name.Tag); ok {
|
|
||||||
reference = tagged.TagStr()
|
|
||||||
} else if digested, ok := ref.(name.Digest); ok {
|
|
||||||
reference = digested.DigestStr()
|
|
||||||
}
|
|
||||||
|
|
||||||
if reference != "" {
|
|
||||||
cacheKey := buildManifestCacheKeyWithPlatform(ref.Context().String(), reference, platform)
|
|
||||||
if cachedItem := globalCache.Get(cacheKey); cachedItem != nil {
|
|
||||||
desc := &remote.Descriptor{
|
|
||||||
Manifest: cachedItem.Data,
|
|
||||||
}
|
|
||||||
log.Printf("使用缓存的manifest: %s (平台: %s)", ref.String(), platform)
|
|
||||||
return desc, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
desc, err := remote.Get(ref, options...)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if isCacheEnabled() {
|
|
||||||
var reference string
|
|
||||||
if tagged, ok := ref.(name.Tag); ok {
|
|
||||||
reference = tagged.TagStr()
|
|
||||||
} else if digested, ok := ref.(name.Digest); ok {
|
|
||||||
reference = digested.DigestStr()
|
|
||||||
}
|
|
||||||
|
|
||||||
if reference != "" {
|
|
||||||
cacheKey := buildManifestCacheKeyWithPlatform(ref.Context().String(), reference, platform)
|
|
||||||
ttl := getManifestTTL(reference)
|
|
||||||
headers := map[string]string{
|
|
||||||
"Docker-Content-Digest": desc.Digest.String(),
|
|
||||||
}
|
|
||||||
globalCache.Set(cacheKey, desc.Manifest, string(desc.MediaType), headers, ttl)
|
|
||||||
log.Printf("缓存manifest: %s (平台: %s, TTL: %v)", ref.String(), platform, ttl)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return desc, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// StreamImageToGin 流式响应到Gin
|
// StreamImageToGin 流式响应到Gin
|
||||||
func (is *ImageStreamer) StreamImageToGin(ctx context.Context, imageRef string, c *gin.Context, options *StreamOptions) error {
|
func (is *ImageStreamer) StreamImageToGin(ctx context.Context, imageRef string, c *gin.Context, options *StreamOptions) error {
|
||||||
if options == nil {
|
if options == nil {
|
||||||
options = &StreamOptions{}
|
options = &StreamOptions{UseCompressedLayers: true}
|
||||||
}
|
}
|
||||||
|
|
||||||
filename := strings.ReplaceAll(imageRef, "/", "_") + ".tar"
|
filename := strings.ReplaceAll(imageRef, "/", "_") + ".tar"
|
||||||
@@ -320,16 +281,16 @@ func (is *ImageStreamer) streamImageLayers(ctx context.Context, img v1.Image, wr
|
|||||||
|
|
||||||
log.Printf("镜像包含 %d 层", len(layers))
|
log.Printf("镜像包含 %d 层", len(layers))
|
||||||
|
|
||||||
return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef)
|
return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef, options)
|
||||||
}
|
}
|
||||||
|
|
||||||
// streamDockerFormat 生成Docker格式
|
// streamDockerFormat 生成Docker格式
|
||||||
func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string) error {
|
func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, options *StreamOptions) error {
|
||||||
return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil)
|
return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil, options)
|
||||||
}
|
}
|
||||||
|
|
||||||
// streamDockerFormatWithReturn 生成Docker格式并返回manifest和repositories信息
|
// streamDockerFormatWithReturn 生成Docker格式并返回manifest和repositories信息
|
||||||
func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string) error {
|
func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string, options *StreamOptions) error {
|
||||||
configDigest, err := img.ConfigName()
|
configDigest, err := img.ConfigName()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -379,12 +340,24 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
uncompressedSize, err := partial.UncompressedSize(layer)
|
var layerSize int64
|
||||||
if err != nil {
|
var layerReader io.ReadCloser
|
||||||
return err
|
|
||||||
|
// 根据配置选择使用压缩层或未压缩层
|
||||||
|
if options != nil && options.UseCompressedLayers {
|
||||||
|
layerSize, err = layer.Size()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
layerReader, err = layer.Compressed()
|
||||||
|
} else {
|
||||||
|
layerSize, err = partial.UncompressedSize(layer)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
layerReader, err = layer.Uncompressed()
|
||||||
}
|
}
|
||||||
|
|
||||||
layerReader, err := layer.Uncompressed()
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -392,7 +365,7 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
|
|||||||
|
|
||||||
layerTarHeader := &tar.Header{
|
layerTarHeader := &tar.Header{
|
||||||
Name: layerDir + "/layer.tar",
|
Name: layerDir + "/layer.tar",
|
||||||
Size: uncompressedSize,
|
Size: layerSize,
|
||||||
Mode: 0644,
|
Mode: 0644,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -484,7 +457,31 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// streamSingleImageForBatch 为批量下载流式处理单个镜像
|
// processImageForBatch 处理镜像的公共逻辑,用于批量下载
|
||||||
|
func (is *ImageStreamer) processImageForBatch(ctx context.Context, img v1.Image, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
|
||||||
|
layers, err := img.Layers()
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
configFile, err := img.ConfigFile()
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Printf("镜像包含 %d 层", len(layers))
|
||||||
|
|
||||||
|
var manifest map[string]interface{}
|
||||||
|
var repositories map[string]map[string]string
|
||||||
|
|
||||||
|
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories, options)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return manifest, repositories, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
|
func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
|
||||||
ref, err := name.ParseReference(imageRef)
|
ref, err := name.ParseReference(imageRef)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -498,80 +495,28 @@ func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWrite
|
|||||||
return nil, nil, fmt.Errorf("获取镜像描述失败: %w", err)
|
return nil, nil, fmt.Errorf("获取镜像描述失败: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
var manifest map[string]interface{}
|
var img v1.Image
|
||||||
var repositories map[string]map[string]string
|
|
||||||
|
|
||||||
switch desc.MediaType {
|
switch desc.MediaType {
|
||||||
case types.OCIImageIndex, types.DockerManifestList:
|
case types.OCIImageIndex, types.DockerManifestList:
|
||||||
// 处理多架构镜像,复用单个下载的逻辑
|
// 处理多架构镜像
|
||||||
img, err := is.selectPlatformImage(desc, options)
|
img, err = is.selectPlatformImage(desc, options)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, nil, fmt.Errorf("选择平台镜像失败: %w", err)
|
return nil, nil, fmt.Errorf("选择平台镜像失败: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
layers, err := img.Layers()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
configFile, err := img.ConfigFile()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
log.Printf("镜像包含 %d 层", len(layers))
|
|
||||||
|
|
||||||
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
case types.OCIManifestSchema1, types.DockerManifestSchema2:
|
case types.OCIManifestSchema1, types.DockerManifestSchema2:
|
||||||
img, err := desc.Image()
|
img, err = desc.Image()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
|
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
layers, err := img.Layers()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
configFile, err := img.ConfigFile()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
log.Printf("镜像包含 %d 层", len(layers))
|
|
||||||
|
|
||||||
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
default:
|
default:
|
||||||
img, err := desc.Image()
|
img, err = desc.Image()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
|
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
layers, err := img.Layers()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
configFile, err := img.ConfigFile()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
log.Printf("镜像包含 %d 层", len(layers))
|
|
||||||
|
|
||||||
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return manifest, repositories, nil
|
return is.processImageForBatch(ctx, img, tarWriter, imageRef, options)
|
||||||
}
|
}
|
||||||
|
|
||||||
// selectPlatformImage 从多架构镜像中选择合适的平台镜像
|
// selectPlatformImage 从多架构镜像中选择合适的平台镜像
|
||||||
@@ -637,7 +582,6 @@ var globalImageStreamer *ImageStreamer
|
|||||||
// initImageStreamer 初始化镜像下载器
|
// initImageStreamer 初始化镜像下载器
|
||||||
func initImageStreamer() {
|
func initImageStreamer() {
|
||||||
globalImageStreamer = NewImageStreamer(nil)
|
globalImageStreamer = NewImageStreamer(nil)
|
||||||
// 镜像下载器初始化完成
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// formatPlatformText 格式化平台文本
|
// formatPlatformText 格式化平台文本
|
||||||
@@ -652,9 +596,9 @@ func formatPlatformText(platform string) string {
|
|||||||
func initImageTarRoutes(router *gin.Engine) {
|
func initImageTarRoutes(router *gin.Engine) {
|
||||||
imageAPI := router.Group("/api/image")
|
imageAPI := router.Group("/api/image")
|
||||||
{
|
{
|
||||||
imageAPI.GET("/download/:image", RateLimitMiddleware(globalLimiter), handleDirectImageDownload)
|
imageAPI.GET("/download/:image", handleDirectImageDownload)
|
||||||
imageAPI.GET("/info/:image", RateLimitMiddleware(globalLimiter), handleImageInfo)
|
imageAPI.GET("/info/:image", handleImageInfo)
|
||||||
imageAPI.POST("/batch", RateLimitMiddleware(globalLimiter), handleSimpleBatchDownload)
|
imageAPI.POST("/batch", handleSimpleBatchDownload)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -669,6 +613,7 @@ func handleDirectImageDownload(c *gin.Context) {
|
|||||||
imageRef := strings.ReplaceAll(imageParam, "_", "/")
|
imageRef := strings.ReplaceAll(imageParam, "_", "/")
|
||||||
platform := c.Query("platform")
|
platform := c.Query("platform")
|
||||||
tag := c.DefaultQuery("tag", "")
|
tag := c.DefaultQuery("tag", "")
|
||||||
|
useCompressed := c.DefaultQuery("compressed", "true") == "true"
|
||||||
|
|
||||||
if tag != "" && !strings.Contains(imageRef, ":") && !strings.Contains(imageRef, "@") {
|
if tag != "" && !strings.Contains(imageRef, ":") && !strings.Contains(imageRef, "@") {
|
||||||
imageRef = imageRef + ":" + tag
|
imageRef = imageRef + ":" + tag
|
||||||
@@ -694,8 +639,9 @@ func handleDirectImageDownload(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
options := &StreamOptions{
|
options := &StreamOptions{
|
||||||
Platform: platform,
|
Platform: platform,
|
||||||
Compression: false,
|
Compression: false,
|
||||||
|
UseCompressedLayers: useCompressed,
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx := c.Request.Context()
|
ctx := c.Request.Context()
|
||||||
@@ -711,8 +657,9 @@ func handleDirectImageDownload(c *gin.Context) {
|
|||||||
// handleSimpleBatchDownload 处理批量下载
|
// handleSimpleBatchDownload 处理批量下载
|
||||||
func handleSimpleBatchDownload(c *gin.Context) {
|
func handleSimpleBatchDownload(c *gin.Context) {
|
||||||
var req struct {
|
var req struct {
|
||||||
Images []string `json:"images" binding:"required"`
|
Images []string `json:"images" binding:"required"`
|
||||||
Platform string `json:"platform"`
|
Platform string `json:"platform"`
|
||||||
|
UseCompressedLayers *bool `json:"useCompressedLayers"`
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := c.ShouldBindJSON(&req); err != nil {
|
if err := c.ShouldBindJSON(&req); err != nil {
|
||||||
@@ -746,14 +693,20 @@ func handleSimpleBatchDownload(c *gin.Context) {
|
|||||||
if !batchImageDebouncer.ShouldAllow(userID, contentKey) {
|
if !batchImageDebouncer.ShouldAllow(userID, contentKey) {
|
||||||
c.JSON(http.StatusTooManyRequests, gin.H{
|
c.JSON(http.StatusTooManyRequests, gin.H{
|
||||||
"error": "批量下载请求过于频繁,请稍后再试",
|
"error": "批量下载请求过于频繁,请稍后再试",
|
||||||
"retry_after": 30,
|
"retry_after": 60,
|
||||||
})
|
})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
useCompressed := true // 默认启用原始压缩层
|
||||||
|
if req.UseCompressedLayers != nil {
|
||||||
|
useCompressed = *req.UseCompressedLayers
|
||||||
|
}
|
||||||
|
|
||||||
options := &StreamOptions{
|
options := &StreamOptions{
|
||||||
Platform: req.Platform,
|
Platform: req.Platform,
|
||||||
Compression: false,
|
Compression: false,
|
||||||
|
UseCompressedLayers: useCompressed,
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx := c.Request.Context()
|
ctx := c.Request.Context()
|
||||||
@@ -833,7 +786,7 @@ func handleImageInfo(c *gin.Context) {
|
|||||||
// StreamMultipleImages 批量下载多个镜像
|
// StreamMultipleImages 批量下载多个镜像
|
||||||
func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []string, writer io.Writer, options *StreamOptions) error {
|
func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []string, writer io.Writer, options *StreamOptions) error {
|
||||||
if options == nil {
|
if options == nil {
|
||||||
options = &StreamOptions{}
|
options = &StreamOptions{UseCompressedLayers: true}
|
||||||
}
|
}
|
||||||
|
|
||||||
var finalWriter io.Writer = writer
|
var finalWriter io.Writer = writer
|
||||||
|
|||||||
19
src/main.go
19
src/main.go
@@ -84,6 +84,9 @@ func main() {
|
|||||||
})
|
})
|
||||||
}))
|
}))
|
||||||
|
|
||||||
|
// 全局限流中间件 - 应用到所有路由
|
||||||
|
router.Use(RateLimitMiddleware(globalLimiter))
|
||||||
|
|
||||||
// 初始化监控端点
|
// 初始化监控端点
|
||||||
initHealthRoutes(router)
|
initHealthRoutes(router)
|
||||||
|
|
||||||
@@ -113,15 +116,15 @@ func main() {
|
|||||||
RegisterSearchRoute(router)
|
RegisterSearchRoute(router)
|
||||||
|
|
||||||
// 注册Docker认证路由(/token*)
|
// 注册Docker认证路由(/token*)
|
||||||
router.Any("/token", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin)
|
router.Any("/token", ProxyDockerAuthGin)
|
||||||
router.Any("/token/*path", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin)
|
router.Any("/token/*path", ProxyDockerAuthGin)
|
||||||
|
|
||||||
// 注册Docker Registry代理路由
|
// 注册Docker Registry代理路由
|
||||||
router.Any("/v2/*path", RateLimitMiddleware(globalLimiter), ProxyDockerRegistryGin)
|
router.Any("/v2/*path", ProxyDockerRegistryGin)
|
||||||
|
|
||||||
|
|
||||||
// 注册NoRoute处理器
|
// 注册NoRoute处理器
|
||||||
router.NoRoute(RateLimitMiddleware(globalLimiter), handler)
|
router.NoRoute(handler)
|
||||||
|
|
||||||
cfg := GetConfig()
|
cfg := GetConfig()
|
||||||
fmt.Printf("🚀 HubProxy 启动成功\n")
|
fmt.Printf("🚀 HubProxy 启动成功\n")
|
||||||
@@ -171,11 +174,11 @@ func handler(c *gin.Context) {
|
|||||||
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
|
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
proxy(c, rawPath)
|
proxyRequest(c, rawPath)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
func proxy(c *gin.Context, u string) {
|
func proxyRequest(c *gin.Context, u string) {
|
||||||
proxyWithRedirect(c, u, 0)
|
proxyWithRedirect(c, u, 0)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -295,9 +298,7 @@ func proxyWithRedirect(c *gin.Context, u string, redirectCount int) {
|
|||||||
c.Status(resp.StatusCode)
|
c.Status(resp.StatusCode)
|
||||||
|
|
||||||
// 直接流式转发
|
// 直接流式转发
|
||||||
if _, err := io.Copy(c.Writer, resp.Body); err != nil {
|
io.Copy(c.Writer, resp.Body)
|
||||||
fmt.Printf("直接代理失败: %v\n", err)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -399,6 +399,67 @@
|
|||||||
100% { transform: rotate(360deg); }
|
100% { transform: rotate(360deg); }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* 切换开关样式 */
|
||||||
|
.switch-container {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.75rem;
|
||||||
|
margin-bottom: 1.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.switch {
|
||||||
|
position: relative;
|
||||||
|
display: inline-block;
|
||||||
|
width: 50px;
|
||||||
|
height: 24px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.switch input {
|
||||||
|
opacity: 0;
|
||||||
|
width: 0;
|
||||||
|
height: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.slider {
|
||||||
|
position: absolute;
|
||||||
|
cursor: pointer;
|
||||||
|
top: 0;
|
||||||
|
left: 0;
|
||||||
|
right: 0;
|
||||||
|
bottom: 0;
|
||||||
|
background-color: var(--muted);
|
||||||
|
transition: 0.2s;
|
||||||
|
border-radius: 24px;
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
}
|
||||||
|
|
||||||
|
.slider:before {
|
||||||
|
position: absolute;
|
||||||
|
content: "";
|
||||||
|
height: 18px;
|
||||||
|
width: 18px;
|
||||||
|
left: 2px;
|
||||||
|
bottom: 2px;
|
||||||
|
background-color: white;
|
||||||
|
transition: 0.2s;
|
||||||
|
border-radius: 50%;
|
||||||
|
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
|
||||||
|
}
|
||||||
|
|
||||||
|
input:checked + .slider {
|
||||||
|
background-color: var(--primary);
|
||||||
|
}
|
||||||
|
|
||||||
|
input:checked + .slider:before {
|
||||||
|
transform: translateX(26px);
|
||||||
|
}
|
||||||
|
|
||||||
|
.switch-label {
|
||||||
|
font-weight: 500;
|
||||||
|
color: var(--foreground);
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
.hidden {
|
.hidden {
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
@@ -559,6 +620,14 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="switch-container">
|
||||||
|
<label class="switch">
|
||||||
|
<input type="checkbox" id="compressedToggle" checked>
|
||||||
|
<span class="slider"></span>
|
||||||
|
</label>
|
||||||
|
<label for="compressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
<button type="submit" class="btn btn-primary btn-full" id="downloadBtn">
|
<button type="submit" class="btn btn-primary btn-full" id="downloadBtn">
|
||||||
<span id="downloadText">立即下载</span>
|
<span id="downloadText">立即下载</span>
|
||||||
<span id="downloadLoading" class="loading hidden"></span>
|
<span id="downloadLoading" class="loading hidden"></span>
|
||||||
@@ -595,6 +664,14 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="switch-container">
|
||||||
|
<label class="switch">
|
||||||
|
<input type="checkbox" id="batchCompressedToggle" checked>
|
||||||
|
<span class="slider"></span>
|
||||||
|
</label>
|
||||||
|
<label for="batchCompressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
<button type="submit" class="btn btn-primary btn-full" id="batchDownloadBtn">
|
<button type="submit" class="btn btn-primary btn-full" id="batchDownloadBtn">
|
||||||
<span id="batchDownloadText">开始下载</span>
|
<span id="batchDownloadText">开始下载</span>
|
||||||
<span id="batchDownloadLoading" class="loading hidden"></span>
|
<span id="batchDownloadLoading" class="loading hidden"></span>
|
||||||
@@ -651,12 +728,18 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function buildDownloadUrl(imageName, platform = '') {
|
function buildDownloadUrl(imageName, platform = '', useCompressed = true) {
|
||||||
const encodedImage = imageName.replace(/\//g, '_');
|
const encodedImage = imageName.replace(/\//g, '_');
|
||||||
let url = `/api/image/download/${encodedImage}`;
|
let url = `/api/image/download/${encodedImage}`;
|
||||||
|
|
||||||
|
const params = new URLSearchParams();
|
||||||
if (platform && platform.trim()) {
|
if (platform && platform.trim()) {
|
||||||
url += `?platform=${encodeURIComponent(platform.trim())}`;
|
params.append('platform', platform.trim());
|
||||||
|
}
|
||||||
|
params.append('compressed', useCompressed.toString());
|
||||||
|
|
||||||
|
if (params.toString()) {
|
||||||
|
url += '?' + params.toString();
|
||||||
}
|
}
|
||||||
|
|
||||||
return url;
|
return url;
|
||||||
@@ -672,11 +755,12 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
const platform = document.getElementById('platformInput').value.trim();
|
const platform = document.getElementById('platformInput').value.trim();
|
||||||
|
const useCompressed = document.getElementById('compressedToggle').checked;
|
||||||
|
|
||||||
hideStatus('singleStatus');
|
hideStatus('singleStatus');
|
||||||
setButtonLoading('downloadBtn', 'downloadText', 'downloadLoading', true);
|
setButtonLoading('downloadBtn', 'downloadText', 'downloadLoading', true);
|
||||||
|
|
||||||
const downloadUrl = buildDownloadUrl(imageName, platform);
|
const downloadUrl = buildDownloadUrl(imageName, platform, useCompressed);
|
||||||
|
|
||||||
const link = document.createElement('a');
|
const link = document.createElement('a');
|
||||||
link.href = downloadUrl;
|
link.href = downloadUrl;
|
||||||
@@ -711,9 +795,11 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
const platform = document.getElementById('batchPlatformInput').value.trim();
|
const platform = document.getElementById('batchPlatformInput').value.trim();
|
||||||
|
const useCompressed = document.getElementById('batchCompressedToggle').checked;
|
||||||
|
|
||||||
const options = {
|
const options = {
|
||||||
images: images
|
images: images,
|
||||||
|
useCompressedLayers: useCompressed
|
||||||
};
|
};
|
||||||
|
|
||||||
if (platform) {
|
if (platform) {
|
||||||
|
|||||||
@@ -132,23 +132,33 @@ func (i *IPRateLimiter) cleanupRoutine() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// extractIPFromAddress 从地址中提取纯IP,去除端口号
|
// extractIPFromAddress 从地址中提取纯IP
|
||||||
func extractIPFromAddress(address string) string {
|
func extractIPFromAddress(address string) string {
|
||||||
// 处理IPv6地址 [::1]:8080 格式
|
if host, _, err := net.SplitHostPort(address); err == nil {
|
||||||
if strings.HasPrefix(address, "[") {
|
return host
|
||||||
if endIndex := strings.Index(address, "]"); endIndex != -1 {
|
|
||||||
return address[1:endIndex]
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// 处理IPv4地址 192.168.1.1:8080 格式
|
|
||||||
if lastColon := strings.LastIndex(address, ":"); lastColon != -1 {
|
|
||||||
return address[:lastColon]
|
|
||||||
}
|
|
||||||
|
|
||||||
return address
|
return address
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// normalizeIPForRateLimit 标准化IP地址用于限流:IPv4保持不变,IPv6标准化为/64网段
|
||||||
|
func normalizeIPForRateLimit(ipStr string) string {
|
||||||
|
ip := net.ParseIP(ipStr)
|
||||||
|
if ip == nil {
|
||||||
|
return ipStr // 解析失败,返回原值
|
||||||
|
}
|
||||||
|
|
||||||
|
if ip.To4() != nil {
|
||||||
|
return ipStr // IPv4保持不变
|
||||||
|
}
|
||||||
|
|
||||||
|
// IPv6:标准化为 /64 网段
|
||||||
|
ipv6 := ip.To16()
|
||||||
|
for i := 8; i < 16; i++ {
|
||||||
|
ipv6[i] = 0 // 清零后64位
|
||||||
|
}
|
||||||
|
return ipv6.String() + "/64"
|
||||||
|
}
|
||||||
|
|
||||||
// isIPInCIDRList 检查IP是否在CIDR列表中
|
// isIPInCIDRList 检查IP是否在CIDR列表中
|
||||||
func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
|
func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
|
||||||
// 先提取纯IP地址
|
// 先提取纯IP地址
|
||||||
@@ -181,15 +191,18 @@ func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
|
|||||||
return rate.NewLimiter(rate.Inf, i.b), true
|
return rate.NewLimiter(rate.Inf, i.b), true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 标准化IP用于限流:IPv4保持不变,IPv6标准化为/64网段
|
||||||
|
normalizedIP := normalizeIPForRateLimit(cleanIP)
|
||||||
|
|
||||||
now := time.Now()
|
now := time.Now()
|
||||||
|
|
||||||
i.mu.RLock()
|
i.mu.RLock()
|
||||||
entry, exists := i.ips[cleanIP]
|
entry, exists := i.ips[normalizedIP]
|
||||||
i.mu.RUnlock()
|
i.mu.RUnlock()
|
||||||
|
|
||||||
if exists {
|
if exists {
|
||||||
i.mu.Lock()
|
i.mu.Lock()
|
||||||
if entry, stillExists := i.ips[cleanIP]; stillExists {
|
if entry, stillExists := i.ips[normalizedIP]; stillExists {
|
||||||
entry.lastAccess = now
|
entry.lastAccess = now
|
||||||
i.mu.Unlock()
|
i.mu.Unlock()
|
||||||
return entry.limiter, true
|
return entry.limiter, true
|
||||||
@@ -198,7 +211,7 @@ func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
i.mu.Lock()
|
i.mu.Lock()
|
||||||
if entry, exists := i.ips[cleanIP]; exists {
|
if entry, exists := i.ips[normalizedIP]; exists {
|
||||||
entry.lastAccess = now
|
entry.lastAccess = now
|
||||||
i.mu.Unlock()
|
i.mu.Unlock()
|
||||||
return entry.limiter, true
|
return entry.limiter, true
|
||||||
@@ -208,7 +221,7 @@ func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
|
|||||||
limiter: rate.NewLimiter(i.r, i.b),
|
limiter: rate.NewLimiter(i.r, i.b),
|
||||||
lastAccess: now,
|
lastAccess: now,
|
||||||
}
|
}
|
||||||
i.ips[cleanIP] = entry
|
i.ips[normalizedIP] = entry
|
||||||
i.mu.Unlock()
|
i.mu.Unlock()
|
||||||
|
|
||||||
return entry.limiter, true
|
return entry.limiter, true
|
||||||
@@ -217,6 +230,14 @@ func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
|
|||||||
// RateLimitMiddleware 速率限制中间件
|
// RateLimitMiddleware 速率限制中间件
|
||||||
func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
|
func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
|
||||||
return func(c *gin.Context) {
|
return func(c *gin.Context) {
|
||||||
|
// 静态文件豁免:跳过限流检查
|
||||||
|
path := c.Request.URL.Path
|
||||||
|
if path == "/" || path == "/favicon.ico" || path == "/images.html" || path == "/search.html" ||
|
||||||
|
strings.HasPrefix(path, "/public/") {
|
||||||
|
c.Next()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
// 获取客户端真实IP
|
// 获取客户端真实IP
|
||||||
var ip string
|
var ip string
|
||||||
|
|
||||||
@@ -237,15 +258,22 @@ func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
|
|||||||
ip = c.ClientIP()
|
ip = c.ClientIP()
|
||||||
}
|
}
|
||||||
|
|
||||||
// 提取纯IP地址(去除端口号)
|
// 提取纯IP地址(去除可能存在的端口)
|
||||||
cleanIP := extractIPFromAddress(ip)
|
cleanIP := extractIPFromAddress(ip)
|
||||||
|
|
||||||
// 日志记录请求IP和头信息
|
// 日志记录请求IP和头信息
|
||||||
fmt.Printf("请求IP: %s (去除端口后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
|
normalizedIP := normalizeIPForRateLimit(cleanIP)
|
||||||
ip,
|
if cleanIP != normalizedIP {
|
||||||
cleanIP,
|
fmt.Printf("请求IP: %s (提纯后: %s, 限流段: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
|
||||||
c.GetHeader("X-Forwarded-For"),
|
ip, cleanIP, normalizedIP,
|
||||||
c.GetHeader("X-Real-IP"))
|
c.GetHeader("X-Forwarded-For"),
|
||||||
|
c.GetHeader("X-Real-IP"))
|
||||||
|
} else {
|
||||||
|
fmt.Printf("请求IP: %s (提纯后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
|
||||||
|
ip, cleanIP,
|
||||||
|
c.GetHeader("X-Forwarded-For"),
|
||||||
|
c.GetHeader("X-Real-IP"))
|
||||||
|
}
|
||||||
|
|
||||||
// 获取限流器并检查是否允许访问
|
// 获取限流器并检查是否允许访问
|
||||||
ipLimiter, allowed := limiter.GetLimiter(cleanIP)
|
ipLimiter, allowed := limiter.GetLimiter(cleanIP)
|
||||||
@@ -259,11 +287,8 @@ func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// 智能限流判断:检查是否应该跳过限流计数
|
// 检查限流
|
||||||
shouldSkip := smartLimiter.ShouldSkipRateLimit(cleanIP, c.Request.URL.Path)
|
if !ipLimiter.Allow() {
|
||||||
|
|
||||||
// 只有在不跳过的情况下才检查限流
|
|
||||||
if !shouldSkip && !ipLimiter.Allow() {
|
|
||||||
c.JSON(429, gin.H{
|
c.JSON(429, gin.H{
|
||||||
"error": "请求频率过快,暂时限制访问",
|
"error": "请求频率过快,暂时限制访问",
|
||||||
})
|
})
|
||||||
@@ -275,25 +300,4 @@ func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ApplyRateLimit 应用限流到特定路由
|
|
||||||
func ApplyRateLimit(router *gin.Engine, path string, method string, handler gin.HandlerFunc) {
|
|
||||||
// 使用全局限流器
|
|
||||||
limiter := globalLimiter
|
|
||||||
if limiter == nil {
|
|
||||||
limiter = initGlobalLimiter()
|
|
||||||
}
|
|
||||||
|
|
||||||
// 根据HTTP方法应用限流
|
|
||||||
switch method {
|
|
||||||
case "GET":
|
|
||||||
router.GET(path, RateLimitMiddleware(limiter), handler)
|
|
||||||
case "POST":
|
|
||||||
router.POST(path, RateLimitMiddleware(limiter), handler)
|
|
||||||
case "PUT":
|
|
||||||
router.PUT(path, RateLimitMiddleware(limiter), handler)
|
|
||||||
case "DELETE":
|
|
||||||
router.DELETE(path, RateLimitMiddleware(limiter), handler)
|
|
||||||
default:
|
|
||||||
router.Any(path, RateLimitMiddleware(limiter), handler)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,108 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
"sync"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// SmartRateLimit 智能限流会话管理
|
|
||||||
type SmartRateLimit struct {
|
|
||||||
sessions sync.Map
|
|
||||||
}
|
|
||||||
|
|
||||||
// PullSession Docker拉取会话
|
|
||||||
type PullSession struct {
|
|
||||||
LastManifestTime time.Time
|
|
||||||
RequestCount int
|
|
||||||
}
|
|
||||||
|
|
||||||
// 全局智能限流实例
|
|
||||||
var smartLimiter = &SmartRateLimit{}
|
|
||||||
|
|
||||||
const (
|
|
||||||
// manifest请求后的活跃窗口时间
|
|
||||||
activeWindowDuration = 3 * time.Minute
|
|
||||||
// 活跃窗口内最大免费blob请求数(防止滥用)
|
|
||||||
maxFreeBlobRequests = 100
|
|
||||||
sessionCleanupInterval = 10 * time.Minute
|
|
||||||
sessionExpireTime = 30 * time.Minute
|
|
||||||
)
|
|
||||||
|
|
||||||
func init() {
|
|
||||||
go smartLimiter.cleanupSessions()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ShouldSkipRateLimit 判断是否应该跳过限流计数
|
|
||||||
func (s *SmartRateLimit) ShouldSkipRateLimit(ip, path string) bool {
|
|
||||||
requestType, _ := parseRequestInfo(path)
|
|
||||||
|
|
||||||
if requestType != "manifests" && requestType != "blobs" {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
sessionKey := ip
|
|
||||||
sessionInterface, _ := s.sessions.LoadOrStore(sessionKey, &PullSession{})
|
|
||||||
session := sessionInterface.(*PullSession)
|
|
||||||
|
|
||||||
now := time.Now()
|
|
||||||
|
|
||||||
if requestType == "manifests" {
|
|
||||||
session.LastManifestTime = now
|
|
||||||
session.RequestCount = 0
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
if requestType == "blobs" {
|
|
||||||
if !session.LastManifestTime.IsZero() &&
|
|
||||||
now.Sub(session.LastManifestTime) <= activeWindowDuration {
|
|
||||||
|
|
||||||
session.RequestCount++
|
|
||||||
if session.RequestCount <= maxFreeBlobRequests {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseRequestInfo(path string) (requestType, imageRef string) {
|
|
||||||
path = strings.TrimPrefix(path, "/v2/")
|
|
||||||
|
|
||||||
if idx := strings.Index(path, "/manifests/"); idx != -1 {
|
|
||||||
return "manifests", path[:idx]
|
|
||||||
}
|
|
||||||
if idx := strings.Index(path, "/blobs/"); idx != -1 {
|
|
||||||
return "blobs", path[:idx]
|
|
||||||
}
|
|
||||||
if idx := strings.Index(path, "/tags/"); idx != -1 {
|
|
||||||
return "tags", path[:idx]
|
|
||||||
}
|
|
||||||
|
|
||||||
return "unknown", ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// cleanupSessions 定期清理过期会话,防止内存泄露
|
|
||||||
func (s *SmartRateLimit) cleanupSessions() {
|
|
||||||
ticker := time.NewTicker(sessionCleanupInterval)
|
|
||||||
defer ticker.Stop()
|
|
||||||
|
|
||||||
for range ticker.C {
|
|
||||||
now := time.Now()
|
|
||||||
expiredKeys := make([]string, 0)
|
|
||||||
|
|
||||||
s.sessions.Range(func(key, value interface{}) bool {
|
|
||||||
session := value.(*PullSession)
|
|
||||||
if !session.LastManifestTime.IsZero() &&
|
|
||||||
now.Sub(session.LastManifestTime) > sessionExpireTime {
|
|
||||||
expiredKeys = append(expiredKeys, key.(string))
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
})
|
|
||||||
|
|
||||||
for _, key := range expiredKeys {
|
|
||||||
s.sessions.Delete(key)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -71,14 +71,6 @@ func buildManifestCacheKey(imageRef, reference string) string {
|
|||||||
return buildCacheKey("manifest", key)
|
return buildCacheKey("manifest", key)
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildManifestCacheKeyWithPlatform(imageRef, reference, platform string) string {
|
|
||||||
if platform == "" {
|
|
||||||
platform = "default"
|
|
||||||
}
|
|
||||||
key := fmt.Sprintf("%s:%s@%s", imageRef, reference, platform)
|
|
||||||
return buildCacheKey("manifest", key)
|
|
||||||
}
|
|
||||||
|
|
||||||
func getManifestTTL(reference string) time.Duration {
|
func getManifestTTL(reference string) time.Duration {
|
||||||
cfg := GetConfig()
|
cfg := GetConfig()
|
||||||
defaultTTL := 30 * time.Minute
|
defaultTTL := 30 * time.Minute
|
||||||
@@ -99,7 +91,6 @@ func getManifestTTL(reference string) time.Duration {
|
|||||||
return 10 * time.Minute
|
return 10 * time.Minute
|
||||||
}
|
}
|
||||||
|
|
||||||
// 普通tag: 中等缓存时间
|
|
||||||
return defaultTTL
|
return defaultTTL
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -151,3 +142,27 @@ func isCacheEnabled() bool {
|
|||||||
func isTokenCacheEnabled() bool {
|
func isTokenCacheEnabled() bool {
|
||||||
return isCacheEnabled()
|
return isCacheEnabled()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 定期清理过期缓存,防止内存泄漏
|
||||||
|
func init() {
|
||||||
|
go func() {
|
||||||
|
ticker := time.NewTicker(20 * time.Minute)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
for range ticker.C {
|
||||||
|
now := time.Now()
|
||||||
|
expiredKeys := make([]string, 0)
|
||||||
|
|
||||||
|
globalCache.cache.Range(func(key, value interface{}) bool {
|
||||||
|
if cached := value.(*CachedItem); now.After(cached.ExpiresAt) {
|
||||||
|
expiredKeys = append(expiredKeys, key.(string))
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
})
|
||||||
|
|
||||||
|
for _, key := range expiredKeys {
|
||||||
|
globalCache.cache.Delete(key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user