Compare commits

..

67 Commits

Author SHA1 Message Date
Loyalsoldier 72d7e3f1b4 Add upload artifact function 2019-12-14 06:43:12 +08:00
loyalsoldier aa363d2d75 show last 10 lines of category-ads-all file 2019-12-14 02:47:35 +08:00
loyalsoldier 10619755c8 fixed with awk pattern 2019-12-14 02:39:22 +08:00
loyalsoldier 299cb070b8 try to fixed grep issue 2019-12-14 02:04:14 +08:00
loyalsoldier ea1de172c1 try to fixed grep issue 2019-12-14 01:59:59 +08:00
loyalsoldier 0400d050ca try to fixed grep issue 2019-12-14 01:47:16 +08:00
loyalsoldier 971bab7096 add more rules 2019-12-14 01:33:23 +08:00
Loyalsoldier 13b5b10770 Update README.md 2019-12-14 00:04:55 +08:00
loyalsoldier 929a4f73e5 fixed GitHub release token bug 2019-12-13 23:51:37 +08:00
loyalsoldier 5a7c7c2a26 fixed release parameters 2019-12-13 23:47:55 +08:00
loyalsoldier 3bd1c362fb fixed curl output bug 2019-12-13 23:42:49 +08:00
loyalsoldier 3edacd304d fixed path chaos 2019-12-13 23:36:57 +08:00
loyalsoldier c1deccb2fd delete shell script 2019-12-13 23:26:42 +08:00
loyalsoldier 7d91d0c277 move shell process into workflow actions 2019-12-13 23:26:24 +08:00
Loyalsoldier ea61cc9bce Update build.yml 2019-12-13 15:23:54 +08:00
Loyalsoldier 00c6275516 Update build.yml 2019-12-13 15:20:58 +08:00
Loyalsoldier 196ced99d7 Update README.md 2019-12-12 13:30:04 +08:00
Loyalsoldier 85489ec3d5 Update README.md 2019-12-12 13:27:38 +08:00
Loyalsoldier 785049ff53 Update build.yml 2019-12-12 13:03:04 +08:00
Loyalsoldier 9afe926c99 fixed escaped path 2019-12-12 12:08:36 +08:00
Loyalsoldier ff3f769df0 fixed missing gfwlist and chinalist rules 2019-12-12 11:38:43 +08:00
Loyalsoldier bdceaf9a2b Update build.yml 2019-12-11 21:54:03 +08:00
Loyalsoldier 414e79aaa4 Update generate-rules-dat.sh 2019-12-11 21:53:06 +08:00
Loyalsoldier 5840608a27 Update README.md 2019-12-11 07:22:13 +08:00
Loyalsoldier fc1aa53a09 Update README.md 2019-12-11 07:09:04 +08:00
Loyalsoldier 8ada2c4db3 Removed ADBLOCK_DOMAINS_URL 2019-12-11 06:45:24 +08:00
Loyalsoldier 907d82634d Add repo description and usage tutorial 2019-12-11 06:37:36 +08:00
loyalsoldier ec90e30c1b changed built method to scheduled time 2019-12-11 05:44:34 +08:00
loyalsoldier af9855776f fixed tag 2019-12-11 05:36:17 +08:00
loyalsoldier 672683e5e6 fixed tag generating method 2019-12-11 05:13:19 +08:00
loyalsoldier 785457c4a1 fixed tag generating method 2019-12-11 05:11:55 +08:00
loyalsoldier 1dd35d35c5 changed trigger method 2019-12-11 05:04:30 +08:00
Loyalsoldier e2f8ea527e Update build.yml 2019-12-11 04:57:55 +08:00
loyalsoldier 30c40ce945 fixed released tag 2019-12-11 04:19:50 +08:00
loyalsoldier 4672f77f83 comment latest tag step 2019-12-11 03:48:45 +08:00
loyalsoldier 02d3400e89 removed adblock blocks 2019-12-11 03:40:32 +08:00
loyalsoldier d1ddb676f0 fixed grep regexp 2019-12-11 03:37:02 +08:00
loyalsoldier 99f4a3f36f added adblock rules 2019-12-11 03:31:13 +08:00
loyalsoldier 3e7c61042f removed info blocks 2019-12-11 03:28:29 +08:00
loyalsoldier 244d2b3584 fixed file path 2019-12-11 03:20:38 +08:00
loyalsoldier 6b2ce83c56 fixed file path 2019-12-11 03:15:21 +08:00
loyalsoldier e47a5adb3a change debug place 2019-12-11 02:56:31 +08:00
loyalsoldier 3863d3a5d9 fixed file path 2019-12-11 02:44:37 +08:00
loyalsoldier 3101e3edfc change GOPATH to GitHub_Workspace 2019-12-11 02:43:22 +08:00
loyalsoldier 64d049f8a6 add GOPATH env 2019-12-11 02:37:52 +08:00
loyalsoldier 5caf66de4b fixed git checkout depth bug 2019-12-11 02:27:05 +08:00
loyalsoldier c22db61188 change workflow file name to active build 2019-12-11 02:25:37 +08:00
loyalsoldier e9db0c8cf1 fixed file path 2019-12-11 02:20:25 +08:00
loyalsoldier 1b52c0eb80 fixed file path 2019-12-10 19:28:01 +08:00
loyalsoldier 3a00e5a9ba fixed GOPATH 2019-12-10 19:12:25 +08:00
loyalsoldier 24e3297c2e comment go environment variable 2019-12-10 18:55:55 +08:00
loyalsoldier 719cb2451e fixed files path 2019-12-10 18:40:24 +08:00
loyalsoldier 1eae1a20ad fixed path 2019-12-10 18:32:28 +08:00
loyalsoldier b716301e40 show more dir info 2019-12-10 18:24:50 +08:00
loyalsoldier 3d60ebfe4a fixed file path 2019-12-10 17:44:41 +08:00
loyalsoldier 72f6ce1d87 fixed file path 2019-12-10 17:35:02 +08:00
loyalsoldier 112b0eea53 fixed minor spaces 2019-12-10 17:28:39 +08:00
loyalsoldier 65cc71e345 changed os back to ubuntu-latest 2019-12-10 17:17:09 +08:00
loyalsoldier caab116d5c testing adblock list 2019-12-10 17:12:39 +08:00
loyalsoldier b89e7de536 changed running os and show files 2019-12-10 17:04:56 +08:00
loyalsoldier 0a32b95979 add more notifications 2019-12-10 16:56:49 +08:00
loyalsoldier 11910af39d fixed v2ray folder non-existence bug 2019-12-10 16:45:51 +08:00
loyalsoldier c8e56d3305 fixed matrix vector 2019-12-10 16:30:22 +08:00
loyalsoldier 54fa0c38c2 Test to build every minute 2019-12-10 16:13:14 +08:00
loyalsoldier bbdc4117c0 chmod to 644 2019-12-10 16:05:23 +08:00
loyalsoldier 44567ac619 generate dat files 2019-12-10 15:40:58 +08:00
Loyalsoldier d1a51f8964 Initial commit 2019-12-10 11:19:36 +08:00
10 changed files with 272 additions and 170 deletions
+141
View File
@@ -0,0 +1,141 @@
name: Build v2ray rules dat files
on:
schedule:
- cron: "0 19 * * *"
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.13
uses: actions/setup-go@v1
with:
go-version: 1.13
id: go
- name: Set GOPATH
run: |
echo "::set-env name=GOPATH::$(dirname $GITHUB_WORKSPACE)"
echo "::set-env name=GREEN::\033[0;32m"
echo "::set-env name=NC::\033[0m"
echo "::set-env name=GEOIP_REPO::github.com/v2ray/geoip"
echo "::set-env name=GEOSITE_REPO::github.com/v2ray/domain-list-community"
echo "::set-env name=GFWLIST_URL::https://cokebar.github.io/gfwlist2dnsmasq/gfwlist_domain.txt"
echo "::set-env name=ADBLOCK_URL::https://raw.githubusercontent.com/h2y/Shadowrocket-ADBlock-Rules/master/factory/resultant/ad.list"
echo "::set-env name=Profiles_URL::https://raw.githubusercontent.com/ConnersHua/Profiles/master/Shadow/Pro.conf"
echo "::set-env name=CHINA_DOMAINS_URL::https://raw.githubusercontent.com/felixonmars/dnsmasq-china-list/master/accelerated-domains.china.conf"
echo "::add-path::$(dirname $GITHUB_WORKSPACE)/bin"
shell: bash
- name: Get GeoLite2
run: |
curl -sSL -O https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip
unzip GeoLite2-Country-CSV.zip
rm -f GeoLite2-Country-CSV.zip
mv GeoLite2* geoip
- name: Generate geoip.dat file
run: |
go get -u -v -insecure $GEOIP_REPO
geoip --country=./geoip/GeoLite2-Country-Locations-en.csv --ipv4=./geoip/GeoLite2-Country-Blocks-IPv4.csv --ipv6=./geoip/GeoLite2-Country-Blocks-IPv6.csv
mkdir -p ./publish
mv ./geoip.dat ./publish/
- name: Download geosite project
run: |
go get -u -v -insecure $GEOSITE_REPO
- name: Get and add gfwlist into geolocation-!cn
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $GFWLIST_URL > gfwlist
echo "include:gfwlist" >> geolocation-\!cn
- name: Get and add proxy domains from @ConnersHua/Profiles into geolocation-!cn
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $Profiles_URL | grep PROXY | grep DOMAIN | cut -d ',' -f 2 > profileproxy
echo "include:profileproxy" >> geolocation-\!cn
- name: Get and add direct domains from @ConnersHua/Profiles into cn
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $Profiles_URL | grep DIRECT | grep DOMAIN | cut -d ',' -f 2 > profiledirect
echo "include:profiledirect" >> cn
- name: Get and add chinalist into cn
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $CHINA_DOMAINS_URL | awk -F '/' '{print $2}' > chinalist
echo "include:chinalist" >> cn
- name: Get and add reject domains from @ConnersHua/Profiles into category-ads-all
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $Profiles_URL | grep REJECT | grep DOMAIN | cut -d ',' -f 2 > profilereject
echo "include:profilereject" >> category-ads-all
- name: Get and add reject domains from @h2y/Shadowrocket-ADBlock-Rules into category-ads-all
run: |
cd $GOPATH/src/$GEOSITE_REPO/data
curl -sSL $ADBLOCK_URL | awk '!/[0-9]+$/ { print }' > adblockreject
echo "include:adblockreject" >> category-ads-all
- name: Build geosite.dat file
run: |
domain-list-community
mv ./dlc.dat ./publish/geosite.dat
- name: List above process results
run: |
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo "list files in data folder"
ls -lah $GOPATH/src/$GEOSITE_REPO/data
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo "list last 10 lines of cn file"
tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/cn
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo "list last 10 lines of geolocation-!cn file"
tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/geolocation-\!cn
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo "list last 10 lines of category-ads-all file"
tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/category-ads-all
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo "list files in publish folder"
ls -lah ./publish
echo ">>>>>>>>>>>>>>>>>>>>>>>>"
echo -e "${GREEN}完成啦!🌈${NC}"
- name: Set release variables
run: |
echo "::set-env name=NAME::Released on $(date +%Y%m%d%H%M)"
echo "::set-env name=TAG_NAME::$(date +%Y%m%d%H%M)"
- name: Release dat files
uses: Ricky-Hao/action-release@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ASSET_PATH: publish
- name: Upload dat files
uses: actions/upload-artifact@master
if: success()
with:
name: dat_files
path: publish
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Loyalsoldier
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+110
View File
@@ -0,0 +1,110 @@
# 简介
**v2ray** 加强版规则文件,可代替 v2ray 官方 `geoip.dat` 和 `geosite.dat` 文件。利用 GitHub Actions 每天自动构建,保证最新。
## 规则文件生成方式
### geoip.dat
- 通过仓库 [@v2ray/geoip](https://github.com/v2ray/geoip) 生成
- 其中 IP 地址来源于 [MaxMind 免费 IP](https://dev.maxmind.com/geoip/geoip2/geolite2/)
- **优点**:由于项目每天自动构建,所以更新速度比官方 `geoip.dat` 要快得多
### geosite.dat
- 通过仓库 [@v2ray/domain-list-community](https://github.com/v2ray/domain-list-community) 生成
- **优点**:由于项目每天自动构建,所以更新速度比官方 `geosite.dat` 要快得多
- **加入大量中国大陆域名**:通过仓库 [@felixonmars/dnsmasq-china-list](https://github.com/felixonmars/dnsmasq-china-list) 生成 `chinalist` 类别域名并加入到 `geosite:cn` 类别中
- **加入最新 GFWList**:通过仓库 [@cokebar/gfwlist2dnsmasq](https://github.com/cokebar/gfwlist2dnsmasq) 生成 `gfwlist` 类别域名并加入到 `geosite:geolocation-!cn` 类别中
## 使用方式
1. [点击进入规则文件发布页面](https://github.com/Loyalsoldier/v2ray-rules-dat/releases),点击下载位于最上方的 `geoip.dat` 和 `geosite.dat`
2. 把下载下来的 `geoip.dat` 和 `geosite.dat` 文件放入到 v2ray 软件的规则文件目录,替换掉原来的 `geoip.dat` 和 `geosite.dat`
3. 修改 v2ray 配置文件。配置参考下面👇
### geoip.dat
跟 v2ray 官方 `geoip.dat` 配置方式相同。
Routing 配置方式:
```json
{
"routing": {
"rules": [
{
"type": "field",
"outboundTag": "Direct",
"ip": [
"geoip:cn",
"geoip:private"
]
}
]
}
}
```
### geosite.dat
跟 v2ray 官方 `geosite.dat` 配置方式相同。
Routing 配置方式:
```json
{
"routing": {
"rules": [
{
"type": "field",
"outboundTag": "Reject",
"domain": [
"geosite:category-ads-all"
]
},
{
"type": "field",
"outboundTag": "Proxy",
"domain": [
"geosite:geolocation-!cn"
]
},
{
"type": "field",
"outboundTag": "Direct",
"domain": [
"geosite:cn"
]
}
]
}
}
```
DNS 配置方式:
```json
{
"dns": {
"servers": [
{
"address": "1.1.1.1",
"port": 53,
"domains": [
"geosite:geolocation-!cn"
]
},
{
"address": "114.114.114.114",
"port": 53,
"domains": [
"geosite:cn"
]
},
"8.8.8.8",
"223.5.5.5"
]
}
}
```
-52
View File
@@ -1,52 +0,0 @@
103.com
123cha.com
95081.com
airasia.com
baid.us
baidu.jp
bussou.com
busytrade.com
cnbeta.com
cnbetacdn.com
cnpolitics.org
dm530.net
duanzhihu.com
dysfz.cc
emacs-china.org
galaxymacau.com
galstars.net
haitum.com
hostloc.com
itiger.com
jiaoyou8.com
kh.google.com
kox.moe
laonanren.com
longbridgeapp.com
longbridgehk.com
lovart.ai
mox.moe
mypikpak.com
mypikpak.net
mysinablog.com
naixi.net
nikke-de.com
nikke-en.com
nikke-fr.com
nikke-global.com
nikke-jp.com
nikke-kr.com
nikke-sea.com
ntrqq.com
nytlog.com
shuangtv.net
suppig.net
top
us.ci
ustclug.org
vol.moe
xclient.info
xjp.cc
yanghengjun.com
ydy.com
yslang.com
View File
-67
View File
@@ -1,67 +0,0 @@
#!/usr/bin/env python3
import sys
print(sys.argv[1], sys.argv[2])
''' Find redundant items in domain lists.
e.g. 'bar.foo.com' is redundant for 'foo.com'.
'''
def load(list):
''' Parse conf file & Prepare data structure
Returns: [ ['abc', 'com'],
['bar', 'foo', 'com'],
... ]
'''
results = []
with open(list, 'r') as f:
for line in f.readlines():
line = line.strip()
if line == '' or line.startswith('#'):
continue
# A domain name is case-insensitive and
# consists of several labels, separated by a full stop
domain_labels = line.lower().split('.')
results.append(domain_labels)
# Sort results by domain labels' length
results.sort(key=len)
return results
def find(labelses, removedDomainFile):
''' Find redundant items by a tree of top-level domain label to sub-level.
`tree` is like { 'com': { 'foo: { 'bar': LEAF },
'abc': LEAF },
'org': ... }
'''
tree = {}
LEAF = 1
for labels in labelses:
domain = '.'.join(labels)
# Init root node as current node
node = tree
while len(labels) > 0:
label = labels.pop()
if label in node:
# If child node is a LEAF node,
# current domain must be an existed domain or a subdomain of an existed.
if node[label] == LEAF:
print(f"Redundant found: {domain} at {'.'.join(labels)}")
with open(removedDomainFile, "a") as f:
f.write(domain)
f.write("\n")
break
else:
# Create a leaf node if current label is last one
if len(labels) == 0:
node[label] = LEAF
# Create a branch node
else:
node[label] = {}
# Iterate to child node
node = node[label]
if __name__ == '__main__':
find(load(sys.argv[1]), sys.argv[2])
-4
View File
@@ -1,4 +0,0 @@
ifanr.com
weibo.com
weixinbridge.com
www.baidu.com
-2
View File
@@ -1,2 +0,0 @@
services.googleapis.cn
supertop.co
-37
View File
@@ -1,37 +0,0 @@
4paradigm.com
addthis.com
addthisedge.com
alimama.alicdn.com
alimama.com
analytics.google.com
app.chat.xiaomi.net
bdtj.tagtic.cn
busuanzi.ibruce.info
cdn.onesignal.com
click.discord.com
click.redditmail.com
ctrip.com
d.ifengimg.com
erebor.douban.com
fabric.io
icons.mydrivers.com
img.alibaba.com
jav321.com
knet.cn
mail.tsinghua.edu.cn
mtalk.google.com
mx.technolutions.net
offer.alibaba.com
pingjs.qq.com
qlogo.cn
resolver.msg.xiaomi.net
s.youtube.com
sf3-ttcdn-tos.pstatp.com
t.co
tagtic.cn
telegra.ph
tongji.baidu.com
tv.sohu.com
ue.yeyoucdn.com
umeng.com
wl.spotify.com
-8
View File
@@ -1,8 +0,0 @@
adsurge.com
applvn.com
cookiebot.com
iads.unitychina.cn
remerge.io
singular.net
usercentrics.com
usercentrics.eu