Compare commits

...

87 Commits

Author SHA1 Message Date
taizan-hokouto
6c7dc03d06 Merge branch 'release/v0.5.5' 2021-07-24 23:00:30 +09:00
taizan-hokouto
f511049eaa Increment version 2021-07-24 22:59:46 +09:00
taizan-hokouto
76118ba196 Update classifiers 2021-07-24 22:59:26 +09:00
taizan-hokouto
83b10ab2f3 Merge branch 'feature/client' into develop 2021-07-24 22:50:28 +09:00
taizan-hokouto
604c52e608 Update requirements 2021-07-24 22:46:56 +09:00
taizan-hokouto
8949599232 Update pipfile 2021-07-24 22:46:56 +09:00
taizan-hokouto
c9c235061c Add tests 2021-07-24 22:46:56 +09:00
taizan-hokouto
328889689f Make it possible to use customized client 2021-07-24 22:46:56 +09:00
taizan-hokouto
e865c25a4e Merge branch 'develop' 2021-07-01 23:40:47 +09:00
taizan-hokouto
6d581c22f9 Increment version 2021-07-01 23:39:36 +09:00
taizan-hokouto
87aadeeb58 Enhance channelID extraction 2021-07-01 22:44:41 +09:00
taizan-hokouto
d331131cfe Merge branch 'master' into develop 2021-07-01 20:51:16 +09:00
taizan-hokuto
0dce1b009a Merge pull request #47 from taizan-hokuto/dependabot/pip/urllib3-1.26.5
Bump urllib3 from 1.26.4 to 1.26.5
2021-06-08 19:21:46 +09:00
dependabot[bot]
bd77cd2058 Bump urllib3 from 1.26.4 to 1.26.5
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.4 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.4...1.26.5)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-02 03:37:37 +00:00
taizan-hokuto
8e47e8d5db Merge pull request #44 from taizan-hokuto/dependabot/pip/urllib3-1.26.4
Bump urllib3 from 1.26.3 to 1.26.4
2021-04-25 12:54:19 +09:00
dependabot[bot]
946dd155d8 Bump urllib3 from 1.26.3 to 1.26.4
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.3 to 1.26.4.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.3...1.26.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-06 18:25:46 +00:00
taizan-hokouto
3565153597 Merge branch 'master' into develop 2021-02-20 21:20:10 +09:00
taizan-hokouto
f6c6ec5603 Modify .gitignore 2021-02-20 21:19:14 +09:00
taizan-hokuto
1cd0bab60b Ignore .gitignore 2021-02-20 21:05:23 +09:00
taizan-hokuto
1c246eea6d Merge pull request #40 from mark-ignacio/gitignore
add .gitignore for __pycache__
2021-02-20 21:04:07 +09:00
Mark Ignacio
3a365243d9 add .gitignore for __pycache__ 2021-02-13 10:08:52 -08:00
taizan-hokuto
7f8882c9e9 Merge pull request #38 from taizan-hokuto/dependabot/pip/cryptography-3.3.2
Bump cryptography from 3.3.1 to 3.3.2
2021-02-12 21:37:23 +09:00
taizan-hokouto
bc401ac80f Merge tag 'fix_param' into develop 2021-02-11 02:02:14 +09:00
taizan-hokouto
0a9837adca Merge branch 'hotfix/fix_param' 2021-02-11 02:02:14 +09:00
taizan-hokouto
f4bf30c0e9 Increment version 2021-02-11 01:57:45 +09:00
taizan-hokouto
acfba74821 Delete tests 2021-02-11 01:56:43 +09:00
taizan-hokouto
f46845c777 Fix parameters for live 2021-02-11 01:56:32 +09:00
dependabot[bot]
74f1536553 Bump cryptography from 3.3.1 to 3.3.2
Bumps [cryptography](https://github.com/pyca/cryptography) from 3.3.1 to 3.3.2.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/3.3.1...3.3.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-10 02:41:39 +00:00
taizan-hokuto
af4c2fe4b9 Merge pull request #36 from taizan-hokuto/dependabot/pip/bleach-3.3.0
Bump bleach from 3.2.1 to 3.3.0
2021-02-04 00:46:52 +09:00
dependabot[bot]
b7c656536d Bump bleach from 3.2.1 to 3.3.0
Bumps [bleach](https://github.com/mozilla/bleach) from 3.2.1 to 3.3.0.
- [Release notes](https://github.com/mozilla/bleach/releases)
- [Changelog](https://github.com/mozilla/bleach/blob/master/CHANGES)
- [Commits](https://github.com/mozilla/bleach/compare/v3.2.1...v3.3.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-02 23:12:20 +00:00
taizan-hokouto
faf875c0f5 Merge tag 'v0.5.2' into develop
v0.5.2
2021-01-17 22:41:20 +09:00
taizan-hokouto
b3ebe3879d Merge branch 'release/v0.5.2' 2021-01-17 22:41:20 +09:00
taizan-hokouto
da79895e55 Increment version 2021-01-17 22:40:15 +09:00
taizan-hokouto
aaa7421fdf Add replay_continuation parameter 2021-01-17 22:38:04 +09:00
taizan-hokuto
b9f213f047 Merge pull request #32 from miyuk/develop-add-replay-continuation
Add replay_continuation parameter
2021-01-15 00:11:04 +09:00
miyuk
fee070b299 Add replay_continuation parameter 2021-01-14 20:06:32 +09:00
taizan-hokouto
275e28b635 Merge tag 'v0.5.1' into develop
v0.5.1
2021-01-09 22:14:56 +09:00
taizan-hokouto
808e599be6 Merge branch 'release/v0.5.1' 2021-01-09 22:14:55 +09:00
taizan-hokouto
5cb6f7f123 Increment version 2021-01-09 22:14:30 +09:00
taizan-hokouto
a2f1c658f0 Merge branch 'master' into develop 2021-01-09 22:13:15 +09:00
taizan-hokouto
05de644d77 Merge branch 'hotfix/fix' 2021-01-09 22:13:15 +09:00
taizan-hokouto
b908855566 Delete unnecessary line 2021-01-09 22:12:33 +09:00
taizan-hokouto
8d93bfcb95 Merge branch 'master' into develop 2021-01-09 22:10:28 +09:00
taizan-hokouto
bf68859f38 Merge branch 'hotfix/fix' 2021-01-09 22:10:28 +09:00
taizan-hokouto
78fbe97b66 Fix process of fetching archived chat 2021-01-09 22:09:31 +09:00
taizan-hokouto
166a256c1c Merge tag 'v0.5.0' into develop
v0.5.0
2020-12-13 22:29:25 +09:00
taizan-hokouto
b7f2967a4f Merge branch 'release/v0.5.0' 2020-12-13 22:29:25 +09:00
taizan-hokouto
0a8ff3abdc Increment version 2020-12-13 22:28:39 +09:00
taizan-hokouto
9b38a5428d Add python version 2020-12-13 22:08:10 +09:00
taizan-hokuto
9311bf1993 Merge pull request #26 from zecktos/patch-1
Fix for python3.9
2020-12-13 22:04:18 +09:00
zecktos
ee839da7c9 Fix for python3.9
'encoding' is deprecated and removed in Python 3.9 
could fix this https://github.com/taizan-hokuto/pytchat/issues/24
2020-12-13 13:39:58 +01:00
taizan-hokouto
2ae77b3850 Merge branch 'hotfix/readme' 2020-12-13 14:22:05 +09:00
taizan-hokouto
afd7cea635 Merge branch 'master' into develop 2020-12-13 14:22:05 +09:00
taizan-hokouto
9018ff9ee4 Update README 2020-12-13 14:21:42 +09:00
taizan-hokouto
b676912d64 Merge branch 'master' into develop 2020-12-05 21:32:13 +09:00
taizan-hokouto
d75e1db304 Merge branch 'hotfix/rem' 2020-12-05 21:32:12 +09:00
taizan-hokouto
0d7dd26ede Increment version 2020-12-05 21:29:12 +09:00
taizan-hokouto
24b9f45e4a Remove entry point 2020-12-05 21:28:59 +09:00
taizan-hokouto
821cc57024 Merge tag 'v0.4.8' into develop
v0.4.8
2020-12-05 21:11:42 +09:00
taizan-hokouto
f050f5d650 Merge branch 'release/v0.4.8' 2020-12-05 21:11:42 +09:00
taizan-hokouto
c4eb78e112 Increment version 2020-12-05 21:11:18 +09:00
taizan-hokouto
40b0138a92 Merge branch 'master' into develop 2020-12-05 21:10:01 +09:00
taizan-hokouto
1d80dce913 Merge branch 'hotfix/dat' 2020-12-05 21:10:00 +09:00
taizan-hokouto
8b18f4476a Update README 2020-12-05 21:07:11 +09:00
taizan-hokouto
35a27fee3e Remove files 2020-12-05 20:59:47 +09:00
taizan-hokouto
eca766ff1a Update requirements 2020-12-05 14:45:24 +09:00
taizan-hokouto
139dc7fce2 Fix tests 2020-12-05 14:44:25 +09:00
taizan-hokouto
2c8a883ee3 Format 2020-12-05 14:43:56 +09:00
taizan-hokouto
865e4b5fab Remove files 2020-12-05 14:43:33 +09:00
taizan-hokouto
02d48ceccc Fix process 2020-12-05 14:42:02 +09:00
taizan-hokouto
bc3f16e86b Move functions 2020-12-05 14:39:55 +09:00
taizan-hokouto
1e6ce58f8b Merge branch 'master' into develop 2020-11-19 19:13:54 +09:00
taizan-hokouto
4db9486853 Merge branch 'hotfix/readme' 2020-11-19 19:13:53 +09:00
taizan-hokouto
775b79642e Update README 2020-11-19 19:12:54 +09:00
taizan-hokouto
423a128882 Merge tag 'v0.4.7' into develop
v0.4.7
2020-11-18 01:25:10 +09:00
taizan-hokouto
aaf9860bdc Merge branch 'release/v0.4.7' 2020-11-18 01:25:10 +09:00
taizan-hokouto
83ad4dcf1f Increment version 2020-11-18 01:24:37 +09:00
taizan-hokouto
765251b872 Merge branch 'feature/pipenv' into develop 2020-11-18 01:17:35 +09:00
taizan-hokouto
7ea88fead2 Modify requirements 2020-11-18 01:16:49 +09:00
taizan-hokouto
ea67e3e54e Add pipenv files 2020-11-18 01:16:01 +09:00
taizan-hokouto
7cf780ee87 Merge branch 'master' into develop 2020-11-17 01:11:22 +09:00
taizan-hokouto
6fcc1393de Merge branch 'master' into develop 2020-11-17 01:01:56 +09:00
taizan-hokouto
661d1e4b81 Fix tests 2020-11-17 00:54:32 +09:00
taizan-hokouto
966320cab5 Merge branch 'master' into develop 2020-11-16 23:32:32 +09:00
taizan-hokouto
3ad6b7e845 Merge branch 'master' into develop 2020-11-16 23:29:50 +09:00
taizan-hokouto
7b0708ec46 Merge branch 'master' into develop 2020-11-16 23:17:37 +09:00
taizan-hokouto
13812bdad3 Merge tag 'v0.4.5' into develop
v0.4.5
2020-11-16 01:50:50 +09:00
106 changed files with 64833 additions and 66937 deletions

View File

@@ -9,7 +9,7 @@ jobs:
strategy:
max-parallel: 4
matrix:
python-version: [3.7, 3.8]
python-version: [3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2

39
.gitignore vendored Normal file
View File

@@ -0,0 +1,39 @@
# Byte-compiled / optimized / DLL files
__pycache__/
.pytest_cache/
#logs
log.txt
*.log
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
# VSCode
.vscode
# develop
*.egg-info/
# Pypi dist
dist/
README.rst
temporary/
# Pypi wheel
build/
exclude/

13
Pipfile Normal file
View File

@@ -0,0 +1,13 @@
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
httpx = {extras = ["http2"]}
[dev-packages]
pytest-mock = "*"
pytest-httpx = "*"
wheel = "*"
twine = "*"

410
Pipfile.lock generated Normal file
View File

@@ -0,0 +1,410 @@
{
"_meta": {
"hash": {
"sha256": "74b83f2e50bc16f8d90c06ddc775d24ee427f8481a2501f62170bf5b76a2f1bd"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"anyio": {
"hashes": [
"sha256:929a6852074397afe1d989002aa96d457e3e1e5441357c60d03e7eea0e65e1b0",
"sha256:ae57a67583e5ff8b4af47666ff5651c3732d45fd26c929253748e796af860374"
],
"markers": "python_full_version >= '3.6.2'",
"version": "==3.3.0"
},
"certifi": {
"hashes": [
"sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee",
"sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8"
],
"version": "==2021.5.30"
},
"h11": {
"hashes": [
"sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6",
"sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"
],
"markers": "python_version >= '3.6'",
"version": "==0.12.0"
},
"h2": {
"hashes": [
"sha256:61e0f6601fa709f35cdb730863b4e5ec7ad449792add80d1410d4174ed139af5",
"sha256:875f41ebd6f2c44781259005b157faed1a5031df3ae5aa7bcb4628a6c0782f14"
],
"version": "==3.2.0"
},
"hpack": {
"hashes": [
"sha256:0edd79eda27a53ba5be2dfabf3b15780928a0dff6eb0c60a3d6767720e970c89",
"sha256:8eec9c1f4bfae3408a3f30500261f7e6a65912dc138526ea054f9ad98892e9d2"
],
"version": "==3.0.0"
},
"httpcore": {
"hashes": [
"sha256:b0d16f0012ec88d8cc848f5a55f8a03158405f4bca02ee49bc4ca2c1fda49f3e",
"sha256:db4c0dcb8323494d01b8c6d812d80091a31e520033e7b0120883d6f52da649ff"
],
"markers": "python_version >= '3.6'",
"version": "==0.13.6"
},
"httpx": {
"extras": [
"http2"
],
"hashes": [
"sha256:979afafecb7d22a1d10340bafb403cf2cb75aff214426ff206521fc79d26408c",
"sha256:9f99c15d33642d38bce8405df088c1c4cfd940284b4290cacbfb02e64f4877c6"
],
"index": "pypi",
"version": "==0.18.2"
},
"hyperframe": {
"hashes": [
"sha256:5187962cb16dcc078f23cb5a4b110098d546c3f41ff2d4038a9896893bbd0b40",
"sha256:a9f5c17f2cc3c719b917c4f33ed1c61bd1f8dfac4b1bd23b7c80b3400971b41f"
],
"version": "==5.2.0"
},
"idna": {
"hashes": [
"sha256:14475042e284991034cb48e06f6851428fb14c4dc953acd9be9a5e95c7b6dd7a",
"sha256:467fbad99067910785144ce333826c71fb0e63a425657295239737f7ecd125f3"
],
"version": "==3.2"
},
"rfc3986": {
"extras": [
"idna2008"
],
"hashes": [
"sha256:270aaf10d87d0d4e095063c65bf3ddbc6ee3d0b226328ce21e036f946e421835",
"sha256:a86d6e1f5b1dc238b218b012df0aa79409667bb209e58da56d0b94704e712a97"
],
"version": "==1.5.0"
},
"sniffio": {
"hashes": [
"sha256:471b71698eac1c2112a40ce2752bb2f4a4814c22a54a3eed3676bc0f5ca9f663",
"sha256:c4666eecec1d3f50960c6bdf61ab7bc350648da6c126e3cf6898d8cd4ddcd3de"
],
"markers": "python_version >= '3.5'",
"version": "==1.2.0"
}
},
"develop": {
"anyio": {
"hashes": [
"sha256:929a6852074397afe1d989002aa96d457e3e1e5441357c60d03e7eea0e65e1b0",
"sha256:ae57a67583e5ff8b4af47666ff5651c3732d45fd26c929253748e796af860374"
],
"markers": "python_full_version >= '3.6.2'",
"version": "==3.3.0"
},
"atomicwrites": {
"hashes": [
"sha256:6d1784dea7c0c8d4a5172b6c620f40b6e4cbfdf96d783691f2e1302a7b88e197",
"sha256:ae70396ad1a434f9c7046fd2dd196fc04b12f9e91ffb859164193be8b6168a7a"
],
"markers": "sys_platform == 'win32'",
"version": "==1.4.0"
},
"attrs": {
"hashes": [
"sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1",
"sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==21.2.0"
},
"bleach": {
"hashes": [
"sha256:306483a5a9795474160ad57fce3ddd1b50551e981eed8e15a582d34cef28aafa",
"sha256:ae976d7174bba988c0b632def82fdc94235756edfb14e6558a9c5be555c9fb78"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==3.3.1"
},
"certifi": {
"hashes": [
"sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee",
"sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8"
],
"version": "==2021.5.30"
},
"charset-normalizer": {
"hashes": [
"sha256:88fce3fa5b1a84fdcb3f603d889f723d1dd89b26059d0123ca435570e848d5e1",
"sha256:c46c3ace2d744cfbdebceaa3c19ae691f53ae621b39fd7570f59d14fb7f2fd12"
],
"markers": "python_version >= '3'",
"version": "==2.0.3"
},
"colorama": {
"hashes": [
"sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b",
"sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"
],
"markers": "platform_system == 'Windows' and sys_platform == 'win32'",
"version": "==0.4.4"
},
"docutils": {
"hashes": [
"sha256:686577d2e4c32380bb50cbb22f575ed742d58168cee37e99117a854bcd88f125",
"sha256:cf316c8370a737a022b72b56874f6602acf974a37a9fba42ec2876387549fc61"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==0.17.1"
},
"h11": {
"hashes": [
"sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6",
"sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"
],
"markers": "python_version >= '3.6'",
"version": "==0.12.0"
},
"httpcore": {
"hashes": [
"sha256:b0d16f0012ec88d8cc848f5a55f8a03158405f4bca02ee49bc4ca2c1fda49f3e",
"sha256:db4c0dcb8323494d01b8c6d812d80091a31e520033e7b0120883d6f52da649ff"
],
"markers": "python_version >= '3.6'",
"version": "==0.13.6"
},
"httpx": {
"extras": [
"http2"
],
"hashes": [
"sha256:979afafecb7d22a1d10340bafb403cf2cb75aff214426ff206521fc79d26408c",
"sha256:9f99c15d33642d38bce8405df088c1c4cfd940284b4290cacbfb02e64f4877c6"
],
"index": "pypi",
"version": "==0.18.2"
},
"idna": {
"hashes": [
"sha256:14475042e284991034cb48e06f6851428fb14c4dc953acd9be9a5e95c7b6dd7a",
"sha256:467fbad99067910785144ce333826c71fb0e63a425657295239737f7ecd125f3"
],
"version": "==3.2"
},
"importlib-metadata": {
"hashes": [
"sha256:079ada16b7fc30dfbb5d13399a5113110dab1aa7c2bc62f66af75f0b717c8cac",
"sha256:9f55f560e116f8643ecf2922d9cd3e1c7e8d52e683178fecd9d08f6aa357e11e"
],
"markers": "python_version >= '3.6'",
"version": "==4.6.1"
},
"iniconfig": {
"hashes": [
"sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3",
"sha256:bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32"
],
"version": "==1.1.1"
},
"keyring": {
"hashes": [
"sha256:045703609dd3fccfcdb27da201684278823b72af515aedec1a8515719a038cb8",
"sha256:8f607d7d1cc502c43a932a275a56fe47db50271904513a379d39df1af277ac48"
],
"markers": "python_version >= '3.6'",
"version": "==23.0.1"
},
"packaging": {
"hashes": [
"sha256:7dc96269f53a4ccec5c0670940a4281106dd0bb343f47b7471f779df49c2fbe7",
"sha256:c86254f9220d55e31cc94d69bade760f0847da8000def4dfe1c6b872fd14ff14"
],
"markers": "python_version >= '3.6'",
"version": "==21.0"
},
"pkginfo": {
"hashes": [
"sha256:37ecd857b47e5f55949c41ed061eb51a0bee97a87c969219d144c0e023982779",
"sha256:e7432f81d08adec7297633191bbf0bd47faf13cd8724c3a13250e51d542635bd"
],
"version": "==1.7.1"
},
"pluggy": {
"hashes": [
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.13.1"
},
"py": {
"hashes": [
"sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
"sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.10.0"
},
"pygments": {
"hashes": [
"sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f",
"sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e"
],
"markers": "python_version >= '3.5'",
"version": "==2.9.0"
},
"pyparsing": {
"hashes": [
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.4.7"
},
"pytest": {
"hashes": [
"sha256:50bcad0a0b9c5a72c8e4e7c9855a3ad496ca6a881a3641b4260605450772c54b",
"sha256:91ef2131a9bd6be8f76f1f08eac5c5317221d6ad1e143ae03894b862e8976890"
],
"markers": "python_version >= '3.6'",
"version": "==6.2.4"
},
"pytest-httpx": {
"hashes": [
"sha256:1e135b8779060091fa1c87d8aff7904921e8bea95fce5e971a0262764d064b12",
"sha256:e262932f2d3ce380da8273c7bacbcfdc2c94e167fa94da29571caaf1f4d3ba27"
],
"index": "pypi",
"version": "==0.12.0"
},
"pytest-mock": {
"hashes": [
"sha256:30c2f2cc9759e76eee674b81ea28c9f0b94f8f0445a1b87762cadf774f0df7e3",
"sha256:40217a058c52a63f1042f0784f62009e976ba824c418cced42e88d5f40ab0e62"
],
"index": "pypi",
"version": "==3.6.1"
},
"pywin32-ctypes": {
"hashes": [
"sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942",
"sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98"
],
"markers": "sys_platform == 'win32'",
"version": "==0.2.0"
},
"readme-renderer": {
"hashes": [
"sha256:63b4075c6698fcfa78e584930f07f39e05d46f3ec97f65006e430b595ca6348c",
"sha256:92fd5ac2bf8677f310f3303aa4bce5b9d5f9f2094ab98c29f13791d7b805a3db"
],
"version": "==29.0"
},
"requests": {
"hashes": [
"sha256:6c1246513ecd5ecd4528a0906f910e8f0f9c6b8ec72030dc9fd154dc1a6efd24",
"sha256:b8aa58f8cf793ffd8782d3d8cb19e66ef36f7aba4353eec859e74678b01b07a7"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
"version": "==2.26.0"
},
"requests-toolbelt": {
"hashes": [
"sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f",
"sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"
],
"version": "==0.9.1"
},
"rfc3986": {
"extras": [
"idna2008"
],
"hashes": [
"sha256:270aaf10d87d0d4e095063c65bf3ddbc6ee3d0b226328ce21e036f946e421835",
"sha256:a86d6e1f5b1dc238b218b012df0aa79409667bb209e58da56d0b94704e712a97"
],
"version": "==1.5.0"
},
"six": {
"hashes": [
"sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
"sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.16.0"
},
"sniffio": {
"hashes": [
"sha256:471b71698eac1c2112a40ce2752bb2f4a4814c22a54a3eed3676bc0f5ca9f663",
"sha256:c4666eecec1d3f50960c6bdf61ab7bc350648da6c126e3cf6898d8cd4ddcd3de"
],
"markers": "python_version >= '3.5'",
"version": "==1.2.0"
},
"toml": {
"hashes": [
"sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.10.2"
},
"tqdm": {
"hashes": [
"sha256:5aa445ea0ad8b16d82b15ab342de6b195a722d75fc1ef9934a46bba6feafbc64",
"sha256:8bb94db0d4468fea27d004a0f1d1c02da3cdedc00fe491c0de986b76a04d6b0a"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==4.61.2"
},
"twine": {
"hashes": [
"sha256:087328e9bb405e7ce18527a2dca4042a84c7918658f951110b38bc135acab218",
"sha256:4caec0f1ed78dc4c9b83ad537e453d03ce485725f2aea57f1bb3fdde78dae936"
],
"index": "pypi",
"version": "==3.4.2"
},
"urllib3": {
"hashes": [
"sha256:39fb8672126159acb139a7718dd10806104dec1e2f0f6c88aab05d17df10c8d4",
"sha256:f57b4c16c62fa2760b7e3d97c35b255512fb6b59a259730f36ba32ce9f8e342f"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.6"
},
"webencodings": {
"hashes": [
"sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78",
"sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"
],
"version": "==0.5.1"
},
"wheel": {
"hashes": [
"sha256:78b5b185f0e5763c26ca1e324373aadd49182ca90e825f7853f4b2509215dc0e",
"sha256:e11eefd162658ea59a60a0f6c7d493a7190ea4b9a85e335b33489d9f17e0245e"
],
"index": "pypi",
"version": "==0.36.2"
},
"zipp": {
"hashes": [
"sha256:957cfda87797e389580cb8b9e3870841ca991e2125350677b2ca83a0e99390a3",
"sha256:f5812b1e007e48cff63449a5e9f4e7ebea716b4111f9c4f9a645f91d579bf0c4"
],
"markers": "python_version >= '3.6'",
"version": "==3.5.0"
}
}
}

View File

@@ -3,6 +3,8 @@ pytchat
pytchat is a python library for fetching youtube live chat.
<br><br><br>
## Description
pytchat is a python library for fetching youtube live chat
without using Selenium or BeautifulSoup.
@@ -22,23 +24,6 @@ pip install pytchat
```
## Examples
### CLI
+ One-liner command.
+ Save chat data to html with embedded custom emojis.
+ Show chat stream (--echo option).
```bash
$ pytchat -v uIx8l2xlYVY -o "c:/temp/"
# options:
# -v : Video ID or URL that includes ID
# -o : output directory (default path: './')
# --echo : Show chats.
# saved filename is [video_id].html
```
### Fetch chat data (see [wiki](https://github.com/taizan-hokuto/pytchat/wiki/PytchatCore))
```python
@@ -74,8 +59,6 @@ while chat.is_alive():
+ YT API compatible chat processor ([CompatibleProcessor](https://github.com/taizan-hokuto/pytchat/wiki/CompatibleProcessor))
+ Extract archived chat data ([Extractor](https://github.com/taizan-hokuto/pytchat/wiki/Extractor))
## Structure of Default Processor
Each item can be got with `sync_items()` function.
@@ -205,19 +188,3 @@ Structure of author object.
[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
## Contributes
Great thanks:
Most of source code of CLI refer to:
[PetterKraabol / Twitch-Chat-Downloader](https://github.com/PetterKraabol/Twitch-Chat-Downloader)
Progress bar in CLI is based on:
[vladignatyev/progress.py](https://gist.github.com/vladignatyev/06860ec2040cb497f0f3)
## Author
[taizan-hokuto](https://github.com/taizan-hokuto)
[twitter:@taizan205](https://twitter.com/taizan205)

View File

@@ -1,8 +1,8 @@
"""
pytchat is a lightweight python library to browse youtube livechat without Selenium or BeautifulSoup.
"""
__copyright__ = 'Copyright (C) 2019, 2020 taizan-hokuto'
__version__ = '0.4.6'
__copyright__ = 'Copyright (C) 2019, 2020, 2021 taizan-hokuto'
__version__ = '0.5.5'
__license__ = 'MIT'
__author__ = 'taizan-hokuto'
__author_email__ = '55448286+taizan-hokuto@users.noreply.github.com'
@@ -26,7 +26,6 @@ from .exceptions import (
)
from .api import (
cli,
config,
LiveChat,
LiveChatAsync,
@@ -34,14 +33,12 @@ from .api import (
CompatibleProcessor,
DummyProcessor,
DefaultProcessor,
Extractor,
HTMLArchiver,
TSVArchiver,
JsonfileArchiver,
SimpleDisplayProcessor,
SpeedCalculator,
SuperchatCalculator,
VideoInfo,
create
)
# flake8: noqa

View File

@@ -1,4 +1,3 @@
from . import cli
from . import config
from .core import create
from .core_multithread.livechat import LiveChat
@@ -13,11 +12,9 @@ from .processors.jsonfile_archiver import JsonfileArchiver
from .processors.simple_display_processor import SimpleDisplayProcessor
from .processors.speed.calculator import SpeedCalculator
from .processors.superchat.calculator import SuperchatCalculator
from .tool.extract.extractor import Extractor
from .tool.videoinfo import VideoInfo
__all__ = [
cli,
config,
LiveChat,
LiveChatAsync,
@@ -25,14 +22,12 @@ __all__ = [
CompatibleProcessor,
DummyProcessor,
DefaultProcessor,
Extractor,
HTMLArchiver,
TSVArchiver,
JsonfileArchiver,
SimpleDisplayProcessor,
SpeedCalculator,
SuperchatCalculator,
VideoInfo,
create
]

View File

@@ -1,71 +0,0 @@
import argparse
try:
from asyncio import CancelledError
except ImportError:
from asyncio.futures import CancelledError
import os
from .arguments import Arguments
from .echo import Echo
from .. exceptions import InvalidVideoIdException
from .. import __version__
from .cli_extractor import CLIExtractor
'''
Most of CLI modules refer to
Petter Kraabøl's Twitch-Chat-Downloader
https://github.com/PetterKraabol/Twitch-Chat-Downloader
(MIT License)
'''
def main():
# Arguments
parser = argparse.ArgumentParser(description=f'pytchat v{__version__}')
parser.add_argument('-v', f'--{Arguments.Name.VIDEO_IDS}', type=str,
help='Video ID (or URL that includes Video ID). You can specify multiple video IDs by '
'separating them with commas without spaces.\n'
'If ID starts with a hyphen (-), enclose the ID in square brackets.')
parser.add_argument('-o', f'--{Arguments.Name.OUTPUT}', type=str,
help='Output directory (end with "/"). default="./"', default='./')
parser.add_argument(f'--{Arguments.Name.DEBUG}', action='store_true',
help='Debug mode. Stop when exceptions have occurred and save error data (".dat" file).')
parser.add_argument(f'--{Arguments.Name.VERSION}', action='store_true',
help='Show version.')
parser.add_argument(f'--{Arguments.Name.ECHO}', action='store_true',
help='Display chats of specified video.')
Arguments(parser.parse_args().__dict__)
if Arguments().print_version:
print(f'pytchat v{__version__} © 2019, 2020 taizan-hokuto')
return
if not Arguments().video_ids:
parser.print_help()
return
# Echo
if Arguments().echo:
if len(Arguments().video_ids) > 1:
print("When using --echo option, only one video ID can be specified.")
return
try:
Echo(Arguments().video_ids[0]).run()
except InvalidVideoIdException as e:
print("Invalid video id:", str(e))
except Exception as e:
print(type(e), str(e))
if Arguments().debug:
raise
finally:
return
# Extractor
if not os.path.exists(Arguments().output):
print("\nThe specified directory does not exist.:{}\n".format(Arguments().output))
return
try:
CLIExtractor().run()
except CancelledError as e:
print(str(e))

View File

@@ -1,45 +0,0 @@
from typing import Optional, Dict, Union, List
from .singleton import Singleton
'''
This modules refer to
Petter Kraabøl's Twitch-Chat-Downloader
https://github.com/PetterKraabol/Twitch-Chat-Downloader
(MIT License)
'''
class Arguments(metaclass=Singleton):
"""
Arguments singleton
"""
class Name:
VERSION: str = 'version'
OUTPUT: str = 'output_dir'
VIDEO_IDS: str = 'video_id'
DEBUG: bool = 'debug'
ECHO: bool = 'echo'
def __init__(self,
arguments: Optional[Dict[str, Union[str, bool, int]]] = None):
"""
Initialize arguments
:param arguments: Arguments from cli
(Optional to call singleton instance without parameters)
"""
if arguments is None:
print('Error: arguments were not provided')
exit()
self.print_version: bool = arguments[Arguments.Name.VERSION]
self.output: str = arguments[Arguments.Name.OUTPUT]
self.video_ids: List[int] = []
self.debug: bool = arguments[Arguments.Name.DEBUG]
self.echo: bool = arguments[Arguments.Name.ECHO]
# Videos
if arguments[Arguments.Name.VIDEO_IDS]:
self.video_ids = [video_id
for video_id in arguments[Arguments.Name.VIDEO_IDS].split(',')]

View File

@@ -1,121 +0,0 @@
import asyncio
import os
import signal
import traceback
from httpcore import ReadTimeout as HCReadTimeout, NetworkError as HCNetworkError
from json.decoder import JSONDecodeError
from pathlib import Path
from .arguments import Arguments
from .progressbar import ProgressBar
from .. import util
from .. exceptions import InvalidVideoIdException, NoContents, PatternUnmatchError, UnknownConnectionError
from .. processors.html_archiver import HTMLArchiver
from .. tool.extract.extractor import Extractor
from .. tool.videoinfo import VideoInfo
from .. util.extract_video_id import extract_video_id
class CLIExtractor:
def run(self) -> None:
ex = None
pbar = None
for counter, video_id in enumerate(Arguments().video_ids):
if len(Arguments().video_ids) > 1:
print(f"\n{'-' * 10} video:{counter + 1} of {len(Arguments().video_ids)} {'-' * 10}")
try:
video_id = extract_video_id(video_id)
separated_path = str(Path(Arguments().output)) + os.path.sep
path = util.checkpath(separated_path + video_id + '.html')
try:
info = VideoInfo(video_id)
except (PatternUnmatchError, JSONDecodeError) as e:
print("Cannot parse video information.:{} {}".format(video_id, type(e)))
if Arguments().debug:
util.save(str(e.doc), "ERR", ".dat")
continue
except Exception as e:
print("Cannot parse video information.:{} {}".format(video_id, type(e)))
continue
print(f"\n"
f" video_id: {video_id}\n"
f" channel: {info.get_channel_name()}\n"
f" title: {info.get_title()}\n"
f" output path: {path}")
duration = info.get_duration()
pbar = ProgressBar(total=(duration * 1000), status_txt="Extracting")
ex = Extractor(video_id,
callback=pbar.disp,
div=10)
signal.signal(signal.SIGINT, (lambda a, b: self.cancel(ex, pbar)))
data = ex.extract()
if data == [] or data is None:
continue
pbar.reset("#", "=", total=1000, status_txt="Rendering ")
processor = HTMLArchiver(path, callback=pbar.disp)
processor.process(
[{'video_id': None,
'timeout': 1,
'chatdata': (action["replayChatItemAction"]["actions"][0] for action in data)}]
)
processor.finalize()
pbar.reset('#', '#', status_txt='Completed ')
pbar.close()
print()
if pbar.is_cancelled():
print("\nThe extraction process has been discontinued.\n")
except InvalidVideoIdException:
print("Invalid Video ID or URL:", video_id)
except NoContents as e:
print(f"Abort:{str(e)}:[{video_id}]")
except (JSONDecodeError, PatternUnmatchError) as e:
print("{}:{}".format(e.msg, video_id))
if Arguments().debug:
filename = util.save(e.doc, "ERR_", ".dat")
traceback.print_exc()
print(f"Saved error data: {filename}")
except (UnknownConnectionError, HCNetworkError, HCReadTimeout) as e:
if Arguments().debug:
traceback.print_exc()
print(f"An unknown network error occurred during the processing of [{video_id}]. : " + str(e))
except Exception as e:
print(f"Abort:{str(type(e))} {str(e)[:80]}")
if Arguments().debug:
traceback.print_exc()
finally:
clear_tasks()
return
def cancel(self, ex=None, pbar=None) -> None:
'''Called when keyboard interrupted has occurred.
'''
print("\nKeyboard interrupted.\n")
if ex and pbar:
ex.cancel()
pbar.cancel()
def clear_tasks():
'''
Clear remained tasks.
Called when internal exception has occurred or
after each extraction process is completed.
'''
async def _shutdown():
tasks = [t for t in asyncio.all_tasks()
if t is not asyncio.current_task()]
for task in tasks:
task.cancel()
try:
loop = asyncio.get_event_loop()
loop.run_until_complete(_shutdown())
except Exception as e:
print(str(e))
if Arguments().debug:
traceback.print_exc()

View File

@@ -1,22 +0,0 @@
import pytchat
from ..exceptions import ChatDataFinished, NoContents
from ..util.extract_video_id import extract_video_id
class Echo:
def __init__(self, video_id):
self.video_id = extract_video_id(video_id)
def run(self):
livechat = pytchat.create(self.video_id)
while livechat.is_alive():
chatdata = livechat.get()
for c in chatdata.sync_items():
print(f"{c.datetime} [{c.author.name}] {c.message} {c.amountString}")
try:
livechat.raise_for_status()
except (ChatDataFinished, NoContents):
print("Chat finished.")
except Exception as e:
print(type(e), str(e))

View File

@@ -1,54 +0,0 @@
'''
This code is based on
vladignatyev/progress.py
https://gist.github.com/vladignatyev/06860ec2040cb497f0f3
(MIT License)
'''
import shutil
import sys
class ProgressBar:
def __init__(self, total, status_txt):
self._bar_len = 60
self._cancelled = False
self.reset(total=total, status_txt=status_txt)
def reset(self, symbol_done="=", symbol_space=" ", total=100, status_txt=''):
self._console_width = shutil.get_terminal_size(fallback=(80, 24)).columns
self._symbol_done = symbol_done
self._symbol_space = symbol_space
self._total = total
self._status_txt = status_txt
self._count = 0
def disp(self, _, fetched):
self._progress(fetched, self._total)
def _progress(self, fillin, total):
if total == 0 or self._cancelled:
return
self._count += fillin
filled_len = int(round(self._bar_len * self._count / float(total)))
percents = round(100.0 * self._count / float(total), 1)
if percents > 100:
percents = 100.0
if filled_len > self._bar_len:
filled_len = self._bar_len
bar = self._symbol_done * filled_len + \
self._symbol_space * (self._bar_len - filled_len)
disp = f" [{bar}] {percents:>5.1f}% ...{self._status_txt} "[:self._console_width - 1] + '\r'
sys.stdout.write(disp)
sys.stdout.flush()
def close(self):
if not self._cancelled:
self._progress(self._total, self._total)
def cancel(self):
self._cancelled = True
def is_cancelled(self):
return self._cancelled

View File

@@ -1,21 +0,0 @@
'''
This modules refer to
Petter Kraabøl's Twitch-Chat-Downloader
https://github.com/PetterKraabol/Twitch-Chat-Downloader
(MIT License)
'''
class Singleton(type):
"""
Abstract class for singletons
"""
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]
def get_instance(cls, *args, **kwargs):
cls.__call__(*args, **kwargs)

View File

@@ -1,8 +1,14 @@
import logging # noqa
from . import mylogger
from base64 import a85decode as dc
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 Edg/86.0.622.63,gzip(gfe)',
}
m_headers = {
'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Mobile Safari/537.36 Edg/91.0.864.59',
}
_sml = dc(b"BQS?8F#ks-GB\\6`H#IhIF^eo7@rH3;H#IhIF^eor06T''Ch\\'(?XmbXF>%9<FC/iuG%G#jBOQ!ICLqcS5tQB2;gCZ)?UdXC;f$GR3)MM2<(0>O7mh!,G@+K5?SO9T@okV").decode()
_smr = dc(b"BQS?8F#ks-GB\\6`H#IhIF^eo7@rH3;H#IhIF^eor06T''Ch\\'(?XmbXF>%9<FC/iuG%G#jBOQ!iEb03+@<k(QAU-F)8U=fDGsP557S5F7CiNH7;)D3N77^*B6YU@\\?WfBr0emZX=#^").decode()
def logger(module_name: str, loglevel=None):

View File

@@ -1,5 +1,5 @@
from .pytchat import PytchatCore
from .. util.extract_video_id import extract_video_id
from .. util import extract_video_id
def create(video_id: str, **kwargs):

View File

@@ -3,19 +3,17 @@ import json
import signal
import time
import traceback
import urllib.parse
from ..parser.live import Parser
from .. import config
from .. import exceptions
from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
from ..util.extract_video_id import extract_video_id
from .. import util
headers = config.headers
MAX_RETRY = 10
class PytchatCore:
'''
@@ -31,9 +29,13 @@ class PytchatCore:
processor : ChatProcessor
client : httpx.Client
The client for connecting youtube.
You can specify any customized httpx client (e.g. coolies, user agent).
interruptable : bool
Allows keyboard interrupts.
Set this parameter to False if your own threading program causes
Set this parameter to False if your own multi-threading program causes
the problem.
force_replay : bool
@@ -46,54 +48,65 @@ class PytchatCore:
If True, when exceptions occur, the exception is held internally,
and can be raised by raise_for_status().
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes
---------
_is_alive : bool
Flag to stop getting chat.
'''
_setup_finished = False
def __init__(self, video_id,
seektime=-1,
processor=DefaultProcessor(),
client = httpx.Client(http2=True),
interruptable=True,
force_replay=False,
topchat_only=False,
hold_exception=True,
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = extract_video_id(video_id)
self._client = client
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
else:
self.processor = processor
self._is_alive = True
self._is_replay = force_replay
self._is_replay = force_replay or (replay_continuation is not None)
self._hold_exception = hold_exception
self._exception_holder = None
self._parser = Parser(
is_replay=self._is_replay,
exception_holder=self._exception_holder
)
self._first_fetch = True
self._fetch_url = "live_chat/get_live_chat?continuation="
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self.continuation = replay_continuation
if interruptable:
signal.signal(signal.SIGINT, lambda a, b: self.terminate())
self._setup()
def _setup(self):
if not self.continuation:
time.sleep(0.1) # sleep shortly to prohibit skipping fetching data
"""Fetch first continuation parameter,
create and start _listen loop.
"""
self.continuation = liveparam.getparam(self._video_id, 3)
self.continuation = liveparam.getparam(
self._video_id,
channel_id=util.get_channelid(self._client, self._video_id),
past_sec=3)
def _get_chat_component(self):
''' Fetch chat data and store them into buffer,
get next continuaiton parameter and loop.
@@ -103,9 +116,8 @@ class PytchatCore:
parameter for next chat data
'''
try:
with httpx.Client(http2=True) as client:
if self.continuation and self._is_alive:
contents = self._get_contents(self.continuation, client, headers)
contents = self._get_contents(self.continuation, self._client, headers)
metadata, chatdata = self._parser.parse(contents)
timeout = metadata['timeoutMs'] / 1000
chat_component = {
@@ -114,6 +126,7 @@ class PytchatCore:
"chatdata": chatdata
}
self.continuation = metadata.get('continuation')
self._last_offset_ms = metadata.get('last_offset_ms', 0)
return chat_component
except exceptions.ChatParseException as e:
self._logger.debug(f"[{self._video_id}]{str(e)}")
@@ -131,40 +144,43 @@ class PytchatCore:
-------
'continuationContents' which includes metadata & chat data.
'''
livechat_json = (
self._get_livechat_json(continuation, client, headers)
)
contents = self._parser.get_contents(livechat_json)
livechat_json = self._get_livechat_json(
continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = "live_chat_replay/get_live_chat_replay?continuation="
self._fetch_url = config._smr
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only)
livechat_json = (self._get_livechat_json(continuation, client, headers))
self._video_id, self.seektime, self._topchat_only, util.get_channelid(client, self._video_id))
livechat_json = self._get_livechat_json(
continuation, client, replay=True, offset_ms=self.seektime * 1000)
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json))
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (self._get_livechat_json(
reload_continuation, client, headers))
contents = self._parser.get_contents(livechat_json)
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
def _get_livechat_json(self, continuation, client, headers):
def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
'''
continuation = urllib.parse.quote(continuation)
livechat_json = None
err = None
url = f"https://www.youtube.com/{self._fetch_url}{continuation}&pbj=1"
if offset_ms < 0:
offset_ms = 0
param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
for _ in range(MAX_RETRY + 1):
with client:
try:
livechat_json = client.get(url, headers=headers).json()
response = client.post(self._fetch_url, json=param)
livechat_json = response.json()
break
except (json.JSONDecodeError, httpx.ConnectTimeout, httpx.ReadTimeout, httpx.ConnectError) as e:
err = e
@@ -190,6 +206,8 @@ class PytchatCore:
return self._is_alive
def terminate(self):
if not self.is_alive():
return
self._is_alive = False
self.processor.finalize()

View File

@@ -5,17 +5,16 @@ import json
import signal
import time
import traceback
import urllib.parse
from asyncio import Queue
from concurrent.futures import CancelledError
from .buffer import Buffer
from ..parser.live import Parser
from .. import config
from .. import exceptions
from .. import util
from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
from ..util.extract_video_id import extract_video_id
headers = config.headers
MAX_RETRY = 10
@@ -63,6 +62,10 @@ class LiveChatAsync:
topchat_only : bool
If True, get only top chat.
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes
---------
_is_alive : bool
@@ -75,6 +78,7 @@ class LiveChatAsync:
seektime=-1,
processor=DefaultProcessor(),
buffer=None,
client = httpx.AsyncClient(http2=True),
interruptable=True,
callback=None,
done_callback=None,
@@ -83,8 +87,10 @@ class LiveChatAsync:
force_replay=False,
topchat_only=False,
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = extract_video_id(video_id)
self._client:httpx.AsyncClient = client
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
@@ -96,15 +102,18 @@ class LiveChatAsync:
self._exception_handler = exception_handler
self._direct_mode = direct_mode
self._is_alive = True
self._is_replay = force_replay
self._is_replay = force_replay or (replay_continuation is not None)
self._parser = Parser(is_replay=self._is_replay)
self._pauser = Queue()
self._pauser.put_nowait(None)
self._first_fetch = True
self._fetch_url = "live_chat/get_live_chat?continuation="
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self.exception = None
self.continuation = replay_continuation
LiveChatAsync._logger = logger
if exception_handler:
@@ -144,8 +153,14 @@ class LiveChatAsync:
"""Fetch first continuation parameter,
create and start _listen loop.
"""
initial_continuation = liveparam.getparam(self._video_id, 3)
await self._listen(initial_continuation)
if not self.continuation:
channel_id = await util.get_channelid_async(self._client, self._video_id)
self.continuation = liveparam.getparam(
self._video_id,
channel_id,
past_sec=3)
await self._listen(self.continuation)
async def _listen(self, continuation):
''' Fetch chat data and store them into buffer,
@@ -157,13 +172,14 @@ class LiveChatAsync:
parameter for next chat data
'''
try:
async with httpx.AsyncClient(http2=True) as client:
async with self._client as client:
while(continuation and self._is_alive):
continuation = await self._check_pause(continuation)
contents = await self._get_contents(
continuation, client, headers)
contents = await self._get_contents(continuation, client, headers) #Q#
metadata, chatdata = self._parser.parse(contents)
continuation = metadata.get('continuation')
if continuation:
self.continuation = continuation
timeout = metadata['timeoutMs'] / 1000
chat_component = {
"video_id": self._video_id,
@@ -182,17 +198,16 @@ class LiveChatAsync:
await self._buffer.put(chat_component)
diff_time = timeout - (time.time() - time_mark)
await asyncio.sleep(diff_time)
continuation = metadata.get('continuation')
self._last_offset_ms = metadata.get('last_offset_ms', 0)
except exceptions.ChatParseException as e:
self._logger.debug(f"[{self._video_id}]{str(e)}")
raise
except Exception:
self._logger.error(f"{traceback.format_exc(limit = -1)}")
self._logger.error(f"{traceback.format_exc(limit=-1)}")
raise
self._logger.debug(f"[{self._video_id}] finished fetching chat.")
async def _check_pause(self, continuation):
if self._pauser.empty():
'''pause'''
@@ -202,8 +217,12 @@ class LiveChatAsync:
'''
self._pauser.put_nowait(None)
if not self._is_replay:
continuation = liveparam.getparam(
self._video_id, 3, self._topchat_only)
async with self._client as client:
channel_id = await util.get_channelid_async(client, self.video_id)
continuation = liveparam.getparam(self._video_id,
channel_id,
past_sec=3)
return continuation
async def _get_contents(self, continuation, client, headers):
@@ -215,46 +234,50 @@ class LiveChatAsync:
-------
'continuationContents' which includes metadata & chatdata.
'''
livechat_json = await self._get_livechat_json(continuation, client, headers)
contents = self._parser.get_contents(livechat_json)
livechat_json = await self._get_livechat_json(continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = "live_chat_replay/get_live_chat_replay?continuation="
self._fetch_url = config._smr
channelid = await util.get_channelid_async(client, self._video_id)
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only)
self._video_id, self.seektime, self._topchat_only, channelid)
livechat_json = (await self._get_livechat_json(
continuation, client, headers))
continuation, client, replay=True, offset_ms=self.seektime * 1000))
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json))
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (await self._get_livechat_json(
reload_continuation, client, headers))
contents = self._parser.get_contents(livechat_json)
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
async def _get_livechat_json(self, continuation, client, headers):
async def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
'''
continuation = urllib.parse.quote(continuation)
livechat_json = None
url = f"https://www.youtube.com/{self._fetch_url}{continuation}&pbj=1"
if offset_ms < 0:
offset_ms = 0
param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
for _ in range(MAX_RETRY + 1):
try:
resp = await client.get(url, headers=headers)
resp = await client.post(self._fetch_url, json=param)
livechat_json = resp.json()
break
except (json.JSONDecodeError, httpx.HTTPError):
await asyncio.sleep(1)
await asyncio.sleep(2)
continue
else:
self._logger.error(f"[{self._video_id}]"
f"Exceeded retry count.")
return None
raise exceptions.RetryExceedMaxCount()
return livechat_json
async def _callback_loop(self, callback):
@@ -319,6 +342,8 @@ class LiveChatAsync:
self._logger.debug(f'[{self._video_id}] cancelled:{sender}')
def terminate(self):
if not self.is_alive():
return
if self._pauser.empty():
self._pauser.put_nowait(None)
self._is_alive = False
@@ -330,9 +355,6 @@ class LiveChatAsync:
self.terminate()
def _task_finished(self):
'''
Terminate fetching chats.
'''
if self.is_alive():
self.terminate()
try:

View File

@@ -3,7 +3,6 @@ import json
import signal
import time
import traceback
import urllib.parse
from concurrent.futures import CancelledError, ThreadPoolExecutor
from queue import Queue
from threading import Event
@@ -11,10 +10,10 @@ from .buffer import Buffer
from ..parser.live import Parser
from .. import config
from .. import exceptions
from .. import util
from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
from ..util.extract_video_id import extract_video_id
headers = config.headers
MAX_RETRY = 10
@@ -61,6 +60,10 @@ class LiveChat:
topchat_only : bool
If True, get only top chat.
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes
---------
_executor : ThreadPoolExecutor
@@ -75,6 +78,7 @@ class LiveChat:
def __init__(self, video_id,
seektime=-1,
processor=DefaultProcessor(),
client = httpx.Client(http2=True),
buffer=None,
interruptable=True,
callback=None,
@@ -82,9 +86,11 @@ class LiveChat:
direct_mode=False,
force_replay=False,
topchat_only=False,
logger=config.logger(__name__)
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = extract_video_id(video_id)
self._client = client
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
@@ -96,15 +102,19 @@ class LiveChat:
self._executor = ThreadPoolExecutor(max_workers=2)
self._direct_mode = direct_mode
self._is_alive = True
self._is_replay = force_replay
self._is_replay = force_replay or (replay_continuation is not None)
self._parser = Parser(is_replay=self._is_replay)
self._pauser = Queue()
self._pauser.put_nowait(None)
self._first_fetch = True
self._fetch_url = "live_chat/get_live_chat?continuation="
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._event = Event()
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self._event = Event()
self.continuation = replay_continuation
self.exception = None
if interruptable:
signal.signal(signal.SIGINT, lambda a, b: self.terminate())
@@ -139,8 +149,12 @@ class LiveChat:
"""Fetch first continuation parameter,
create and start _listen loop.
"""
initial_continuation = liveparam.getparam(self._video_id, 3)
self._listen(initial_continuation)
if not self.continuation:
self.continuation = liveparam.getparam(
self._video_id,
channel_id=util.get_channelid(self._client, self._video_id),
past_sec=3)
self._listen(self.continuation)
def _listen(self, continuation):
''' Fetch chat data and store them into buffer,
@@ -152,11 +166,14 @@ class LiveChat:
parameter for next chat data
'''
try:
with httpx.Client(http2=True) as client:
with self._client as client:
while(continuation and self._is_alive):
continuation = self._check_pause(continuation)
contents = self._get_contents(continuation, client, headers)
metadata, chatdata = self._parser.parse(contents)
continuation = metadata.get('continuation')
if continuation:
self.continuation = continuation
timeout = metadata['timeoutMs'] / 1000
chat_component = {
"video_id": self._video_id,
@@ -175,7 +192,7 @@ class LiveChat:
self._buffer.put(chat_component)
diff_time = timeout - (time.time() - time_mark)
self._event.wait(diff_time if diff_time > 0 else 0)
continuation = metadata.get('continuation')
self._last_offset_ms = metadata.get('last_offset_ms', 0)
except exceptions.ChatParseException as e:
self._logger.debug(f"[{self._video_id}]{str(e)}")
raise
@@ -185,7 +202,6 @@ class LiveChat:
self._logger.debug(f"[{self._video_id}] finished fetching chat.")
def _check_pause(self, continuation):
if self._pauser.empty():
'''pause'''
@@ -195,7 +211,10 @@ class LiveChat:
'''
self._pauser.put_nowait(None)
if not self._is_replay:
continuation = liveparam.getparam(self._video_id, 3)
continuation = liveparam.getparam(
self._video_id, channel_id=util.get_channelid(httpx.Client(http2=True), self._video_id),
past_sec=3, topchat_only=self._topchat_only)
return continuation
def _get_contents(self, continuation, client, headers):
@@ -207,39 +226,42 @@ class LiveChat:
-------
'continuationContents' which includes metadata & chat data.
'''
livechat_json = (
self._get_livechat_json(continuation, client, headers)
)
contents = self._parser.get_contents(livechat_json)
livechat_json = self._get_livechat_json(
continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = "live_chat_replay/get_live_chat_replay?continuation="
self._fetch_url = config._smr
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only)
livechat_json = (self._get_livechat_json(continuation, client, headers))
self._video_id, self.seektime, self._topchat_only, util.get_channelid(client, self._video_id))
livechat_json = self._get_livechat_json(
continuation, client, replay=True, offset_ms=self.seektime * 1000)
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json))
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (self._get_livechat_json(
reload_continuation, client, headers))
contents = self._parser.get_contents(livechat_json)
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
def _get_livechat_json(self, continuation, client, headers):
def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
'''
continuation = urllib.parse.quote(continuation)
livechat_json = None
url = f"https://www.youtube.com/{self._fetch_url}{continuation}&pbj=1"
if offset_ms < 0:
offset_ms = 0
param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
for _ in range(MAX_RETRY + 1):
with client:
try:
livechat_json = client.get(url, headers=headers).json()
response = client.post(self._fetch_url, json=param)
livechat_json = response.json()
break
except (json.JSONDecodeError, httpx.HTTPError):
time.sleep(2)
@@ -312,9 +334,8 @@ class LiveChat:
self._logger.debug(f'[{self._video_id}] cancelled:{sender}')
def terminate(self):
'''
Terminate fetching chats.
'''
if not self.is_alive():
return
if self._pauser.empty():
self._pauser.put_nowait(None)
self._is_alive = False

View File

@@ -1,55 +1,37 @@
from .pb.header_pb2 import Header
from .pb.replay_pb2 import Continuation
from . import enc
from base64 import urlsafe_b64encode as b64enc
from urllib.parse import quote
import base64
'''
Generate continuation parameter of youtube replay chat.
Author: taizan-hokuto
ver 0.0.1 2019.10.05 : Initial release.
ver 0.0.2 2020.05.30 : Use Protocol Buffers.
'''
def _gen_vid(video_id) -> str:
header = Header()
header.info.video.id = video_id
header.terminator = 1
return base64.urlsafe_b64encode(header.SerializeToString()).decode()
def _header(video_id, channel_id) -> str:
S1_3 = enc.rs(1, video_id)
S1_5 = enc.rs(1, channel_id) + enc.rs(2, video_id)
S1 = enc.rs(3, S1_3) + enc.rs(5, S1_5)
S3 = enc.rs(48687757, enc.rs(1, video_id))
header_replay = enc.rs(1, S1) + enc.rs(3, S3) + enc.nm(4, 1)
return b64enc(header_replay)
def _build(video_id, seektime, topchat_only) -> str:
chattype = 1
timestamp = 0
if topchat_only:
chattype = 4
fetch_before_start = 3
def _build(video_id, seektime, topchat_only, channel_id) -> str:
chattype = 4 if topchat_only else 1
if seektime < 0:
fetch_before_start = 4
elif seektime == 0:
timestamp = 1
else:
seektime = 0
timestamp = int(seektime * 1000000)
continuation = Continuation()
entity = continuation.entity
entity.header = _gen_vid(video_id)
entity.timestamp = timestamp
entity.s6 = 0
entity.s7 = 0
entity.s8 = 0
entity.s9 = fetch_before_start
entity.s10 = ''
entity.s12 = chattype
entity.chattype.value = chattype
entity.s15 = 0
return quote(
base64.urlsafe_b64encode(continuation.SerializeToString()).decode())
header = enc.rs(3, _header(video_id, channel_id))
timestamp = enc.nm(5, timestamp)
s6 = enc.nm(6, 0)
s7 = enc.nm(7, 0)
s8 = enc.nm(8, 0)
s9 = enc.nm(9, 4)
s10 = enc.rs(10, enc.nm(4, 0))
chattype = enc.rs(14, enc.nm(1, 4))
s15 = enc.nm(15, 0)
entity = b''.join((header, timestamp, s6, s7, s8, s9, s10, chattype, s15))
continuation = enc.rs(156074452, entity)
return quote(b64enc(continuation).decode())
def getparam(video_id, seektime=-1, topchat_only=False) -> str:
def getparam(video_id, seektime=0, topchat_only=False, channel_id='') -> str:
'''
Parameter
---------
@@ -59,4 +41,4 @@ def getparam(video_id, seektime=-1, topchat_only=False) -> str:
topchat_only : bool
if True, fetch only 'top chat'
'''
return _build(video_id, seektime, topchat_only)
return _build(video_id, seektime, topchat_only, channel_id)

24
pytchat/paramgen/enc.py Normal file
View File

@@ -0,0 +1,24 @@
def vn(val):
if val < 0:
raise ValueError
buf = b''
while val >> 7:
m = val & 0xFF | 0x80
buf += m.to_bytes(1, 'big')
val >>= 7
buf += val.to_bytes(1, 'big')
return buf
def tp(a, b, ary):
return vn((b << 3) | a) + ary
def rs(a, ary):
if isinstance(ary, str):
ary = ary.encode()
return tp(2, a, vn(len(ary)) + ary)
def nm(a, ary):
return tp(0, a, vn(ary))

View File

@@ -1,69 +1,51 @@
from .pb.header_pb2 import Header
from .pb.live_pb2 import Continuation
from urllib.parse import quote
import base64
import random
import time
'''
Generate continuation parameter of youtube live chat.
Author: taizan-hokuto
ver 0.0.1 2019.10.05 : Initial release.
ver 0.0.2 2020.05.30 : Use Protocol Buffers.
'''
from . import enc
from base64 import urlsafe_b64encode as b64enc
from urllib.parse import quote
def _gen_vid(video_id) -> str:
"""generate video_id parameter.
Parameter
---------
video_id : str
Return
---------
str : base64 encoded video_id parameter.
"""
header = Header()
header.info.video.id = video_id
header.terminator = 1
return base64.urlsafe_b64encode(header.SerializeToString()).decode()
def _header(video_id, channel_id) -> str:
S1_3 = enc.rs(1, video_id)
S1_5 = enc.rs(1, channel_id) + enc.rs(2, video_id)
S1 = enc.rs(3, S1_3) + enc.rs(5, S1_5)
S3 = enc.rs(48687757, enc.rs(1, video_id))
header_replay = enc.rs(1, S1) + enc.rs(3, S3) + enc.nm(4, 1)
return b64enc(header_replay)
def _build(video_id, ts1, ts2, ts3, ts4, ts5, topchat_only) -> str:
chattype = 1
if topchat_only:
chattype = 4
continuation = Continuation()
entity = continuation.entity
def _build(video_id, channel_id, ts1, ts2, ts3, ts4, ts5, topchat_only) -> str:
chattype = 4 if topchat_only else 1
entity.header = _gen_vid(video_id)
entity.timestamp1 = ts1
entity.s6 = 0
entity.s7 = 0
entity.s8 = 1
entity.body.b1 = 0
entity.body.b2 = 0
entity.body.b3 = 0
entity.body.b4 = 0
entity.body.b7 = ''
entity.body.b8 = 0
entity.body.b9 = ''
entity.body.timestamp2 = ts2
entity.body.b11 = 3
entity.body.b15 = 0
entity.timestamp3 = ts3
entity.timestamp4 = ts4
entity.s13 = chattype
entity.chattype.value = chattype
entity.s17 = 0
entity.str19.value = 0
entity.timestamp5 = ts5
b1 = enc.nm(1, 0)
b2 = enc.nm(2, 0)
b3 = enc.nm(3, 0)
b4 = enc.nm(4, 0)
b7 = enc.rs(7, '')
b8 = enc.nm(8, 0)
b9 = enc.rs(9, '')
timestamp2 = enc.nm(10, ts2)
b11 = enc.nm(11, 3)
b15 = enc.nm(15, 0)
return quote(
base64.urlsafe_b64encode(continuation.SerializeToString()).decode()
)
header = enc.rs(3, _header(video_id, channel_id))
timestamp1 = enc.nm(5, ts1)
s6 = enc.nm(6, 0)
s7 = enc.nm(7, 0)
s8 = enc.nm(8, 1)
body = enc.rs(9, b''.join(
(b1, b2, b3, b4, b7, b8, b9, timestamp2, b11, b15)))
timestamp3 = enc.nm(10, ts3)
timestamp4 = enc.nm(11, ts4)
s13 = enc.nm(13, chattype)
chattype = enc.rs(16, enc.nm(1, chattype))
s17 = enc.nm(17, 0)
str19 = enc.rs(19, enc.nm(1, 0))
timestamp5 = enc.nm(20, ts5)
entity = b''.join((header, timestamp1, s6, s7, s8, body, timestamp3,
timestamp4, s13, chattype, s17, str19, timestamp5))
continuation = enc.rs(119693434, entity)
return quote(b64enc(continuation).decode())
def _times(past_sec):
@@ -76,7 +58,7 @@ def _times(past_sec):
return list(map(lambda x: int(x * 1000000), [_ts1, _ts2, _ts3, _ts4, _ts5]))
def getparam(video_id, past_sec=0, topchat_only=False) -> str:
def getparam(video_id, channel_id, past_sec=0, topchat_only=False) -> str:
'''
Parameter
---------
@@ -85,4 +67,4 @@ def getparam(video_id, past_sec=0, topchat_only=False) -> str:
topchat_only : bool
if True, fetch only 'top chat'
'''
return _build(video_id, *_times(past_sec), topchat_only)
return _build(video_id, channel_id, *_times(past_sec), topchat_only)

View File

@@ -1,159 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: header.proto
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='header.proto',
package='',
syntax='proto3',
serialized_options=None,
create_key=_descriptor._internal_create_key,
serialized_pb=b'\n\x0cheader.proto\"\x13\n\x05Video\x12\n\n\x02id\x18\x01 \x01(\t\"#\n\nHeaderInfo\x12\x15\n\x05video\x18\x01 \x01(\x0b\x32\x06.Video\"7\n\x06Header\x12\x19\n\x04info\x18\x01 \x01(\x0b\x32\x0b.HeaderInfo\x12\x12\n\nterminator\x18\x04 \x01(\x05\x62\x06proto3'
)
_VIDEO = _descriptor.Descriptor(
name='Video',
full_name='Video',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='id', full_name='Video.id', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=16,
serialized_end=35,
)
_HEADERINFO = _descriptor.Descriptor(
name='HeaderInfo',
full_name='HeaderInfo',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='video', full_name='HeaderInfo.video', index=0,
number=1, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=37,
serialized_end=72,
)
_HEADER = _descriptor.Descriptor(
name='Header',
full_name='Header',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='info', full_name='Header.info', index=0,
number=1, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='terminator', full_name='Header.terminator', index=1,
number=4, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=74,
serialized_end=129,
)
_HEADERINFO.fields_by_name['video'].message_type = _VIDEO
_HEADER.fields_by_name['info'].message_type = _HEADERINFO
DESCRIPTOR.message_types_by_name['Video'] = _VIDEO
DESCRIPTOR.message_types_by_name['HeaderInfo'] = _HEADERINFO
DESCRIPTOR.message_types_by_name['Header'] = _HEADER
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Video = _reflection.GeneratedProtocolMessageType('Video', (_message.Message,), {
'DESCRIPTOR' : _VIDEO,
'__module__' : 'header_pb2'
# @@protoc_insertion_point(class_scope:Video)
})
_sym_db.RegisterMessage(Video)
HeaderInfo = _reflection.GeneratedProtocolMessageType('HeaderInfo', (_message.Message,), {
'DESCRIPTOR' : _HEADERINFO,
'__module__' : 'header_pb2'
# @@protoc_insertion_point(class_scope:HeaderInfo)
})
_sym_db.RegisterMessage(HeaderInfo)
Header = _reflection.GeneratedProtocolMessageType('Header', (_message.Message,), {
'DESCRIPTOR' : _HEADER,
'__module__' : 'header_pb2'
# @@protoc_insertion_point(class_scope:Header)
})
_sym_db.RegisterMessage(Header)
# @@protoc_insertion_point(module_scope)

View File

@@ -1,381 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: live.proto
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='live.proto',
package='live',
syntax='proto3',
serialized_options=None,
create_key=_descriptor._internal_create_key,
serialized_pb=b'\n\nlive.proto\x12\x04live\"\x88\x01\n\x04\x42ody\x12\n\n\x02\x62\x31\x18\x01 \x01(\x05\x12\n\n\x02\x62\x32\x18\x02 \x01(\x05\x12\n\n\x02\x62\x33\x18\x03 \x01(\x05\x12\n\n\x02\x62\x34\x18\x04 \x01(\x05\x12\n\n\x02\x62\x37\x18\x07 \x01(\t\x12\n\n\x02\x62\x38\x18\x08 \x01(\x05\x12\n\n\x02\x62\x39\x18\t \x01(\t\x12\x12\n\ntimestamp2\x18\n \x01(\x03\x12\x0b\n\x03\x62\x31\x31\x18\x0b \x01(\x05\x12\x0b\n\x03\x62\x31\x35\x18\x0f \x01(\x05\"\x19\n\x08\x43hatType\x12\r\n\x05value\x18\x01 \x01(\x05\"\x16\n\x05STR19\x12\r\n\x05value\x18\x01 \x01(\x05\"\x8a\x02\n\x12\x43ontinuationEntity\x12\x0e\n\x06header\x18\x03 \x01(\t\x12\x12\n\ntimestamp1\x18\x05 \x01(\x03\x12\n\n\x02s6\x18\x06 \x01(\x05\x12\n\n\x02s7\x18\x07 \x01(\x05\x12\n\n\x02s8\x18\x08 \x01(\x05\x12\x18\n\x04\x62ody\x18\t \x01(\x0b\x32\n.live.Body\x12\x12\n\ntimestamp3\x18\n \x01(\x03\x12\x12\n\ntimestamp4\x18\x0b \x01(\x03\x12\x0b\n\x03s13\x18\r \x01(\x05\x12 \n\x08\x63hattype\x18\x10 \x01(\x0b\x32\x0e.live.ChatType\x12\x0b\n\x03s17\x18\x11 \x01(\x05\x12\x1a\n\x05str19\x18\x13 \x01(\x0b\x32\x0b.live.STR19\x12\x12\n\ntimestamp5\x18\x14 \x01(\x03\";\n\x0c\x43ontinuation\x12+\n\x06\x65ntity\x18\xfa\xc0\x89\x39 \x01(\x0b\x32\x18.live.ContinuationEntityb\x06proto3'
)
_BODY = _descriptor.Descriptor(
name='Body',
full_name='live.Body',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='b1', full_name='live.Body.b1', index=0,
number=1, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b2', full_name='live.Body.b2', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b3', full_name='live.Body.b3', index=2,
number=3, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b4', full_name='live.Body.b4', index=3,
number=4, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b7', full_name='live.Body.b7', index=4,
number=7, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b8', full_name='live.Body.b8', index=5,
number=8, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b9', full_name='live.Body.b9', index=6,
number=9, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp2', full_name='live.Body.timestamp2', index=7,
number=10, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b11', full_name='live.Body.b11', index=8,
number=11, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='b15', full_name='live.Body.b15', index=9,
number=15, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=21,
serialized_end=157,
)
_CHATTYPE = _descriptor.Descriptor(
name='ChatType',
full_name='live.ChatType',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='value', full_name='live.ChatType.value', index=0,
number=1, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=159,
serialized_end=184,
)
_STR19 = _descriptor.Descriptor(
name='STR19',
full_name='live.STR19',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='value', full_name='live.STR19.value', index=0,
number=1, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=186,
serialized_end=208,
)
_CONTINUATIONENTITY = _descriptor.Descriptor(
name='ContinuationEntity',
full_name='live.ContinuationEntity',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='header', full_name='live.ContinuationEntity.header', index=0,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp1', full_name='live.ContinuationEntity.timestamp1', index=1,
number=5, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s6', full_name='live.ContinuationEntity.s6', index=2,
number=6, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s7', full_name='live.ContinuationEntity.s7', index=3,
number=7, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s8', full_name='live.ContinuationEntity.s8', index=4,
number=8, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='body', full_name='live.ContinuationEntity.body', index=5,
number=9, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp3', full_name='live.ContinuationEntity.timestamp3', index=6,
number=10, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp4', full_name='live.ContinuationEntity.timestamp4', index=7,
number=11, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s13', full_name='live.ContinuationEntity.s13', index=8,
number=13, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='chattype', full_name='live.ContinuationEntity.chattype', index=9,
number=16, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s17', full_name='live.ContinuationEntity.s17', index=10,
number=17, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='str19', full_name='live.ContinuationEntity.str19', index=11,
number=19, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp5', full_name='live.ContinuationEntity.timestamp5', index=12,
number=20, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=211,
serialized_end=477,
)
_CONTINUATION = _descriptor.Descriptor(
name='Continuation',
full_name='live.Continuation',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='entity', full_name='live.Continuation.entity', index=0,
number=119693434, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=479,
serialized_end=538,
)
_CONTINUATIONENTITY.fields_by_name['body'].message_type = _BODY
_CONTINUATIONENTITY.fields_by_name['chattype'].message_type = _CHATTYPE
_CONTINUATIONENTITY.fields_by_name['str19'].message_type = _STR19
_CONTINUATION.fields_by_name['entity'].message_type = _CONTINUATIONENTITY
DESCRIPTOR.message_types_by_name['Body'] = _BODY
DESCRIPTOR.message_types_by_name['ChatType'] = _CHATTYPE
DESCRIPTOR.message_types_by_name['STR19'] = _STR19
DESCRIPTOR.message_types_by_name['ContinuationEntity'] = _CONTINUATIONENTITY
DESCRIPTOR.message_types_by_name['Continuation'] = _CONTINUATION
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Body = _reflection.GeneratedProtocolMessageType('Body', (_message.Message,), {
'DESCRIPTOR' : _BODY,
'__module__' : 'live_pb2'
# @@protoc_insertion_point(class_scope:live.Body)
})
_sym_db.RegisterMessage(Body)
ChatType = _reflection.GeneratedProtocolMessageType('ChatType', (_message.Message,), {
'DESCRIPTOR' : _CHATTYPE,
'__module__' : 'live_pb2'
# @@protoc_insertion_point(class_scope:live.ChatType)
})
_sym_db.RegisterMessage(ChatType)
STR19 = _reflection.GeneratedProtocolMessageType('STR19', (_message.Message,), {
'DESCRIPTOR' : _STR19,
'__module__' : 'live_pb2'
# @@protoc_insertion_point(class_scope:live.STR19)
})
_sym_db.RegisterMessage(STR19)
ContinuationEntity = _reflection.GeneratedProtocolMessageType('ContinuationEntity', (_message.Message,), {
'DESCRIPTOR' : _CONTINUATIONENTITY,
'__module__' : 'live_pb2'
# @@protoc_insertion_point(class_scope:live.ContinuationEntity)
})
_sym_db.RegisterMessage(ContinuationEntity)
Continuation = _reflection.GeneratedProtocolMessageType('Continuation', (_message.Message,), {
'DESCRIPTOR' : _CONTINUATION,
'__module__' : 'live_pb2'
# @@protoc_insertion_point(class_scope:live.Continuation)
})
_sym_db.RegisterMessage(Continuation)
# @@protoc_insertion_point(module_scope)

View File

@@ -1,215 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: replay.proto
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='replay.proto',
package='replay',
syntax='proto3',
serialized_options=None,
create_key=_descriptor._internal_create_key,
serialized_pb=b'\n\x0creplay.proto\x12\x06replay\"\x19\n\x08\x43hatType\x12\r\n\x05value\x18\x01 \x01(\x05\"\xb2\x01\n\x12\x43ontinuationEntity\x12\x0e\n\x06header\x18\x03 \x01(\t\x12\x11\n\ttimestamp\x18\x05 \x01(\x03\x12\n\n\x02s6\x18\x06 \x01(\x05\x12\n\n\x02s7\x18\x07 \x01(\x05\x12\n\n\x02s8\x18\x08 \x01(\x05\x12\n\n\x02s9\x18\t \x01(\x05\x12\x0b\n\x03s10\x18\n \x01(\t\x12\x0b\n\x03s12\x18\x0c \x01(\x05\x12\"\n\x08\x63hattype\x18\x0e \x01(\x0b\x32\x10.replay.ChatType\x12\x0b\n\x03s15\x18\x0f \x01(\x05\"=\n\x0c\x43ontinuation\x12-\n\x06\x65ntity\x18\xd4\x83\xb6J \x01(\x0b\x32\x1a.replay.ContinuationEntityb\x06proto3'
)
_CHATTYPE = _descriptor.Descriptor(
name='ChatType',
full_name='replay.ChatType',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='value', full_name='replay.ChatType.value', index=0,
number=1, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=24,
serialized_end=49,
)
_CONTINUATIONENTITY = _descriptor.Descriptor(
name='ContinuationEntity',
full_name='replay.ContinuationEntity',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='header', full_name='replay.ContinuationEntity.header', index=0,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='timestamp', full_name='replay.ContinuationEntity.timestamp', index=1,
number=5, type=3, cpp_type=2, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s6', full_name='replay.ContinuationEntity.s6', index=2,
number=6, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s7', full_name='replay.ContinuationEntity.s7', index=3,
number=7, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s8', full_name='replay.ContinuationEntity.s8', index=4,
number=8, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s9', full_name='replay.ContinuationEntity.s9', index=5,
number=9, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s10', full_name='replay.ContinuationEntity.s10', index=6,
number=10, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s12', full_name='replay.ContinuationEntity.s12', index=7,
number=12, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='chattype', full_name='replay.ContinuationEntity.chattype', index=8,
number=14, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
_descriptor.FieldDescriptor(
name='s15', full_name='replay.ContinuationEntity.s15', index=9,
number=15, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=52,
serialized_end=230,
)
_CONTINUATION = _descriptor.Descriptor(
name='Continuation',
full_name='replay.Continuation',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='entity', full_name='replay.Continuation.entity', index=0,
number=156074452, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=232,
serialized_end=293,
)
_CONTINUATIONENTITY.fields_by_name['chattype'].message_type = _CHATTYPE
_CONTINUATION.fields_by_name['entity'].message_type = _CONTINUATIONENTITY
DESCRIPTOR.message_types_by_name['ChatType'] = _CHATTYPE
DESCRIPTOR.message_types_by_name['ContinuationEntity'] = _CONTINUATIONENTITY
DESCRIPTOR.message_types_by_name['Continuation'] = _CONTINUATION
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
ChatType = _reflection.GeneratedProtocolMessageType('ChatType', (_message.Message,), {
'DESCRIPTOR' : _CHATTYPE,
'__module__' : 'replay_pb2'
# @@protoc_insertion_point(class_scope:replay.ChatType)
})
_sym_db.RegisterMessage(ChatType)
ContinuationEntity = _reflection.GeneratedProtocolMessageType('ContinuationEntity', (_message.Message,), {
'DESCRIPTOR' : _CONTINUATIONENTITY,
'__module__' : 'replay_pb2'
# @@protoc_insertion_point(class_scope:replay.ContinuationEntity)
})
_sym_db.RegisterMessage(ContinuationEntity)
Continuation = _reflection.GeneratedProtocolMessageType('Continuation', (_message.Message,), {
'DESCRIPTOR' : _CONTINUATION,
'__module__' : 'replay_pb2'
# @@protoc_insertion_point(class_scope:replay.Continuation)
})
_sym_db.RegisterMessage(Continuation)
# @@protoc_insertion_point(module_scope)

View File

@@ -1,14 +0,0 @@
syntax = "proto3";
message Video {
string id = 1;
}
message HeaderInfo {
Video video = 1;
}
message Header {
HeaderInfo info = 1;
int32 terminator = 4;
}

View File

@@ -1,45 +0,0 @@
syntax = "proto3";
package live;
message Body {
int32 b1 = 1;
int32 b2 = 2;
int32 b3 = 3;
int32 b4 = 4;
string b7 = 7;
int32 b8 = 8;
string b9 = 9;
int64 timestamp2 = 10;
int32 b11 = 11;
int32 b15 = 15;
}
message ChatType {
int32 value = 1;
}
message STR19 {
int32 value = 1;
}
message ContinuationEntity {
string header = 3;
int64 timestamp1 = 5;
int32 s6 = 6;
int32 s7 = 7;
int32 s8 = 8;
Body body = 9;
int64 timestamp3 = 10;
int64 timestamp4 = 11;
int32 s13 = 13;
ChatType chattype = 16;
int32 s17 = 17;
STR19 str19 = 19;
int64 timestamp5 = 20;
}
message Continuation {
ContinuationEntity entity = 119693434;
}

View File

@@ -1,24 +0,0 @@
syntax = "proto3";
package replay;
message ChatType {
int32 value = 1;
}
message ContinuationEntity {
string header = 3;
int64 timestamp = 5;
int32 s6 = 6;
int32 s7 = 7;
int32 s8 = 8;
int32 s9 = 9;
string s10 = 10;
int32 s12 = 12;
ChatType chattype = 14;
int32 s15 = 15;
}
message Continuation {
ContinuationEntity entity = 156074452;
}

View File

@@ -28,11 +28,12 @@ class Parser:
def get_contents(self, jsn):
if jsn is None:
self.raise_exception(exceptions.IllegalFunctionCall('Called with none JSON object.'))
if jsn['response']['responseContext'].get('errors'):
if jsn.get("responseContext", {}).get("errors"):
raise exceptions.ResponseContextError(
'The video_id would be wrong, or video is deleted or private.')
contents = jsn['response'].get('continuationContents')
return contents
contents = jsn.get('continuationContents')
visitor_data = jsn.get("responseContext", {}).get("visitorData")
return contents, visitor_data
def parse(self, contents):
"""
@@ -85,6 +86,7 @@ class Parser:
'''Broadcasting end or cannot fetch chat stream'''
self.raise_exception(exceptions.NoContents('Chat data stream is empty.'))
cont = contents['liveChatContinuation']['continuations'][0]
if cont.get("liveChatReplayContinuationData"):
# chat data exist.
return None
@@ -97,23 +99,22 @@ class Parser:
def _create_data(self, metadata, contents):
actions = contents['liveChatContinuation'].get('actions')
if self.is_replay:
interval = self._get_interval(actions)
metadata.setdefault("timeoutMs", interval)
last_offset_ms = self._get_lastoffset(actions)
metadata.setdefault("timeoutMs", 5000)
metadata.setdefault("last_offset_ms", last_offset_ms)
"""Archived chat has different structures than live chat,
so make it the same format."""
chatdata = [action["replayChatItemAction"]["actions"][0]
for action in actions]
else:
metadata.setdefault('timeoutMs', 10000)
metadata.setdefault('timeoutMs', 5000)
chatdata = actions
return metadata, chatdata
def _get_interval(self, actions: list):
if actions is None:
def _get_lastoffset(self, actions: list):
if actions:
return int(actions[-1]["replayChatItemAction"]["videoOffsetTimeMsec"])
return 0
start = int(actions[0]["replayChatItemAction"]["videoOffsetTimeMsec"])
last = int(actions[-1]["replayChatItemAction"]["videoOffsetTimeMsec"])
return (last - start)
def raise_exception(self, exception):
if self.exception_holder is None:

View File

@@ -1,5 +1,6 @@
import datetime
import pytz
from datetime import datetime, timedelta, timezone
TZ_UTC = timezone(timedelta(0), 'UTC')
class BaseRenderer:
@@ -62,13 +63,13 @@ class BaseRenderer:
if badges:
for badge in badges:
author_type = badge["liveChatAuthorBadgeRenderer"]["accessibility"]["accessibilityData"]["label"]
if author_type == '確認済み':
if author_type == 'VERIFIED' or author_type == '確認済み':
isVerified = True
if author_type == '所有者':
if author_type == 'OWNER' or author_type == '所有者':
isChatOwner = True
if 'メンバー' in author_type:
if 'メンバー' in author_type or 'MEMBER' in author_type:
isChatSponsor = True
if author_type == 'モデレーター':
if author_type == 'MODERATOR' or author_type == 'モデレーター':
isChatModerator = True
return isVerified, isChatOwner, isChatSponsor, isChatModerator
@@ -76,6 +77,6 @@ class BaseRenderer:
return self.renderer.get('id')
def get_publishedat(self, timestamp):
dt = datetime.datetime.fromtimestamp(int(timestamp) / 1000000)
return dt.astimezone(pytz.utc).isoformat(
dt = datetime.fromtimestamp(int(timestamp) / 1000000)
return dt.astimezone(TZ_UTC).isoformat(
timespec='milliseconds').replace('+00:00', 'Z')

View File

@@ -1,12 +1,12 @@
'''
YouTubeスーパーチャットで使用される通貨の記号とレート検索用の略号の
対応表
Table of symbols for the currencies used in YouTube Superchat.
Key
YouTubeスーパーチャットで使用される通貨の記号
(アルファベットで終わる場合、0xA0(&npsp)が付く)
Currency symbols used in YouTube Super Chat
If it ends with an alphabet, it will be followed by 0xA0(&npsp).
Value:
fxtext: 3文字の通貨略称
jptest: 日本語テキスト
fxtext: ISO 4217 currency code
jptest: japanese text
'''
symbols = {
"$": {"fxtext": "USD", "jptext": "米・ドル"},

View File

@@ -7,6 +7,7 @@ from .renderer.paidmessage import LiveChatPaidMessageRenderer
from .renderer.paidsticker import LiveChatPaidStickerRenderer
from .renderer.legacypaid import LiveChatLegacyPaidMessageRenderer
from .renderer.membership import LiveChatMembershipItemRenderer
from .renderer.donation import LiveChatDonationAnnouncementRenderer
from .. chat_processor import ChatProcessor
from ... import config
@@ -124,7 +125,8 @@ class DefaultProcessor(ChatProcessor):
"liveChatPaidMessageRenderer": LiveChatPaidMessageRenderer(),
"liveChatPaidStickerRenderer": LiveChatPaidStickerRenderer(),
"liveChatLegacyPaidMessageRenderer": LiveChatLegacyPaidMessageRenderer(),
"liveChatMembershipItemRenderer": LiveChatMembershipItemRenderer()
"liveChatMembershipItemRenderer": LiveChatMembershipItemRenderer(),
"liveChatDonationAnnouncementRenderer": LiveChatDonationAnnouncementRenderer(),
}
def process(self, chat_components: list):

View File

@@ -0,0 +1,6 @@
from .base import BaseRenderer
class LiveChatDonationAnnouncementRenderer(BaseRenderer):
def settype(self):
self.chat.type = "donation"

View File

@@ -86,7 +86,7 @@ class HTMLArchiver(ChatProcessor):
Count of total lines written to the file.
"""
if chat_components is None or len(chat_components) == 0:
return self.save_path ,self.counter
return self.save_path, self.counter
for c in self.processor.process(chat_components).items:
self.tmp_fp.write(
self._parse_html_line((

View File

@@ -82,16 +82,17 @@ class RingQueue:
class SpeedCalculator(ChatProcessor, RingQueue):
"""
チャットの勢いを計算する。
Calculate the momentum of the chat.
一定期間のチャットデータのうち、最初のチャットの投稿時刻と
最後のチャットの投稿時刻の差を、チャット数で割り返し
1分あたりの速度に換算する。
Divide the difference between the time of the first chat and
the time of the last chat in the chat data over a period of
time by the number of chats and convert it to speed per minute.
Parameter
----------
capacity : int
RingQueueに格納するチャット勢い算出用データの最大数
Maximum number of data for calculating chat momentum
to be stored in RingQueue.
"""
def __init__(self, capacity=10):
@@ -111,17 +112,17 @@ class SpeedCalculator(ChatProcessor, RingQueue):
def _calc_speed(self):
"""
RingQueue内のチャット勢い算出用データリストを元に、
チャット速度を計算して返す
Calculates the chat speed based on the data list for calculating
the chat momentum in RingQueue.
Return
---------------------------
チャット速度(1分間で換算したチャット数)
Chat speed (number of chats converted per minute)
"""
try:
# キュー内の総チャット数
# Total number of chats in the queue
total = sum(item['chat_count'] for item in self.items)
# キュー内の最初と最後のチャットの時間差
# Interval between the first and last chats in the queue
duration = (self.items[self.last_pos]['endtime'] - self.items[self.first_pos]['starttime'])
if duration != 0:
return int(total * 60 / duration)
@@ -131,19 +132,12 @@ class SpeedCalculator(ChatProcessor, RingQueue):
def _put_chatdata(self, actions):
"""
チャットデータからタイムスタンプを読み取り、勢い測定用のデータを組み立て、
RingQueueに投入する。
200円以上のスパチャはtickerとmessageの2つのデータが生成されるが、
tickerの方は時刻データの場所が異なることを利用し、勢いの集計から除外している。
Parameter
---------
actions : List[dict]
チャットデータ(addChatItemAction) のリスト
List of addChatItemActions
"""
def _put_emptydata():
'''
チャットデータがない場合に空のデータをキューに投入する。
'''
timestamp_now = int(time.time())
self.put({
'chat_count': 0,
@@ -152,9 +146,6 @@ class SpeedCalculator(ChatProcessor, RingQueue):
})
def _get_timestamp(action: dict):
"""
チャットデータから時刻データを取り出す。
"""
try:
item = action['addChatItemAction']['item']
timestamp = int(item[list(item.keys())[0]]['timestampUsec'])
@@ -166,32 +157,24 @@ class SpeedCalculator(ChatProcessor, RingQueue):
_put_emptydata()
return
# actions内の時刻データを持つチャットデータの数
counter = 0
# actions内の最初のチャットデータの時刻
starttime = None
# actions内の最後のチャットデータの時刻
endtime = None
for action in actions:
# チャットデータからtimestampUsecを読み取る
# Get timestampUsec from chatdata
gettime = _get_timestamp(action)
# 時刻のないデータだった場合は次の行のデータで読み取り試行
if gettime is None:
continue
# 最初に有効な時刻を持つデータのtimestampをstarttimeに設定
if starttime is None:
starttime = gettime
# 最後のtimestampを設定(途中で時刻のないデータの場合もあるので上書きしていく)
endtime = gettime
# チャットの数をインクリメント
counter += 1
# チャット速度用のデータをRingQueueに送る
if starttime is None or endtime is None:
_put_emptydata()
return

View File

@@ -1,176 +0,0 @@
import asyncio
import httpx
import socket
from . import parser
from . block import Block
from . worker import ExtractWorker
from . patch import Patch
from ... import config
from ... paramgen import arcparam
from ... exceptions import UnknownConnectionError
from concurrent.futures import CancelledError
from json import JSONDecodeError
from urllib.parse import quote
headers = config.headers
REPLAY_URL = "https://www.youtube.com/live_chat_replay/" \
"get_live_chat_replay?continuation="
MAX_RETRY_COUNT = 3
# Set to avoid duplicate parameters
param_set = set()
def _split(start, end, count, min_interval_sec=120):
"""
Split section from `start` to `end` into `count` pieces,
and returns the beginning of each piece.
The `count` is adjusted so that the length of each piece
is no smaller than `min_interval`.
Returns:
--------
List of the offset of each block's first chat data.
"""
if not (isinstance(start, int) or isinstance(start, float)) or \
not (isinstance(end, int) or isinstance(end, float)):
raise ValueError("start/end must be int or float")
if not isinstance(count, int):
raise ValueError("count must be int")
if start > end:
raise ValueError("end must be equal to or greater than start.")
if count < 1:
raise ValueError("count must be equal to or greater than 1.")
if (end - start) / count < min_interval_sec:
count = int((end - start) / min_interval_sec)
if count == 0:
count = 1
interval = (end - start) / count
if count == 1:
return [start]
return sorted(list(set([int(start + interval * j)
for j in range(count)])))
def ready_blocks(video_id, duration, div, callback):
param_set.clear()
if div <= 0:
raise ValueError
async def _get_blocks(video_id, duration, div, callback):
async with httpx.AsyncClient(http2=True) as session:
tasks = [_create_block(session, video_id, seektime, callback)
for seektime in _split(-1, duration, div)]
return await asyncio.gather(*tasks)
async def _create_block(session, video_id, seektime, callback):
continuation = arcparam.getparam(video_id, seektime=seektime)
url = f"{REPLAY_URL}{quote(continuation)}&pbj=1"
err = None
for _ in range(MAX_RETRY_COUNT):
try:
if continuation in param_set:
next_continuation, actions = None, []
break
param_set.add(continuation)
resp = await session.get(url, headers=headers, timeout=10)
next_continuation, actions = parser.parse(resp.json())
break
except JSONDecodeError:
await asyncio.sleep(3)
except httpx.HTTPError as e:
err = e
await asyncio.sleep(3)
else:
cancel()
raise UnknownConnectionError("Abort:" + str(err))
if actions:
first = parser.get_offset(actions[0])
last = parser.get_offset(actions[-1])
if callback:
callback(actions, last - first)
return Block(
continuation=next_continuation,
chat_data=actions,
first=first,
last=last
)
"""
fetch initial blocks.
"""
loop = asyncio.get_event_loop()
blocks = loop.run_until_complete(
_get_blocks(video_id, duration, div, callback))
return blocks
def fetch_patch(callback, blocks, video_id):
async def _allocate_workers():
workers = [
ExtractWorker(
fetch=_fetch, block=block,
blocks=blocks, video_id=video_id
)
for block in blocks
]
async with httpx.AsyncClient() as session:
tasks = [worker.run(session) for worker in workers]
return await asyncio.gather(*tasks)
async def _fetch(continuation, session) -> Patch:
url = f"{REPLAY_URL}{quote(continuation)}&pbj=1"
err = None
for _ in range(MAX_RETRY_COUNT):
try:
if continuation in param_set:
continuation, actions = None, []
break
param_set.add(continuation)
resp = await session.get(url, headers=config.headers)
continuation, actions = parser.parse(resp.json())
break
except JSONDecodeError:
await asyncio.sleep(3)
except httpx.HTTPError as e:
err = e
await asyncio.sleep(3)
except socket.error as error:
print("socket error", error.errno)
await asyncio.sleep(3)
else:
cancel()
raise UnknownConnectionError("Abort:" + str(err))
if actions:
last = parser.get_offset(actions[-1])
first = parser.get_offset(actions[0])
if callback:
callback(actions, last - first)
return Patch(actions, continuation, first, last)
return Patch(continuation=continuation)
"""
allocate workers and assign blocks.
"""
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(_allocate_workers())
except CancelledError:
pass
async def _shutdown():
tasks = [t for t in asyncio.all_tasks()
if t is not asyncio.current_task()]
for task in tasks:
task.cancel()
def cancel():
loop = asyncio.get_event_loop()
loop.create_task(_shutdown())

View File

@@ -1,56 +0,0 @@
class Block:
"""Block object represents something like a box
to join chunk of chatdata.
Parameter:
---------
first : int :
videoOffsetTimeMs of the first chat_data
(chat_data[0])
last : int :
videoOffsetTimeMs of the last chat_data.
(chat_data[-1])
this value increases as fetching chatdata progresses.
end : int :
target videoOffsetTimeMs of last chat data for extract,
equals to first videoOffsetTimeMs of next block.
when extract worker reaches this offset, stop fetching.
continuation : str :
continuation param of last chat data.
chat_data : list
done : bool :
whether this block has been fetched.
remaining : int :
remaining data to extract.
equals end - last.
is_last : bool :
whether this block is the last one in blocklist.
during_split : bool :
whether this block is in the process of during_split.
while True, this block is excluded from duplicate split procedure.
"""
__slots__ = ['first', 'last', 'end', 'continuation', 'chat_data', 'remaining',
'done', 'is_last', 'during_split']
def __init__(self, first=0, last=0, end=0,
continuation='', chat_data=[], is_last=False,
during_split=False):
self.first = first
self.last = last
self.end = end
self.continuation = continuation
self.chat_data = chat_data
self.done = False
self.remaining = self.end - self.last
self.is_last = is_last
self.during_split = during_split

View File

@@ -1,153 +0,0 @@
from . import parser
def check_duplicate(chatdata):
max_range = len(chatdata) - 1
tbl_offset = [None] * max_range
tbl_id = [None] * max_range
tbl_type = [None] * max_range
def create_table(chatdata, max_range):
for i in range(max_range):
tbl_offset[i] = parser.get_offset(chatdata[i])
tbl_id[i] = parser.get_id(chatdata[i])
tbl_type[i] = parser.get_type(chatdata[i])
def is_duplicate(i, j):
return (
tbl_offset[i] == tbl_offset[j]
and tbl_id[i] == tbl_id[j]
and tbl_type[i] == tbl_type[j]
)
print("creating table...")
create_table(chatdata, max_range)
print("searching duplicate data...")
return [{"i": {
"index": i, "id": parser.get_id(chatdata[i]),
"offsetTime": parser.get_offset(chatdata[i]),
"type": parser.get_type(chatdata[i])
},
"j":{
"index": j, "id": parser.get_id(chatdata[j]),
"offsetTime": parser.get_offset(chatdata[j]),
"type": parser.get_type(chatdata[j])
}
}
for i in range(max_range) for j in range(i + 1, max_range)
if is_duplicate(i, j)]
def check_duplicate_offset(chatdata):
max_range = len(chatdata)
tbl_offset = [None] * max_range
tbl_id = [None] * max_range
tbl_type = [None] * max_range
def create_table(chatdata, max_range):
for i in range(max_range):
tbl_offset[i] = parser.get_offset(chatdata[i])
tbl_id[i] = parser.get_id(chatdata[i])
tbl_type[i] = parser.get_type(chatdata[i])
def is_duplicate(i, j):
return (
tbl_offset[i] == tbl_offset[j]
and tbl_id[i] == tbl_id[j]
)
print("creating table...")
create_table(chatdata, max_range)
print("searching duplicate data...")
return [{
"index": i, "id": tbl_id[i],
"offsetTime": tbl_offset[i],
"type:": tbl_type[i]
}
for i in range(max_range - 1)
if is_duplicate(i, i + 1)]
def remove_duplicate_head(blocks):
if len(blocks) == 0 or len(blocks) == 1:
return blocks
def is_duplicate_head(index):
if len(blocks[index].chat_data) == 0:
return True
elif len(blocks[index + 1].chat_data) == 0:
return False
id_0 = parser.get_id(blocks[index].chat_data[0])
id_1 = parser.get_id(blocks[index + 1].chat_data[0])
type_0 = parser.get_type(blocks[index].chat_data[0])
type_1 = parser.get_type(blocks[index + 1].chat_data[0])
return (
blocks[index].first == blocks[index + 1].first
and id_0 == id_1
and type_0 == type_1
)
ret = [blocks[i] for i in range(len(blocks) - 1)
if (len(blocks[i].chat_data) > 0
and not is_duplicate_head(i))]
ret.append(blocks[-1])
return ret
def remove_duplicate_tail(blocks):
if len(blocks) == 0 or len(blocks) == 1:
return blocks
def is_duplicate_tail(index):
if len(blocks[index].chat_data) == 0:
return True
elif len(blocks[index - 1].chat_data) == 0:
return False
id_0 = parser.get_id(blocks[index - 1].chat_data[-1])
id_1 = parser.get_id(blocks[index].chat_data[-1])
type_0 = parser.get_type(blocks[index - 1].chat_data[-1])
type_1 = parser.get_type(blocks[index].chat_data[-1])
return (
blocks[index - 1].last == blocks[index].last
and id_0 == id_1
and type_0 == type_1
)
ret = [blocks[i] for i in range(0, len(blocks))
if i == 0 or not is_duplicate_tail(i)]
return ret
def remove_overlap(blocks):
"""
Fix overlapped blocks after ready_blocks().
Align the last offset of each block to the first offset
of next block (equals `end` offset of each block).
"""
if len(blocks) == 0 or len(blocks) == 1:
return blocks
for block in blocks:
if block.is_last:
break
if len(block.chat_data) == 0:
continue
block_end = block.end
if block.last >= block_end:
for line in reversed(block.chat_data):
if parser.get_offset(line) < block_end:
break
block.chat_data.pop()
block.last = parser.get_offset(line)
block.remaining = 0
block.done = True
block.continuation = None
return blocks
def _dump(blocks):
print("---------- first last end---")
for i, block in enumerate(blocks):
print(
f"block[{i:3}] {block.first:>10} {block.last:>10} {block.end:>10}")

View File

@@ -1,96 +0,0 @@
from typing import Generator
from . import asyncdl
from . import duplcheck
from .. videoinfo import VideoInfo
from ... import config
from ... exceptions import InvalidVideoIdException
from ... util.extract_video_id import extract_video_id
logger = config.logger(__name__)
headers = config.headers
class Extractor:
def __init__(self, video_id, div=1, callback=None, processor=None):
if not isinstance(div, int) or div < 1:
raise ValueError('div must be positive integer.')
elif div > 10:
div = 10
self.video_id = extract_video_id(video_id)
self.div = div
self.callback = callback
self.processor = processor
self.duration = self._get_duration_of_video(video_id)
self.blocks = []
def _get_duration_of_video(self, video_id):
duration = 0
try:
duration = VideoInfo(video_id).get_duration()
except InvalidVideoIdException:
raise
return duration
def _ready_blocks(self):
blocks = asyncdl.ready_blocks(
self.video_id, self.duration, self.div, self.callback)
self.blocks = [block for block in blocks if block]
return self
def _remove_duplicate_head(self):
self.blocks = duplcheck.remove_duplicate_head(self.blocks)
return self
def _set_block_end(self):
if len(self.blocks) > 0:
for i in range(len(self.blocks) - 1):
self.blocks[i].end = self.blocks[i + 1].first
self.blocks[-1].end = self.duration * 1000
self.blocks[-1].is_last = True
return self
def _remove_overlap(self):
self.blocks = duplcheck.remove_overlap(self.blocks)
return self
def _download_blocks(self):
asyncdl.fetch_patch(self.callback, self.blocks, self.video_id)
return self
def _remove_duplicate_tail(self):
self.blocks = duplcheck.remove_duplicate_tail(self.blocks)
return self
def _get_chatdata(self) -> Generator:
for block in self.blocks:
for chatdata in block.chat_data:
yield chatdata
def _execute_extract_operations(self):
return (
self._ready_blocks()
._remove_duplicate_head()
._set_block_end()
._remove_overlap()
._download_blocks()
._remove_duplicate_tail()
._get_chatdata()
)
def extract(self):
if self.duration == 0:
print("\nCannot extract chat data:\n The specified video has not yet been archived.")
return []
data = self._execute_extract_operations()
if self.processor is None:
return data
ret = self.processor.process(
[{'video_id': None,
'timeout': 1,
'chatdata': (action["replayChatItemAction"]["actions"][0] for action in data)}]
)
self.processor.finalize()
return ret
def cancel(self):
asyncdl.cancel()

View File

@@ -1,55 +0,0 @@
from ... import config
from ... import exceptions
logger = config.logger(__name__)
def parse(jsn):
"""
Parse replay chat data.
Parameter:
----------
jsn : dict
JSON of replay chat data.
Returns:
------
continuation : str
actions : list
"""
if jsn is None:
raise ValueError("parameter JSON is None")
if jsn['response']['responseContext'].get('errors'):
raise exceptions.ResponseContextError(
'video_id is invalid or private/deleted.')
contents = jsn['response'].get('continuationContents')
if contents is None:
raise exceptions.NoContents('No chat data.')
cont = contents['liveChatContinuation']['continuations'][0]
if cont is None:
raise exceptions.NoContinuation('No Continuation')
metadata = cont.get('liveChatReplayContinuationData')
if metadata:
continuation = metadata.get("continuation")
actions = contents['liveChatContinuation'].get('actions')
return continuation, actions
return None, []
def get_offset(item):
return int(item['replayChatItemAction']["videoOffsetTimeMsec"])
def get_id(item):
a = list(item['replayChatItemAction']["actions"][0].values())[0].get('item')
if a:
return list(a.values())[0].get('id')
return None
def get_type(item):
a = list(item['replayChatItemAction']["actions"][0].values())[0].get('item')
if a:
return list(a.keys())[0]
return None

View File

@@ -1,55 +0,0 @@
from . import parser
from . block import Block
from typing import NamedTuple
class Patch(NamedTuple):
"""
Patch represents chunk of chat data
which is fetched by asyncdl.fetch_patch._fetch().
"""
chats: list = []
continuation: str = None
first: int = None
last: int = None
def fill(block: Block, patch: Patch):
block_end = block.end
if patch.last < block_end or block.is_last:
set_patch(block, patch)
return
for line in reversed(patch.chats):
line_offset = parser.get_offset(line)
if line_offset < block_end:
break
patch.chats.pop()
set_patch(block, patch._replace(
continuation=None,
last=line_offset
)
)
block.remaining = 0
block.done = True
def split(parent_block: Block, child_block: Block, patch: Patch):
parent_block.during_split = False
if patch.first <= parent_block.last:
''' When patch overlaps with parent_block,
discard this block. '''
child_block.continuation = None
''' Leave child_block.during_split == True
to exclude from during_split sequence. '''
return
child_block.during_split = False
child_block.first = patch.first
parent_block.end = patch.first
fill(child_block, patch)
def set_patch(block: Block, patch: Patch):
block.continuation = patch.continuation
block.chat_data.extend(patch.chats)
block.last = patch.last
block.remaining = block.end - block.last

View File

@@ -1,92 +0,0 @@
from . block import Block
from . patch import fill, split
from ... paramgen import arcparam
from typing import Tuple
class ExtractWorker:
"""
ExtractWorker associates a download session with a block.
When the worker finishes fetching, the block
being fetched is splitted and assigned the free worker.
Parameter
----------
fetch : func :
extract function of asyncdl
block : Block :
Block object that includes chat_data
blocks : list :
List of Block(s)
video_id : str :
parent_block : Block :
the block from which current block is splitted
"""
__slots__ = ['block', 'fetch', 'blocks', 'video_id', 'parent_block']
def __init__(self, fetch, block, blocks, video_id):
self.block = block
self.fetch = fetch
self.blocks = blocks
self.video_id = video_id
self.parent_block = None
async def run(self, session):
while self.block.continuation:
patch = await self.fetch(
self.block.continuation, session)
if patch.continuation is None:
"""TODO : make the worker assigned to the last block
to work more than twice as possible.
"""
break
if self.parent_block:
split(self.parent_block, self.block, patch)
self.parent_block = None
else:
fill(self.block, patch)
if self.block.continuation is None:
"""finished fetching this block """
self.block.done = True
self.block = _search_new_block(self)
def _search_new_block(worker) -> Block:
index, undone_block = _get_undone_block(worker.blocks)
if undone_block is None:
return Block(continuation=None)
mean = (undone_block.last + undone_block.end) / 2
continuation = arcparam.getparam(worker.video_id, seektime=mean / 1000)
worker.parent_block = undone_block
worker.parent_block.during_split = True
new_block = Block(
end=undone_block.end,
chat_data=[],
continuation=continuation,
during_split=True,
is_last=worker.parent_block.is_last)
'''swap last block'''
if worker.parent_block.is_last:
worker.parent_block.is_last = False
worker.blocks.insert(index + 1, new_block)
return new_block
def _get_undone_block(blocks) -> Tuple[int, Block]:
min_interval_ms = 120000
max_remaining = 0
undone_block = None
index_undone_block = 0
for index, block in enumerate(blocks):
if block.done or block.during_split:
continue
remaining = block.remaining
if remaining > max_remaining and remaining > min_interval_ms:
index_undone_block = index
undone_block = block
max_remaining = remaining
return index_undone_block, undone_block

View File

@@ -1,201 +0,0 @@
import httpx
import json
import re
import time
from .. import config
from ..exceptions import InvalidVideoIdException, PatternUnmatchError, UnknownConnectionError
from ..util.extract_video_id import extract_video_id
headers = config.headers
pattern = re.compile(r"['\"]PLAYER_CONFIG['\"]:\s*({.*})")
pattern2 = re.compile(r"yt\.setConfig\((\{[\s\S]*?\})\);")
item_channel_id = [
"videoDetails",
"embeddedPlayerOverlayVideoDetailsRenderer",
"channelThumbnailEndpoint",
"channelThumbnailEndpoint",
"urlEndpoint",
"urlEndpoint",
"url"
]
item_renderer = [
"embedPreview",
"thumbnailPreviewRenderer"
]
item_response = [
"args",
"embedded_player_response"
]
item_response2 = [
"PLAYER_VARS",
"embedded_player_response"
]
item_author_image = [
"videoDetails",
"embeddedPlayerOverlayVideoDetailsRenderer",
"channelThumbnail",
"thumbnails",
0,
"url"
]
item_thumbnail = [
"defaultThumbnail",
"thumbnails",
2,
"url"
]
item_channel_name = [
"videoDetails",
"embeddedPlayerOverlayVideoDetailsRenderer",
"expandedRenderer",
"embeddedPlayerOverlayVideoDetailsExpandedRenderer",
"title",
"runs",
0,
"text"
]
item_moving_thumbnail = [
"movingThumbnail",
"thumbnails",
0,
"url"
]
class VideoInfo:
'''
VideoInfo object retrieves YouTube video information.
Parameter
---------
video_id : str
Exception
---------
InvalidVideoIdException :
Occurs when video_id does not exist on YouTube.
'''
def __init__(self, video_id):
self.video_id = extract_video_id(video_id)
self.client = httpx.Client(http2=True)
self.new_pattern_text = False
err = None
for _ in range(3):
try:
text = self._get_page_text(self.video_id)
self._parse(text)
break
except (InvalidVideoIdException, UnknownConnectionError) as e:
raise e
except Exception as e:
err = e
time.sleep(2)
pass
else:
raise err
def _get_page_text(self, video_id):
url = f"https://www.youtube.com/embed/{video_id}"
err = None
for _ in range(3):
try:
resp = self.client.get(url, headers=headers)
resp.raise_for_status()
break
except httpx.HTTPError as e:
err = e
time.sleep(3)
else:
raise UnknownConnectionError(str(err))
return resp.text
def _parse(self, text):
result = re.search(pattern, text)
if result is None:
result = re.search(pattern2, text)
if result is None:
raise PatternUnmatchError(doc=text)
else:
self.new_pattern_text = True
decoder = json.JSONDecoder()
if self.new_pattern_text:
res = decoder.raw_decode(result.group(1))[0]
else:
res = decoder.raw_decode(result.group(1)[:-1])[0]
if self.new_pattern_text:
response = self._get_item(res, item_response2)
else:
response = self._get_item(res, item_response)
if response is None:
if self.new_pattern_text:
self._check_video_is_private(res.get("PLAYER_VARS"))
else:
self._check_video_is_private(res.get("args"))
self._renderer = self._get_item(json.loads(response), item_renderer)
if self._renderer is None:
raise InvalidVideoIdException(
f"No renderer found in video_id: [{self.video_id}].")
def _check_video_is_private(self, args):
if args and args.get("video_id"):
raise InvalidVideoIdException(
f"video_id [{self.video_id}] is private or deleted.")
raise InvalidVideoIdException(
f"video_id [{self.video_id}] is invalid.")
def _get_item(self, dict_body, items: list):
for item in items:
if dict_body is None:
break
if isinstance(dict_body, dict):
dict_body = dict_body.get(item)
continue
if isinstance(item, int) and \
isinstance(dict_body, list) and \
len(dict_body) > item:
dict_body = dict_body[item]
continue
return None
return dict_body
def get_duration(self):
duration_seconds = self._renderer.get("videoDurationSeconds")
if duration_seconds:
'''Fetched value is string, so cast to integer.'''
return int(duration_seconds)
'''When key is not found, explicitly returns None.'''
return None
def get_title(self):
if self._renderer.get("title"):
return [''.join(run["text"])
for run in self._renderer["title"]["runs"]][0]
return None
def get_channel_id(self):
channel_url = self._get_item(self._renderer, item_channel_id)
if channel_url:
return channel_url[9:]
return None
def get_author_image(self):
return self._get_item(self._renderer, item_author_image)
def get_thumbnail(self):
return self._get_item(self._renderer, item_thumbnail)
def get_channel_name(self):
return self._get_item(self._renderer, item_channel_name)
def get_moving_thumbnail(self):
return self._get_item(self._renderer, item_moving_thumbnail)

View File

@@ -3,10 +3,24 @@ import httpx
import json
import os
import re
from urllib.parse import quote
from .. import config
from .. exceptions import InvalidVideoIdException
PATTERN = re.compile(r"(.*)\(([0-9]+)\)$")
PATTERN_YTURL = re.compile(r"((?<=(v|V)/)|(?<=be/)|(?<=(\?|\&)v=)|(?<=embed/))([\w-]+)")
PATTERN_CHANNEL = re.compile(r"\\\"channelId\\\":\\\"(.{24})\\\"")
PATTERN_M_CHANNEL = re.compile(r"\"channelId\":\"(.{24})\"")
YT_VIDEO_ID_LENGTH = 11
CLIENT_VERSION = ''.join(("2.", (datetime.datetime.today() - datetime.timedelta(days=1)).strftime("%Y%m%d"), ".01.00"))
UA = config.headers["user-agent"]
def extract(url):
_session = httpx.Client(http2=True)
@@ -17,8 +31,9 @@ def extract(url):
def save(data, filename, extention) -> str:
save_filename = filename + "_" + (datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S')) + extention
with open(save_filename ,mode='w', encoding='utf-8') as f:
save_filename = filename + "_" + \
(datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S')) + extention
with open(save_filename, mode='w', encoding='utf-8') as f:
f.writelines(data)
return save_filename
@@ -39,3 +54,94 @@ def checkpath(filepath):
body = f'{body}({str(counter)})'
newpath = os.path.join(os.path.dirname(filepath), body + extention)
return newpath
def get_param(continuation, replay=False, offsetms: int = 0, dat=''):
if offsetms < 0:
offsetms = 0
ret = {
"context": {
"client": {
"visitorData": dat,
"userAgent": UA,
"clientName": "WEB",
"clientVersion": CLIENT_VERSION,
},
},
"continuation": continuation,
}
if replay:
ret.setdefault("currentPlayerState", {
"playerOffsetMs": str(int(offsetms))})
return ret
def extract_video_id(url_or_id: str) -> str:
ret = ''
if '[' in url_or_id:
url_or_id = url_or_id.replace('[', '').replace(']', '')
if type(url_or_id) != str:
raise TypeError(f"{url_or_id}: URL or VideoID must be str, but {type(url_or_id)} is passed.")
if len(url_or_id) == YT_VIDEO_ID_LENGTH:
return url_or_id
match = re.search(PATTERN_YTURL, url_or_id)
if match is None:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
try:
ret = match.group(4)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
if ret is None or len(ret) != YT_VIDEO_ID_LENGTH:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
return ret
def get_channelid(client, video_id):
resp = client.get("https://www.youtube.com/embed/{}".format(quote(video_id)), headers=config.headers)
match = re.search(PATTERN_CHANNEL, resp.text)
try:
if match is None:
raise IndexError
ret = match.group(1)
except IndexError:
ret = get_channelid_2nd(client, video_id)
return ret
def get_channelid_2nd(client, video_id):
resp = client.get("https://m.youtube.com/watch?v={}".format(quote(video_id)), headers=config.m_headers)
match = re.search(PATTERN_M_CHANNEL, resp.text)
if match is None:
raise InvalidVideoIdException(f"Cannot find channel id for video id:{video_id}. This video id seems to be invalid.")
try:
ret = match.group(1)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {video_id}")
return ret
async def get_channelid_async(client, video_id):
resp = await client.get("https://www.youtube.com/embed/{}".format(quote(video_id)), headers=config.headers)
match = re.search(PATTERN_CHANNEL, resp.text)
try:
if match is None:
raise IndexError
ret = match.group(1)
except IndexError:
ret = await get_channelid_async_2nd(client, video_id)
return ret
async def get_channelid_async_2nd(client, video_id):
resp = await client.get("https://m.youtube.com/watch?v={}".format(quote(video_id)), headers=config.m_headers)
match = re.search(PATTERN_M_CHANNEL, resp.text)
if match is None:
raise InvalidVideoIdException(f"Cannot find channel id for video id:{video_id}. This video id seems to be invalid.")
try:
ret = match.group(1)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {video_id}")
return ret

View File

@@ -1,28 +0,0 @@
import re
from .. exceptions import InvalidVideoIdException
PATTERN = re.compile(r"((?<=(v|V)/)|(?<=be/)|(?<=(\?|\&)v=)|(?<=embed/))([\w-]+)")
YT_VIDEO_ID_LENGTH = 11
def extract_video_id(url_or_id: str) -> str:
ret = ''
if '[' in url_or_id:
url_or_id = url_or_id.replace('[', '').replace(']', '')
if type(url_or_id) != str:
raise TypeError(f"{url_or_id}: URL or VideoID must be str, but {type(url_or_id)} is passed.")
if len(url_or_id) == YT_VIDEO_ID_LENGTH:
return url_or_id
match = re.search(PATTERN, url_or_id)
if match is None:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
try:
ret = match.group(4)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
if ret is None or len(ret) != YT_VIDEO_ID_LENGTH:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
return ret

View File

@@ -1,4 +1 @@
httpx[http2]==0.16.1
protobuf==3.14.0
pytz
urllib3
httpx[http2]

View File

@@ -1,2 +1,2 @@
pytest-mock==3.3.1
pytest-httpx==0.10.0
pytest-mock
pytest-httpx

View File

@@ -52,14 +52,10 @@ setup(
'Programming Language :: Python',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'License :: OSI Approved :: MIT License',
],
description="a python library for fetching youtube live chat.",
entry_points=
'''
[console_scripts]
pytchat=pytchat.cli:main
''',
install_requires=_requirements(),
keywords='youtube livechat asyncio',
license=license,

View File

@@ -1,32 +0,0 @@
import json
import httpx
import pytchat.config as config
from pytchat.paramgen import arcparam
from pytchat.parser.live import Parser
def test_arcparam_0(mocker):
param = arcparam.getparam("01234567890", -1)
assert param == "op2w0wQmGhxDZzhLRFFvTE1ERXlNelExTmpjNE9UQWdBUT09SARgAXICCAE%3D"
def test_arcparam_1(mocker):
param = arcparam.getparam("01234567890", seektime=100000)
assert param == "op2w0wQtGhxDZzhLRFFvTE1ERXlNelExTmpjNE9UQWdBUT09KIDQ28P0AkgDYAFyAggB"
def test_arcparam_2(mocker):
param = arcparam.getparam("SsjCnHOk-Sk", seektime=100)
url = f"https://www.youtube.com/live_chat_replay/get_live_chat_replay?continuation={param}&pbj=1"
resp = httpx.Client(http2=True).get(url, headers=config.headers)
jsn = json.loads(resp.text)
parser = Parser(is_replay=True)
contents = parser.get_contents(jsn)
_, chatdata = parser.parse(contents)
test_id = chatdata[0]["addChatItemAction"]["item"]["liveChatTextMessageRenderer"]["id"]
assert test_id == "CjoKGkNMYXBzZTdudHVVQ0Zjc0IxZ0FkTnFnQjVREhxDSnlBNHV2bnR1VUNGV0dnd2dvZDd3NE5aZy0w"
def test_arcparam_3(mocker):
param = arcparam.getparam("01234567890")
assert param == "op2w0wQmGhxDZzhLRFFvTE1ERXlNelExTmpjNE9UQWdBUT09SARgAXICCAE%3D"

View File

@@ -1,140 +0,0 @@
from pytchat.processors.superchat.calculator import SuperchatCalculator
get_item = SuperchatCalculator()._get_item
dict_test = {
'root':{
'node0' : 'value0',
'node1' : 'value1',
'node2' : {
'node2-0' : 'value2-0'
},
'node3' : [
{'node3-0' : 'value3-0'},
{'node3-1' :
{'node3-1-0' : 'value3-1-0'}
}
],
'node4' : [],
'node5' : [
[
{'node5-1-0' : 'value5-1-0'},
{'node5-1-1' : 'value5-1-1'},
],
{'node5-0' : 'value5-0'},
]
}
}
items_test0 = [
'root',
'node1'
]
items_test_not_found0 = [
'root',
'other_data'
]
items_test_nest = [
'root',
'node2',
'node2-0'
]
items_test_list0 = [
'root',
'node3',
1,
'node3-1'
]
items_test_list1 = [
'root',
'node3',
1,
'node3-1',
'node3-1-0'
]
items_test_list2 = [
'root',
'node4',
None
]
items_test_list3 = [
'root',
'node4'
]
items_test_list_nest = [
'root',
'node5',
0,
1,
'node5-1-1'
]
items_test_list_nest_not_found1 = [
'root',
'node5',
0,
1,
'node5-1-1',
'nodez'
]
items_test_not_found1 = [
'root',
'node3',
2,
'node3-1',
'node3-1-0'
]
items_test_not_found2 = [
'root',
'node3',
2,
'node3-1',
'node3-1-0',
'nodex'
]
def test_get_items_0():
assert get_item(dict_test, items_test0) == 'value1'
def test_get_items_1():
assert get_item(dict_test, items_test_not_found0) is None
def test_get_items_2():
assert get_item(dict_test, items_test_nest) == 'value2-0'
def test_get_items_3():
assert get_item(
dict_test, items_test_list0) == {'node3-1-0' : 'value3-1-0'}
def test_get_items_4():
assert get_item(dict_test, items_test_list1) == 'value3-1-0'
def test_get_items_5():
assert get_item(dict_test, items_test_not_found1) == None
def test_get_items_6():
assert get_item(dict_test, items_test_not_found2) == None
def test_get_items_7():
assert get_item(dict_test, items_test_list2) == None
def test_get_items_8():
assert get_item(dict_test, items_test_list_nest) == 'value5-1-1'
def test_get_items_9():
assert get_item(dict_test, items_test_list_nest_not_found1) == None
def test_get_items_10():
assert get_item(dict_test, items_test_list3) == []

View File

@@ -6,50 +6,55 @@ parse = SuperchatCalculator()._parse
def _open_file(path):
with open(path,mode ='r',encoding = 'utf-8') as f:
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def load_chatdata(filepath):
parser = Parser(is_replay=True)
#print(json.loads(_open_file(filepath)))
contents = parser.get_contents( json.loads(_open_file(filepath)))
# print(json.loads(_open_file(filepath)))
contents = parser.get_contents(json.loads(_open_file(filepath)))[0]
return parser.parse(contents)[1]
def test_parse_1():
renderer ={"purchaseAmountText":{"simpleText":"¥2,000"}}
symbol ,amount = parse(renderer)
renderer = {"purchaseAmountText": {"simpleText": "¥2,000"}}
symbol, amount = parse(renderer)
assert symbol == ''
assert amount == 2000.0
def test_parse_2():
renderer ={"purchaseAmountText":{"simpleText":"ABC\x0a200"}}
symbol ,amount = parse(renderer)
renderer = {"purchaseAmountText": {"simpleText": "ABC\x0a200"}}
symbol, amount = parse(renderer)
assert symbol == 'ABC\x0a'
assert amount == 200.0
def test_process_0():
"""
parse superchat data
"""
chat_component = {
'video_id':'',
'timeout':10,
'chatdata':load_chatdata(r"tests/testdata/calculator/superchat_0.json")
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/superchat_0.json")
}
assert SuperchatCalculator().process([chat_component])=={'': 6800.0, '': 2.0}
assert SuperchatCalculator().process([chat_component]) == {
'': 6800.0, '': 2.0}
def test_process_1():
"""
parse no superchat data
"""
chat_component = {
'video_id':'',
'timeout':10,
'chatdata':load_chatdata(r"tests/testdata/calculator/text_only.json")
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/text_only.json")
}
assert SuperchatCalculator().process([chat_component])=={}
assert SuperchatCalculator().process([chat_component]) == {}
def test_process_2():
"""
@@ -57,12 +62,10 @@ def test_process_2():
"""
try:
chat_component = {
'video_id':'',
'timeout':10,
'chatdata':load_chatdata(r"tests/testdata/calculator/replay_end.json")
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/replay_end.json")
}
assert False
SuperchatCalculator().process([chat_component])
except ChatParseException:
assert True

View File

@@ -11,7 +11,7 @@ def test_textmessage(mocker):
_json = _open_file("tests/testdata/compatible/textmessage.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -51,7 +51,7 @@ def test_newsponcer(mocker):
_json = _open_file("tests/testdata/compatible/newSponsor.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -88,7 +88,7 @@ def test_newsponcer_rev(mocker):
_json = _open_file("tests/testdata/compatible/newSponsor_rev.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -125,7 +125,7 @@ def test_superchat(mocker):
_json = _open_file("tests/testdata/compatible/superchat.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -164,7 +164,7 @@ def test_unregistered_currency(mocker):
_json = _open_file("tests/testdata/unregistered_currency.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",

View File

@@ -0,0 +1,24 @@
import pytchat
from pytchat.processors.compatible.processor import CompatibleProcessor
root_keys = ('kind', 'etag', 'nextPageToken', 'pollingIntervalMillis', 'pageInfo', 'items')
item_keys = ('kind', 'etag', 'id', 'snippet', 'authorDetails')
snippet_keys = ('type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage', 'textMessageDetails')
author_details_keys = ('channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor', 'isChatModerator')
def test_compatible_processor():
stream = pytchat.create("Hj-wnLIYKjw", seektime = 6000, processor=CompatibleProcessor())
while stream.is_alive():
chat = stream.get()
for key in chat.keys():
assert key in root_keys
for key in chat["items"][0].keys():
assert key in item_keys
for key in chat["items"][0]["snippet"].keys():
assert key in snippet_keys
for key in chat["items"][0]["authorDetails"].keys():
assert key in author_details_keys
break

View File

@@ -18,7 +18,7 @@ def test_textmessage(mocker):
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/textmessage.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -47,7 +47,7 @@ def test_textmessage_replay_member(mocker):
parser = Parser(is_replay=True)
_json = _open_file("tests/testdata/default/replay_member_text.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -79,7 +79,7 @@ def test_superchat(mocker):
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/superchat.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -121,7 +121,7 @@ def test_supersticker(mocker):
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/supersticker.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -162,7 +162,7 @@ def test_sponsor(mocker):
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/newSponsor_current.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
@@ -195,7 +195,7 @@ def test_sponsor_legacy(mocker):
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/newSponsor_lagacy.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,

View File

@@ -0,0 +1,94 @@
import asyncio
import pytchat
from concurrent.futures import CancelledError
from pytchat.core_multithread.livechat import LiveChat
from pytchat.core_async.livechat import LiveChatAsync
cases = [
{
"video_id":"1X7oL0hDnMg", "seektime":1620,
"result1":{'textMessage': 84},
"result2":{'': 83, 'MODERATOR': 1}
},
{
"video_id":"Hj-wnLIYKjw", "seektime":420,
"result1":{'superChat': 1, 'newSponsor': 6, 'textMessage': 63, 'donation': 1},
"result2":{'': 59, 'MEMBER': 12}
},
{
"video_id":"S8dmq5YIUoc", "seektime":3,
"result1":{'textMessage': 86},
"result2":{'': 62, 'MEMBER': 21, 'OWNER': 2, 'VERIFIED': 1}
},{
"video_id":"yLrstz80MKs", "seektime":30,
"result1":{'superSticker': 8, 'superChat': 2, 'textMessage': 67},
"result2":{'': 73, 'MEMBER': 4}
}
]
def test_archived_stream():
for case in cases:
stream = pytchat.create(video_id=case["video_id"], seektime=case["seektime"])
while stream.is_alive():
chat = stream.get()
agg1 = {}
agg2 = {}
for c in chat.items:
if c.type in agg1:
agg1[c.type] += 1
else:
agg1[c.type] = 1
if c.author.type in agg2:
agg2[c.author.type] += 1
else:
agg2[c.author.type] = 1
break
assert agg1 == case["result1"]
assert agg2 == case["result2"]
def test_archived_stream_multithread():
for case in cases:
stream = LiveChat(video_id=case["video_id"], seektime=case["seektime"])
while stream.is_alive():
chat = stream.get()
agg1 = {}
agg2 = {}
for c in chat.items:
if c.type in agg1:
agg1[c.type] += 1
else:
agg1[c.type] = 1
if c.author.type in agg2:
agg2[c.author.type] += 1
else:
agg2[c.author.type] = 1
break
assert agg1 == case["result1"]
assert agg2 == case["result2"]
def test_async_live_stream():
async def test_loop():
for case in cases:
stream = LiveChatAsync(video_id=case["video_id"], seektime=case["seektime"])
while stream.is_alive():
chat = await stream.get()
agg1 = {}
agg2 = {}
for c in chat.items:
if c.type in agg1:
agg1[c.type] += 1
else:
agg1[c.type] = 1
if c.author.type in agg2:
agg2[c.author.type] += 1
else:
agg2[c.author.type] = 1
break
assert agg1 == case["result1"]
assert agg2 == case["result2"]
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test_loop())
except CancelledError:
assert True

View File

@@ -1,134 +0,0 @@
import json
from pytchat.tool.extract import duplcheck
from pytchat.tool.extract import parser
from pytchat.tool.extract.block import Block
from pytchat.tool.extract.duplcheck import _dump
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def test_overlap():
"""
test overlap data
operation : [0] [2] [3] [4] -> last :align to end
[1] , [5] -> no change
"""
def load_chatdata(filename):
return parser.parse(
json.loads(_open_file(
"tests/testdata/extract_duplcheck/overlap/" + filename))
)[1]
blocks = (
Block(first=0, last=12771, end=9890,
chat_data=load_chatdata("dp0-0.json")),
Block(first=9890, last=15800, end=20244,
chat_data=load_chatdata("dp0-1.json")),
Block(first=20244, last=45146, end=32476,
chat_data=load_chatdata("dp0-2.json")),
Block(first=32476, last=50520, end=41380,
chat_data=load_chatdata("dp0-3.json")),
Block(first=41380, last=62875, end=52568,
chat_data=load_chatdata("dp0-4.json")),
Block(first=52568, last=62875, end=54000,
chat_data=load_chatdata("dp0-5.json"), is_last=True)
)
result = duplcheck.remove_overlap(blocks)
# dp0-0.json has item offset time is 9890 (equals block[0].end = block[1].first),
# but must be aligne to the most close and smaller value:9779.
assert result[0].last == 9779
assert result[1].last == 15800
assert result[2].last == 32196
assert result[3].last == 41116
assert result[4].last == 52384
# the last block must be always added to result.
assert result[5].last == 62875
def test_duplicate_head():
def load_chatdata(filename):
return parser.parse(
json.loads(_open_file(
"tests/testdata/extract_duplcheck/head/" + filename))
)[1]
"""
test duplicate head data
operation : [0] , [1] -> discard [0]
[1] , [2] -> discard [1]
[2] , [3] -> append [2]
[3] , [4] -> discard [3]
[4] , [5] -> append [4]
append [5]
result : [2] , [4] , [5]
"""
# chat data offsets are ignored.
blocks = (
Block(first=0, last=2500, chat_data=load_chatdata("dp0-0.json")),
Block(first=0, last=38771, chat_data=load_chatdata("dp0-1.json")),
Block(first=0, last=45146, chat_data=load_chatdata("dp0-2.json")),
Block(first=20244, last=60520, chat_data=load_chatdata("dp0-3.json")),
Block(first=20244, last=62875, chat_data=load_chatdata("dp0-4.json")),
Block(first=52568, last=62875, chat_data=load_chatdata("dp0-5.json"))
)
_dump(blocks)
result = duplcheck.remove_duplicate_head(blocks)
assert len(result) == 3
assert result[0].first == blocks[2].first
assert result[0].last == blocks[2].last
assert result[1].first == blocks[4].first
assert result[1].last == blocks[4].last
assert result[2].first == blocks[5].first
assert result[2].last == blocks[5].last
def test_duplicate_tail():
"""
test duplicate tail data
operation : append [0]
[0] , [1] -> discard [1]
[1] , [2] -> append [2]
[2] , [3] -> discard [3]
[3] , [4] -> append [4]
[4] , [5] -> discard [5]
result : [0] , [2] , [4]
"""
def load_chatdata(filename):
return parser.parse(
json.loads(_open_file(
"tests/testdata/extract_duplcheck/head/" + filename))
)[1]
# chat data offsets are ignored.
blocks = (
Block(first=0, last=2500, chat_data=load_chatdata("dp0-0.json")),
Block(first=1500, last=2500, chat_data=load_chatdata("dp0-1.json")),
Block(first=10000, last=45146, chat_data=load_chatdata("dp0-2.json")),
Block(first=20244, last=45146, chat_data=load_chatdata("dp0-3.json")),
Block(first=20244, last=62875, chat_data=load_chatdata("dp0-4.json")),
Block(first=52568, last=62875, chat_data=load_chatdata("dp0-5.json"))
)
result = duplcheck.remove_duplicate_tail(blocks)
_dump(result)
assert len(result) == 3
assert result[0].first == blocks[0].first
assert result[0].last == blocks[0].last
assert result[1].first == blocks[2].first
assert result[1].last == blocks[2].last
assert result[2].first == blocks[4].first
assert result[2].last == blocks[4].last

View File

@@ -1,239 +0,0 @@
import json
from pytchat.tool.extract import parser
from pytchat.tool.extract.block import Block
from pytchat.tool.extract.patch import Patch, split
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def load_chatdata(filename):
return parser.parse(
json.loads(_open_file("tests/testdata/fetch_patch/" + filename))
)[1]
def test_split_0():
"""
Normal case
~~~~~~ before ~~~~~~
@parent_block (# = already fetched)
first last end
|########----------------------------------------|
@child_block
first = last = 0 end (=parent_end)
| |
@fetched patch
|-- patch --|
|
|
V
~~~~~~ after ~~~~~~
@parent_block
first last end (after split)
|########------------|
@child_block
first last end
|###########---------------|
@fetched patch
|-- patch --|
"""
parent = Block(first=0, last=4000, end=60000,
continuation='parent', during_split=True)
child = Block(first=0, last=0, end=60000,
continuation='mean', during_split=True)
patch = Patch(chats=load_chatdata('pt0-5.json'),
first=32500, last=34000, continuation='patch')
split(parent, child, patch)
assert child.continuation == 'patch'
assert parent.last < child.first
assert parent.end == child.first
assert child.first < child.last
assert child.last < child.end
assert parent.during_split is False
assert child.during_split is False
def test_split_1():
"""patch.first <= parent_block.last
While awaiting at run()->asyncdl._fetch()
fetching parent_block proceeds,
and parent.block.last exceeds patch.first.
In this case, fetched patch is all discarded,
and worker searches other processing block again.
~~~~~~ before ~~~~~~
patch.first
first | last end
|####################|#####|---------------------|
^
@child_block
first = last = 0 end (=parent_end)
| |
@fetched patch
|-- patch --|
|
|
V
~~~~~~ after ~~~~~~
@parent_block
first last end
|###########################|--------------------|
@child_block
.............. ->  discard all data
"""
parent = Block(first=0, last=33000, end=60000, continuation='parent', during_split=True)
child = Block(first=0, last=0, end=60000, continuation='mean', during_split=True)
patch = Patch(chats=load_chatdata('pt0-5.json'),
first=32500, last=34000, continuation='patch')
split(parent, child, patch)
assert parent.last == 33000 # no change
assert parent.end == 60000 # no change
assert child.continuation is None
assert parent.during_split is False
assert child.during_split is True # exclude during_split sequence
def test_split_2():
"""child_block.end < patch.last:
Case the last offset of patch exceeds child_block.end.
In this case, remove overlapped data of patch.
~~~~~~ before ~~~~~~
@parent_block (# = already fetched)
first last end (before split)
|########------------------------------|
@child_block
first = last = 0 end (=parent_end)
| |
continuation:succeed from patch
@fetched patch
|-------- patch --------|
|
|
V
~~~~~~ after ~~~~~~
@parent_block
first last end (after split)
|########------------|
@child_block old patch.end
first last=end |
|#################|...... cut extra data.
^
continuation : None (extract complete)
@fetched patch
|-------- patch --------|
"""
parent = Block(first=0, last=4000, end=33500, continuation='parent', during_split=True)
child = Block(first=0, last=0, end=33500, continuation='mean', during_split=True)
patch = Patch(chats=load_chatdata('pt0-5.json'),
first=32500, last=34000, continuation='patch')
split(parent, child, patch)
assert child.continuation is None
assert parent.last < child.first
assert parent.end == child.first
assert child.first < child.last
assert child.last < child.end
assert child.continuation is None
assert parent.during_split is False
assert child.during_split is False
def test_split_none():
"""patch.last <= parent_block.last
While awaiting at run()->asyncdl._fetch()
fetching parent_block proceeds,
and parent.block.last exceeds patch.first.
In this case, fetched patch is all discarded,
and worker searches other processing block again.
~~~~~~ before ~~~~~~
patch.first
first | last end
|####################|###################|-------|
^
@child_block
first = last = 0 end (=parent_end)
| |
@fetched patch
|-- patch --|
patch.last < parent_block.last .
|
|
V
~~~~~~ after ~~~~~~
@parent_block
first last end (before split)
|########################################|-------|
@child_block
............ -> discard all data.
"""
parent = Block(first=0, last=40000, end=60000, continuation='parent', during_split=True)
child = Block(first=0, last=0, end=60000, continuation='mean', during_split=True)
patch = Patch(chats=load_chatdata('pt0-5.json'),
first=32500, last=34000, continuation='patch')
split(parent, child, patch)
assert parent.last == 40000 # no change
assert parent.end == 60000 # no change
assert child.continuation is None
assert parent.during_split is False
assert child.during_split is True # exclude during_split sequence

View File

@@ -1,4 +1,4 @@
from pytchat.util.extract_video_id import extract_video_id
from pytchat.util import extract_video_id
from pytchat.exceptions import InvalidVideoIdException
VALID_TEST_PATTERNS = (

View File

@@ -1,48 +0,0 @@
import asyncio
import json
from pytest_httpx import HTTPXMock
from concurrent.futures import CancelledError
from pytchat.core_multithread.livechat import LiveChat
from pytchat.core_async.livechat import LiveChatAsync
from pytchat.exceptions import ResponseContextError
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def add_response_file(httpx_mock: HTTPXMock, jsonfile_path: str):
testdata = json.loads(_open_file(jsonfile_path))
httpx_mock.add_response(json=testdata)
def test_async(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/paramgen_firstread.json')
async def test_loop():
try:
chat = LiveChatAsync(video_id='__test_id__')
_ = await chat.get()
assert chat.is_alive()
chat.terminate()
assert not chat.is_alive()
except ResponseContextError:
assert False
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test_loop())
except CancelledError:
assert True
def test_multithread(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/paramgen_firstread.json')
try:
chat = LiveChat(video_id='__test_id__')
_ = chat.get()
assert chat.is_alive()
chat.terminate()
assert not chat.is_alive()
except ResponseContextError:
assert False

View File

@@ -1,113 +0,0 @@
import asyncio
import json
from pytest_httpx import HTTPXMock
from concurrent.futures import CancelledError
from pytchat.core_multithread.livechat import LiveChat
from pytchat.core_async.livechat import LiveChatAsync
from pytchat.processors.dummy_processor import DummyProcessor
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def add_response_file(httpx_mock: HTTPXMock, jsonfile_path: str):
testdata = json.loads(_open_file(jsonfile_path))
httpx_mock.add_response(json=testdata)
def test_async_live_stream(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/test_stream.json')
async def test_loop():
chat = LiveChatAsync(video_id='__test_id__', processor=DummyProcessor())
chats = await chat.get()
rawdata = chats[0]["chatdata"]
assert list(rawdata[0]["addChatItemAction"]["item"].keys())[
0] == "liveChatTextMessageRenderer"
assert list(rawdata[1]["addChatItemAction"]["item"].keys())[
0] == "liveChatTextMessageRenderer"
assert list(rawdata[2]["addChatItemAction"]["item"].keys())[
0] == "liveChatPlaceholderItemRenderer"
assert list(rawdata[3]["addLiveChatTickerItemAction"]["item"].keys())[
0] == "liveChatTickerPaidMessageItemRenderer"
assert list(rawdata[4]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidMessageRenderer"
assert list(rawdata[5]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidStickerRenderer"
assert list(rawdata[6]["addLiveChatTickerItemAction"]["item"].keys())[
0] == "liveChatTickerSponsorItemRenderer"
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test_loop())
except CancelledError:
assert True
def test_async_replay_stream(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/finished_live.json')
add_response_file(httpx_mock, 'tests/testdata/chatreplay.json')
async def test_loop():
chat = LiveChatAsync(video_id='__test_id__', processor=DummyProcessor())
chats = await chat.get()
rawdata = chats[0]["chatdata"]
# assert fetching replaychat data
assert list(rawdata[0]["addChatItemAction"]["item"].keys())[
0] == "liveChatTextMessageRenderer"
assert list(rawdata[14]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidMessageRenderer"
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test_loop())
except CancelledError:
assert True
def test_async_force_replay(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/test_stream.json')
add_response_file(httpx_mock, 'tests/testdata/chatreplay.json')
async def test_loop():
chat = LiveChatAsync(
video_id='__test_id__', processor=DummyProcessor(), force_replay=True)
chats = await chat.get()
rawdata = chats[0]["chatdata"]
# assert fetching replaychat data
assert list(rawdata[14]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidMessageRenderer"
# assert not mix livechat data
assert list(rawdata[2]["addChatItemAction"]["item"].keys())[
0] != "liveChatPlaceholderItemRenderer"
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test_loop())
except CancelledError:
assert True
def test_multithread_live_stream(httpx_mock: HTTPXMock):
add_response_file(httpx_mock, 'tests/testdata/test_stream.json')
chat = LiveChat(video_id='__test_id__', processor=DummyProcessor())
chats = chat.get()
rawdata = chats[0]["chatdata"]
# assert fetching livachat data
assert list(rawdata[0]["addChatItemAction"]["item"].keys())[
0] == "liveChatTextMessageRenderer"
assert list(rawdata[1]["addChatItemAction"]["item"].keys())[
0] == "liveChatTextMessageRenderer"
assert list(rawdata[2]["addChatItemAction"]["item"].keys())[
0] == "liveChatPlaceholderItemRenderer"
assert list(rawdata[3]["addLiveChatTickerItemAction"]["item"].keys())[
0] == "liveChatTickerPaidMessageItemRenderer"
assert list(rawdata[4]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidMessageRenderer"
assert list(rawdata[5]["addChatItemAction"]["item"].keys())[
0] == "liveChatPaidStickerRenderer"
assert list(rawdata[6]["addLiveChatTickerItemAction"]["item"].keys())[
0] == "liveChatTickerSponsorItemRenderer"
chat.terminate()

View File

@@ -1,9 +0,0 @@
import pytest
from pytchat.paramgen import liveparam
def test_liveparam_0(mocker):
_ts1= 1546268400
param = liveparam._build("01234567890",
*([_ts1*1000000 for i in range(5)]), topchat_only=False)
test_param="0ofMyANcGhxDZzhLRFFvTE1ERXlNelExTmpjNE9UQWdBUT09KIC41tWqyt8CQAFKC1CAuNbVqsrfAlgDUIC41tWqyt8CWIC41tWqyt8CaAGCAQIIAZoBAKABgLjW1arK3wI%3D"
assert test_param == param

View File

@@ -12,29 +12,23 @@ def _open_file(path):
def test_finishedlive(*mock):
'''配信が終了した動画を正しく処理できるか'''
_text = _open_file('tests/testdata/finished_live.json')
_text = json.loads(_text)
try:
parser.parse(parser.get_contents(_text))
parser.parse(parser.get_contents(_text)[0])
assert False
except NoContents:
assert True
def test_parsejson(*mock):
'''jsonを正常にパースできるか'''
_text = _open_file('tests/testdata/paramgen_firstread.json')
_text = json.loads(_text)
try:
parser.parse(parser.get_contents(_text))
jsn = _text
timeout = jsn["response"]["continuationContents"]["liveChatContinuation"]["continuations"][0]["timedContinuationData"]["timeoutMs"]
continuation = jsn["response"]["continuationContents"]["liveChatContinuation"][
"continuations"][0]["timedContinuationData"]["continuation"]
assert timeout == 5035
assert continuation == "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%3D"
s, _ = parser.parse(parser.get_contents(_text)[0])
assert s['timeoutMs'] == 5035
assert s['continuation'] == "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%3D"
except Exception:
assert False

View File

@@ -15,7 +15,7 @@ def test_speed_1(mocker):
_json = _open_file("tests/testdata/speed/speedtest_normal.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,
@@ -32,7 +32,7 @@ def test_speed_2(mocker):
_json = _open_file("tests/testdata/speed/speedtest_undefined.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,
@@ -49,7 +49,7 @@ def test_speed_3(mocker):
_json = _open_file("tests/testdata/speed/speedtest_empty.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json)))
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,

View File

@@ -0,0 +1,16 @@
import json
import pytchat
from pytchat.parser.live import Parser
from pytchat.processors.speed.calculator import SpeedCalculator
parser = Parser(is_replay=False)
def test_speed_1():
stream = pytchat.create("Hj-wnLIYKjw", seektime = 6000,processor=SpeedCalculator())
while stream.is_alive():
speed = stream.get()
assert speed > 100
break
test_speed_1()

View File

@@ -1,101 +0,0 @@
from json.decoder import JSONDecodeError
from pytchat.tool.videoinfo import VideoInfo
from pytchat.exceptions import InvalidVideoIdException
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def _set_test_data(filepath, mocker):
_text = _open_file(filepath)
response_mock = mocker.Mock()
response_mock.status_code = 200
response_mock.text = _text
mocker.patch('httpx.Client.get').return_value = response_mock
def test_archived_page(mocker):
_set_test_data('tests/testdata/videoinfo/archived_page.txt', mocker)
info = VideoInfo('__test_id__')
actual_thumbnail_url = 'https://i.ytimg.com/vi/fzI9FNjXQ0o/hqdefault.jpg'
assert info.video_id == '__test_id__'
assert info.get_channel_name() == 'GitHub'
assert info.get_thumbnail() == actual_thumbnail_url
assert info.get_title() == 'GitHub Arctic Code Vault'
assert info.get_channel_id() == 'UC7c3Kb6jYCRj4JOHHZTxKsQ'
assert info.get_duration() == 148
def test_live_page(mocker):
_set_test_data('tests/testdata/videoinfo/live_page.txt', mocker)
info = VideoInfo('__test_id__')
'''live page: duration==0'''
assert info.get_duration() == 0
assert info.video_id == '__test_id__'
assert info.get_channel_name() == 'BGM channel'
assert info.get_thumbnail() == \
'https://i.ytimg.com/vi/fEvM-OUbaKs/hqdefault_live.jpg'
assert info.get_title() == (
'Coffee Jazz Music - Chill Out Lounge Jazz Music Radio'
' - 24/7 Live Stream - Slow Jazz')
assert info.get_channel_id() == 'UCQINXHZqCU5i06HzxRkujfg'
def test_invalid_video_id(mocker):
'''Test case invalid video_id is specified.'''
_set_test_data(
'tests/testdata/videoinfo/invalid_video_id_page.txt', mocker)
try:
_ = VideoInfo('__test_id__')
assert False
except InvalidVideoIdException:
assert True
def test_no_info(mocker):
'''Test case the video page has renderer, but no info.'''
_set_test_data(
'tests/testdata/videoinfo/no_info_page.txt', mocker)
info = VideoInfo('__test_id__')
assert info.video_id == '__test_id__'
assert info.get_channel_name() is None
assert info.get_thumbnail() is None
assert info.get_title() is None
assert info.get_channel_id() is None
assert info.get_duration() is None
def test_collapsed_data(mocker):
'''Test case the video page's info is collapsed.'''
_set_test_data(
'tests/testdata/videoinfo/collapsed_page.txt', mocker)
try:
_ = VideoInfo('__test_id__')
assert False
except JSONDecodeError:
assert True
def test_pattern_unmatch(mocker):
'''Test case the pattern for extraction is unmatched.'''
_set_test_data(
'tests/testdata/videoinfo/pattern_unmatch.txt', mocker)
try:
_ = VideoInfo('__test_id__')
assert False
except JSONDecodeError:
assert True
def test_extradata_handling(mocker):
'''Test case the extracted data are JSON lines.'''
_set_test_data(
'tests/testdata/videoinfo/extradata_page.txt', mocker)
try:
_ = VideoInfo('__test_id__')
assert True
except JSONDecodeError as e:
print(e.doc)
assert False

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3320,5 +3319,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -85,5 +84,4 @@
]
}
}
}
}

View File

@@ -15,7 +15,6 @@
"url": "/live_chat_replay/get_live_chat_replay?continuation=op2w0wRyGjxDZzhhRFFvTFUzTnFRMjVJVDJzdFUyc2FFLXFvM2JrQkRRb0xVM05xUTI1SVQyc3RVMnNnQVElM0QlM0QoATAAOABAAEgEUhwIABAAGAAgACoOc3RhdGljY2hlY2tzdW1AAFgDYAFoAHIECAEQAHgA"
}
},
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -3096,9 +3095,8 @@
}
]
}
}
},
"csn": "CAXIXa-uHZnC4wL5nZ7oBQ",
"url": "\/live_chat_replay\/get_live_chat_replay?continuation=op2w0wRyGjxDZzhhRFFvTFUzTnFRMjVJVDJzdFUyc2FFLXFvM2JrQkRRb0xVM05xUTI1SVQyc3RVMnNnQVElM0QlM0QoATAAOABAAEgEUhwIABAAGAAgACoOc3RhdGljY2hlY2tzdW1AAFgDYAFoAHIECAEQAHgA",
"url": "/live_chat_replay/get_live_chat_replay?continuation=op2w0wRyGjxDZzhhRFFvTFUzTnFRMjVJVDJzdFUyc2FFLXFvM2JrQkRRb0xVM05xUTI1SVQyc3RVMnNnQVElM0QlM0QoATAAOABAAEgEUhwIABAAGAAgACoOc3RhdGljY2hlY2tzdW1AAFgDYAFoAHIECAEQAHgA",
"xsrf_token": "QUFFLUhqbTZWWEFiT3ZxaDAtY09pRzZXSUotZC1uclFMQXxBQ3Jtc0tsOFZYN09CWFlBd2NKSFB4R3hmN3dUY2xXaW9tbzdFZlZBTllDcnBhMG9WUXVkZGZ5RGRIYkxSajBiNVpsNU5PV3hNYkhUZGJybTVEYWM2MWREbTRUYnc3XzRpeUJVbFpNR0dod1RPbGtVLWJhdkhtUVpVN0JKVGNSQVRSY0ZsODhodEwxaWdjN0pHZThlbEJVXzJMc2VXZGtQOXc="
}

View File

@@ -4,7 +4,7 @@
"st": 100
}
},
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -1838,9 +1838,8 @@
}
}
},
"trackingParams": "CAAQ0b4BIhMI1LbVw_aa5QIV2cxMAh2AtAj8"
},
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAOBAhrKAUNrd1NJUW9ZVlVOdVVsRlpTRlJ1VWt4VFJqQmpURXAzVFc1bFpFTm5FZ1V2YkdsMlpTb25DaGhWUTI1U1VWbElWRzVTVEZOR01HTk1TbmROYm1Wa1EyY1NDMDFETTNkVlNpMUNXRTVGR2tPcXVjRzlBVDBLTzJoMGRIQnpPaTh2ZDNkM0xubHZkWFIxWW1VdVkyOXRMMnhwZG1WZlkyaGhkRDkyUFUxRE0zZFZTaTFDV0U1RkptbHpYM0J2Y0c5MWREMHhJQUklM0QwAUopCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoAUPOaw8P2muUCWANoAYIBAggB",
"trackingParams": "CAAQ0b4BIhMI1LbVw_aa5QIV2cxMAh2AtAj8",
"url": "/live_chat/get_live_chat?continuation=0ofMyAOBAhrKAUNrd1NJUW9ZVlVOdVVsRlpTRlJ1VWt4VFJqQmpURXAzVFc1bFpFTm5FZ1V2YkdsMlpTb25DaGhWUTI1U1VWbElWRzVTVEZOR01HTk1TbmROYm1Wa1EyY1NDMDFETTNkVlNpMUNXRTVGR2tPcXVjRzlBVDBLTzJoMGRIQnpPaTh2ZDNkM0xubHZkWFIxWW1VdVkyOXRMMnhwZG1WZlkyaGhkRDkyUFUxRE0zZFZTaTFDV0U1RkptbHpYM0J2Y0c5MWREMHhJQUklM0QwAUopCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoAUPOaw8P2muUCWANoAYIBAggB",
"csn": "PvujXbH0OIazqQHXgJ64DQ",
"endpoint": {
"commandMetadata": {

View File

@@ -4,7 +4,7 @@
"st": 100
}
},
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -1805,7 +1805,7 @@
}
},
"trackingParams": "CAAQ0b4BIhMI1LbVw_aa5QIV2cxMAh2AtAj8"
},
,
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAOBAhrKAUNrd1NJUW9ZVlVOdVVsRlpTRlJ1VWt4VFJqQmpURXAzVFc1bFpFTm5FZ1V2YkdsMlpTb25DaGhWUTI1U1VWbElWRzVTVEZOR01HTk1TbmROYm1Wa1EyY1NDMDFETTNkVlNpMUNXRTVGR2tPcXVjRzlBVDBLTzJoMGRIQnpPaTh2ZDNkM0xubHZkWFIxWW1VdVkyOXRMMnhwZG1WZlkyaGhkRDkyUFUxRE0zZFZTaTFDV0U1RkptbHpYM0J2Y0c5MWREMHhJQUklM0QwAUopCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoAUPOaw8P2muUCWANoAYIBAggB",
"csn": "PvujXbH0OIazqQHXgJ64DQ",
"endpoint": {

View File

@@ -15,9 +15,9 @@
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -278,5 +278,4 @@
]
}
}
}
}

View File

@@ -15,9 +15,8 @@
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -193,5 +192,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"serviceTrackingParams": [
{
@@ -154,7 +153,6 @@
}
]
}
}
},
"endpoint": {
"commandMetadata": {
@@ -168,7 +166,7 @@
},
"csn": "n96GXabRGouFlQTigY2YDg",
"xsrf_token": "QUFFLUhqbHNNWTF3NFJqc2h3cGE1NE9FWGdaWk5mRlVhUXxBQ3Jtc0tuTWhZNFcyTW1iZnA3ZnFTYUFudVFEUVE0cnFEOVBGcEU1MEh0Zlh4bll1amVmRl9OMkxZV3pKV1ZSbExBeDctTl95NGtBVnJZdlNxeS1KdWVNempEN2N6MHhaU1laV3hnVkZPeHp1OHVDTGVFSGUyOGduT0szbDV5N05LYUZTdzdoTDRwV1VJWndaVjdQVGRjNWVpR0YwUXgtZXc=",
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D",
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D",
"timing": {
"info": {
"st": 81

View File

@@ -1 +0,0 @@
{"author": {"badgeUrl": "", "type": "", "isVerified": false, "isChatOwner": false, "isChatSponsor": false, "isChatModerator": false, "channelId": "author_channel_id", "channelUrl": "http://www.youtube.com/channel/author_channel_id", "name": "author_name", "imageUrl": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"}, "type": "superChat", "id": "dummy_id", "timestamp": 1570678496000, "elapsedTime": "", "datetime": "2019-10-10 12:34:56", "message": "dummy_message", "messageEx": ["dummy_message"], "amountValue": 800.0, "amountString": "¥800", "currency": "JPY", "bgColor": 4280150454, "colors": {"headerBackgroundColor": 4278239141, "headerTextColor": 4278190080, "bodyBackgroundColor": 4280150454, "bodyTextColor": 4278190080, "timestampColor": 2147483648, "authorNameTextColor": 2315255808}}

View File

@@ -1 +0,0 @@
[{"author": {"badgeUrl": "", "type": "", "isVerified": false, "isChatOwner": false, "isChatSponsor": false, "isChatModerator": false, "channelId": "author_channel_id", "channelUrl": "http://www.youtube.com/channel/author_channel_id", "name": "author_name", "imageUrl": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"}, "type": "superChat", "id": "dummy_id", "timestamp": 1570678496000, "elapsedTime": "", "datetime": "2019-10-10 12:34:56", "message": "dummy_message", "messageEx": ["dummy_message"], "amountValue": 800.0, "amountString": "¥800", "currency": "JPY", "bgColor": 4280150454, "colors": {"headerBackgroundColor": 4278239141, "headerTextColor": 4278190080, "bodyBackgroundColor": 4280150454, "bodyTextColor": 4278190080, "timestampColor": 2147483648, "authorNameTextColor": 2315255808}}]

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -96,5 +95,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -78,5 +77,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": "data"
},
@@ -108,5 +107,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -180,5 +179,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -95,5 +94,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -75,5 +74,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -6124,5 +6123,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -2525,5 +2524,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -1427,5 +1426,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -6124,5 +6123,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -2525,5 +2524,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -1427,5 +1426,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

View File

@@ -1,5 +1,4 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
@@ -3074,5 +3073,4 @@
]
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More