Compare commits

...

550 Commits

Author SHA1 Message Date
taizan-hokouto
0a9837adca Merge branch 'hotfix/fix_param' 2021-02-11 02:02:14 +09:00
taizan-hokouto
f4bf30c0e9 Increment version 2021-02-11 01:57:45 +09:00
taizan-hokouto
acfba74821 Delete tests 2021-02-11 01:56:43 +09:00
taizan-hokouto
f46845c777 Fix parameters for live 2021-02-11 01:56:32 +09:00
taizan-hokuto
af4c2fe4b9 Merge pull request #36 from taizan-hokuto/dependabot/pip/bleach-3.3.0
Bump bleach from 3.2.1 to 3.3.0
2021-02-04 00:46:52 +09:00
dependabot[bot]
b7c656536d Bump bleach from 3.2.1 to 3.3.0
Bumps [bleach](https://github.com/mozilla/bleach) from 3.2.1 to 3.3.0.
- [Release notes](https://github.com/mozilla/bleach/releases)
- [Changelog](https://github.com/mozilla/bleach/blob/master/CHANGES)
- [Commits](https://github.com/mozilla/bleach/compare/v3.2.1...v3.3.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-02 23:12:20 +00:00
taizan-hokouto
b3ebe3879d Merge branch 'release/v0.5.2' 2021-01-17 22:41:20 +09:00
taizan-hokouto
da79895e55 Increment version 2021-01-17 22:40:15 +09:00
taizan-hokouto
aaa7421fdf Add replay_continuation parameter 2021-01-17 22:38:04 +09:00
taizan-hokuto
b9f213f047 Merge pull request #32 from miyuk/develop-add-replay-continuation
Add replay_continuation parameter
2021-01-15 00:11:04 +09:00
miyuk
fee070b299 Add replay_continuation parameter 2021-01-14 20:06:32 +09:00
taizan-hokouto
275e28b635 Merge tag 'v0.5.1' into develop
v0.5.1
2021-01-09 22:14:56 +09:00
taizan-hokouto
808e599be6 Merge branch 'release/v0.5.1' 2021-01-09 22:14:55 +09:00
taizan-hokouto
5cb6f7f123 Increment version 2021-01-09 22:14:30 +09:00
taizan-hokouto
a2f1c658f0 Merge branch 'master' into develop 2021-01-09 22:13:15 +09:00
taizan-hokouto
05de644d77 Merge branch 'hotfix/fix' 2021-01-09 22:13:15 +09:00
taizan-hokouto
b908855566 Delete unnecessary line 2021-01-09 22:12:33 +09:00
taizan-hokouto
8d93bfcb95 Merge branch 'master' into develop 2021-01-09 22:10:28 +09:00
taizan-hokouto
bf68859f38 Merge branch 'hotfix/fix' 2021-01-09 22:10:28 +09:00
taizan-hokouto
78fbe97b66 Fix process of fetching archived chat 2021-01-09 22:09:31 +09:00
taizan-hokouto
166a256c1c Merge tag 'v0.5.0' into develop
v0.5.0
2020-12-13 22:29:25 +09:00
taizan-hokouto
b7f2967a4f Merge branch 'release/v0.5.0' 2020-12-13 22:29:25 +09:00
taizan-hokouto
0a8ff3abdc Increment version 2020-12-13 22:28:39 +09:00
taizan-hokouto
9b38a5428d Add python version 2020-12-13 22:08:10 +09:00
taizan-hokuto
9311bf1993 Merge pull request #26 from zecktos/patch-1
Fix for python3.9
2020-12-13 22:04:18 +09:00
zecktos
ee839da7c9 Fix for python3.9
'encoding' is deprecated and removed in Python 3.9 
could fix this https://github.com/taizan-hokuto/pytchat/issues/24
2020-12-13 13:39:58 +01:00
taizan-hokouto
2ae77b3850 Merge branch 'hotfix/readme' 2020-12-13 14:22:05 +09:00
taizan-hokouto
afd7cea635 Merge branch 'master' into develop 2020-12-13 14:22:05 +09:00
taizan-hokouto
9018ff9ee4 Update README 2020-12-13 14:21:42 +09:00
taizan-hokouto
b676912d64 Merge branch 'master' into develop 2020-12-05 21:32:13 +09:00
taizan-hokouto
d75e1db304 Merge branch 'hotfix/rem' 2020-12-05 21:32:12 +09:00
taizan-hokouto
0d7dd26ede Increment version 2020-12-05 21:29:12 +09:00
taizan-hokouto
24b9f45e4a Remove entry point 2020-12-05 21:28:59 +09:00
taizan-hokouto
821cc57024 Merge tag 'v0.4.8' into develop
v0.4.8
2020-12-05 21:11:42 +09:00
taizan-hokouto
f050f5d650 Merge branch 'release/v0.4.8' 2020-12-05 21:11:42 +09:00
taizan-hokouto
c4eb78e112 Increment version 2020-12-05 21:11:18 +09:00
taizan-hokouto
40b0138a92 Merge branch 'master' into develop 2020-12-05 21:10:01 +09:00
taizan-hokouto
1d80dce913 Merge branch 'hotfix/dat' 2020-12-05 21:10:00 +09:00
taizan-hokouto
8b18f4476a Update README 2020-12-05 21:07:11 +09:00
taizan-hokouto
35a27fee3e Remove files 2020-12-05 20:59:47 +09:00
taizan-hokouto
eca766ff1a Update requirements 2020-12-05 14:45:24 +09:00
taizan-hokouto
139dc7fce2 Fix tests 2020-12-05 14:44:25 +09:00
taizan-hokouto
2c8a883ee3 Format 2020-12-05 14:43:56 +09:00
taizan-hokouto
865e4b5fab Remove files 2020-12-05 14:43:33 +09:00
taizan-hokouto
02d48ceccc Fix process 2020-12-05 14:42:02 +09:00
taizan-hokouto
bc3f16e86b Move functions 2020-12-05 14:39:55 +09:00
taizan-hokouto
1e6ce58f8b Merge branch 'master' into develop 2020-11-19 19:13:54 +09:00
taizan-hokouto
4db9486853 Merge branch 'hotfix/readme' 2020-11-19 19:13:53 +09:00
taizan-hokouto
775b79642e Update README 2020-11-19 19:12:54 +09:00
taizan-hokouto
423a128882 Merge tag 'v0.4.7' into develop
v0.4.7
2020-11-18 01:25:10 +09:00
taizan-hokouto
aaf9860bdc Merge branch 'release/v0.4.7' 2020-11-18 01:25:10 +09:00
taizan-hokouto
83ad4dcf1f Increment version 2020-11-18 01:24:37 +09:00
taizan-hokouto
765251b872 Merge branch 'feature/pipenv' into develop 2020-11-18 01:17:35 +09:00
taizan-hokouto
7ea88fead2 Modify requirements 2020-11-18 01:16:49 +09:00
taizan-hokouto
ea67e3e54e Add pipenv files 2020-11-18 01:16:01 +09:00
taizan-hokouto
a5c7ba52c8 Merge branch 'hotfix/test' 2020-11-17 01:11:22 +09:00
taizan-hokouto
7cf780ee87 Merge branch 'master' into develop 2020-11-17 01:11:22 +09:00
taizan-hokouto
c37201fa03 Remove tests 2020-11-17 01:10:54 +09:00
taizan-hokouto
6fcc1393de Merge branch 'master' into develop 2020-11-17 01:01:56 +09:00
taizan-hokouto
a474899268 Merge branch 'hotfix/tests' 2020-11-17 01:00:39 +09:00
taizan-hokouto
3f72eb0e00 Remove tests 2020-11-17 00:59:48 +09:00
taizan-hokouto
661d1e4b81 Fix tests 2020-11-17 00:54:32 +09:00
taizan-hokouto
4652a56bc6 Merge branch 'hotfix/json' 2020-11-16 23:32:32 +09:00
taizan-hokouto
966320cab5 Merge branch 'master' into develop 2020-11-16 23:32:32 +09:00
taizan-hokouto
35218a66da Remove unnecessary import 2020-11-16 23:32:14 +09:00
taizan-hokouto
3432609588 Merge branch 'hotfix/json' 2020-11-16 23:29:50 +09:00
taizan-hokouto
3ad6b7e845 Merge branch 'master' into develop 2020-11-16 23:29:50 +09:00
taizan-hokouto
48669e5f53 Fix tests 2020-11-16 23:29:24 +09:00
taizan-hokouto
7b0708ec46 Merge branch 'master' into develop 2020-11-16 23:17:37 +09:00
taizan-hokouto
f46df3ae42 Merge branch 'hotfix/json' 2020-11-16 23:17:36 +09:00
taizan-hokouto
96c028bd5d Increment version 2020-11-16 23:17:10 +09:00
taizan-hokouto
402dc15d7a Add tests 2020-11-16 23:11:51 +09:00
taizan-hokouto
6088ab6932 Fix jsonifying 2020-11-16 22:50:53 +09:00
taizan-hokouto
13812bdad3 Merge tag 'v0.4.5' into develop
v0.4.5
2020-11-16 01:50:50 +09:00
taizan-hokouto
d98d34d8b3 Merge branch 'release/v0.4.5' 2020-11-16 01:50:49 +09:00
taizan-hokouto
24fa104e84 Increment version 2020-11-16 01:50:25 +09:00
taizan-hokouto
b4dad8c641 Merge branch 'feature/archiver' into develop 2020-11-16 01:49:34 +09:00
taizan-hokouto
3550cd6d91 Use temporary file to reduce memory usage 2020-11-16 01:37:31 +09:00
taizan-hokouto
2815b48e0e Return filename 2020-11-16 01:36:59 +09:00
taizan-hokouto
650e6ccb65 Remove unnecessary lines 2020-11-16 01:17:10 +09:00
taizan-hokouto
4a00a19a43 Change argument name 2020-11-16 01:16:09 +09:00
taizan-hokouto
b067eda7b6 Separate modules 2020-11-16 01:15:36 +09:00
taizan-hokouto
1b6bc86e76 Fix handling exception 2020-11-15 23:49:36 +09:00
taizan-hokouto
da2b513bcc Reduce delay 2020-11-15 19:52:00 +09:00
taizan-hokouto
6adae578ef Return generator instead of list 2020-11-15 19:50:53 +09:00
taizan-hokuto
128a834841 Merge branch 'hotfix/fix' 2020-11-15 16:54:24 +09:00
taizan-hokuto
086a14115f Merge tag 'fix' into develop 2020-11-15 16:54:24 +09:00
taizan-hokuto
6a392f3e1a Increment version 2020-11-15 16:53:36 +09:00
taizan-hokuto
93127a703c Revert 2020-11-15 16:53:03 +09:00
taizan-hokuto
e4ddbaf8ae Merge branch 'develop' 2020-11-15 16:39:07 +09:00
taizan-hokuto
ec75058605 Merge pull request #22 from wakamezake/github_actions
Add GitHub actions
2020-11-15 16:05:13 +09:00
taizan-hokouto
2b62e5dc5e Merge branch 'feature/pr_22' into develop 2020-11-15 15:59:52 +09:00
taizan-hokouto
8d7874096e Fix datetime tests 2020-11-15 15:59:28 +09:00
taizan-hokouto
99fcab83c8 Revert 2020-11-15 15:49:39 +09:00
wakamezake
3027bc0579 change timezone utc to jst 2020-11-15 15:39:16 +09:00
wakamezake
b1b70a4e76 delete cache 2020-11-15 15:39:16 +09:00
wakamezake
de41341d84 typo 2020-11-15 15:39:16 +09:00
wakamezake
a03d43b081 version up 2020-11-15 15:39:16 +09:00
wakamezake
f60aaade7f init 2020-11-15 15:39:16 +09:00
wakamezake
d3c34086ff change timezone utc to jst 2020-11-15 11:29:12 +09:00
wakamezake
6b58c9bcf5 delete cache 2020-11-15 10:50:14 +09:00
wakamezake
c2cba1651e Merge remote-tracking branch 'upstream/master' into github_actions 2020-11-15 10:40:00 +09:00
taizan-hokouto
ada3eb437d Merge branch 'hotfix/test_requirements' 2020-11-15 09:22:38 +09:00
taizan-hokouto
c1517d5be8 Merge branch 'master' into develop 2020-11-15 09:22:38 +09:00
taizan-hokouto
351034d1e6 Increment version 2020-11-15 09:21:58 +09:00
taizan-hokouto
c1db5a0c47 Update requirements.txt and requirements_test.txt 2020-11-15 09:18:01 +09:00
wakamezake
088dce712a typo 2020-11-14 18:08:41 +09:00
wakamezake
425e880b09 version up 2020-11-14 18:07:30 +09:00
wakamezake
62ec78abee init 2020-11-14 18:04:49 +09:00
taizan-hokouto
c84a32682c Merge branch 'hotfix/fix_prompt' 2020-11-08 12:31:52 +09:00
taizan-hokouto
74277b2afe Merge branch 'master' into develop 2020-11-08 12:31:52 +09:00
taizan-hokouto
cd20b74b2a Increment version 2020-11-08 12:31:16 +09:00
taizan-hokouto
06f54fd985 Remove unnecessary console output 2020-11-08 12:30:40 +09:00
taizan-hokouto
98b0470703 Merge tag 'emoji' into develop
v0.4.1
2020-11-06 19:58:45 +09:00
taizan-hokouto
bb4113b53c Merge branch 'hotfix/emoji' 2020-11-06 19:58:44 +09:00
taizan-hokouto
07f4382ed4 Increment version 2020-11-06 19:57:16 +09:00
taizan-hokouto
d40720616b Fix emoji encoding 2020-11-06 19:56:54 +09:00
taizan-hokouto
eebe7c79bd Merge branch 'master' into develop 2020-11-05 22:19:11 +09:00
taizan-hokouto
6c9e327e36 Merge branch 'hotfix/fix_readme' 2020-11-05 22:19:11 +09:00
taizan-hokouto
e9161c0ddd Update README 2020-11-05 22:18:54 +09:00
taizan-hokouto
c8b75dcf0e Merge branch 'master' into develop 2020-11-05 00:14:50 +09:00
taizan-hokouto
30cb7d7043 Merge branch 'hotfix/fix_readme' 2020-11-05 00:14:50 +09:00
taizan-hokouto
19d5b74beb Update README 2020-11-05 00:14:36 +09:00
taizan-hokouto
d5c3e45edc Merge branch 'master' into develop 2020-11-03 20:21:53 +09:00
taizan-hokouto
1d479fc15c Merge branch 'hotfix/fix_readme' 2020-11-03 20:21:52 +09:00
taizan-hokouto
20a20ddd08 Update README 2020-11-03 20:21:39 +09:00
taizan-hokouto
00c239f974 Merge branch 'master' into develop 2020-11-03 20:10:48 +09:00
taizan-hokouto
67b766b32c Merge branch 'hotfix/fix_readme' 2020-11-03 20:10:48 +09:00
taizan-hokouto
249aa0d147 Update README 2020-11-03 20:10:34 +09:00
taizan-hokouto
c708a588d8 Merge tag 'v0.4.0' into develop
v0.4.0
2020-11-03 18:20:10 +09:00
taizan-hokouto
cb15df525f Merge branch 'release/v0.4.0' 2020-11-03 18:20:09 +09:00
taizan-hokouto
fcddc1516b Increment version 2020-11-03 18:19:43 +09:00
taizan-hokouto
a7732efd07 Merge branch 'feature/new_method' into develop 2020-11-03 18:18:43 +09:00
taizan-hokouto
0a2f4e8418 Update tests 2020-11-03 18:14:17 +09:00
taizan-hokouto
0c0ba0dfe6 Update README 2020-11-03 18:13:25 +09:00
taizan-hokouto
02827b174e Update tests 2020-11-03 18:13:09 +09:00
taizan-hokouto
81dee8a218 Fix comments 2020-11-03 16:51:30 +09:00
taizan-hokouto
5eb8bdbd0e Fix parsing info 2020-11-03 15:44:44 +09:00
taizan-hokouto
a37602e666 Fix keyboard interrupt process 2020-11-03 11:57:24 +09:00
taizan-hokouto
306b69198e Update README 2020-11-03 01:59:16 +09:00
taizan-hokouto
175e457052 Improve processing custom emojis 2020-11-02 22:44:09 +09:00
taizan-hokouto
5633a48618 Implement finalize() 2020-11-02 22:08:17 +09:00
taizan-hokouto
d7e608e8a1 Flake8 2020-11-02 00:26:46 +09:00
taizan-hokouto
213427fab3 Flake8 2020-11-02 00:26:27 +09:00
taizan-hokouto
3427c6fb69 Remove unnecessary line 2020-11-02 00:25:31 +09:00
taizan-hokouto
603c4470b7 Flake8 2020-11-02 00:25:05 +09:00
taizan-hokouto
37c8b7ae45 Use client instead of direct httpx 2020-11-01 21:58:41 +09:00
taizan-hokouto
d362152c77 Change module name 2020-11-01 19:29:09 +09:00
taizan-hokouto
8f5c3f312a Add --echo option to cli 2020-10-29 01:40:43 +09:00
taizan-hokouto
15a1d5c210 Implement exception holder 2020-10-29 01:39:07 +09:00
taizan-hokouto
499cf26fa8 Integrate httpx exceptions 2020-10-26 23:39:33 +09:00
taizan-hokouto
90596be880 Fix comment 2020-10-26 22:49:31 +09:00
taizan-hokouto
50d7b097e6 Remove unnecessary module 2020-10-26 22:34:43 +09:00
taizan-hokouto
b8d5ec5465 Remove unnecessary lines 2020-10-26 22:34:25 +09:00
taizan-hokouto
3200c5654f Change structure of default processor 2020-10-24 19:12:00 +09:00
taizan-hokouto
4905b1e4d8 Add simple core module 2020-10-24 18:07:54 +09:00
taizan-hokouto
16df63c14e Fix comments 2020-10-24 16:10:04 +09:00
taizan-hokouto
e950dff9d2 Merge tag 'fix_json' into develop
v0.3.2
2020-10-06 01:30:16 +09:00
taizan-hokouto
39d99ad4af Merge branch 'hotfix/fix_json' 2020-10-06 01:30:15 +09:00
taizan-hokouto
3675c91240 Increment version 2020-10-06 01:24:31 +09:00
taizan-hokouto
46258f625a Fix import module 2020-10-06 01:24:04 +09:00
taizan-hokouto
2cc161b589 Increment version 2020-10-06 01:20:25 +09:00
taizan-hokouto
115277e5e1 Fix handling internal error and keyboard interrupt 2020-10-06 01:19:45 +09:00
taizan-hokouto
ebf0e7c181 Fix handling json decode error and pattern unmatch 2020-10-05 21:38:51 +09:00
taizan-hokouto
b418898eef Merge tag 'filepath' into develop
v0.3.0
2020-10-04 11:33:59 +09:00
taizan-hokouto
3106b3e545 Merge branch 'hotfix/filepath' 2020-10-04 11:33:58 +09:00
taizan-hokouto
50816a661d Increment version 2020-10-04 11:30:07 +09:00
taizan-hokouto
6755bc8bb2 Make sure to pass fixed filepath to processor 2020-10-04 11:29:52 +09:00
taizan-hokouto
d62e7730ab Merge tag 'fix' into develop
v0.2.9
2020-10-04 10:32:54 +09:00
taizan-hokouto
26be989b9b Merge branch 'hotfix/fix' 2020-10-04 10:32:53 +09:00
taizan-hokouto
73ad0a1f44 Increment version 2020-10-04 10:22:34 +09:00
taizan-hokouto
66b185ebf7 Fix constructing filepath 2020-10-04 10:20:14 +09:00
taizan_hokuto
8bd82713e2 Merge tag 'fix' into develop
v0.2.7
2020-10-03 22:42:48 +09:00
taizan_hokuto
71650c39f7 Merge branch 'hotfix/fix' 2020-10-03 22:42:48 +09:00
taizan_hokuto
488445c73b Increment version 2020-10-03 22:41:53 +09:00
taizan_hokuto
075e811efe Delete unnecessary code 2020-10-03 22:41:12 +09:00
taizan_hokuto
9f9b83f185 Merge tag 'pattern' into develop
v0.2.6
2020-10-03 22:35:46 +09:00
taizan_hokuto
58d9bf7fdb Merge branch 'hotfix/pattern' 2020-10-03 22:35:46 +09:00
taizan_hokuto
b3e6275de7 Increment version 2020-10-03 22:35:22 +09:00
taizan_hokuto
748778f545 Fix pattern matching 2020-10-03 22:04:09 +09:00
taizan-hokuto
b2a68d0a74 Merge tag 'network' into develop
v0.2.5
2020-09-14 00:40:40 +09:00
taizan-hokuto
e29b3b8377 Merge branch 'hotfix/network' 2020-09-14 00:40:40 +09:00
taizan-hokuto
0859ed5fb1 Increment version 2020-09-14 00:29:21 +09:00
taizan-hokuto
a80d5ba080 Fix handling network error 2020-09-14 00:28:41 +09:00
taizan-hokuto
ac2924824e Merge tag 'memory' into develop
v0.2.4
2020-09-12 02:12:47 +09:00
taizan-hokuto
b7e6043a71 Merge branch 'hotfix/memory' 2020-09-12 02:12:46 +09:00
taizan-hokuto
820ba35013 Increment version 2020-09-12 02:02:07 +09:00
taizan-hokuto
ecd2d130bf Clear set each time the extraction changes 2020-09-12 01:57:55 +09:00
taizan-hokuto
1d410b6e68 Merge tag 'not_quit' into develop
v0.2.3
2020-09-12 00:57:49 +09:00
taizan-hokuto
f77a2c889b Merge branch 'hotfix/not_quit' 2020-09-12 00:57:48 +09:00
taizan-hokuto
47d5ab288f Increment version 2020-09-12 00:49:37 +09:00
taizan-hokuto
5f53fd24dd Format 2020-09-12 00:48:40 +09:00
taizan-hokuto
11a9d0e2d7 Fix a problem with extraction not completing 2020-09-12 00:42:30 +09:00
taizan-hokuto
6f18de46f7 Merge tag 'continue_error' into develop
v0.2.2
2020-09-11 00:21:07 +09:00
taizan-hokuto
480c9e15b8 Merge branch 'hotfix/continue_error' 2020-09-11 00:21:07 +09:00
taizan-hokuto
35aa7636f6 Increment version 2020-09-11 00:20:24 +09:00
taizan-hokuto
8fee67c2d4 Fix handling video info error 2020-09-11 00:18:09 +09:00
taizan-hokuto
74bfdd07e2 Merge tag 'v0.2.1' into develop
v0.2.1
2020-09-09 22:23:02 +09:00
taizan-hokuto
d3f1643a40 Merge branch 'release/v0.2.1' 2020-09-09 22:23:01 +09:00
taizan-hokuto
eb29f27493 Increment version 2020-09-09 22:22:31 +09:00
taizan-hokuto
8adf75ab83 Merge branch 'feature/pbar' into develop 2020-09-09 22:20:36 +09:00
taizan-hokuto
2e05803d75 Remove unnecessary option 2020-09-09 22:20:09 +09:00
taizan-hokuto
f16c0ee73a Fix progress bar line feed and remove pbar option 2020-09-09 22:19:10 +09:00
taizan-hokuto
a338f2b782 Merge tag 'v0.2.0' into develop
v0.2.0
2020-09-07 23:35:45 +09:00
taizan-hokuto
864ccddfd7 Merge branch 'release/v0.2.0' 2020-09-07 23:35:44 +09:00
taizan-hokuto
339df69e36 Increment version 2020-09-07 23:35:14 +09:00
taizan-hokuto
76a5b0cd18 Merge branch 'feature/new_item' into develop 2020-09-07 23:34:16 +09:00
taizan-hokuto
be0ab2431b Delete test for unuse module 2020-09-07 23:33:26 +09:00
taizan-hokuto
2edb60c592 Delete unuse modules 2020-09-07 23:31:32 +09:00
taizan-hokuto
2c6c3a1ca3 Delete old progress bar 2020-09-07 23:30:49 +09:00
taizan-hokuto
4be540793d Delete unnecessary blank lines 2020-09-07 23:30:30 +09:00
taizan-hokuto
08b86fe596 Make it possible to switch progress bar 2020-09-07 23:29:48 +09:00
taizan-hokuto
157f3b9952 Fix handling when missing id and type 2020-09-07 23:28:03 +09:00
taizan-hokuto
8f3ca2662a Merge tag 'pbar' into develop
v0.1.9
2020-09-06 18:58:34 +09:00
taizan-hokuto
c4b015861c Merge branch 'hotfix/pbar' 2020-09-06 18:58:33 +09:00
taizan-hokuto
3aa413d59e Increment version 2020-09-06 18:54:10 +09:00
taizan-hokuto
03ba285a16 Fix callback handling 2020-09-06 18:53:35 +09:00
taizan-hokuto
5fe0ee5aa8 Merge tag 'v0.1.8' into develop
v0.1.8
2020-09-06 18:27:58 +09:00
taizan-hokuto
4e829a25d4 Merge branch 'release/v0.1.8' 2020-09-06 18:27:57 +09:00
taizan-hokuto
15132a9bb8 Increment version 2020-09-06 18:27:08 +09:00
taizan-hokuto
64ace9dad6 Update progress bar 2020-09-06 18:25:16 +09:00
taizan-hokuto
9a2e96d3a0 Merge tag 'extract_vid' into develop
v0.1.7
2020-09-04 01:55:42 +09:00
taizan-hokuto
a3695a59b8 Merge branch 'hotfix/extract_vid' 2020-09-04 01:55:41 +09:00
taizan-hokuto
bc8655ed62 Increment version 2020-09-04 01:53:14 +09:00
taizan-hokuto
3bdc465740 Devide exception handling 2020-09-04 01:52:53 +09:00
taizan-hokuto
235d6b7212 Fix extract video info 2020-09-04 01:46:10 +09:00
taizan-hokuto
9f0754da57 Merge tag 'http2' into develop
v0.1.6
2020-09-03 21:27:48 +09:00
taizan-hokuto
306b0a4564 Merge branch 'hotfix/http2' 2020-09-03 21:27:48 +09:00
taizan-hokuto
1c49387f1a Increment version 2020-09-03 21:24:42 +09:00
taizan-hokuto
300d96e56c Fix requirements.txt 2020-09-03 21:24:21 +09:00
taizan-hokuto
0e301f48a8 Merge tag 'v0.1.5' into develop
v0.1.5
2020-09-03 20:16:56 +09:00
taizan-hokuto
a790ab13a9 Merge branch 'release/v0.1.5' 2020-09-03 20:16:55 +09:00
taizan-hokuto
0456300d19 Increment version 2020-09-03 20:15:38 +09:00
taizan-hokuto
2ef1e7028f Restore setup 2020-09-03 19:59:18 +09:00
taizan-hokuto
9413c4a186 Merge branch 'feature/add_progressbar' into develop 2020-09-03 19:54:35 +09:00
taizan-hokuto
8a8cef399f Format 2020-09-03 19:48:34 +09:00
taizan-hokuto
3bcad12cf6 Add cli option 2020-09-03 19:31:34 +09:00
taizan-hokuto
4eb18279fe Add progress bar 2020-09-03 00:57:26 +09:00
taizan-hokuto
e9ed564e1b Merge branch 'feature/httpx' into develop 2020-08-30 22:17:57 +09:00
taizan-hokuto
95f975c93d Use httpx 2020-08-30 22:16:58 +09:00
taizan-hokuto
8012e1d191 Merge branch 'master' into feature/httpx 2020-08-22 12:41:57 +09:00
taizan-hokuto
f9480ea1eb Merge branch 'hotfix/cli_handle_live' 2020-08-21 22:25:48 +09:00
taizan-hokuto
404727c49c Merge tag 'cli_handle_live' into develop
v0.1.4
2020-08-21 22:25:48 +09:00
taizan-hokuto
6b924a88ef Increment version 2020-08-21 22:25:06 +09:00
taizan-hokuto
56294d6a67 Fix extracting video_id 2020-08-21 22:23:33 +09:00
taizan-hokuto
283443e374 Merge pull request #15 from EtianAM/patch-1
Fix videoinfo.py and CLI download
2020-08-21 19:34:19 +09:00
Etian Daniel Alavardo Mtz
89b51c420f Avoid changing the type of result.
However, if this argument is used elsewhere in the code it should be corrected.
2020-08-20 22:39:32 -05:00
Etian Daniel Alavardo Mtz
96474f10c6 Fix videoinfo.py
A bit ugly, but I couldn't solve it any other way. I'm bad with regex.
2020-08-20 22:29:59 -05:00
taizan-hokuto
5f78a99507 Merge tag 'exist_dir' into develop
v0.1.3
2020-08-06 00:32:07 +09:00
taizan-hokuto
78373bf45c Merge branch 'hotfix/exist_dir' 2020-08-06 00:32:06 +09:00
taizan-hokuto
3e11deed8f Increment version 2020-08-06 00:31:21 +09:00
taizan-hokuto
6daa375adf Handle exception when specified directory not found 2020-08-06 00:30:43 +09:00
taizan-hokuto
497d84015e Merge branch 'master' into develop 2020-07-27 00:26:57 +09:00
taizan-hokuto
a90bda674d Merge pull request #12 from mark-ignacio/mit
Add LICENSE file
2020-07-27 00:22:56 +09:00
Mark Ignacio
48543b7866 add LICENSE file 2020-07-26 07:09:28 -07:00
taizan-hokuto
5d3c7b5abd Merge tag 'fix_color' into develop
v0.1.2
2020-07-24 22:43:09 +09:00
taizan-hokuto
8df7062873 Merge branch 'hotfix/fix_color' 2020-07-24 22:43:09 +09:00
taizan-hokuto
b788f692ad Increment version 2020-07-24 22:42:26 +09:00
taizan-hokuto
713215f1d7 Fix supersticker bgColor 2020-07-24 22:41:07 +09:00
taizan-hokuto
f16ef60f11 Merge tag 'fix_cli' into develop
v0.1.1
2020-07-24 16:43:14 +09:00
taizan-hokuto
9bbdb6c4de Merge branch 'hotfix/fix_cli' 2020-07-24 16:43:14 +09:00
taizan-hokuto
2200abf204 Increment version 2020-07-24 16:40:54 +09:00
taizan-hokuto
3ed0cb2c35 Fix setting save path 2020-07-24 16:40:09 +09:00
taizan-hokuto
5fa4d051ee Merge tag 'v0.1.0' into develop
v0.1.0
2020-07-24 16:27:14 +09:00
taizan-hokuto
cd6d522055 Merge branch 'release/v0.1.0' 2020-07-24 16:27:14 +09:00
taizan-hokuto
aa8a4fb592 Increment version 2020-07-24 16:26:09 +09:00
taizan-hokuto
92a01aa4d9 Merge tag 'fix_exception_handling' into develop 2020-07-24 15:20:08 +09:00
taizan-hokuto
dbde072828 Merge branch 'hotfix/fix_exception_handling' 2020-07-24 15:20:08 +09:00
taizan-hokuto
e3f9f95fb1 Fix exception handling 2020-07-24 15:19:32 +09:00
taizan-hokuto
fa02116ab4 Merge branch 'feature/url_pattern' into develop 2020-07-24 14:52:06 +09:00
taizan-hokuto
d8656161cd Update README 2020-07-24 14:04:13 +09:00
taizan-hokuto
174d9f27c0 Add tests 2020-07-24 14:03:20 +09:00
taizan-hokuto
0abf8dd9f0 Make it possible to extract video id from url 2020-07-24 14:03:07 +09:00
taizan-hokuto
5ab653a1b2 Merge branch 'feature/extend_processor' into develop 2020-07-23 16:35:37 +09:00
taizan-hokuto
6e6bb8e019 Add tests 2020-07-23 16:20:38 +09:00
taizan-hokuto
ee4b696fc5 Add colors attribute 2020-07-23 16:20:12 +09:00
taizan-hokuto
fd1d283caa Merge branch 'hotfix/meta_tag' 2020-07-13 23:04:19 +09:00
taizan-hokuto
85966186b5 Merge tag 'meta_tag' into develop
v0.0.9.1
2020-07-13 23:04:19 +09:00
taizan-hokuto
71341d2876 Increment version 2020-07-13 23:03:46 +09:00
taizan-hokuto
8882c82f8b Fix place of meta tag 2020-07-13 23:03:20 +09:00
taizan-hokuto
cf6ed24864 Merge branch 'release/v0.0.9' 2020-07-13 01:55:13 +09:00
taizan-hokuto
584b9c5591 Merge tag 'v0.0.9' into develop
v0.0.9
2020-07-13 01:55:13 +09:00
taizan-hokuto
167c8acb93 Incerment version 2020-07-13 01:52:38 +09:00
taizan-hokuto
75a31bd245 Merge branch 'feature/emoji_embedding' into develop 2020-07-13 01:45:07 +09:00
taizan-hokuto
366d75c2bb Update README 2020-07-13 01:44:49 +09:00
taizan-hokuto
b7ff2b6537 Restore logging settings 2020-07-13 00:59:20 +09:00
taizan-hokuto
5dfd883fc9 Remove unnecessary line 2020-07-12 23:47:02 +09:00
taizan-hokuto
133a8afb27 Make it possible to embed custom emojis in HTML 2020-07-12 23:24:43 +09:00
taizan-hokuto
971e4bdf39 Add finalize function to processor 2020-07-12 23:23:05 +09:00
taizan-hokuto
f78bfde59e Merge branch 'hotfix/type_comment' 2020-06-18 02:07:46 +09:00
taizan-hokuto
a7379fd93f Merge tag 'type_comment' into develop 2020-06-18 02:07:46 +09:00
taizan-hokuto
1cc3661d35 Fix comment 2020-06-18 02:06:27 +09:00
taizan-hokuto
6c781483a9 Merge branch 'release/v0.0.8' 2020-06-18 00:17:54 +09:00
taizan-hokuto
5c3280f858 Merge tag 'v0.0.8' into develop
v0.0.8
2020-06-18 00:17:54 +09:00
taizan-hokuto
7500f79de0 Increment version 2020-06-18 00:16:57 +09:00
taizan-hokuto
94d4eebd0f Implement raise_for_status() 2020-06-17 23:56:07 +09:00
taizan-hokuto
2474207691 Format code 2020-06-04 23:10:26 +09:00
taizan-hokuto
e6dbc8772e Merge branch 'feature/use_protbuf' into develop 2020-05-31 22:58:20 +09:00
taizan-hokuto
8f91e031f3 Modify tests 2020-05-31 22:57:28 +09:00
taizan-hokuto
870d1f3fbe Modify parameters for archived chat 2020-05-31 22:57:12 +09:00
taizan-hokuto
141dbcd2da Lint 2020-05-31 19:45:01 +09:00
taizan-hokuto
6eb848f1c9 Modify checking timestamp 2020-05-31 19:43:09 +09:00
taizan-hokuto
8d7fc03fe0 Remove unecessary parser 2020-05-31 01:13:15 +09:00
taizan-hokuto
970e63cb38 Use protocol buffers 2020-05-31 00:33:46 +09:00
taizan-hokuto
1c817b6476 Merge branch 'release/v0.0.7.2' 2020-05-22 02:39:53 +09:00
taizan-hokuto
51eff10eeb Merge tag 'v0.0.7.2' into develop
v0.0.7.2
2020-05-22 02:39:53 +09:00
taizan-hokuto
18b88200a8 Increment version 2020-05-22 02:29:41 +09:00
taizan-hokuto
c95d70a232 Merge branch 'hotfix/#7_cli_index_outof_range' 2020-05-22 02:28:28 +09:00
taizan-hokuto
7640586591 Merge branch 'master' into develop 2020-05-22 02:28:28 +09:00
taizan-hokuto
f7ec14e166 Fix for #7 2020-05-22 02:27:52 +09:00
taizan-hokuto
a4dacdb7d7 Merge tag 'v0.0.7.1' into develop
v0.0.7.1
2020-05-06 01:24:55 +09:00
taizan-hokuto
785a82b618 Merge branch 'release/v0.0.7.1' 2020-05-06 01:24:54 +09:00
taizan-hokuto
faf886eebd Increment version 2020-05-06 01:24:30 +09:00
taizan-hokuto
8a627414cb Merge tag 'sponsor_text' into develop 2020-05-06 01:23:37 +09:00
taizan-hokuto
d14262cbcb Merge branch 'hotfix/sponsor_text' 2020-05-06 01:23:37 +09:00
taizan-hokuto
da7c694dfb Modify parsing membership 2020-05-06 01:23:19 +09:00
taizan-hokuto
9aa35b9756 Merge tag 'v0.0.7' into develop
v0.0.7
2020-05-05 22:59:16 +09:00
taizan-hokuto
f0a1a509a0 Merge branch 'release/v0.0.7' 2020-05-05 22:59:16 +09:00
taizan-hokuto
5ebca605ac Increment version 2020-05-05 22:58:29 +09:00
taizan-hokuto
3826b32ab9 Merge tag 'membership_renderer' into develop 2020-05-05 22:51:16 +09:00
taizan-hokuto
a46c82d3c0 Merge branch 'hotfix/membership_renderer' 2020-05-05 22:51:16 +09:00
taizan-hokuto
206d052907 Modify parsing membership 2020-05-05 22:47:12 +09:00
taizan-hokuto
141d7a9299 Merge tag 'termination' into develop 2020-05-05 21:18:46 +09:00
taizan-hokuto
04457eaa5c Merge branch 'hotfix/termination' 2020-05-05 21:18:46 +09:00
taizan-hokuto
bd32c75833 Modify termination 2020-05-05 21:16:06 +09:00
taizan-hokuto
84bae4ad2a Modify bytes combination 2020-04-18 00:55:56 +09:00
taizan-hokuto
d72608bf0a Merge tag 'json_decode_error' into develop
v0.0.6.6
2020-03-14 09:43:37 +09:00
taizan-hokuto
3243d69d7a Merge branch 'hotfix/json_decode_error' 2020-03-14 09:43:37 +09:00
taizan-hokuto
6e1b735ebc Increment version 2020-03-14 09:42:53 +09:00
taizan-hokuto
c54481dad5 Add header html and show progress 2020-03-14 09:26:28 +09:00
taizan-hokuto
78604c84d4 Fix testdata path separator 2020-03-14 08:16:19 +09:00
taizan-hokuto
21d93613a2 Handling JSONDecodeError 2020-03-14 08:00:31 +09:00
taizan-hokuto
56bf721330 Merge tag 'argparse' into develop
v0.0.6.5
2020-03-10 01:58:25 +09:00
taizan-hokuto
5f50598f79 Merge branch 'hotfix/argparse' 2020-03-10 01:58:24 +09:00
taizan-hokuto
5e8c438c6b Increment version 2020-03-10 01:57:55 +09:00
taizan-hokuto
23e47f6fb0 Fix parsing video_id which starts with '-' 2020-03-10 01:57:01 +09:00
taizan-hokuto
74dfe0a612 Modify requirements.txt 2020-03-10 01:06:36 +09:00
taizan-hokuto
725af25d81 Merge tag 'v0.0.6.4' into develop
v0.0.6.4
2020-03-08 23:43:01 +09:00
taizan-hokuto
316fc5594a Merge branch 'release/v0.0.6.4' 2020-03-08 23:43:00 +09:00
taizan-hokuto
44dffc7650 Increment version 2020-03-08 23:42:28 +09:00
taizan-hokuto
102d8c48c4 Merge branch 'feature/commandline-tool' into develop 2020-03-08 23:39:47 +09:00
taizan-hokuto
f8822a053f Add desription to README.md 2020-03-08 23:33:50 +09:00
taizan-hokuto
9d624f771a Implement CLI 2020-03-08 23:18:30 +09:00
taizan-hokuto
778d4db28b Merge tag 'fix_resume' into develop
v0.0.6.3
2020-03-08 14:34:08 +09:00
taizan-hokuto
36e0fd5c54 Merge branch 'hotfix/fix_resume' 2020-03-08 14:34:07 +09:00
taizan-hokuto
4252643273 Increment version 2020-03-08 14:31:49 +09:00
taizan-hokuto
c88fd8bc4e Fix resume 2020-03-08 14:31:24 +09:00
taizan-hokuto
af3b6d4271 Merge tag 'full_of_que_exception' into develop
v0.0.6.2
2020-03-07 22:58:13 +09:00
taizan-hokuto
331e825c97 Merge branch 'hotfix/full_of_que_exception' 2020-03-07 22:58:13 +09:00
taizan-hokuto
4019ad4b9d Increment version 2020-03-07 22:49:18 +09:00
taizan-hokuto
1074178afc Fix handling full que exception 2020-03-07 22:16:46 +09:00
taizan-hokuto
55a58f532d Increment version 2020-02-29 08:15:39 +09:00
taizan-hokuto
b302454083 Expose Extractor 2020-02-29 05:55:29 +09:00
taizan-hokuto
ff9e7de796 Change VideoInfo functions to accessor style 2020-02-29 03:34:25 +09:00
taizan-hokuto
fe2047502a Limit lines 2020-02-28 01:17:54 +09:00
taizan-hokuto
5480e3e9ed Modify video info 2020-02-28 01:04:18 +09:00
taizan-hokuto
18c08f45ad Move extract method into class 2020-02-26 23:47:33 +09:00
taizan-hokuto
a9831c6a27 Expose superchat calculator 2020-02-26 22:31:44 +09:00
taizan-hokuto
60976b2584 Move directory of speed calculator 2020-02-26 22:25:03 +09:00
taizan-hokuto
92abf7499c Rename superchat calculator 2020-02-26 22:23:09 +09:00
taizan-hokuto
4416e1a79c Merge branch 'feature/fix_json_archive_processor' into develop 2020-02-26 22:18:32 +09:00
taizan-hokuto
f7f9c1cda3 Fix testdata path 2020-02-26 22:12:41 +09:00
taizan-hokuto
de35537be8 Rename modules 2020-02-26 22:08:36 +09:00
taizan-hokuto
61d4e06470 Fix JsonfileArchiveProcessor:
Rename to `JsonfileArchiver`
Add tests
2020-02-26 21:42:27 +09:00
taizan-hokuto
3c95242ddf Implement Superchat Calculator 2020-02-24 13:56:58 +09:00
taizan-hokuto
af4afb4636 Merge branch 'feature/mining' into develop 2020-02-22 17:24:44 +09:00
taizan-hokuto
05e1c908a5 Use generator comprehension 2020-02-22 17:21:01 +09:00
taizan-hokuto
e770d95fe8 Implement mining 2020-02-22 17:10:40 +09:00
taizan-hokuto
eae485b914 Return continuation even if no chat data 2020-02-16 23:31:25 +09:00
taizan-hokuto
d8c1c4491d Delete unnesessary file 2020-02-16 22:55:45 +09:00
taizan-hokuto
3e941c2cf1 Merge branch 'feature/downloader' into develop 2020-02-16 21:38:19 +09:00
taizan-hokuto
8b617551ad Add channelName parsing 2020-02-16 21:37:35 +09:00
taizan-hokuto
c4cf424702 Aggregate return values with patch class 2020-02-16 20:43:12 +09:00
taizan-hokuto
6fdb3bf8cf Implement cancell download 2020-02-13 21:15:39 +09:00
taizan-hokuto
b1292b4329 Rename functions 2020-02-11 12:43:11 +09:00
taizan-hokuto
339d04ad75 Handling JSONDecodeError 2020-02-11 11:26:29 +09:00
taizan-hokuto
abb7565e3a Implement postprocessing 2020-02-09 21:58:09 +09:00
taizan-hokuto
ee77807dbd Improve dlworker efficiency 2020-02-09 20:26:21 +09:00
taizan-hokuto
2c598bc8f7 Change construct method of videoinfo 2020-02-09 15:14:33 +09:00
taizan-hokuto
c7bfae9f2a Modify MANIFEST.in 2020-02-03 22:05:43 +09:00
taizan-hokuto
eaa7bdc8b6 Deligate processing extra chat data to Block class 2020-02-03 22:01:54 +09:00
taizan-hokuto
4a8e353098 Change process order 2020-02-03 00:37:52 +09:00
taizan-hokuto
24f08ecbdb Add test 2020-02-02 22:36:26 +09:00
taizan-hokuto
e8510f1116 Delete unnecessary lines 2020-02-02 12:07:56 +09:00
taizan-hokuto
f1d8393971 Divide download module 2020-02-02 00:38:22 +09:00
taizan-hokuto
04aedc82e8 Divide modules 2020-02-01 21:08:27 +09:00
taizan-hokuto
228773295d Check if keys exist 2020-02-01 16:29:43 +09:00
taizan-hokuto
59defc568c Merge tag 'cannot_fetch_at_0_multithread' into develop
v0.0.5.3
2020-02-01 01:41:41 +09:00
taizan-hokuto
9de75788f2 Merge branch 'hotfix/cannot_fetch_at_0_multithread' 2020-02-01 01:41:40 +09:00
taizan-hokuto
76f0c0e658 Increment version 2020-02-01 01:36:35 +09:00
taizan-hokuto
0d8ecb778f Fix for #1 : core_multithread 2020-02-01 01:25:42 +09:00
taizan-hokuto
a3eca8f05d Merge tag 'cannot_fetch_at_0' into develop
v0.0.5.2
2020-02-01 00:01:05 +09:00
taizan-hokuto
bbf7a2906a Merge branch 'hotfix/cannot_fetch_at_0' 2020-02-01 00:01:04 +09:00
taizan-hokuto
1862b83eac Increment version 2020-01-31 23:58:07 +09:00
taizan-hokuto
053ff5291f Modify url for japanese wiki 2020-01-31 23:54:03 +09:00
taizan-hokuto
4e47d4a262 Fix for #1 2020-01-31 23:38:51 +09:00
taizan-hokuto
436e8df4c9 Simplify getting timestamp 2020-01-28 22:36:53 +09:00
taizan-hokuto
5ab8cfe736 Add command 2020-01-28 21:48:04 +09:00
taizan-hokuto
15b517e905 Aggregate variable 2020-01-28 21:13:58 +09:00
taizan-hokuto
214a3d2be3 Change function to inline 2020-01-27 00:15:46 +09:00
taizan-hokuto
e968325d1f Add videoinfo items 2020-01-26 23:56:02 +09:00
taizan-hokuto
a56dc89477 Implement 'download' function as entry point 2020-01-26 23:31:13 +09:00
taizan-hokuto
38253e1d18 Modify usage of videoinfo 2020-01-26 23:29:50 +09:00
taizan-hokuto
cc78551e90 Change function to inline 2020-01-26 22:23:21 +09:00
taizan-hokuto
6e37ef5d4f Implement function to check duplication of chat 2020-01-26 22:12:43 +09:00
taizan-hokuto
c126d5b825 Cover all actions when checking id and type 2020-01-26 22:11:16 +09:00
taizan-hokuto
a89503fe9e Move files 2020-01-26 20:14:39 +09:00
taizan-hokuto
1d7678c954 Implement module for getting video information 2020-01-26 20:06:04 +09:00
taizan-hokuto
dea98c33d7 Fix handling when specified video length is too long 2020-01-26 15:11:29 +09:00
taizan-hokuto
5ba61db4f3 Modify remove_overwrap 2020-01-26 14:21:18 +09:00
taizan-hokuto
03b901d59c Implement callback parameter 2020-01-26 13:49:42 +09:00
taizan-hokuto
540f16c1a0 Make it possible to use method chain 2020-01-26 12:08:10 +09:00
taizan-hokuto
cc8bba8f63 Rewrite downloader 2020-01-26 08:03:18 +09:00
taizan-hokuto
22b3ec2994 Add test downloader 2020-01-24 01:14:23 +09:00
taizan-hokuto
9d494446e1 Implement base downloader 2020-01-23 02:00:50 +09:00
taizan-hokuto
956c7e2640 Update README.md 2020-01-18 17:50:31 +09:00
taizan-hokuto
03537c0a06 Merge tag 'v0.0.5.1.3' into develop
v0.0.5.1.3
2020-01-18 14:45:34 +09:00
taizan-hokuto
f7d1830226 Merge branch 'release/v0.0.5.1.3' 2020-01-18 14:45:33 +09:00
taizan-hokuto
76b126faf2 Increment version 2020-01-18 14:43:06 +09:00
taizan-hokuto
bbd01d6523 Increment version 2020-01-18 13:12:52 +09:00
taizan-hokuto
f8fa0e394e Delete json_display_processor 2020-01-18 13:02:43 +09:00
taizan-hokuto
efdf07e3de Make it possible to set custom logger 2020-01-18 12:38:36 +09:00
taizan-hokuto
2573cc18de Fix setting exception_handler 2020-01-18 09:22:16 +09:00
taizan-hokuto
1c5852421b Undo _set_exception_handler 2020-01-18 09:18:20 +09:00
taizan-hokuto
970d111e1b Alert default processor attribute error
: delete default exception handler

Alert default processor attribute error
: delete default exception handler

Delete unnecessary lines

Delete unnecessary lines
2020-01-18 08:39:36 +09:00
taizan-hokuto
1643dd1ad1 Switch author type by icon type 2020-01-13 18:32:36 +09:00
taizan-hokuto
0272319fa6 Update README 2020-01-13 13:09:51 +09:00
taizan-hokuto
fb0edef136 Merge branch 'release/v0.0.5.0' 2020-01-13 09:38:10 +09:00
taizan-hokuto
260a2b35a9 Merge tag 'v0.0.5.0' into develop
v0.0.5.0
2020-01-13 09:38:10 +09:00
taizan-hokuto
e03d39475e Increment version 2020-01-13 09:37:34 +09:00
taizan-hokuto
2462b8aca0 Merge branch 'hotfix/fix_defaultprocessor_display_amount' 2020-01-13 08:19:42 +09:00
taizan-hokuto
a1024c8734 Merge tag 'fix_defaultprocessor_display_amount' into develop 2020-01-13 08:19:42 +09:00
taizan-hokuto
6b3ca00d35 Fix superchat/sticker display amount 2020-01-13 08:19:03 +09:00
taizan-hokuto
385634b709 Fix MANIFEST.in 2020-01-12 14:55:29 +09:00
taizan-hokuto
c1a78a2743 Fix setup.py 2020-01-12 14:40:26 +09:00
taizan-hokuto
7961801e0c Increment version 2020-01-12 14:38:21 +09:00
taizan-hokuto
5fe4e7af04 Fix description 2020-01-12 14:26:15 +09:00
taizan-hokuto
892dfb8a91 Fix setup.py 2020-01-11 14:23:32 +09:00
taizan-hokuto
fddab22a1f Delete unnecessary file 2020-01-11 13:29:34 +09:00
taizan-hokuto
7194948066 Modify setup.py 2020-01-11 13:23:55 +09:00
taizan-hokuto
a836d92194 Increment version 2020-01-11 05:33:50 +09:00
taizan-hokuto
c408cb2713 Increment version 2020-01-11 05:04:10 +09:00
taizan-hokuto
c3d2238ead Merge branch 'feature/switch_topchat' into develop 2020-01-11 04:54:23 +09:00
taizan-hokuto
6c8d390fc7 Modify test 2020-01-11 04:41:39 +09:00
taizan-hokuto
ff1ee70d7e Implement 'topchat_only' parameter
: make it possible to select whether to get only top chat.
2020-01-11 04:22:48 +09:00
taizan-hokuto
404623546e Exclude test requirements 2020-01-10 01:10:12 +09:00
taizan-hokuto
3f9f64d19c Increment version 2020-01-09 00:17:18 +09:00
taizan-hokuto
7996c6adad Add test 2020-01-09 00:15:36 +09:00
taizan-hokuto
50d55da7dc Add currency 2020-01-08 23:55:50 +09:00
taizan-hokuto
c92e735715 Merge branch 'release/v0.0.4.3' 2020-01-08 01:27:06 +09:00
taizan-hokuto
d4a1d00e28 Increment version 2020-01-08 01:25:34 +09:00
taizan-hokuto
60c389f3f7 Change debug mode 2020-01-08 01:25:19 +09:00
taizan-hokuto
705bfe0bed Modify MANIFEST.in 2020-01-08 01:23:19 +09:00
taizan-hokuto
0f7a0218b6 Delete unnecessary lines 2020-01-08 01:12:58 +09:00
taizan-hokuto
89d2f8978f Modify README 2020-01-08 01:02:05 +09:00
taizan-hokuto
6befc2de95 Merge branch 'feature/force_replay_mode' into develop 2020-01-08 00:47:48 +09:00
taizan-hokuto
a0c5ea035a Fix comment 2020-01-08 00:46:23 +09:00
taizan-hokuto
3b27c81166 Add tests 2020-01-08 00:44:50 +09:00
taizan-hokuto
26fefddddf Implement force_replay_mode 2020-01-04 13:23:32 +09:00
taizan-hokuto
5d86fb4b71 Fix parameter switching and tests 2020-01-04 09:28:44 +09:00
taizan-hokuto
b5e302cdf3 Make it possible to retrieve chat before broadcast
by specifying negative number in seektime
2020-01-04 00:41:58 +09:00
taizan-hokuto
5d228589f1 Delete unnecessary file 2020-01-04 00:25:22 +09:00
taizan-hokuto
fd8ecec0c5 Merge branch 'feature/warning_deprecation_replaychat' into develop 2020-01-04 00:04:00 +09:00
taizan-hokuto
a1e48b56e6 Add warning for deprecating replaychat 2020-01-03 23:45:08 +09:00
taizan-hokuto
9c41536533 Merge branch 'feature/integrate' into develop 2020-01-03 22:12:15 +09:00
taizan-hokuto
2c684d04b5 Intgegrate replaychat into livechat (multithread) 2020-01-03 02:09:39 +09:00
taizan-hokuto
30708470f2 Change comments 2020-01-02 22:43:23 +09:00
taizan-hokuto
d742a9fdf3 Fix seek time param 2020-01-02 22:27:21 +09:00
taizan-hokuto
2fdd834caf Extract method _check_pause() 2020-01-02 22:08:26 +09:00
taizan-hokuto
4c558491a3 Change exception message 2020-01-02 21:17:37 +09:00
taizan-hokuto
0fc9d14780 Fix handling exception 2020-01-02 21:14:22 +09:00
taizan-hokuto
18400724b1 Modify metadata selection 2020-01-02 21:08:53 +09:00
taizan-hokuto
7b7323abf8 Delete debug line 2020-01-02 20:52:36 +09:00
taizan-hokuto
fc5979c025 Moved livechat_json part 2020-01-02 20:51:59 +09:00
taizan-hokuto
f4dc5e9d4a Delete unnecessary lines 2020-01-02 20:31:51 +09:00
taizan-hokuto
347707a514 Delete unnecessary lines 2020-01-02 20:08:45 +09:00
taizan-hokuto
7766a39c9c Integrate replaychat into livechat 2020-01-02 19:35:58 +09:00
taizan-hokuto
48b6f2c24e Add comment 2020-01-02 15:46:45 +09:00
taizan-hokuto
907f8aba0b Rename function name 2020-01-02 15:42:32 +09:00
taizan-hokuto
2616e4c4b5 Adjust amount of first fetching chat 2020-01-02 15:20:34 +09:00
taizan-hokuto
d6ea673f98 Fix getting arcparam when resume 2020-01-02 15:12:48 +09:00
taizan-hokuto
2bb481a228 Disable _pauser when callback is unset 2020-01-02 14:27:36 +09:00
taizan-hokuto
7308a87a61 Implement pause/pauser to livechat 2020-01-02 13:15:41 +09:00
taizan-hokuto
9751289eca Integrate _get_initial_continuation 2020-01-02 12:47:40 +09:00
taizan-hokuto
044fe97aa5 Merge branch 'release/0.0.4.2' 2020-01-02 10:25:26 +09:00
taizan-hokuto
28e3289790 Merge branch 'release/0.0.4.2' into develop 2020-01-02 10:25:06 +09:00
taizan-hokuto
585a4be7dc Increment version 2020-01-02 10:22:29 +09:00
taizan-hokuto
b84a82341e Fix README 2020-01-01 21:13:39 +09:00
taizan-hokuto
b4f3307b1c Fix comment 2020-01-01 20:08:07 +09:00
taizan-hokuto
be7ac97c62 Modify tuple comprehension 2019-12-31 01:01:27 +09:00
taizan-hokuto
f8de4e7e39 Merge branch 'hotfix' into develop 2019-12-30 19:12:17 +09:00
taizan-hokuto
ac0f052aa0 Merge branch 'hotfix' 2019-12-30 19:11:39 +09:00
taizan-hokuto
1cc0338a8e Export DefaultProcessor 2019-12-30 19:09:12 +09:00
taizan-hokuto
f6b8229998 Merge branch 'release' 2019-12-30 18:33:41 +09:00
taizan-hokuto
f8bcc8a453 Merge branch 'release' into develop 2019-12-30 18:32:43 +09:00
taizan-hokuto
f24c5f9e30 Increment version 2019-12-30 18:32:17 +09:00
taizan-hokuto
5268961854 Delete unnecessary lines of old logger 2019-12-30 17:54:53 +09:00
taizan-hokuto
733f754e11 Merge branch 'feature/fix_replaychat_is_alive' into develop 2019-12-30 17:32:50 +09:00
taizan-hokuto
582d0b749d Add comment 2019-12-30 17:32:23 +09:00
taizan-hokuto
b8bc00d880 Fix README 2019-12-30 17:27:25 +09:00
taizan-hokuto
ce96d94e23 Fix termination of fetching chat data 2019-12-30 17:10:34 +09:00
taizan-hokuto
7af92f14c0 Merge branch 'feature/config_logger' into develop 2019-12-30 15:01:32 +09:00
taizan-hokuto
7305e4178b Change description of getting logger 2019-12-30 14:38:02 +09:00
taizan-hokuto
a835d58e10 Merge branch 'feature/combinator' into develop 2019-12-30 11:04:25 +09:00
taizan-hokuto
4e956b8d84 Implement Combinator, DummyProcessor 2019-12-30 11:02:29 +09:00
taizan-hokuto
c4f1194a53 Merge branch 'feature/fix_massage_ex' into develop 2019-12-30 10:36:22 +09:00
taizan-hokuto
90b10a9f8f Integrate rendering message and message_ex 2019-12-27 02:19:08 +09:00
taizan-hokuto
b576c3f928 Merge branch 'develop' 2019-12-24 00:57:06 +09:00
taizan-hokuto
c0728e1366 Increment version 2019-12-24 00:55:00 +09:00
taizan-hokuto
fff09d4c27 Fix README 2019-12-24 00:52:53 +09:00
taizan-hokuto
810b6c8c6b Fix README 2019-12-24 00:29:10 +09:00
taizan-hokuto
dfada86caf Merge branch 'develop' 2019-12-22 02:56:54 +09:00
taizan-hokuto
91aa944df5 Increment version 2019-12-22 02:54:20 +09:00
taizan-hokuto
6ac5191e85 Change debug mode 2019-12-22 01:58:42 +09:00
taizan-hokuto
fff3e0371f Export SpeedCalculator 2019-12-22 01:39:35 +09:00
taizan-hokuto
a70efe8a67 Fix comments 2019-12-21 23:42:51 +09:00
taizan-hokuto
dc47f4debe Fix README 2019-12-21 22:40:08 +09:00
taizan-hokuto
ab5a2a8df2 Fix syntax error 2019-12-21 20:12:30 +09:00
taizan-hokuto
5a79f26fa7 Fix calculation algorithm 2019-12-21 20:06:55 +09:00
taizan-hokuto
18666199b7 Add test SpeedCalculator 2019-12-21 02:13:15 +09:00
taizan-hokuto
b357bccb98 Add test json 2019-12-20 23:57:10 +09:00
taizan-hokuto
3c1f079d5f Extends ChatProcessor explicitly 2019-12-20 21:33:32 +09:00
taizan-hokuto
289841a000 Export api speed_calculator 2019-12-20 01:13:50 +09:00
taizan-hokuto
ee0ff7fe74 Merge branch 'develop' 2019-12-19 23:14:37 +09:00
taizan-hokuto
c0870ce537 Fix README 2019-12-19 23:14:05 +09:00
taizan-hokuto
de6ef2490e Merge branch 'develop' 2019-12-19 02:03:09 +09:00
taizan-hokuto
b8bdbdc36f Increment version 2019-12-19 02:02:32 +09:00
taizan-hokuto
9f5d3f323e Fix README 2019-12-19 02:00:41 +09:00
taizan-hokuto
cf9aae3322 Merge branch 'develop' 2019-12-19 01:57:08 +09:00
taizan-hokuto
6ac2315936 Increment version 2019-12-19 01:53:55 +09:00
taizan-hokuto
50c8e34080 Fix README 2019-12-19 01:51:31 +09:00
taizan-hokuto
2d3da91d51 Fix calculation algorithm 2019-12-19 01:21:49 +09:00
taizan-hokuto
3ac71985ff Implement SpeedCalculator 2019-12-17 21:29:20 +09:00
taizan-hokuto
13bdf0376b Merge branch 'develop' 2019-12-01 22:58:53 +09:00
taizan-hokuto
b2ffdaec0c Increment version 2019-12-01 22:54:43 +09:00
taizan-hokuto
c85786679f Merge branch 'feature/1' into develop 2019-12-01 22:50:06 +09:00
taizan-hokuto
c7a7886672 Fix superSticker rendering 2019-12-01 22:47:01 +09:00
taizan-hokuto
12996fb44d Merge branch 'feature/1' 2019-11-22 00:17:15 +09:00
taizan-hokuto
c884ef7288 Merge branch 'feature/1' into develop 2019-11-22 00:16:20 +09:00
taizan-hokuto
2cd9e98fc2 Increment version 2019-11-22 00:15:57 +09:00
taizan-hokuto
2ac4c99ab4 Increment version 2019-11-22 00:04:18 +09:00
taizan-hokuto
51bf8ad738 Update README 2019-11-21 23:37:35 +09:00
taizan-hokuto
2e70e74bcd Update README 2019-11-21 23:04:12 +09:00
taizan-hokuto
a39d6cb420 Use list comprehension 2019-11-21 22:46:15 +09:00
taizan-hokuto
5dd0cb45b7 Implement messageEx 2019-11-21 22:35:27 +09:00
taizan-hokuto
24873651a6 Fix comments 2019-11-21 20:47:42 +09:00
taizan-hokuto
0e060bf998 Use logger when errors occur 2019-11-20 23:59:16 +09:00
taizan-hokuto
817fed9d1d Make functions private. 2019-11-19 20:53:37 +09:00
taizan-hokuto
823f7fefa4 Fix comments 2019-11-19 20:36:54 +09:00
taizan-hokuto
aa894fc52b Fix comments 2019-11-15 00:58:36 +09:00
120 changed files with 71260 additions and 7625 deletions

27
.github/workflows/run_test.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
name: Run All UnitTest
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python-version: [3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r requirements_test.txt
- name: Test with pytest
run: |
export PYTHONPATH=./
pytest --verbose --color=yes

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2020 taizan-hokuto
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,3 +1,5 @@
include requirements.txt include requirements.txt
include requirements_test.txt include requirements_test.txt
include README.md
global-exclude tests/*
global-exclude pytchat/testrun*.py

13
Pipfile Normal file
View File

@@ -0,0 +1,13 @@
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
httpx = {extras = ["http2"], version = "0.16.1"}
[dev-packages]
pytest-mock = "*"
pytest-httpx = "*"
wheel = "*"
twine = "*"

424
Pipfile.lock generated Normal file
View File

@@ -0,0 +1,424 @@
{
"_meta": {
"hash": {
"sha256": "e1eb34f14c75998519a90838b283ccd23bd168afa8e4837f956c5c4df66376f9"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"certifi": {
"hashes": [
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.12.5"
},
"h11": {
"hashes": [
"sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6",
"sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"
],
"version": "==0.12.0"
},
"h2": {
"hashes": [
"sha256:61e0f6601fa709f35cdb730863b4e5ec7ad449792add80d1410d4174ed139af5",
"sha256:875f41ebd6f2c44781259005b157faed1a5031df3ae5aa7bcb4628a6c0782f14"
],
"version": "==3.2.0"
},
"hpack": {
"hashes": [
"sha256:0edd79eda27a53ba5be2dfabf3b15780928a0dff6eb0c60a3d6767720e970c89",
"sha256:8eec9c1f4bfae3408a3f30500261f7e6a65912dc138526ea054f9ad98892e9d2"
],
"version": "==3.0.0"
},
"httpcore": {
"hashes": [
"sha256:37ae835fb370049b2030c3290e12ed298bf1473c41bb72ca4aa78681eba9b7c9",
"sha256:93e822cd16c32016b414b789aeff4e855d0ccbfc51df563ee34d4dbadbb3bcdc"
],
"version": "==0.12.3"
},
"httpx": {
"extras": [
"http2"
],
"hashes": [
"sha256:126424c279c842738805974687e0518a94c7ae8d140cd65b9c4f77ac46ffa537",
"sha256:9cffb8ba31fac6536f2c8cde30df859013f59e4bcc5b8d43901cb3654a8e0a5b"
],
"index": "pypi",
"version": "==0.16.1"
},
"hyperframe": {
"hashes": [
"sha256:5187962cb16dcc078f23cb5a4b110098d546c3f41ff2d4038a9896893bbd0b40",
"sha256:a9f5c17f2cc3c719b917c4f33ed1c61bd1f8dfac4b1bd23b7c80b3400971b41f"
],
"version": "==5.2.0"
},
"idna": {
"hashes": [
"sha256:5205d03e7bcbb919cc9c19885f9920d622ca52448306f2377daede5cf3faac16",
"sha256:c5b02147e01ea9920e6b0a3f1f7bb833612d507592c837a6c49552768f4054e1"
],
"version": "==3.1"
},
"rfc3986": {
"extras": [
"idna2008"
],
"hashes": [
"sha256:112398da31a3344dc25dbf477d8df6cb34f9278a94fee2625d89e4514be8bb9d",
"sha256:af9147e9aceda37c91a05f4deb128d4b4b49d6b199775fd2d2927768abdc8f50"
],
"version": "==1.4.0"
},
"sniffio": {
"hashes": [
"sha256:471b71698eac1c2112a40ce2752bb2f4a4814c22a54a3eed3676bc0f5ca9f663",
"sha256:c4666eecec1d3f50960c6bdf61ab7bc350648da6c126e3cf6898d8cd4ddcd3de"
],
"version": "==1.2.0"
}
},
"develop": {
"attrs": {
"hashes": [
"sha256:31b2eced602aa8423c2aea9c76a724617ed67cf9513173fd3a4f03e3a929c7e6",
"sha256:832aa3cde19744e49938b91fea06d69ecb9e649c93ba974535d08ad92164f700"
],
"version": "==20.3.0"
},
"bleach": {
"hashes": [
"sha256:6123ddc1052673e52bab52cdc955bcb57a015264a1c57d37bea2f6b817af0125",
"sha256:98b3170739e5e83dd9dc19633f074727ad848cbedb6026708c8ac2d3b697a433"
],
"index": "pypi",
"version": "==3.3.0"
},
"certifi": {
"hashes": [
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.12.5"
},
"cffi": {
"hashes": [
"sha256:00a1ba5e2e95684448de9b89888ccd02c98d512064b4cb987d48f4b40aa0421e",
"sha256:00e28066507bfc3fe865a31f325c8391a1ac2916219340f87dfad602c3e48e5d",
"sha256:045d792900a75e8b1e1b0ab6787dd733a8190ffcf80e8c8ceb2fb10a29ff238a",
"sha256:0638c3ae1a0edfb77c6765d487fee624d2b1ee1bdfeffc1f0b58c64d149e7eec",
"sha256:105abaf8a6075dc96c1fe5ae7aae073f4696f2905fde6aeada4c9d2926752362",
"sha256:155136b51fd733fa94e1c2ea5211dcd4c8879869008fc811648f16541bf99668",
"sha256:1a465cbe98a7fd391d47dce4b8f7e5b921e6cd805ef421d04f5f66ba8f06086c",
"sha256:1d2c4994f515e5b485fd6d3a73d05526aa0fcf248eb135996b088d25dfa1865b",
"sha256:2c24d61263f511551f740d1a065eb0212db1dbbbbd241db758f5244281590c06",
"sha256:51a8b381b16ddd370178a65360ebe15fbc1c71cf6f584613a7ea08bfad946698",
"sha256:594234691ac0e9b770aee9fcdb8fa02c22e43e5c619456efd0d6c2bf276f3eb2",
"sha256:5cf4be6c304ad0b6602f5c4e90e2f59b47653ac1ed9c662ed379fe48a8f26b0c",
"sha256:64081b3f8f6f3c3de6191ec89d7dc6c86a8a43911f7ecb422c60e90c70be41c7",
"sha256:6bc25fc545a6b3d57b5f8618e59fc13d3a3a68431e8ca5fd4c13241cd70d0009",
"sha256:798caa2a2384b1cbe8a2a139d80734c9db54f9cc155c99d7cc92441a23871c03",
"sha256:7c6b1dece89874d9541fc974917b631406233ea0440d0bdfbb8e03bf39a49b3b",
"sha256:7ef7d4ced6b325e92eb4d3502946c78c5367bc416398d387b39591532536734e",
"sha256:840793c68105fe031f34d6a086eaea153a0cd5c491cde82a74b420edd0a2b909",
"sha256:8d6603078baf4e11edc4168a514c5ce5b3ba6e3e9c374298cb88437957960a53",
"sha256:9cc46bc107224ff5b6d04369e7c595acb700c3613ad7bcf2e2012f62ece80c35",
"sha256:9f7a31251289b2ab6d4012f6e83e58bc3b96bd151f5b5262467f4bb6b34a7c26",
"sha256:9ffb888f19d54a4d4dfd4b3f29bc2c16aa4972f1c2ab9c4ab09b8ab8685b9c2b",
"sha256:a5ed8c05548b54b998b9498753fb9cadbfd92ee88e884641377d8a8b291bcc01",
"sha256:a7711edca4dcef1a75257b50a2fbfe92a65187c47dab5a0f1b9b332c5919a3fb",
"sha256:af5c59122a011049aad5dd87424b8e65a80e4a6477419c0c1015f73fb5ea0293",
"sha256:b18e0a9ef57d2b41f5c68beefa32317d286c3d6ac0484efd10d6e07491bb95dd",
"sha256:b4e248d1087abf9f4c10f3c398896c87ce82a9856494a7155823eb45a892395d",
"sha256:ba4e9e0ae13fc41c6b23299545e5ef73055213e466bd107953e4a013a5ddd7e3",
"sha256:c6332685306b6417a91b1ff9fae889b3ba65c2292d64bd9245c093b1b284809d",
"sha256:d5ff0621c88ce83a28a10d2ce719b2ee85635e85c515f12bac99a95306da4b2e",
"sha256:d9efd8b7a3ef378dd61a1e77367f1924375befc2eba06168b6ebfa903a5e59ca",
"sha256:df5169c4396adc04f9b0a05f13c074df878b6052430e03f50e68adf3a57aa28d",
"sha256:ebb253464a5d0482b191274f1c8bf00e33f7e0b9c66405fbffc61ed2c839c775",
"sha256:ec80dc47f54e6e9a78181ce05feb71a0353854cc26999db963695f950b5fb375",
"sha256:f032b34669220030f905152045dfa27741ce1a6db3324a5bc0b96b6c7420c87b",
"sha256:f60567825f791c6f8a592f3c6e3bd93dd2934e3f9dac189308426bd76b00ef3b",
"sha256:f803eaa94c2fcda012c047e62bc7a51b0bdabda1cad7a92a522694ea2d76e49f"
],
"version": "==1.14.4"
},
"chardet": {
"hashes": [
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
],
"version": "==4.0.0"
},
"colorama": {
"hashes": [
"sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b",
"sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"
],
"version": "==0.4.4"
},
"cryptography": {
"hashes": [
"sha256:0003a52a123602e1acee177dc90dd201f9bb1e73f24a070db7d36c588e8f5c7d",
"sha256:0e85aaae861d0485eb5a79d33226dd6248d2a9f133b81532c8f5aae37de10ff7",
"sha256:594a1db4511bc4d960571536abe21b4e5c3003e8750ab8365fafce71c5d86901",
"sha256:69e836c9e5ff4373ce6d3ab311c1a2eed274793083858d3cd4c7d12ce20d5f9c",
"sha256:788a3c9942df5e4371c199d10383f44a105d67d401fb4304178020142f020244",
"sha256:7e177e4bea2de937a584b13645cab32f25e3d96fc0bc4a4cf99c27dc77682be6",
"sha256:83d9d2dfec70364a74f4e7c70ad04d3ca2e6a08b703606993407bf46b97868c5",
"sha256:84ef7a0c10c24a7773163f917f1cb6b4444597efd505a8aed0a22e8c4780f27e",
"sha256:9e21301f7a1e7c03dbea73e8602905a4ebba641547a462b26dd03451e5769e7c",
"sha256:9f6b0492d111b43de5f70052e24c1f0951cb9e6022188ebcb1cc3a3d301469b0",
"sha256:a69bd3c68b98298f490e84519b954335154917eaab52cf582fa2c5c7efc6e812",
"sha256:b4890d5fb9b7a23e3bf8abf5a8a7da8e228f1e97dc96b30b95685df840b6914a",
"sha256:c366df0401d1ec4e548bebe8f91d55ebcc0ec3137900d214dd7aac8427ef3030",
"sha256:dc42f645f8f3a489c3dd416730a514e7a91a59510ddaadc09d04224c098d3302"
],
"version": "==3.3.1"
},
"docutils": {
"hashes": [
"sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af",
"sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc"
],
"version": "==0.16"
},
"h11": {
"hashes": [
"sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6",
"sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"
],
"version": "==0.12.0"
},
"httpcore": {
"hashes": [
"sha256:37ae835fb370049b2030c3290e12ed298bf1473c41bb72ca4aa78681eba9b7c9",
"sha256:93e822cd16c32016b414b789aeff4e855d0ccbfc51df563ee34d4dbadbb3bcdc"
],
"version": "==0.12.3"
},
"httpx": {
"extras": [
"http2"
],
"hashes": [
"sha256:126424c279c842738805974687e0518a94c7ae8d140cd65b9c4f77ac46ffa537",
"sha256:9cffb8ba31fac6536f2c8cde30df859013f59e4bcc5b8d43901cb3654a8e0a5b"
],
"index": "pypi",
"version": "==0.16.1"
},
"idna": {
"hashes": [
"sha256:5205d03e7bcbb919cc9c19885f9920d622ca52448306f2377daede5cf3faac16",
"sha256:c5b02147e01ea9920e6b0a3f1f7bb833612d507592c837a6c49552768f4054e1"
],
"version": "==3.1"
},
"iniconfig": {
"hashes": [
"sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3",
"sha256:bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32"
],
"version": "==1.1.1"
},
"jeepney": {
"hashes": [
"sha256:7d59b6622675ca9e993a6bd38de845051d315f8b0c72cca3aef733a20b648657",
"sha256:aec56c0eb1691a841795111e184e13cad504f7703b9a64f63020816afa79a8ae"
],
"markers": "sys_platform == 'linux'",
"version": "==0.6.0"
},
"keyring": {
"hashes": [
"sha256:9acb3e1452edbb7544822b12fd25459078769e560fa51f418b6d00afaa6178df",
"sha256:9f44660a5d4931bdc14c08a1d01ef30b18a7a8147380710d8c9f9531e1f6c3c0"
],
"version": "==22.0.1"
},
"packaging": {
"hashes": [
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
],
"version": "==20.9"
},
"pkginfo": {
"hashes": [
"sha256:029a70cb45c6171c329dfc890cde0879f8c52d6f3922794796e06f577bb03db4",
"sha256:9fdbea6495622e022cc72c2e5e1b735218e4ffb2a2a69cde2694a6c1f16afb75"
],
"version": "==1.7.0"
},
"pluggy": {
"hashes": [
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
],
"version": "==0.13.1"
},
"py": {
"hashes": [
"sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
"sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
],
"version": "==1.10.0"
},
"pycparser": {
"hashes": [
"sha256:2d475327684562c3a96cc71adf7dc8c4f0565175cf86b6d7a404ff4c771f15f0",
"sha256:7582ad22678f0fcd81102833f60ef8d0e57288b6b5fb00323d101be910e35705"
],
"version": "==2.20"
},
"pygments": {
"hashes": [
"sha256:bc9591213a8f0e0ca1a5e68a479b4887fdc3e75d0774e5c71c31920c427de435",
"sha256:df49d09b498e83c1a73128295860250b0b7edd4c723a32e9bc0d295c7c2ec337"
],
"version": "==2.7.4"
},
"pyparsing": {
"hashes": [
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
],
"version": "==2.4.7"
},
"pytest": {
"hashes": [
"sha256:9d1edf9e7d0b84d72ea3dbcdfd22b35fb543a5e8f2a60092dd578936bf63d7f9",
"sha256:b574b57423e818210672e07ca1fa90aaf194a4f63f3ab909a2c67ebb22913839"
],
"version": "==6.2.2"
},
"pytest-httpx": {
"hashes": [
"sha256:0a7c56e559b23efbf857054cd74de60a7c540694a162423f89c70da6ad358d8e",
"sha256:d32e8f6fb7e028f0313f5f5a2d463c8673eb43fd11a9bfe8527299717a7764c4"
],
"index": "pypi",
"version": "==0.10.1"
},
"pytest-mock": {
"hashes": [
"sha256:024e405ad382646318c4281948aadf6fe1135632bea9cc67366ea0c4098ef5f2",
"sha256:a4d6d37329e4a893e77d9ffa89e838dd2b45d5dc099984cf03c703ac8411bb82"
],
"index": "pypi",
"version": "==3.3.1"
},
"readme-renderer": {
"hashes": [
"sha256:267854ac3b1530633c2394ead828afcd060fc273217c42ac36b6be9c42cd9a9d",
"sha256:6b7e5aa59210a40de72eb79931491eaf46fefca2952b9181268bd7c7c65c260a"
],
"version": "==28.0"
},
"requests": {
"hashes": [
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
],
"version": "==2.25.1"
},
"requests-toolbelt": {
"hashes": [
"sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f",
"sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"
],
"version": "==0.9.1"
},
"rfc3986": {
"extras": [
"idna2008"
],
"hashes": [
"sha256:112398da31a3344dc25dbf477d8df6cb34f9278a94fee2625d89e4514be8bb9d",
"sha256:af9147e9aceda37c91a05f4deb128d4b4b49d6b199775fd2d2927768abdc8f50"
],
"version": "==1.4.0"
},
"secretstorage": {
"hashes": [
"sha256:30cfdef28829dad64d6ea1ed08f8eff6aa115a77068926bcc9f5225d5a3246aa",
"sha256:5c36f6537a523ec5f969ef9fad61c98eb9e017bc601d811e53aa25bece64892f"
],
"markers": "sys_platform == 'linux'",
"version": "==3.3.0"
},
"six": {
"hashes": [
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
],
"version": "==1.15.0"
},
"sniffio": {
"hashes": [
"sha256:471b71698eac1c2112a40ce2752bb2f4a4814c22a54a3eed3676bc0f5ca9f663",
"sha256:c4666eecec1d3f50960c6bdf61ab7bc350648da6c126e3cf6898d8cd4ddcd3de"
],
"version": "==1.2.0"
},
"toml": {
"hashes": [
"sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
],
"version": "==0.10.2"
},
"tqdm": {
"hashes": [
"sha256:4621f6823bab46a9cc33d48105753ccbea671b68bab2c50a9f0be23d4065cb5a",
"sha256:fe3d08dd00a526850568d542ff9de9bbc2a09a791da3c334f3213d8d0bbbca65"
],
"version": "==4.56.0"
},
"twine": {
"hashes": [
"sha256:34352fd52ec3b9d29837e6072d5a2a7c6fe4290e97bba46bb8d478b5c598f7ab",
"sha256:ba9ff477b8d6de0c89dd450e70b2185da190514e91c42cc62f96850025c10472"
],
"index": "pypi",
"version": "==3.2.0"
},
"urllib3": {
"hashes": [
"sha256:1b465e494e3e0d8939b50680403e3aedaa2bc434b7d5af64dfd3c958d7f5ae80",
"sha256:de3eedaad74a2683334e282005cd8d7f22f4d55fa690a2a1020a416cb0a47e73"
],
"version": "==1.26.3"
},
"webencodings": {
"hashes": [
"sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78",
"sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"
],
"version": "==0.5.1"
},
"wheel": {
"hashes": [
"sha256:906864fb722c0ab5f2f9c35b2c65e3af3c009402c108a709c0aca27bc2c9187b",
"sha256:aaef9b8c36db72f8bf7f1e54f85f875c4d466819940863ca0b3f3f77f0a1646f"
],
"index": "pypi",
"version": "==0.36.1"
}
}
}

137
README.md
View File

@@ -2,120 +2,66 @@ pytchat
======= =======
pytchat is a python library for fetching youtube live chat. pytchat is a python library for fetching youtube live chat.
<br><br><br>
## Description ## Description
pytchat is a python library for fetching youtube live chat pytchat is a python library for fetching youtube live chat
without using youtube api, Selenium or BeautifulSoup. without using Selenium or BeautifulSoup.
Other features: Other features:
+ Customizable chat data processors including yt api compatible one. + Customizable [chat data processors](https://github.com/taizan-hokuto/pytchat/wiki/ChatProcessor) including youtube api compatible one.
+ Available on asyncio context. + Available on asyncio context.
+ Quick fetching of initial chat data by generating continuation params + Quick fetching of initial chat data by generating continuation params
instead of web scraping. instead of web scraping.
For more detailed information, see [wiki](https://github.com/taizan-hokuto/pytchat/wiki). For more detailed information, see [wiki](https://github.com/taizan-hokuto/pytchat/wiki). <br>
[wiki (Japanese)](https://github.com/taizan-hokuto/pytchat/wiki/Home_jp)
## Install ## Install
```python ```python
pip install pytchat pip install pytchat
``` ```
## Demo
![demo](https://taizan-hokuto.github.io/statics/demo.gif "demo")
## Examples ## Examples
### on-demand mode
```python
from pytchat import LiveChat
chat = LiveChat("G1w62uEMZ74")
### Fetch chat data (see [wiki](https://github.com/taizan-hokuto/pytchat/wiki/PytchatCore))
```python
import pytchat
chat = pytchat.create(video_id="uIx8l2xlYVY")
while chat.is_alive(): while chat.is_alive():
data = chat.get() for c in chat.get().sync_items():
for c in data.items: print(f"{c.datetime} [{c.author.name}]- {c.message}")
print(f"{c.datetime} [{c.author.name}]-{c.message} {c.amountString}")
data.tick()
``` ```
### callback mode
### Output JSON format string (feature of [DefaultProcessor](https://github.com/taizan-hokuto/pytchat/wiki/DefaultProcessor))
```python ```python
from pytchat import LiveChat import pytchat
import time import time
chat = LiveChat("G1w62uEMZ74", callback = func) chat = pytchat.create(video_id="uIx8l2xlYVY")
while chat.is_alive(): while chat.is_alive():
#other background operation here. print(chat.get().json())
time.sleep(3) time.sleep(5)
'''
def func(data): # Each chat item can also be output in JSON format.
for c in data.items: for c in chat.get().items:
print(f"{c.datetime} [{c.author.name}]-{c.message} {c.amountString}") print(c.json())
data.tick() '''
```
### asyncio context:
```python
from pytchat import LiveChatAsync
import asyncio
async def main():
chat = LiveChatAsync("G1w62uEMZ74", callback = func)
while chat.is_alive():
#other background operation here.
await asyncio.sleep(3)
async def func(data):
for c in data.items:
print(f"{c.datetime} [{c.author.name}]-{c.message} {c.amountString}")
await data.tick_async()
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
``` ```
### youtube api compatible processor: ### other
```python + Fetch chat with a buffer ([LiveChat](https://github.com/taizan-hokuto/pytchat/wiki/LiveChat))
from pytchat import LiveChat, CompatibleProcessor
chat = LiveChat("G1w62uEMZ74", + Use with asyncio ([LiveChatAsync](https://github.com/taizan-hokuto/pytchat/wiki/LiveChatAsync))
processor = CompatibleProcessor() )
while chat.is_alive(): + YT API compatible chat processor ([CompatibleProcessor](https://github.com/taizan-hokuto/pytchat/wiki/CompatibleProcessor))
data = chat.get()
polling = data["pollingIntervalMillis"]/1000
for c in data["items"]:
if c.get("snippet"):
print(f"[{c['authorDetails']['displayName']}]"
f"-{c['snippet']['displayMessage']}")
time.sleep(polling/len(data["items"]))
```
### replay:
```python
from pytchat import ReplayChatAsync
import asyncio
async def main(): ## Structure of Default Processor
chat = ReplayChatAsync("G1w62uEMZ74", seektime = 1000, callback = func) Each item can be got with `sync_items()` function.
while chat.is_alive():
#other background operation here.
await asyncio.sleep(3)
async def func(data):
for count in range(0,len(data.items)):
c= data.items[count]
if count!=len(data.items):
tick=data.items[count+1].timestamp -data.items[count].timestamp
else:
tick=0
print(f"<{c.timestampText}> [{c.author.name}]-{c.message} {c.amountString}")
await asyncio.sleep(tick/1000)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
```
## Chatdata Structure of Default Processor
Structure of each item which got from items() function.
<table> <table>
<tr> <tr>
<th>name</th> <th>name</th>
@@ -137,6 +83,11 @@ Structure of each item which got from items() function.
<td>str</td> <td>str</td>
<td>emojis are represented by ":(shortcut text):"</td> <td>emojis are represented by ":(shortcut text):"</td>
</tr> </tr>
<tr>
<td>messageEx</td>
<td>str</td>
<td>list of message texts and emoji dicts(id, txt, url).</td>
</tr>
<tr> <tr>
<td>timestamp</td> <td>timestamp</td>
<td>int</td> <td>int</td>
@@ -145,26 +96,26 @@ Structure of each item which got from items() function.
<tr> <tr>
<td>datetime</td> <td>datetime</td>
<td>str</td> <td>str</td>
<td>ex. "2019-10-10 12:34:56"</td> <td>e.g. "2019-10-10 12:34:56"</td>
</tr> </tr>
<td>timestampText</td> <td>elapsedTime</td>
<td>str</td> <td>str</td>
<td>elapsed time. (ex. "1:02:27")</td> <td>elapsed time. (e.g. "1:02:27") *Replay Only.</td>
</tr> </tr>
<tr> <tr>
<td>amountValue</td> <td>amountValue</td>
<td>float</td> <td>float</td>
<td>ex. 1,234.0</td> <td>e.g. 1,234.0</td>
</tr> </tr>
<tr> <tr>
<td>amountString</td> <td>amountString</td>
<td>str</td> <td>str</td>
<td>ex. "$ 1,234"</td> <td>e.g. "$ 1,234"</td>
</tr> </tr>
<tr> <tr>
<td>currency</td> <td>currency</td>
<td>str</td> <td>str</td>
<td><a href="https://en.wikipedia.org/wiki/ISO_4217">ISO 4217 currency codes</a> (ex. "USD")</td> <td><a href="https://en.wikipedia.org/wiki/ISO_4217">ISO 4217 currency codes</a> (e.g. "USD")</td>
</tr> </tr>
<tr> <tr>
<td>bgColor</td> <td>bgColor</td>
@@ -193,7 +144,7 @@ Structure of author object.
<tr> <tr>
<td>channelId</td> <td>channelId</td>
<td>str</td> <td>str</td>
<td></td> <td>*chatter's channel ID.</td>
</tr> </tr>
<tr> <tr>
<td>channelUrl</td> <td>channelUrl</td>
@@ -236,8 +187,4 @@ Structure of author object.
[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE) [![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
## Author
[taizan-hokuto](https://github.com/taizan-hokuto)
[twitter:@taizan205](https://twitter.com/taizan205)

View File

@@ -1,22 +1,44 @@
""" """
pytchat is a python library for fetching youtube live chat without using yt api, Selenium, or BeautifulSoup. pytchat is a lightweight python library to browse youtube livechat without Selenium or BeautifulSoup.
""" """
__copyright__ = 'Copyright (C) 2019 taizan-hokuto' __copyright__ = 'Copyright (C) 2019, 2020 taizan-hokuto'
__version__ = '0.0.3.1' __version__ = '0.5.3'
__license__ = 'MIT' __license__ = 'MIT'
__author__ = 'taizan-hokuto' __author__ = 'taizan-hokuto'
__author_email__ = '55448286+taizan-hokuto@users.noreply.github.com' __author_email__ = '55448286+taizan-hokuto@users.noreply.github.com'
__url__ = 'https://github.com/taizan-hokuto/pytchat' __url__ = 'https://github.com/taizan-hokuto/pytchat'
__all__ = ["core_async","core_multithread","processors"]
from .exceptions import (
ChatParseException,
ResponseContextError,
NoContents,
NoContinuation,
IllegalFunctionCall,
InvalidVideoIdException,
UnknownConnectionError,
RetryExceedMaxCount,
ChatDataFinished,
ReceivedUnknownContinuation,
FailedExtractContinuation,
VideoInfoParseError,
PatternUnmatchError
)
from .api import ( from .api import (
config,
LiveChat, LiveChat,
LiveChatAsync, LiveChatAsync,
ReplayChat,
ReplayChatAsync,
ChatProcessor, ChatProcessor,
CompatibleProcessor, CompatibleProcessor,
DummyProcessor,
DefaultProcessor,
HTMLArchiver,
TSVArchiver,
JsonfileArchiver,
SimpleDisplayProcessor, SimpleDisplayProcessor,
JsonfileArchiveProcessor SpeedCalculator,
) SuperchatCalculator,
create
)
# flake8: noqa

View File

@@ -1,10 +1,34 @@
from . import config
from .core import create
from .core_multithread.livechat import LiveChat from .core_multithread.livechat import LiveChat
from .core_async.livechat import LiveChatAsync from .core_async.livechat import LiveChatAsync
from .core_multithread.replaychat import ReplayChat
from .core_async.replaychat import ReplayChatAsync
from .processors.chat_processor import ChatProcessor from .processors.chat_processor import ChatProcessor
from .processors.default.processor import DefaultProcessor
from .processors.compatible.processor import CompatibleProcessor from .processors.compatible.processor import CompatibleProcessor
from .processors.default.processor import DefaultProcessor
from .processors.dummy_processor import DummyProcessor
from .processors.html_archiver import HTMLArchiver
from .processors.tsv_archiver import TSVArchiver
from .processors.jsonfile_archiver import JsonfileArchiver
from .processors.simple_display_processor import SimpleDisplayProcessor from .processors.simple_display_processor import SimpleDisplayProcessor
from .processors.jsonfile_archive_processor import JsonfileArchiveProcessor from .processors.speed.calculator import SpeedCalculator
from .processors.superchat.calculator import SuperchatCalculator
__all__ = [
config,
LiveChat,
LiveChatAsync,
ChatProcessor,
CompatibleProcessor,
DummyProcessor,
DefaultProcessor,
HTMLArchiver,
TSVArchiver,
JsonfileArchiver,
SimpleDisplayProcessor,
SpeedCalculator,
SuperchatCalculator,
create
]
# flake8: noqa

View File

@@ -1,4 +1,14 @@
import logging import logging # noqa
LOGGER_MODE = logging.ERROR from . import mylogger
from base64 import a85decode as dc
headers = { headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36'} 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 Edg/86.0.622.63,gzip(gfe)',
}
_sml = dc(b"BQS?8F#ks-GB\\6`H#IhIF^eo7@rH3;H#IhIF^eor06T''Ch\\'(?XmbXF>%9<FC/iuG%G#jBOQ!ICLqcS5tQB2;gCZ)?UdXC;f$GR3)MM2<(0>O7mh!,G@+K5?SO9T@okV").decode()
_smr = dc(b"BQS?8F#ks-GB\\6`H#IhIF^eo7@rH3;H#IhIF^eor06T''Ch\\'(?XmbXF>%9<FC/iuG%G#jBOQ!iEb03+@<k(QAU-F)8U=fDGsP557S5F7CiNH7;)D3N77^*B6YU@\\?WfBr0emZX=#^").decode()
def logger(module_name: str, loglevel=None):
module_logger = mylogger.get_logger(module_name, loglevel=loglevel)
return module_logger

View File

@@ -0,0 +1,38 @@
from logging import NullHandler, getLogger, StreamHandler, FileHandler
import logging
from datetime import datetime
def get_logger(modname, loglevel=logging.DEBUG):
logger = getLogger(modname)
if loglevel is None:
logger.addHandler(NullHandler())
return logger
logger.setLevel(loglevel)
# create handler1 for showing info
handler1 = StreamHandler()
my_formatter = MyFormatter()
handler1.setFormatter(my_formatter)
handler1.setLevel(loglevel)
logger.addHandler(handler1)
# create handler2 for recording log file
if loglevel <= logging.DEBUG:
handler2 = FileHandler(filename="log.txt", encoding='utf-8')
handler2.setLevel(logging.ERROR)
handler2.setFormatter(my_formatter)
logger.addHandler(handler2)
return logger
class MyFormatter(logging.Formatter):
def format(self, record):
timestamp = (
datetime.fromtimestamp(record.created)).strftime("%m-%d %H:%M:%S")
module = (record.module).ljust(15)
funcname = (record.funcName).ljust(18)
lineno = str(record.lineno).rjust(4)
message = record.getMessage()
return timestamp + '| ' + module + ' { ' + funcname + ':' + lineno + '} - ' + message

7
pytchat/core/__init__.py Normal file
View File

@@ -0,0 +1,7 @@
from .pytchat import PytchatCore
from .. util import extract_video_id
def create(video_id: str, **kwargs):
_vid = extract_video_id(video_id)
return PytchatCore(_vid, **kwargs)

216
pytchat/core/pytchat.py Normal file
View File

@@ -0,0 +1,216 @@
import httpx
import json
import signal
import time
import traceback
from ..parser.live import Parser
from .. import config
from .. import exceptions
from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
from .. import util
headers = config.headers
MAX_RETRY = 10
class PytchatCore:
'''
Parameter
---------
video_id : str
seektime : int
start position of fetching chat (seconds).
This option is valid for archived chat only.
If negative value, chat data posted before the start of the broadcast
will be retrieved as well.
processor : ChatProcessor
interruptable : bool
Allows keyboard interrupts.
Set this parameter to False if your own threading program causes
the problem.
force_replay : bool
force to fetch archived chat data, even if specified video is live.
topchat_only : bool
If True, get only top chat.
hold_exception : bool [default:True]
If True, when exceptions occur, the exception is held internally,
and can be raised by raise_for_status().
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes
---------
_is_alive : bool
Flag to stop getting chat.
'''
def __init__(self, video_id,
seektime=-1,
processor=DefaultProcessor(),
interruptable=True,
force_replay=False,
topchat_only=False,
hold_exception=True,
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
else:
self.processor = processor
self._is_alive = True
self._is_replay = force_replay or (replay_continuation is not None)
self._hold_exception = hold_exception
self._exception_holder = None
self._parser = Parser(
is_replay=self._is_replay,
exception_holder=self._exception_holder
)
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self.continuation = replay_continuation
if interruptable:
signal.signal(signal.SIGINT, lambda a, b: self.terminate())
self._setup()
def _setup(self):
if not self.continuation:
time.sleep(0.1) # sleep shortly to prohibit skipping fetching data
"""Fetch first continuation parameter,
create and start _listen loop.
"""
self.continuation = liveparam.getparam(
self._video_id,
channel_id=util.get_channelid(httpx.Client(http2=True), self._video_id),
past_sec=3)
def _get_chat_component(self):
''' Fetch chat data and store them into buffer,
get next continuaiton parameter and loop.
Parameter
---------
continuation : str
parameter for next chat data
'''
try:
with httpx.Client(http2=True) as client:
if self.continuation and self._is_alive:
contents = self._get_contents(self.continuation, client, headers)
metadata, chatdata = self._parser.parse(contents)
timeout = metadata['timeoutMs'] / 1000
chat_component = {
"video_id": self._video_id,
"timeout": timeout,
"chatdata": chatdata
}
self.continuation = metadata.get('continuation')
self._last_offset_ms = metadata.get('last_offset_ms', 0)
return chat_component
except exceptions.ChatParseException as e:
self._logger.debug(f"[{self._video_id}]{str(e)}")
self._raise_exception(e)
except Exception as e:
self._logger.error(f"{traceback.format_exc(limit=-1)}")
self._raise_exception(e)
def _get_contents(self, continuation, client, headers):
'''Get 'continuationContents' from livechat json.
If contents is None at first fetching,
try to fetch archive chat data.
Return:
-------
'continuationContents' which includes metadata & chat data.
'''
livechat_json = (
self._get_livechat_json(continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = config._smr
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only, util.get_channelid(client, self._video_id))
livechat_json = self._get_livechat_json(continuation, client, replay=True, offset_ms=self.seektime * 1000)
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (self._get_livechat_json(
reload_continuation, client, headers))
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
'''
livechat_json = None
err = None
if offset_ms < 0:
offset_ms = 0
param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
for _ in range(MAX_RETRY + 1):
with httpx.Client(http2=True) as client:
try:
response = client.post(self._fetch_url, json=param)
livechat_json = json.loads(response.text)
break
except (json.JSONDecodeError, httpx.ConnectTimeout, httpx.ReadTimeout, httpx.ConnectError) as e:
err = e
time.sleep(2)
continue
else:
self._logger.error(f"[{self._video_id}]"
f"Exceeded retry count. Last error: {str(err)}")
self._raise_exception(exceptions.RetryExceedMaxCount())
return livechat_json
def get(self):
if self.is_alive():
chat_component = self._get_chat_component()
return self.processor.process([chat_component])
else:
return []
def is_replay(self):
return self._is_replay
def is_alive(self):
return self._is_alive
def terminate(self):
self._is_alive = False
self.processor.finalize()
def raise_for_status(self):
if self._exception_holder is not None:
raise self._exception_holder
def _raise_exception(self, exception: Exception = None):
self.terminate()
if self._hold_exception is False:
raise exception
self._exception_holder = exception

View File

@@ -1,28 +1,38 @@
import asyncio import asyncio
class Buffer(asyncio.Queue): class Buffer(asyncio.Queue):
''' '''
チャットデータを格納するバッファの役割を持つFIFOキュー Buffer for storing chat data.
Parameter Parameter
--------- ---------
maxsize : int maxsize : int
格納するチャットブロックの最大個数。0の場合は無限。 Maximum number of chat blocks to be stored.
最大値を超える場合は古いチャットブロックから破棄される。 If it exceeds the maximum, the oldest chat block will be discarded.
''' '''
def __init__(self,maxsize = 0):
def __init__(self, maxsize=0):
super().__init__(maxsize) super().__init__(maxsize)
async def put(self,item): async def put(self, item):
if item is None: if item is None:
return return
if super().full(): if super().full():
super().get_nowait() super().get_nowait()
await super().put(item) await super().put(item)
def put_nowait(self, item):
if item is None:
return
if super().full():
super().get_nowait()
super().put_nowait(item)
async def get(self): async def get(self):
ret = [] ret = []
ret.append(await super().get()) ret.append(await super().get())
while not super().empty(): while not super().empty():
ret.append(super().get_nowait()) ret.append(super().get_nowait())
return ret return ret

View File

@@ -1,298 +1,369 @@
import aiohttp, asyncio
import datetime import asyncio
import httpx
import json import json
import random
import signal import signal
import threading
import time import time
import traceback import traceback
import urllib.parse from asyncio import Queue
from aiohttp.client_exceptions import ClientConnectorError
from concurrent.futures import CancelledError from concurrent.futures import CancelledError
from .buffer import Buffer from .buffer import Buffer
from ..parser.live import Parser from ..parser.live import Parser
from .. import config from .. import config
from .. import mylogger from .. import exceptions
from ..exceptions import ChatParseException,IllegalFunctionCall from .. import util
from ..paramgen import liveparam from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
MAX_RETRY = 10
headers = config.headers headers = config.headers
MAX_RETRY = 10
class LiveChatAsync: class LiveChatAsync:
'''asyncio(aiohttp)を利用してYouTubeのライブ配信のチャットデータを取得する。 '''LiveChatAsync object fetches chat data and stores them
in a buffer with asyncio.
Parameter Parameter
--------- ---------
video_id : str video_id : str
動画ID
seektime : int
start position of fetching chat (seconds).
This option is valid for archived chat only.
If negative value, chat data posted before the start of the broadcast
will be retrieved as well.
processor : ChatProcessor processor : ChatProcessor
チャットデータを加工するオブジェクト
buffer : Buffer(maxsize:20[default]) buffer : Buffer
チャットデータchat_componentを格納するバッファ。 buffer of chat data fetched background.
maxsize : 格納できるchat_componentの個数
default値20個。1個で約5~10秒分。
interruptable : bool interruptable : bool
Ctrl+Cによる処理中断を行うかどうか。 Allows keyboard interrupts.
Set this parameter to False if your own threading program causes
the problem.
callback : func callback : func
_listen()関数から一定間隔で自動的に呼びだす関数。 function called periodically from _listen().
done_callback : func done_callback : func
listener終了時に呼び出すコールバック。 function called when listener ends.
exception_handler : func exception_handler : func
例外を処理する関数
direct_mode : bool direct_mode : bool
Trueの場合、bufferを使わずにcallbackを呼ぶ。 If True, invoke specified callback function without using buffer.
Trueの場合、callbackの設定が必須 callback is required. If not, IllegalFunctionCall will be raised.
(設定していない場合IllegalFunctionCall例外を発生させる
force_replay : bool
force to fetch archived chat data, even if specified video is live.
topchat_only : bool
If True, get only top chat.
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes Attributes
--------- ---------
_is_alive : bool _is_alive : bool
チャット取得を停止するためのフラグ Flag to stop getting chat.
''' '''
_setup_finished = False _setup_finished = False
def __init__(self, video_id, def __init__(self, video_id,
processor = DefaultProcessor(), seektime=-1,
buffer = None, processor=DefaultProcessor(),
interruptable = True, buffer=None,
callback = None, interruptable=True,
done_callback = None, callback=None,
exception_handler = None, done_callback=None,
direct_mode = False): exception_handler=None,
self.video_id = video_id direct_mode=False,
self.processor = processor force_replay=False,
topchat_only=False,
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
else:
self.processor = processor
self._buffer = buffer self._buffer = buffer
self._callback = callback self._callback = callback
self._done_callback = done_callback self._done_callback = done_callback
self._exception_handler = exception_handler self._exception_handler = exception_handler
self._direct_mode = direct_mode self._direct_mode = direct_mode
self._is_alive = True self._is_alive = True
self._parser = Parser() self._is_replay = force_replay or (replay_continuation is not None)
self._parser = Parser(is_replay=self._is_replay)
self._pauser = Queue()
self._pauser.put_nowait(None)
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self.exception = None
self.continuation = replay_continuation
LiveChatAsync._logger = logger
if exception_handler:
self._set_exception_handler(exception_handler)
if interruptable:
signal.signal(signal.SIGINT,
(lambda a, b: self._keyboard_interrupt()))
self._setup() self._setup()
if not LiveChatAsync._setup_finished:
LiveChatAsync._setup_finished = True
if exception_handler == None:
self._set_exception_handler(self._handle_exception)
else:
self._set_exception_handler(exception_handler)
if interruptable:
signal.signal(signal.SIGINT,
(lambda a, b:asyncio.create_task(
LiveChatAsync.shutdown(None,signal.SIGINT,b))
))
def _setup(self): def _setup(self):
#direct modeがTrueでcallback未設定の場合例外発生。 # An exception is raised when direct mode is true and no callback is set.
if self._direct_mode: if self._direct_mode:
if self._callback is None: if self._callback is None:
raise IllegalFunctionCall( raise exceptions.IllegalFunctionCall(
"direct_mode=Trueの場合callbackの設定が必須です。") "When direct_mode=True, callback parameter is required.")
else: else:
#direct modeがFalseでbufferが未設定ならばデフォルトのbufferを作成 # Create a default buffer if `direct_mode` is False and buffer is not set.
if self._buffer is None: if self._buffer is None:
self._buffer = Buffer(maxsize = 20) self._buffer = Buffer(maxsize=20)
#callbackが指定されている場合はcallbackを呼ぶループタスクを作成 # Create a loop task to call callback if the `callback` param is specified.
if self._callback is None: if self._callback is None:
pass pass
else: else:
#callbackを呼ぶループタスクの開始 # Create a loop task to call callback if the `callback` param is specified.
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
loop.create_task(self._callback_loop(self._callback)) loop.create_task(self._callback_loop(self._callback))
#_listenループタスクの開始 # Start a loop task for _listen()
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
listen_task = loop.create_task(self._startlisten()) self.listen_task = loop.create_task(self._startlisten())
#add_done_callbackの登録 # Register add_done_callback
if self._done_callback is None: if self._done_callback is None:
listen_task.add_done_callback(self.finish) self.listen_task.add_done_callback(self._finish)
else: else:
listen_task.add_done_callback(self._done_callback) self.listen_task.add_done_callback(self._done_callback)
async def _startlisten(self): async def _startlisten(self):
"""最初のcontinuationパラメータを取得し、 """Fetch first continuation parameter,
_listenループを開始する create and start _listen loop.
""" """
initial_continuation = await self._get_initial_continuation() if not self.continuation:
if initial_continuation is None: self.continuation = liveparam.getparam(
self.terminate() self._video_id,
logger.debug(f"[{self.video_id}]No initial continuation.") channel_id=util.get_channelid(httpx.Client(http2=True), self._video_id),
return past_sec=3)
await self._listen(initial_continuation)
async def _get_initial_continuation(self): await self._listen(self.continuation)
''' チャットデータ取得に必要な最初のcontinuationを取得する。'''
try:
initial_continuation = liveparam.getparam(self.video_id)
except ChatParseException as e:
self.terminate()
logger.debug(f"[{self.video_id}]Error:{str(e)}")
return
except KeyError:
logger.debug(f"[{self.video_id}]KeyError:"
f"{traceback.format_exc(limit = -1)}")
self.terminate()
return
return initial_continuation
async def _listen(self, continuation): async def _listen(self, continuation):
''' continuationに紐付いたチャットデータを取得し ''' Fetch chat data and store them into buffer,
チャットデータを格納、 get next continuaiton parameter and loop.
次のcontinuaitonを取得してループする。
Parameter Parameter
--------- ---------
continuation : str continuation : str
次のチャットデータ取得に必要なパラメータ parameter for next chat data
''' '''
try: try:
async with aiohttp.ClientSession() as session: async with httpx.AsyncClient(http2=True) as client:
while(continuation and self._is_alive): while(continuation and self._is_alive):
livechat_json = (await continuation = await self._check_pause(continuation)
self._get_livechat_json(continuation, session, headers) contents = await self._get_contents(continuation, client, headers)
) metadata, chatdata = self._parser.parse(contents)
metadata, chatdata = self._parser.parse( livechat_json ) continuation = metadata.get('continuation')
timeout = metadata['timeoutMs']/1000 if continuation:
self.continuation = continuation
timeout = metadata['timeoutMs'] / 1000
chat_component = { chat_component = {
"video_id" : self.video_id, "video_id": self._video_id,
"timeout" : timeout, "timeout": timeout,
"chatdata" : chatdata "chatdata": chatdata
} }
time_mark =time.time() time_mark = time.time()
if self._direct_mode: if self._direct_mode:
await self._callback( processed_chat = self.processor.process(
self.processor.process([chat_component]) [chat_component])
) if isinstance(processed_chat, tuple):
await self._callback(*processed_chat)
else:
await self._callback(processed_chat)
else: else:
await self._buffer.put(chat_component) await self._buffer.put(chat_component)
diff_time = timeout - (time.time()-time_mark) diff_time = timeout - (time.time() - time_mark)
await asyncio.sleep(diff_time) await asyncio.sleep(diff_time)
continuation = metadata.get('continuation') self._last_offset_ms = metadata.get('last_offset_ms', 0)
except ChatParseException as e: except exceptions.ChatParseException as e:
logger.info(f"{str(e)}video_id:\"{self.video_id}\"") self._logger.debug(f"[{self._video_id}]{str(e)}")
return raise
except (TypeError , json.JSONDecodeError) : except Exception:
logger.error(f"{traceback.format_exc(limit = -1)}") self._logger.error(f"{traceback.format_exc(limit=-1)}")
return raise
logger.debug(f"[{self.video_id}]チャット取得を終了しました。")
async def _get_livechat_json(self, continuation, session, headers): self._logger.debug(f"[{self._video_id}] finished fetching chat.")
async def _check_pause(self, continuation):
if self._pauser.empty():
'''pause'''
await self._pauser.get()
'''resume:
prohibit from blocking by putting None into _pauser.
'''
self._pauser.put_nowait(None)
if not self._is_replay:
async with httpx.AsyncClient(http2=True) as client:
continuation = await liveparam.getparam(self._video_id,
channel_id=util.get_channelid_async(client, self.video_id),
past_sec=3)
return continuation
async def _get_contents(self, continuation, client, headers):
'''Get 'continuationContents' from livechat json.
If contents is None at first fetching,
try to fetch archive chat data.
Return:
-------
'continuationContents' which includes metadata & chatdata.
''' '''
チャットデータが格納されたjsonデータを取得する。 livechat_json = await self._get_livechat_json(continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = config._smr
channelid = await util.get_channelid_async(client, self._video_id)
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only, channelid)
livechat_json = (await self._get_livechat_json(
continuation, client, replay=True, offset_ms=self.seektime * 1000))
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (await self._get_livechat_json(
reload_continuation, client, headers))
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
async def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
''' '''
continuation = urllib.parse.quote(continuation)
livechat_json = None livechat_json = None
status_code = 0 if offset_ms < 0:
url =( offset_ms = 0
f"https://www.youtube.com/live_chat/get_live_chat?" param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
f"continuation={continuation}&pbj=1")
for _ in range(MAX_RETRY + 1): for _ in range(MAX_RETRY + 1):
async with session.get(url ,headers = headers) as resp: try:
try: resp = await client.post(self._fetch_url, json=param)
text = await resp.text() livechat_json = resp.json()
status_code = resp.status break
livechat_json = json.loads(text) except (json.JSONDecodeError, httpx.HTTPError):
break await asyncio.sleep(2)
except (ClientConnectorError,json.JSONDecodeError) : continue
await asyncio.sleep(1)
continue
else: else:
logger.error(f"[{self.video_id}]" self._logger.error(f"[{self._video_id}]"
f"Exceeded retry count. status_code={status_code}") f"Exceeded retry count.")
return None raise exceptions.RetryExceedMaxCount()
return livechat_json return livechat_json
async def _callback_loop(self,callback): async def _callback_loop(self, callback):
""" コンストラクタでcallbackを指定している場合、バックグラウンドで """ If a callback is specified in the constructor,
callbackに指定された関数に一定間隔でチャットデータを投げる。 it throws chat data at regular intervals to the
function specified in the callback in the backgroun
Parameter Parameter
--------- ---------
callback : func callback : func
加工済みのチャットデータを渡す先の関数。 function to which the processed chat data is passed.
""" """
while self.is_alive(): while self.is_alive():
items = await self._buffer.get() items = await self._buffer.get()
data = self.processor.process(items) processed_chat = self.processor.process(items)
await callback(data) if isinstance(processed_chat, tuple):
await self._callback(*processed_chat)
else:
await self._callback(processed_chat)
async def get(self): async def get(self):
""" bufferからデータを取り出し、processorに投げ、 """
加工済みのチャットデータを返す。 Retrieves data from the buffer,
throws it to the processor,
and returns the processed chat data.
Returns Returns
: Processorによって加工されたチャットデータ : Chat data processed by the Processor
""" """
if self._callback is None: if self._callback is None:
items = await self._buffer.get() if self.is_alive():
return self.processor.process(items) items = await self._buffer.get()
raise IllegalFunctionCall( return self.processor.process(items)
"既にcallbackを登録済みのため、get()は実行できません。") else:
return []
raise exceptions.IllegalFunctionCall(
"Callback parameter is already set, so get() cannot be performed.")
def is_replay(self):
return self._is_replay
def pause(self):
if self._callback is None:
return
if not self._pauser.empty():
self._pauser.get_nowait()
def resume(self):
if self._callback is None:
return
if self._pauser.empty():
self._pauser.put_nowait(None)
def is_alive(self): def is_alive(self):
return self._is_alive return self._is_alive
def finish(self,sender): def _finish(self, sender):
'''Listener終了時のコールバック''' '''Called when the _listen() task finished.'''
try: try:
self.terminate() self._task_finished()
except CancelledError: except CancelledError:
logger.debug(f'[{self.video_id}]cancelled:{sender}') self._logger.debug(f'[{self._video_id}] cancelled:{sender}')
def terminate(self): def terminate(self):
''' if self._pauser.empty():
Listenerを終了する。 self._pauser.put_nowait(None)
'''
self._is_alive = False self._is_alive = False
if self._direct_mode == False: self._buffer.put_nowait({})
#bufferにダミーオブジェクトを入れてis_alive()を判定させる self.processor.finalize()
self._buffer.put_nowait({'chatdata':'','timeout':1})
logger.info(f'終了しました:[{self.video_id}]') def _keyboard_interrupt(self):
self.exception = exceptions.ChatDataFinished()
self.terminate()
def _task_finished(self):
if self.is_alive():
self.terminate()
try:
self.listen_task.result()
except Exception as e:
self.exception = e
if not isinstance(e, exceptions.ChatParseException):
self._logger.error(f'Internal exception - {type(e)}{str(e)}')
self._logger.info(f'[{self._video_id}] finished.')
def raise_for_status(self):
if self.exception is not None:
raise self.exception
@classmethod @classmethod
def _set_exception_handler(cls, handler): def _set_exception_handler(cls, handler):
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
#default handler: cls._handle_exception
loop.set_exception_handler(handler) loop.set_exception_handler(handler)
@classmethod
def _handle_exception(cls, loop, context):
#msg = context.get("exception", context["message"])
if not isinstance(context["exception"],CancelledError):
logger.error(f"Caught exception: {context}")
loop= asyncio.get_event_loop()
loop.create_task(cls.shutdown(None,None,None))
@classmethod
async def shutdown(cls, event, sig = None, handler=None):
logger.debug("シャットダウンしています")
tasks = [t for t in asyncio.all_tasks() if t is not
asyncio.current_task()]
[task.cancel() for task in tasks]
logger.debug(f"残っているタスクを終了しています")
await asyncio.gather(*tasks,return_exceptions=True)
loop = asyncio.get_event_loop()
loop.stop()

View File

@@ -1,307 +0,0 @@
import aiohttp, asyncio
import datetime
import json
import random
import signal
import time
import traceback
import urllib.parse
from aiohttp.client_exceptions import ClientConnectorError
from concurrent.futures import CancelledError
from queue import Queue
from .buffer import Buffer
from ..parser.replay import Parser
from .. import config
from .. import mylogger
from ..exceptions import ChatParseException,IllegalFunctionCall
from ..paramgen import arcparam
from ..processors.default.processor import DefaultProcessor
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
MAX_RETRY = 10
headers = config.headers
class ReplayChatAsync:
''' aiohttpを利用してYouTubeのライブ配信のチャットデータを取得する
Parameter
---------
video_id : str
動画ID
processor : ChatProcessor
チャットデータを加工するオブジェクト
buffer : Buffer(maxsize:20[default])
チャットデータchat_componentを格納するバッファ。
maxsize : 格納できるchat_componentの個数
default値20個。1個で約5~10秒分。
interruptable : bool
Ctrl+Cによる処理中断を行うかどうか。
callback : func
_listen()関数から一定間隔で自動的に呼びだす関数。
done_callback : func
listener終了時に呼び出すコールバック。
direct_mode : bool
Trueの場合、bufferを使わずにcallbackを呼ぶ。
Trueの場合、callbackの設定が必須
(設定していない場合IllegalFunctionCall例外を発生させる
Attributes
---------
_executor : ThreadPoolExecutor
チャットデータ取得ループ_listen用のスレッド
_is_alive : bool
チャット取得を終了したか
'''
_setup_finished = False
def __init__(self, video_id,
seektime =0,
processor = DefaultProcessor(),
buffer = Buffer(maxsize = 20),
interruptable = True,
callback = None,
done_callback = None,
exception_handler = None,
direct_mode = False):
self.video_id = video_id
self.seektime= seektime
self.processor = processor
self._buffer = buffer
self._callback = callback
self._done_callback = done_callback
self._exception_handler = exception_handler
self._direct_mode = direct_mode
self._is_alive = True
self._parser = Parser()
self._pauser = Queue()
self._pauser.put_nowait(None)
self._setup()
if not ReplayChatAsync._setup_finished:
ReplayChatAsync._setup_finished = True
if exception_handler == None:
self._set_exception_handler(self._handle_exception)
else:
self._set_exception_handler(exception_handler)
if interruptable:
signal.signal(signal.SIGINT,
(lambda a, b:asyncio.create_task(
ReplayChatAsync.shutdown(None,signal.SIGINT,b))
))
def _setup(self):
#direct modeがTrueでcallback未設定の場合例外発生。
if self._direct_mode:
if self._callback is None:
raise IllegalFunctionCall(
"direct_mode=Trueの場合callbackの設定が必須です。")
else:
#direct modeがFalseでbufferが未設定ならばデフォルトのbufferを作成
if self._buffer is None:
self._buffer = Buffer(maxsize = 20)
#callbackが指定されている場合はcallbackを呼ぶループタスクを作成
if self._callback is None:
pass
else:
#callbackを呼ぶループタスクの開始
loop = asyncio.get_event_loop()
loop.create_task(self._callback_loop(self._callback))
#_listenループタスクの開始
loop = asyncio.get_event_loop()
listen_task = loop.create_task(self._startlisten())
#add_done_callbackの登録
if self._done_callback is None:
listen_task.add_done_callback(self.finish)
else:
listen_task.add_done_callback(self._done_callback)
async def _startlisten(self):
"""最初のcontinuationパラメータを取得し、
_listenループのタスクを作成し開始する
"""
initial_continuation = await self._get_initial_continuation()
if initial_continuation is None:
self.terminate()
logger.debug(f"[{self.video_id}]No initial continuation.")
return
await self._listen(initial_continuation)
async def _get_initial_continuation(self):
''' チャットデータ取得に必要な最初のcontinuationを取得する。'''
try:
initial_continuation = arcparam.get(self.video_id,self.seektime)
except ChatParseException as e:
self.terminate()
logger.debug(f"[{self.video_id}]Error:{str(e)}")
return
except KeyError:
logger.debug(f"[{self.video_id}]KeyError:"
f"{traceback.format_exc(limit = -1)}")
self.terminate()
return
return initial_continuation
async def _listen(self, continuation):
''' continuationに紐付いたチャットデータを取得し
にチャットデータを格納、
次のcontinuaitonを取得してループする
Parameter
---------
continuation : str
次のチャットデータ取得に必要なパラメータ
'''
try:
async with aiohttp.ClientSession() as session:
while(continuation and self._is_alive):
if self._pauser.empty():
#pauseが呼ばれて_pauserが空状態のときは一時停止する
await self._pauser.get()
#resumeが呼ばれて_pauserにitemが入ったら再開する
#直後に_pauserにitem(None)を入れてブロックを防ぐ
self._pauser.put_nowait(None)
livechat_json = (await
self._get_livechat_json(continuation, session, headers)
)
metadata, chatdata = self._parser.parse( livechat_json )
timeout = metadata['timeoutMs']/1000
chat_component = {
"video_id" : self.video_id,
"timeout" : timeout,
"chatdata" : chatdata
}
time_mark =time.time()
if self._direct_mode:
await self._callback(
self.processor.process([chat_component])
)
else:
await self._buffer.put(chat_component)
diff_time = timeout - (time.time()-time_mark)
if diff_time < 0 : diff_time=0
await asyncio.sleep(diff_time)
continuation = metadata.get('continuation')
except ChatParseException as e:
logger.error(f"{str(e)}動画ID:\"{self.video_id}\"")
return
except (TypeError , json.JSONDecodeError) :
logger.error(f"{traceback.format_exc(limit = -1)}")
return
logger.debug(f"[{self.video_id}]チャット取得を終了しました。")
async def _get_livechat_json(self, continuation, session, headers):
'''
チャットデータが格納されたjsonデータを取得する。
'''
continuation = urllib.parse.quote(continuation)
livechat_json = None
status_code = 0
url =(
f"https://www.youtube.com/live_chat_replay/get_live_chat_replay?"
f"continuation={continuation}&pbj=1")
for _ in range(MAX_RETRY + 1):
async with session.get(url ,headers = headers) as resp:
try:
text = await resp.text()
status_code = resp.status
livechat_json = json.loads(text)
break
except (ClientConnectorError,json.JSONDecodeError) :
await asyncio.sleep(1)
continue
else:
logger.error(f"[{self.video_id}]"
f"Exceeded retry count. status_code={status_code}")
return None
return livechat_json
async def _callback_loop(self,callback):
""" コンストラクタでcallbackを指定している場合、バックグラウンドで
callbackに指定された関数に一定間隔でチャットデータを投げる。
Parameter
---------
callback : func
加工済みのチャットデータを渡す先の関数。
"""
while self.is_alive():
items = await self._buffer.get()
data = self.processor.process(items)
await callback(data)
async def get(self):
""" bufferからデータを取り出し、processorに投げ、
加工済みのチャットデータを返す。
Returns
: Processorによって加工されたチャットデータ
"""
if self._callback is None:
items = await self._buffer.get()
return self.processor.process(items)
raise IllegalFunctionCall(
"既にcallbackを登録済みのため、get()は実行できません。")
def pause(self):
if not self._pauser.empty():
self._pauser.get()
def resume(self):
if self._pauser.empty():
self._pauser.put_nowait(None)
def is_alive(self):
return self._is_alive
def finish(self,sender):
'''Listener終了時のコールバック'''
try:
self.terminate()
except CancelledError:
logger.debug(f'[{self.video_id}]cancelled:{sender}')
def terminate(self):
'''
Listenerを終了する。
'''
self._is_alive = False
if self._direct_mode == False:
#bufferにダミーオブジェクトを入れてis_alive()を判定させる
self._buffer.put_nowait({'chatdata':'','timeout':1})
logger.info(f'終了しました:[{self.video_id}]')
@classmethod
def _set_exception_handler(cls, handler):
loop = asyncio.get_event_loop()
#default handler: cls._handle_exception
loop.set_exception_handler(handler)
@classmethod
def _handle_exception(cls, loop, context):
#msg = context.get("exception", context["message"])
if not isinstance(context["exception"],CancelledError):
logger.error(f"Caught exception: {context}")
loop= asyncio.get_event_loop()
loop.create_task(cls.shutdown(None,None,None))
@classmethod
async def shutdown(cls, event, sig = None, handler=None):
logger.debug("シャットダウンしています")
tasks = [t for t in asyncio.all_tasks() if t is not
asyncio.current_task()]
[task.cancel() for task in tasks]
logger.debug(f"残っているタスクを終了しています")
await asyncio.gather(*tasks,return_exceptions=True)
loop = asyncio.get_event_loop()
loop.stop()

View File

@@ -1,31 +1,40 @@
import queue import queue
class Buffer(queue.Queue): class Buffer(queue.Queue):
''' '''
チャットデータを格納するバッファの役割を持つFIFOキュー Buffer for storing chat data.
Parameter Parameter
--------- ---------
max_size : int maxsize : int
格納するチャットブロックの最大個数。0の場合は無限。 Maximum number of chat blocks to be stored.
最大値を超える場合は古いチャットブロックから破棄される。 If it exceeds the maximum, the oldest chat block will be discarded.
''' '''
def __init__(self,maxsize = 0):
def __init__(self, maxsize=0):
super().__init__(maxsize=maxsize) super().__init__(maxsize=maxsize)
def put(self,item): def put(self, item):
if item is None: if item is None:
return return
if super().full(): if super().full():
super().get_nowait() super().get_nowait()
else: else:
super().put(item) super().put(item)
def put_nowait(self, item):
if item is None:
return
if super().full():
super().get_nowait()
else:
super().put_nowait(item)
def get(self): def get(self):
ret = [] ret = []
ret.append(super().get()) ret.append(super().get())
while not super().empty(): while not super().empty():
ret.append(super().get()) ret.append(super().get())
return ret return ret

View File

@@ -1,276 +1,354 @@
import requests import httpx
import datetime
import json import json
import random
import signal import signal
import time import time
import traceback import traceback
import urllib.parse
from concurrent.futures import CancelledError, ThreadPoolExecutor from concurrent.futures import CancelledError, ThreadPoolExecutor
from queue import Queue
from threading import Event
from .buffer import Buffer from .buffer import Buffer
from ..parser.live import Parser from ..parser.live import Parser
from .. import config from .. import config
from .. import mylogger from .. import exceptions
from ..exceptions import ChatParseException,IllegalFunctionCall from .. import util
from ..paramgen import liveparam from ..paramgen import liveparam, arcparam
from ..processors.default.processor import DefaultProcessor from ..processors.default.processor import DefaultProcessor
from ..processors.combinator import Combinator
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
MAX_RETRY = 10
headers = config.headers headers = config.headers
MAX_RETRY = 10
class LiveChat: class LiveChat:
''' スレッドプールを利用してYouTubeのライブ配信のチャットデータを取得する '''
LiveChat object fetches chat data and stores them
in a buffer with ThreadpoolExecutor.
Parameter Parameter
--------- ---------
video_id : str video_id : str
動画ID
seektime : int
start position of fetching chat (seconds).
This option is valid for archived chat only.
If negative value, chat data posted before the start of the broadcast
will be retrieved as well.
processor : ChatProcessor processor : ChatProcessor
チャットデータを加工するオブジェクト
buffer : Buffer(maxsize:20[default]) buffer : Buffer
チャットデータchat_componentを格納するバッファ。 buffer of chat data fetched background.
maxsize : 格納できるchat_componentの個数
default値20個。1個で約5~10秒分。
interruptable : bool interruptable : bool
Ctrl+Cによる処理中断を行うかどうか。 Allows keyboard interrupts.
Set this parameter to False if your own threading program causes
the problem.
callback : func callback : func
_listen()関数から一定間隔で自動的に呼びだす関数。 function called periodically from _listen().
done_callback : func done_callback : func
listener終了時に呼び出すコールバック。 function called when listener ends.
direct_mode : bool direct_mode : bool
Trueの場合、bufferを使わずにcallbackを呼ぶ。 If True, invoke specified callback function without using buffer.
Trueの場合、callbackの設定が必須 callback is required. If not, IllegalFunctionCall will be raised.
(設定していない場合IllegalFunctionCall例外を発生させる
force_replay : bool
force to fetch archived chat data, even if specified video is live.
topchat_only : bool
If True, get only top chat.
replay_continuation : str
If this parameter is not None, the processor will attempt to get chat data from continuation.
This parameter is only allowed in archived mode.
Attributes Attributes
--------- ---------
_executor : ThreadPoolExecutor _executor : ThreadPoolExecutor
チャットデータ取得ループ_listen用のスレッド This is used for _listen() loop.
_is_alive : bool _is_alive : bool
チャット取得を終了したか Flag to stop getting chat.
''' '''
_setup_finished = False _setup_finished = False
#チャット監視中のListenerのリスト
_listeners= []
def __init__(self, video_id, def __init__(self, video_id,
processor = DefaultProcessor(), seektime=-1,
buffer = Buffer(maxsize = 20), processor=DefaultProcessor(),
interruptable = True, buffer=None,
callback = None, interruptable=True,
done_callback = None, callback=None,
direct_mode = False done_callback=None,
): direct_mode=False,
self.video_id = video_id force_replay=False,
self.processor = processor topchat_only=False,
logger=config.logger(__name__),
replay_continuation=None
):
self._video_id = util.extract_video_id(video_id)
self.seektime = seektime
if isinstance(processor, tuple):
self.processor = Combinator(processor)
else:
self.processor = processor
self._buffer = buffer self._buffer = buffer
self._callback = callback self._callback = callback
self._done_callback = done_callback self._done_callback = done_callback
self._executor = ThreadPoolExecutor(max_workers=2) self._executor = ThreadPoolExecutor(max_workers=2)
self._direct_mode = direct_mode self._direct_mode = direct_mode
self._is_alive = True self._is_alive = True
self._parser = Parser() self._is_replay = force_replay or (replay_continuation is not None)
self._parser = Parser(is_replay=self._is_replay)
self._pauser = Queue()
self._pauser.put_nowait(None)
self._first_fetch = replay_continuation is None
self._fetch_url = config._sml if replay_continuation is None else config._smr
self._topchat_only = topchat_only
self._dat = ''
self._last_offset_ms = 0
self._logger = logger
self._event = Event()
self.continuation = replay_continuation
self.exception = None
if interruptable:
signal.signal(signal.SIGINT, lambda a, b: self.terminate())
self._setup() self._setup()
if not LiveChat._setup_finished:
LiveChat._setup_finished = True
if interruptable:
signal.signal(signal.SIGINT, (lambda a, b:
(LiveChat.shutdown(None,signal.SIGINT,b))
))
LiveChat._listeners.append(self)
def _setup(self): def _setup(self):
#direct modeがTrueでcallback未設定の場合例外発生。 # An exception is raised when direct mode is true and no callback is set.
if self._direct_mode: if self._direct_mode:
if self._callback is None: if self._callback is None:
raise IllegalFunctionCall( raise exceptions.IllegalFunctionCall(
"direct_mode=Trueの場合callbackの設定が必須です。") "When direct_mode=True, callback parameter is required.")
else: else:
#direct modeがFalseでbufferが未設定ならばデフォルトのbufferを作成 # Create a default buffer if `direct_mode` is False and buffer is not set.
if self._buffer is None: if self._buffer is None:
self._buffer = Buffer(maxsize = 20) self._buffer = Buffer(maxsize=20)
#callbackが指定されている場合はcallbackを呼ぶループタスクを作成 # Create a loop task to call callback if the `callback` param is specified.
if self._callback is None: if self._callback is None:
pass pass
else: else:
#callbackを呼ぶループタスクの開始 # Start a loop task calling callback function.
self._executor.submit(self._callback_loop,self._callback) self._executor.submit(self._callback_loop, self._callback)
#_listenループタスクの開始 # Start a loop task for _listen()
listen_task = self._executor.submit(self._startlisten) self.listen_task = self._executor.submit(self._startlisten)
#add_done_callbackの登録 # Register add_done_callback
if self._done_callback is None: if self._done_callback is None:
listen_task.add_done_callback(self.finish) self.listen_task.add_done_callback(self._finish)
else: else:
listen_task.add_done_callback(self._done_callback) self.listen_task.add_done_callback(self._done_callback)
def _startlisten(self): def _startlisten(self):
"""最初のcontinuationパラメータを取得し、 time.sleep(0.1) # sleep shortly to prohibit skipping fetching data
_listenループのタスクを作成し開始する """Fetch first continuation parameter,
create and start _listen loop.
""" """
initial_continuation = self._get_initial_continuation() if not self.continuation:
if initial_continuation is None: self.continuation = liveparam.getparam(
self.terminate() self._video_id,
logger.debug(f"[{self.video_id}]No initial continuation.") channel_id=util.get_channelid(httpx.Client(http2=True), self._video_id),
return past_sec=3)
self._listen(initial_continuation) self._listen(self.continuation)
def _get_initial_continuation(self):
''' チャットデータ取得に必要な最初のcontinuationを取得する。'''
try:
initial_continuation = liveparam.getparam(self.video_id)
except ChatParseException as e:
self.terminate()
logger.debug(f"[{self.video_id}]Error:{str(e)}")
return
except KeyError:
logger.debug(f"[{self.video_id}]KeyError:"
f"{traceback.format_exc(limit = -1)}")
self.terminate()
return
return initial_continuation
def _listen(self, continuation): def _listen(self, continuation):
''' continuationに紐付いたチャットデータを取得し ''' Fetch chat data and store them into buffer,
にチャットデータを格納、 get next continuaiton parameter and loop.
次のcontinuaitonを取得してループする
Parameter Parameter
--------- ---------
continuation : str continuation : str
次のチャットデータ取得に必要なパラメータ parameter for next chat data
''' '''
try: try:
with requests.Session() as session: with httpx.Client(http2=True) as client:
while(continuation and self._is_alive): while(continuation and self._is_alive):
livechat_json = ( continuation = self._check_pause(continuation)
self._get_livechat_json(continuation, session, headers) contents = self._get_contents(continuation, client, headers)
) metadata, chatdata = self._parser.parse(contents)
metadata, chatdata = self._parser.parse( livechat_json ) continuation = metadata.get('continuation')
#チャットデータを含むコンポーネントを組み立ててbufferに投入する if continuation:
timeout = metadata['timeoutMs']/1000 self.continuation = continuation
timeout = metadata['timeoutMs'] / 1000
chat_component = { chat_component = {
"video_id" : self.video_id, "video_id": self._video_id,
"timeout" : timeout, "timeout": timeout,
"chatdata" : chatdata "chatdata": chatdata
} }
time_mark =time.time() time_mark = time.time()
if self._direct_mode: if self._direct_mode:
self._callback( processed_chat = self.processor.process(
self.processor.process([chat_component]) [chat_component])
) if isinstance(processed_chat, tuple):
self._callback(*processed_chat)
else:
self._callback(processed_chat)
else: else:
self._buffer.put(chat_component) self._buffer.put(chat_component)
#次のchatを取得するまでsleepする diff_time = timeout - (time.time() - time_mark)
diff_time = timeout - (time.time()-time_mark) self._event.wait(diff_time if diff_time > 0 else 0)
if diff_time < 0 : diff_time=0 self._last_offset_ms = metadata.get('last_offset_ms', 0)
time.sleep(diff_time) except exceptions.ChatParseException as e:
#次のチャットデータのcontinuationパラメータを取り出す。 self._logger.debug(f"[{self._video_id}]{str(e)}")
continuation = metadata.get('continuation') raise
except Exception:
#whileループ先頭に戻る self._logger.error(f"{traceback.format_exc(limit=-1)}")
except ChatParseException as e: raise
logger.error(f"{str(e)}動画ID:\"{self.video_id}\"")
return
except (TypeError , json.JSONDecodeError) :
logger.error(f"{traceback.format_exc(limit = -1)}")
return
logger.debug(f"[{self.video_id}]チャット取得を終了しました。")
def _get_livechat_json(self, continuation, session, headers): self._logger.debug(f"[{self._video_id}] finished fetching chat.")
def _check_pause(self, continuation):
if self._pauser.empty():
'''pause'''
self._pauser.get()
'''resume:
prohibit from blocking by putting None into _pauser.
'''
self._pauser.put_nowait(None)
if not self._is_replay:
continuation = liveparam.getparam(
self._video_id, channel_id=util.get_channelid(httpx.Client(http2=True), self._video_id),
past_sec=3, topchat_only=self._topchat_only)
return continuation
def _get_contents(self, continuation, client, headers):
'''Get 'continuationContents' from livechat json.
If contents is None at first fetching,
try to fetch archive chat data.
Return:
-------
'continuationContents' which includes metadata & chat data.
''' '''
チャットデータが格納されたjsonデータを取得する。 livechat_json = self._get_livechat_json(continuation, client, replay=self._is_replay, offset_ms=self._last_offset_ms)
contents, dat = self._parser.get_contents(livechat_json)
if self._dat == '' and dat:
self._dat = dat
if self._first_fetch:
if contents is None or self._is_replay:
'''Try to fetch archive chat data.'''
self._parser.is_replay = True
self._fetch_url = config._smr
continuation = arcparam.getparam(
self._video_id, self.seektime, self._topchat_only, util.get_channelid(client, self._video_id))
livechat_json = (self._get_livechat_json(
continuation, client, replay=True, offset_ms=self.seektime * 1000))
reload_continuation = self._parser.reload_continuation(
self._parser.get_contents(livechat_json)[0])
if reload_continuation:
livechat_json = (self._get_livechat_json(
reload_continuation, client, headers))
contents, _ = self._parser.get_contents(livechat_json)
self._is_replay = True
self._first_fetch = False
return contents
def _get_livechat_json(self, continuation, client, replay: bool, offset_ms: int = 0):
'''
Get json which includes chat data.
''' '''
continuation = urllib.parse.quote(continuation)
livechat_json = None livechat_json = None
status_code = 0 if offset_ms < 0:
url =( offset_ms = 0
f"https://www.youtube.com/live_chat/get_live_chat?" param = util.get_param(continuation, dat=self._dat, replay=replay, offsetms=offset_ms)
f"continuation={continuation}&pbj=1")
for _ in range(MAX_RETRY + 1): for _ in range(MAX_RETRY + 1):
with session.get(url ,headers = headers) as resp: try:
try: response = client.post(self._fetch_url, json=param)
text = resp.text livechat_json = response.json()
status_code = resp.status_code break
livechat_json = json.loads(text) except (json.JSONDecodeError, httpx.HTTPError):
break time.sleep(2)
except json.JSONDecodeError : continue
time.sleep(1)
continue
else: else:
logger.error(f"[{self.video_id}]" self._logger.error(f"[{self._video_id}]"
f"Exceeded retry count. status_code={status_code}") f"Exceeded retry count.")
return None raise exceptions.RetryExceedMaxCount()
return livechat_json return livechat_json
def _callback_loop(self,callback): def _callback_loop(self, callback):
""" コンストラクタでcallbackを指定している場合、バックグラウンドで """ If a callback is specified in the constructor,
callbackに指定された関数に一定間隔でチャットデータを投げる。 it throws chat data at regular intervals to the
function specified in the callback in the backgroun
Parameter Parameter
--------- ---------
callback : func callback : func
加工済みのチャットデータを渡す先の関数。 function to which the processed chat data is passed.
""" """
while self.is_alive(): while self.is_alive():
items = self._buffer.get() items = self._buffer.get()
data = self.processor.process(items) processed_chat = self.processor.process(items)
callback(data) if isinstance(processed_chat, tuple):
self._callback(*processed_chat)
else:
self._callback(processed_chat)
def get(self): def get(self):
""" bufferからデータを取り出し、processorに投げ、 """
加工済みのチャットデータを返す。 Retrieves data from the buffer,
throws it to the processor,
and returns the processed chat data.
Returns Returns
: Processorによって加工されたチャットデータ : Chat data processed by the Processor
""" """
if self._callback is None: if self._callback is None:
items = self._buffer.get() if self.is_alive():
return self.processor.process(items) items = self._buffer.get()
raise IllegalFunctionCall( return self.processor.process(items)
"既にcallbackを登録済みのため、get()は実行できません。") else:
return []
raise exceptions.IllegalFunctionCall(
"Callback parameter is already set, so get() cannot be performed.")
def is_replay(self):
return self._is_replay
def pause(self):
if self._callback is None:
return
if not self._pauser.empty():
self._pauser.get()
def resume(self):
if self._callback is None:
return
if self._pauser.empty():
self._pauser.put_nowait(None)
def is_alive(self): def is_alive(self):
return self._is_alive return self._is_alive
def finish(self,sender): def _finish(self, sender):
'''Listener終了時のコールバック''' '''Called when the _listen() task finished.'''
try: try:
self.terminate() self._task_finished()
except CancelledError: except CancelledError:
logger.debug(f'[{self.video_id}]cancelled:{sender}') self._logger.debug(f'[{self._video_id}] cancelled:{sender}')
def terminate(self): def terminate(self):
''' if self._pauser.empty():
Listenerを終了する。 self._pauser.put_nowait(None)
'''
self._is_alive = False self._is_alive = False
if self._direct_mode == False: self._buffer.put({})
#bufferにダミーオブジェクトを入れてis_alive()を判定させる self._event.set()
self._buffer.put({'chatdata':'','timeout':1}) self.processor.finalize()
logger.info(f'終了しました:[{self.video_id}]')
@classmethod
def shutdown(cls, event, sig = None, handler=None):
logger.debug("シャットダウンしています")
for t in LiveChat._listeners:
t._is_alive = False
def _task_finished(self):
if self.is_alive():
self.terminate()
try:
self.listen_task.result()
except Exception as e:
self.exception = e
if not isinstance(e, exceptions.ChatParseException):
self._logger.error(f'Internal exception - {type(e)}{str(e)}')
self._logger.info(f'[{self._video_id}] finished.')
def raise_for_status(self):
if self.exception is not None:
raise self.exception

View File

@@ -1,284 +0,0 @@
import requests
import datetime
import json
import random
import signal
import time
import traceback
import urllib.parse
from concurrent.futures import CancelledError, ThreadPoolExecutor
from queue import Queue
from .buffer import Buffer
from ..parser.replay import Parser
from .. import config
from .. import mylogger
from ..exceptions import ChatParseException,IllegalFunctionCall
from ..paramgen import arcparam
from ..processors.default.processor import DefaultProcessor
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
MAX_RETRY = 10
headers = config.headers
class ReplayChat:
''' スレッドプールを利用してYouTubeのライブ配信のチャットデータを取得する
Parameter
---------
video_id : str
動画ID
processor : ChatProcessor
チャットデータを加工するオブジェクト
buffer : Buffer(maxsize:20[default])
チャットデータchat_componentを格納するバッファ。
maxsize : 格納できるchat_componentの個数
default値20個。1個で約5~10秒分。
interruptable : bool
Ctrl+Cによる処理中断を行うかどうか。
callback : func
_listen()関数から一定間隔で自動的に呼びだす関数。
done_callback : func
listener終了時に呼び出すコールバック。
direct_mode : bool
Trueの場合、bufferを使わずにcallbackを呼ぶ。
Trueの場合、callbackの設定が必須
(設定していない場合IllegalFunctionCall例外を発生させる
Attributes
---------
_executor : ThreadPoolExecutor
チャットデータ取得ループ_listen用のスレッド
_is_alive : bool
チャット取得を終了したか
'''
_setup_finished = False
#チャット監視中のListenerのリスト
_listeners= []
def __init__(self, video_id,
seektime =0,
processor = DefaultProcessor(),
buffer = Buffer(maxsize = 20),
interruptable = True,
callback = None,
done_callback = None,
direct_mode = False
):
self.video_id = video_id
self.seektime= seektime
self.processor = processor
self._buffer = buffer
self._callback = callback
self._done_callback = done_callback
self._executor = ThreadPoolExecutor(max_workers=2)
self._direct_mode = direct_mode
self._is_alive = True
self._parser = Parser()
self._pauser = Queue()
self._pauser.put_nowait(None)
self._setup()
if not ReplayChat._setup_finished:
ReplayChat._setup_finished = True
if interruptable:
signal.signal(signal.SIGINT, (lambda a, b:
(ReplayChat.shutdown(None,signal.SIGINT,b))
))
ReplayChat._listeners.append(self)
def _setup(self):
#direct modeがTrueでcallback未設定の場合例外発生。
if self._direct_mode:
if self._callback is None:
raise IllegalFunctionCall(
"direct_mode=Trueの場合callbackの設定が必須です。")
else:
#direct modeがFalseでbufferが未設定ならばデフォルトのbufferを作成
if self._buffer is None:
self._buffer = Buffer(maxsize = 20)
#callbackが指定されている場合はcallbackを呼ぶループタスクを作成
if self._callback is None:
pass
else:
#callbackを呼ぶループタスクの開始
self._executor.submit(self._callback_loop,self._callback)
#_listenループタスクの開始
listen_task = self._executor.submit(self._startlisten)
#add_done_callbackの登録
if self._done_callback is None:
listen_task.add_done_callback(self.finish)
else:
listen_task.add_done_callback(self._done_callback)
def _startlisten(self):
"""最初のcontinuationパラメータを取得し、
_listenループのタスクを作成し開始する
"""
initial_continuation = self._get_initial_continuation()
if initial_continuation is None:
self.terminate()
logger.debug(f"[{self.video_id}]No initial continuation.")
return
self._listen(initial_continuation)
def _get_initial_continuation(self):
''' チャットデータ取得に必要な最初のcontinuationを取得する。'''
try:
initial_continuation = arcparam.get(self.video_id,self.seektime)
except ChatParseException as e:
self.terminate()
logger.debug(f"[{self.video_id}]Error:{str(e)}")
return
except KeyError:
logger.debug(f"[{self.video_id}]KeyError:"
f"{traceback.format_exc(limit = -1)}")
self.terminate()
return
return initial_continuation
def _listen(self, continuation):
''' continuationに紐付いたチャットデータを取得し
にチャットデータを格納、
次のcontinuaitonを取得してループする
Parameter
---------
continuation : str
次のチャットデータ取得に必要なパラメータ
'''
try:
with requests.Session() as session:
while(continuation and self._is_alive):
if self._pauser.empty():
#pauseが呼ばれて_pauserが空状態のときは一時停止する
self._pauser.get()
#resumeが呼ばれて_pauserにitemが入ったら再開する
#直後に_pauserにitem(None)を入れてブロックを防ぐ
self._pauser.put_nowait(None)
livechat_json = (
self._get_livechat_json(continuation, session, headers)
)
metadata, chatdata = self._parser.parse( livechat_json )
#チャットデータを含むコンポーネントを組み立ててbufferに投入する
timeout = metadata['timeoutMs']/1000
chat_component = {
"video_id" : self.video_id,
"timeout" : timeout,
"chatdata" : chatdata
}
time_mark =time.time()
if self._direct_mode:
self._callback(
self.processor.process([chat_component])
)
else:
self._buffer.put(chat_component)
diff_time = timeout - (time.time()-time_mark)
if diff_time < 0 : diff_time=0
time.sleep(diff_time)
continuation = metadata.get('continuation')
except ChatParseException as e:
logger.error(f"{str(e)}動画ID:\"{self.video_id}\"")
return
except (TypeError , json.JSONDecodeError) :
logger.error(f"{traceback.format_exc(limit = -1)}")
return
logger.debug(f"[{self.video_id}]チャット取得を終了しました。")
def _get_livechat_json(self, continuation, session, headers):
'''
チャットデータが格納されたjsonデータを取得する。
'''
continuation = urllib.parse.quote(continuation)
livechat_json = None
status_code = 0
url =(
f"https://www.youtube.com/live_chat_replay/get_live_chat_replay?"
f"continuation={continuation}&pbj=1")
for _ in range(MAX_RETRY + 1):
with session.get(url ,headers = headers) as resp:
try:
text = resp.text
status_code = resp.status_code
livechat_json = json.loads(text)
break
except json.JSONDecodeError :
time.sleep(1)
continue
else:
logger.error(f"[{self.video_id}]"
f"Exceeded retry count. status_code={status_code}")
return None
return livechat_json
def _callback_loop(self,callback):
""" コンストラクタでcallbackを指定している場合、バックグラウンドで
callbackに指定された関数に一定間隔でチャットデータを投げる。
Parameter
---------
callback : func
加工済みのチャットデータを渡す先の関数。
"""
while self.is_alive():
items = self._buffer.get()
data = self.processor.process(items)
callback(data)
def get(self):
""" bufferからデータを取り出し、processorに投げ、
加工済みのチャットデータを返す。
Returns
: Processorによって加工されたチャットデータ
"""
if self._callback is None:
items = self._buffer.get()
return self.processor.process(items)
raise IllegalFunctionCall(
"既にcallbackを登録済みのため、get()は実行できません。")
def pause(self):
if not self._pauser.empty():
self._pauser.get()
def resume(self):
if self._pauser.empty():
self._pauser.put_nowait(None)
def is_alive(self):
return self._is_alive
def finish(self,sender):
'''Listener終了時のコールバック'''
try:
self.terminate()
except CancelledError:
logger.debug(f'[{self.video_id}]cancelled:{sender}')
def terminate(self):
'''
Listenerを終了する。
'''
self._is_alive = False
if self._direct_mode == False:
#bufferにダミーオブジェクトを入れてis_alive()を判定させる
self._buffer.put({'chatdata':'','timeout':1})
logger.info(f'終了しました:[{self.video_id}]')
@classmethod
def shutdown(cls, event, sig = None, handler=None):
logger.debug("シャットダウンしています")
for t in ReplayChat._listeners:
t._is_alive = False

View File

@@ -1,43 +1,81 @@
class ChatParseException(Exception): class ChatParseException(Exception):
''' '''
チャットデータをパースするライブラリが投げる例外の基底クラス Base exception thrown by the parser
''' '''
pass pass
class NoYtinitialdataException(ChatParseException):
'''
配信ページ内にチャットデータurlが見つからないときに投げる例外
'''
pass
class ResponseContextError(ChatParseException): class ResponseContextError(ChatParseException):
''' '''
配信ページでチャットデータ無効の時に投げる例外 Thrown when chat data is invalid.
'''
pass
class NoLivechatRendererException(ChatParseException):
'''
チャットデータのJSON中にlivechatRendererがない時に投げる例外
''' '''
pass pass
class NoContentsException(ChatParseException): class NoContents(ChatParseException):
''' '''
チャットデータのJSON中にContinuationContentsがない時に投げる例外 Thrown when ContinuationContents is missing in JSON.
''' '''
pass pass
class NoContinuationsException(ChatParseException):
class NoContinuation(ChatParseException):
''' '''
チャットデータのContinuationContents中にcontinuationがない時に投げる例外 Thrown when continuation is missing in ContinuationContents.
''' '''
pass pass
class IllegalFunctionCall(Exception): class IllegalFunctionCall(Exception):
''' '''
set_callback()を実行済みにもかかわらず Thrown when get() is called even though
get()を呼び出した場合の例外 set_callback() has been executed.
''' '''
pass pass
class InvalidVideoIdException(Exception):
'''
Thrown when the video_id is not exist (VideoInfo).
'''
def __init__(self, doc):
self.msg = "InvalidVideoIdException"
self.doc = doc
class UnknownConnectionError(Exception):
pass
class RetryExceedMaxCount(Exception):
'''
Thrown when the number of retries exceeds the maximum value.
'''
pass
class ChatDataFinished(ChatParseException):
pass
class ReceivedUnknownContinuation(ChatParseException):
pass
class FailedExtractContinuation(ChatDataFinished):
pass
class VideoInfoParseError(Exception):
'''
Base exception when parsing video info.
'''
class PatternUnmatchError(VideoInfoParseError):
'''
Thrown when failed to parse video info with unmatched pattern.
'''
def __init__(self, doc=''):
self.msg = "PatternUnmatchError"
self.doc = doc

View File

@@ -1,31 +0,0 @@
import logging
import datetime
def get_logger(modname,mode=logging.DEBUG):
logger = logging.getLogger(modname)
if mode == None:
logger.addHandler(logging.NullHandler())
return logger
logger.setLevel(mode)
#create handler1 for showing info
handler1 = logging.StreamHandler()
my_formatter = MyFormatter()
handler1.setFormatter(my_formatter)
handler1.setLevel(mode)
logger.addHandler(handler1)
#create handler2 for recording log file
if mode <= logging.DEBUG:
handler2 = logging.FileHandler(filename="log.txt")
handler2.setLevel(logging.ERROR)
handler2.setFormatter(my_formatter)
logger.addHandler(handler2)
return logger
class MyFormatter(logging.Formatter):
def format(self, record):
s =(datetime.datetime.fromtimestamp(record.created)).strftime("%m-%d %H:%M:%S")+'| '+ (record.module).ljust(15)+(' { '+record.funcName).ljust(20) +":"+str(record.lineno).rjust(4)+'} - '+record.getMessage()
return s

View File

@@ -1,120 +1,44 @@
from . import enc
from base64 import urlsafe_b64encode as b64enc from base64 import urlsafe_b64encode as b64enc
from functools import reduce from urllib.parse import quote
import calendar, datetime, pytz
import math
import random
import urllib.parse
'''
Generate continuation parameter of youtube replay chat.
Author: taizan-hokuto (2019) @taizan205 def _header(video_id, channel_id) -> str:
S1_3 = enc.rs(1, video_id)
S1_5 = enc.rs(1, channel_id) + enc.rs(2, video_id)
S1 = enc.rs(3, S1_3) + enc.rs(5, S1_5)
S3 = enc.rs(48687757, enc.rs(1, video_id))
header_replay = enc.rs(1, S1) + enc.rs(3, S3) + enc.nm(4, 1)
return b64enc(header_replay)
ver 0.0.1 2019.10.05
'''
def _gen_vid(video_id): def _build(video_id, seektime, topchat_only, channel_id) -> str:
"""generate video_id parameter. chattype = 4 if topchat_only else 1
if seektime < 0:
seektime = 0
timestamp = int(seektime * 1000000)
header = enc.rs(3, _header(video_id, channel_id))
timestamp = enc.nm(5, timestamp)
s6 = enc.nm(6, 0)
s7 = enc.nm(7, 0)
s8 = enc.nm(8, 0)
s9 = enc.nm(9, 4)
s10 = enc.rs(10, enc.nm(4, 0))
chattype = enc.rs(14, enc.nm(1, 4))
s15 = enc.nm(15, 0)
entity = b''.join((header, timestamp, s6, s7, s8, s9, s10, chattype, s15))
continuation = enc.rs(156074452, entity)
return quote(b64enc(continuation).decode())
def getparam(video_id, seektime=0, topchat_only=False, channel_id='') -> str:
'''
Parameter Parameter
--------- ---------
video_id : str seektime : int
unit:seconds
Return start position of fetching chat data.
--------- topchat_only : bool
byte[] : base64 encoded video_id parameter. if True, fetch only 'top chat'
""" '''
header_magic = b'\x0A\x0F\x1A\x0D\x0A' return _build(video_id, seektime, topchat_only, channel_id)
header_id = video_id.encode()
header_sep_1 = b'\x1A\x13\xEA\xA8\xDD\xB9\x01\x0D\x0A\x0B'
header_terminator = b'\x20\x01'
item = [
header_magic,
_nval(len(header_id)),
header_id,
header_sep_1,
header_id,
header_terminator
]
return urllib.parse.quote(
b64enc(reduce(lambda x, y: x+y, item)).decode()
).encode()
def _nval(val):
"""convert value to byte array"""
if val<0: raise ValueError
buf = b''
while val >> 7:
m = val & 0xFF | 0x80
buf += m.to_bytes(1,'big')
val >>= 7
buf += val.to_bytes(1,'big')
return buf
def _tzparity(video_id,times):
t=0
for i,s in enumerate(video_id):
ss = ord(s)
if(ss % 2 == 0):
t += ss*(12-i)
else:
t ^= ss*i
return ((times^t) % 2).to_bytes(1,'big')
def get(video_id, seektime = 0, topchatonly = False):
switch_01 = b'\x04' if topchatonly else b'\x01'
if seektime < 0:
raise ValueError('seektime is 0 or positive number.')
if seektime == 0:
times =_nval(1)
switch = b'\x04'
else:
times =_nval(int(seektime*1000000))
switch = b'\x03'
parity = _tzparity(video_id, seektime)
header_magic= b'\xA2\x9D\xB0\xD3\x04'
sep_0 = b'\x1A'
vid = _gen_vid(video_id)
time_tag = b'\x28'
timestamp1 = times
sep_1 = b'\x30\x00\x38\x00\x40\x00\x48'
sep_2 = b'\x52\x1C\x08\x00\x10\x00\x18\x00\x20\x00'
chkstr = b'\x2A\x0E\x73\x74\x61\x74\x69\x63\x63\x68\x65\x63\x6B\x73\x75\x6D\x40'
sep_3 = b'\x00\x58\x03\x60'
sep_4 = b'\x68'+parity+b'\x72\x04\x08'
sep_5 = b'\x10'+parity+b'\x78\x00'
body = [
sep_0,
_nval(len(vid)),
vid,
time_tag,
timestamp1,
sep_1,
switch,
sep_2,
chkstr,
sep_3,
switch_01,
sep_4,
switch_01,
sep_5
]
body = reduce(lambda x, y: x+y, body)
return urllib.parse.quote(
b64enc( header_magic +
_nval(len(body)) +
body
).decode()
)

24
pytchat/paramgen/enc.py Normal file
View File

@@ -0,0 +1,24 @@
def vn(val):
if val < 0:
raise ValueError
buf = b''
while val >> 7:
m = val & 0xFF | 0x80
buf += m.to_bytes(1, 'big')
val >>= 7
buf += val.to_bytes(1, 'big')
return buf
def tp(a, b, ary):
return vn((b << 3) | a) + ary
def rs(a, ary):
if isinstance(ary, str):
ary = ary.encode()
return tp(2, a, vn(len(ary)) + ary)
def nm(a, ary):
return tp(0, a, vn(ary))

View File

@@ -1,166 +1,70 @@
from base64 import urlsafe_b64encode as b64enc
from functools import reduce
import calendar, datetime, pytz
import random import random
import urllib.parse import time
from . import enc
from base64 import urlsafe_b64encode as b64enc
from urllib.parse import quote
'''
Generate continuation parameter of youtube live chat.
Author: taizan-hokuto (2019) @taizan205 def _header(video_id, channel_id) -> str:
S1_3 = enc.rs(1, video_id)
S1_5 = enc.rs(1, channel_id) + enc.rs(2, video_id)
S1 = enc.rs(3, S1_3) + enc.rs(5, S1_5)
S3 = enc.rs(48687757, enc.rs(1, video_id))
header_replay = enc.rs(1, S1) + enc.rs(3, S3) + enc.nm(4, 1)
return b64enc(header_replay)
ver 0.0.1 2019.10.05
'''
def _gen_vid(video_id):
"""generate video_id parameter.
Parameter
---------
video_id : str
Return def _build(video_id, channel_id, ts1, ts2, ts3, ts4, ts5, topchat_only) -> str:
--------- chattype = 4 if topchat_only else 1
byte[] : base64 encoded video_id parameter.
"""
header_magic = b'\x0A\x0F\x0A\x0D\x0A'
header_id = video_id.encode()
header_sep_1 = b'\x1A'
header_sep_2 = b'\x43\xAA\xB9\xC1\xBD\x01\x3D\x0A'
header_suburl = ('https://www.youtube.com/live_chat?v='
f'{video_id}&is_popout=1').encode()
header_terminator = b'\x20\x02'
item = [ b1 = enc.nm(1, 0)
header_magic, b2 = enc.nm(2, 0)
_nval(len(header_id)), b3 = enc.nm(3, 0)
header_id, b4 = enc.nm(4, 0)
header_sep_1, b7 = enc.rs(7, '')
header_sep_2, b8 = enc.nm(8, 0)
_nval(len(header_suburl)), b9 = enc.rs(9, '')
header_suburl, timestamp2 = enc.nm(10, ts2)
header_terminator b11 = enc.nm(11, 3)
] b15 = enc.nm(15, 0)
return urllib.parse.quote( header = enc.rs(3, _header(video_id, channel_id))
b64enc(reduce(lambda x, y: x+y, item)).decode() timestamp1 = enc.nm(5, ts1)
).encode() s6 = enc.nm(6, 0)
s7 = enc.nm(7, 0)
s8 = enc.nm(8, 1)
body = enc.rs(9, b''.join(
(b1, b2, b3, b4, b7, b8, b9, timestamp2, b11, b15)))
timestamp3 = enc.nm(10, ts3)
timestamp4 = enc.nm(11, ts4)
s13 = enc.nm(13, chattype)
chattype = enc.rs(16, enc.nm(1, chattype))
s17 = enc.nm(17, 0)
str19 = enc.rs(19, enc.nm(1, 0))
timestamp5 = enc.nm(20, ts5)
entity = b''.join((header, timestamp1, s6, s7, s8, body, timestamp3,
timestamp4, s13, chattype, s17, str19, timestamp5))
continuation = enc.rs(119693434, entity)
return quote(b64enc(continuation).decode())
def _tzparity(video_id,times):
t=0
for i,s in enumerate(video_id):
ss = ord(s)
if(ss % 2 == 0):
t += ss*(12-i)
else:
t ^= ss*i
return ((times^t) % 2).to_bytes(1,'big')
def _nval(val):
"""convert value to byte array"""
if val<0: raise ValueError
buf = b''
while val >> 7:
m = val & 0xFF | 0x80
buf += m.to_bytes(1,'big')
val >>= 7
buf += val.to_bytes(1,'big')
return buf
def _build(video_id, _ts1, _ts2, _ts3, _ts4, _ts5, topchatonly = False):
#_short_type2
switch_01 = b'\x04' if topchatonly else b'\x01'
parity = _tzparity(video_id, _ts1^_ts2^_ts3^_ts4^_ts5)
header_magic= b'\xD2\x87\xCC\xC8\x03'
sep_0 = b'\x1A'
vid = _gen_vid(video_id)
time_tag = b'\x28'
timestamp1 = _nval(_ts1)
sep_1 = b'\x30\x00\x38\x00\x40\x02\x4A'
un_len = b'\x2B'
sep_2 = b'\x08'+parity+b'\x10\x00\x18\x00\x20\x00'
chkstr = b'\x2A\x0E\x73\x74\x61\x74\x69\x63\x63\x68\x65\x63\x6B\x73\x75\x6D'
sep_3 = b'\x3A\x00\x40\x00\x4A'
sep_4_len = b'\x02'
sep_4 = b'\x08\x01'
ts_2_start = b'\x50'
timestamp2 = _nval(_ts2)
ts_2_end = b'\x58'
sep_5 = b'\x03'
ts_3_start = b'\x50'
timestamp3 = _nval(_ts3)
ts_3_end = b'\x58'
timestamp4 = _nval(_ts4)
sep_6 = b'\x68'
#switch
sep_7 = b'\x82\x01\x04\x08'
#switch
sep_8 = b'\x10\x00'
sep_9 = b'\x88\x01\x00\xA0\x01'
timestamp5 = _nval(_ts5)
body = [
sep_0,
_nval(len(vid)),
vid,
time_tag,
timestamp1,
sep_1,
un_len,
sep_2,
chkstr,
sep_3,
sep_4_len,
sep_4,
ts_2_start,
timestamp2,
ts_2_end,
sep_5,
ts_3_start,
timestamp3,
ts_3_end,
timestamp4,
sep_6,
switch_01,#
sep_7,
switch_01,#
sep_8,
sep_9,
timestamp5
]
body = reduce(lambda x, y: x+y, body)
return urllib.parse.quote(
b64enc( header_magic +
_nval(len(body)) +
body
).decode()
)
def _times(past_sec): def _times(past_sec):
n = int(time.time())
def unixts_now(): _ts1 = n - random.uniform(0, 1 * 3)
now = datetime.datetime.now(pytz.utc) _ts2 = n - random.uniform(0.01, 0.99)
return calendar.timegm(now.utctimetuple()) _ts3 = n - past_sec + random.uniform(0, 1)
_ts4 = n - random.uniform(10 * 60, 60 * 60)
n = unixts_now() _ts5 = n - random.uniform(0.01, 0.99)
return list(map(lambda x: int(x * 1000000), [_ts1, _ts2, _ts3, _ts4, _ts5]))
_ts1= n - random.uniform(0,1*3)
_ts2= n - random.uniform(0.01,0.99)
_ts3= n - past_sec + random.uniform(0,1)
_ts4= n - random.uniform(10*60,60*60)
_ts5= n - random.uniform(0.01,0.99)
return list(map(lambda x:int(x*1000000),[_ts1,_ts2,_ts3,_ts4,_ts5]))
def getparam(video_id,past_sec = 60): def getparam(video_id, channel_id, past_sec=0, topchat_only=False) -> str:
''' '''
Parameter Parameter
--------- ---------
past_sec : int past_sec : int
seconds to load past chat data seconds to load past chat data
topchat_only : bool
if True, fetch only 'top chat'
''' '''
return _build(video_id,*_times(past_sec)) return _build(video_id, channel_id, *_times(past_sec), topchat_only)

View File

@@ -1,39 +1,122 @@
import json """
from .. import config pytchat.parser.live
from .. import mylogger ~~~~~~~~~~~~~~~~~~~
from .. exceptions import ( Parser of live chat JSON.
ResponseContextError, """
NoContentsException,
NoContinuationsException )
from .. import exceptions
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
class Parser: class Parser:
def parse(self, jsn): '''
if jsn is None: Parser of chat json.
return {'timeoutMs':0,'continuation':None},[]
if jsn['response']['responseContext'].get('errors'): Parameter
raise ResponseContextError('動画に接続できません。' ----------
'動画IDが間違っているか、動画が削除非公開の可能性があります。') is_replay : bool
contents=jsn['response'].get('continuationContents')
#配信が終了した場合、もしくはチャットデータが取得できない場合 exception_holder : Object [default:Npne]
The object holding exceptions.
This is passed from the parent livechat object.
'''
__slots__ = ['is_replay', 'exception_holder']
def __init__(self, is_replay, exception_holder=None):
self.is_replay = is_replay
self.exception_holder = exception_holder
def get_contents(self, jsn):
if jsn is None:
self.raise_exception(exceptions.IllegalFunctionCall('Called with none JSON object.'))
if jsn.get("responseContext", {}).get("errors"):
raise exceptions.ResponseContextError(
'The video_id would be wrong, or video is deleted or private.')
contents = jsn.get('continuationContents')
visitor_data = jsn.get("responseContext", {}).get("visitorData")
return contents, visitor_data
def parse(self, contents):
"""
Parameter
----------
+ contents : dict
+ JSON of chat data from YouTube.
Returns
-------
tuple:
+ metadata : dict
+ timeout
+ video_id
+ continuation
+ chatdata : List[dict]
"""
if contents is None: if contents is None:
raise NoContentsException('チャットデータを取得できませんでした。') '''Broadcasting end or cannot fetch chat stream'''
self.raise_exception(exceptions.NoContents('Chat data stream is empty.'))
cont = contents['liveChatContinuation']['continuations'][0] cont = contents['liveChatContinuation']['continuations'][0]
if cont is None: if cont is None:
raise NoContinuationsException('Continuationがありません。') self.raise_exception(exceptions.NoContinuation('No Continuation'))
metadata = (cont.get('invalidationContinuationData') or metadata = (cont.get('invalidationContinuationData')
cont.get('timedContinuationData') or or cont.get('timedContinuationData')
cont.get('reloadContinuationData') or cont.get('reloadContinuationData')
or cont.get('liveChatReplayContinuationData')
) )
if metadata is None: if metadata is None:
if cont.get("playerSeekContinuationData"):
self.raise_exception(exceptions.ChatDataFinished('Finished chat data'))
unknown = list(cont.keys())[0] unknown = list(cont.keys())[0]
if unknown: if unknown:
logger.error(f"Received unknown continuation type:{unknown}") self.raise_exception(exceptions.ReceivedUnknownContinuation(
metadata = cont.get(unknown) f"Received unknown continuation type:{unknown}"))
metadata.setdefault('timeoutMs', 10000) else:
chatdata = contents['liveChatContinuation'].get('actions') self.raise_exception(exceptions.FailedExtractContinuation('Cannot extract continuation data'))
return self._create_data(metadata, contents)
def reload_continuation(self, contents):
"""
When `seektime == 0` or seektime is abbreviated ,
check if fetched chat json has no chat data.
If so, try to fetch playerSeekContinuationData.
This function must be run only first fetching.
"""
if contents is None:
'''Broadcasting end or cannot fetch chat stream'''
self.raise_exception(exceptions.NoContents('Chat data stream is empty.'))
cont = contents['liveChatContinuation']['continuations'][0]
if cont.get("liveChatReplayContinuationData"):
# chat data exist.
return None
# chat data do not exist, get playerSeekContinuationData.
init_cont = cont.get("playerSeekContinuationData")
if init_cont:
return init_cont.get("continuation")
self.raise_exception(exceptions.ChatDataFinished('Finished chat data'))
def _create_data(self, metadata, contents):
actions = contents['liveChatContinuation'].get('actions')
if self.is_replay:
last_offset_ms = self._get_lastoffset(actions)
metadata.setdefault("timeoutMs", 5000)
metadata.setdefault("last_offset_ms", last_offset_ms)
"""Archived chat has different structures than live chat,
so make it the same format."""
chatdata = [action["replayChatItemAction"]["actions"][0]
for action in actions]
else:
metadata.setdefault('timeoutMs', 5000)
chatdata = actions
return metadata, chatdata return metadata, chatdata
def _get_lastoffset(self, actions: list):
if actions:
return int(actions[-1]["replayChatItemAction"]["videoOffsetTimeMsec"])
return 0
def raise_exception(self, exception):
if self.exception_holder is None:
raise exception
self.exception_holder = exception

View File

@@ -1,52 +0,0 @@
import json
from .. import config
from .. import mylogger
from .. exceptions import (
ResponseContextError,
NoContentsException,
NoContinuationsException )
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE)
class Parser:
def parse(self, jsn):
if jsn is None:
return {'timeoutMs':0,'continuation':None},[]
if jsn['response']['responseContext'].get('errors'):
raise ResponseContextError('動画に接続できません。'
'動画IDが間違っているか、動画が削除非公開の可能性があります。')
contents=jsn['response'].get('continuationContents')
#配信が終了した場合、もしくはチャットデータが取得できない場合
if contents is None:
raise NoContentsException('チャットデータを取得できませんでした。')
cont = contents['liveChatContinuation']['continuations'][0]
if cont is None:
raise NoContinuationsException('Continuationがありません。')
metadata = cont.get('liveChatReplayContinuationData')
if metadata is None:
unknown = list(cont.keys())[0]
if unknown:
metadata = cont.get(unknown)
actions = contents['liveChatContinuation'].get('actions')
if actions is None:
raise NoContentsException('チャットデータを取得できませんでした。')
interval = self.get_interval(actions)
metadata.setdefault("timeoutMs",interval)
chatdata = []
for action in actions:
chatdata.append(action["replayChatItemAction"]["actions"][0])
return metadata, chatdata
def get_interval(self, actions: list):
if actions is None:
return 0
start = int(actions[0]["replayChatItemAction"]["videoOffsetTimeMsec"])
last = int(actions[-1]["replayChatItemAction"]["videoOffsetTimeMsec"])
return (last - start)

View File

@@ -1,28 +1,30 @@
class ChatProcessor: class ChatProcessor:
''' '''
Listenerからチャットデータactionsを受け取り Abstract class that processes chat data.
チャットデータを加工するクラスの抽象クラス Receive chat data (actions) from Listener.
''' '''
def process(self, chat_components: list): def process(self, chat_components: list):
''' '''
チャットデータの加工を表すインターフェース。 Interface that represents processing of chat data.
LiveChatオブジェクトから呼び出される。 Called from LiveChat object.
Parameter Parameter
---------- ----------
chat_components: [LIST:component] chat_components: List[component]
component : dict { component : dict {
"video_id" : str "video_id" : str
動画ID
"timeout" : int "timeout" : int
次のチャットの再読み込みまでの時間(秒) Time to fetch next chat (seconds)
"chatdata" : list<object> "chatdata" : List[dict]
チャットデータactionsのリスト List of chat data.
} }
''' '''
pass pass
def finalize(self, *args, **kwargs):
'''
Interface for finalizing the process.
Called when chat fetching finished.
'''
pass

View File

@@ -0,0 +1,42 @@
from .chat_processor import ChatProcessor
class Combinator(ChatProcessor):
'''
Combinator combines multiple chat processors.
Specify processors as tuple at `processor` params of LiveChat object.
For example:
[constructor]
chat = LiveChat("video_id", processor = ( Processor1(), Processor2(), Processor3() ) )
[receive return values]
ret1, ret2, ret3 = chat.get()
The return values are tuple of processed chat data,
the order of return depends on parameter order.
Parameter
---------
processors : Tuple[ChatProcessor]
multiple processors for processing chat data
'''
def __init__(self, processors: tuple):
self.processors = processors
def process(self, chat_components: list):
'''
Called from LiveChat.get() function by user,
or LiveChat._listen() automatically.
Returns
-------
Tuple of chat data processed by each chat processor.
'''
return tuple(processor.process(chat_components)
for processor in self.processors)
def finalize(self, *args, **kwargs):
[processor.finalize(*args, **kwargs)
for processor in self.processors]

View File

@@ -1,20 +1,20 @@
import datetime
import time
from .renderer.textmessage import LiveChatTextMessageRenderer from .renderer.textmessage import LiveChatTextMessageRenderer
from .renderer.paidmessage import LiveChatPaidMessageRenderer from .renderer.paidmessage import LiveChatPaidMessageRenderer
from .renderer.paidsticker import LiveChatPaidStickerRenderer from .renderer.paidsticker import LiveChatPaidStickerRenderer
from .renderer.legacypaid import LiveChatLegacyPaidMessageRenderer from .renderer.legacypaid import LiveChatLegacyPaidMessageRenderer
from ... import mylogger from .renderer.membership import LiveChatMembershipItemRenderer
from .. chat_processor import ChatProcessor
from ... import config from ... import config
logger = mylogger.get_logger(__name__,mode=config.LOGGER_MODE) logger = config.logger(__name__)
class CompatibleProcessor:
class CompatibleProcessor(ChatProcessor):
def process(self, chat_components: list): def process(self, chat_components: list):
chatlist = [] chatlist = []
timeout = 0 timeout = 0
ret={} ret = {}
ret["kind"] = "youtube#liveChatMessageListResponse" ret["kind"] = "youtube#liveChatMessageListResponse"
ret["etag"] = "" ret["etag"] = ""
ret["nextPageToken"] = "" ret["nextPageToken"] = ""
@@ -23,20 +23,24 @@ class CompatibleProcessor:
for chat_component in chat_components: for chat_component in chat_components:
timeout += chat_component.get('timeout', 0) timeout += chat_component.get('timeout', 0)
chatdata = chat_component.get('chatdata') chatdata = chat_component.get('chatdata')
if chatdata is None: break if chatdata is None:
break
for action in chatdata: for action in chatdata:
if action is None: continue if action is None:
if action.get('addChatItemAction') is None: continue continue
if action['addChatItemAction'].get('item') is None: continue if action.get('addChatItemAction') is None:
continue
if action['addChatItemAction'].get('item') is None:
continue
chat = self.parse(action) chat = self.parse(action)
if chat: if chat:
chatlist.append(chat) chatlist.append(chat)
ret["pollingIntervalMillis"] = int(timeout*1000) ret["pollingIntervalMillis"] = int(timeout * 1000)
ret["pageInfo"]={ ret["pageInfo"] = {
"totalResults":len(chatlist), "totalResults": len(chatlist),
"resultsPerPage":len(chatlist), "resultsPerPage": len(chatlist),
} }
ret["items"] = chatlist ret["items"] = chatlist
@@ -47,35 +51,37 @@ class CompatibleProcessor:
action = sitem.get("addChatItemAction") action = sitem.get("addChatItemAction")
if action: if action:
item = action.get("item") item = action.get("item")
if item is None: return None if item is None:
rd={} return None
rd = {}
try: try:
renderer = self.get_renderer(item) renderer = self.get_renderer(item)
if renderer == None: if renderer is None:
return None return None
rd["kind"] = "youtube#liveChatMessage" rd["kind"] = "youtube#liveChatMessage"
rd["etag"] = "" rd["etag"] = ""
rd["id"] = 'LCC.' + renderer.get_id() rd["id"] = 'LCC.' + renderer.get_id()
rd["snippet"] = renderer.get_snippet() rd["snippet"] = renderer.get_snippet()
rd["authorDetails"] = renderer.get_authordetails() rd["authorDetails"] = renderer.get_authordetails()
except (KeyError,TypeError,AttributeError) as e: except (KeyError, TypeError, AttributeError) as e:
logger.error(f"Error: {str(type(e))}-{str(e)}") logger.error(f"Error: {str(type(e))}-{str(e)}")
logger.error(f"item: {sitem}") logger.error(f"item: {sitem}")
return None return None
return rd return rd
def get_renderer(self, item): def get_renderer(self, item):
if item.get("liveChatTextMessageRenderer"): if item.get("liveChatTextMessageRenderer"):
renderer = LiveChatTextMessageRenderer(item) renderer = LiveChatTextMessageRenderer(item)
elif item.get("liveChatPaidMessageRenderer"): elif item.get("liveChatPaidMessageRenderer"):
renderer = LiveChatPaidMessageRenderer(item) renderer = LiveChatPaidMessageRenderer(item)
elif item.get( "liveChatPaidStickerRenderer"): elif item.get("liveChatPaidStickerRenderer"):
renderer = LiveChatPaidStickerRenderer(item) renderer = LiveChatPaidStickerRenderer(item)
elif item.get("liveChatLegacyPaidMessageRenderer"): elif item.get("liveChatLegacyPaidMessageRenderer"):
renderer = LiveChatLegacyPaidMessageRenderer(item) renderer = LiveChatLegacyPaidMessageRenderer(item)
elif item.get("liveChatMembershipItemRenderer"):
renderer = LiveChatMembershipItemRenderer(item)
else: else:
renderer = None renderer = None
return renderer return renderer

View File

@@ -1,83 +1,82 @@
import datetime, pytz from datetime import datetime, timedelta, timezone
TZ_UTC = timezone(timedelta(0), 'UTC')
class BaseRenderer: class BaseRenderer:
def __init__(self, item, chattype): def __init__(self, item, chattype):
self.renderer = list(item.values())[0] self.renderer = list(item.values())[0]
self.chattype = chattype self.chattype = chattype
def get_snippet(self): def get_snippet(self):
message = self.get_message(self.renderer) message = self.get_message(self.renderer)
return { return {
"type" : self.chattype, "type": self.chattype,
"liveChatId" : "", "liveChatId": "",
"authorChannelId" : self.renderer.get("authorExternalChannelId"), "authorChannelId": self.renderer.get("authorExternalChannelId"),
"publishedAt" : self.get_publishedat(self.renderer.get("timestampUsec",0)), "publishedAt": self.get_publishedat(self.renderer.get("timestampUsec", 0)),
"hasDisplayContent" : True, "hasDisplayContent": True,
"displayMessage" : message, "displayMessage": message,
"textMessageDetails": { "textMessageDetails": {
"messageText" : message "messageText": message
} }
} }
def get_authordetails(self): def get_authordetails(self):
authorExternalChannelId = self.renderer.get("authorExternalChannelId") authorExternalChannelId = self.renderer.get("authorExternalChannelId")
#parse subscriber type # parse subscriber type
isVerified, isChatOwner, isChatSponsor, isChatModerator = ( isVerified, isChatOwner, isChatSponsor, isChatModerator = (
self.get_badges(self.renderer) self.get_badges(self.renderer)
) )
return { return {
"channelId" : authorExternalChannelId, "channelId": authorExternalChannelId,
"channelUrl" : "http://www.youtube.com/channel/"+authorExternalChannelId, "channelUrl": "http://www.youtube.com/channel/" + authorExternalChannelId,
"displayName" : self.renderer["authorName"]["simpleText"], "displayName": self.renderer["authorName"]["simpleText"],
"profileImageUrl" : self.renderer["authorPhoto"]["thumbnails"][1]["url"] , "profileImageUrl": self.renderer["authorPhoto"]["thumbnails"][1]["url"],
"isVerified" : isVerified, "isVerified": isVerified,
"isChatOwner" : isChatOwner, "isChatOwner": isChatOwner,
"isChatSponsor" : isChatSponsor, "isChatSponsor": isChatSponsor,
"isChatModerator" : isChatModerator "isChatModerator": isChatModerator
} }
def get_message(self, renderer):
def get_message(self,renderer):
message = '' message = ''
if renderer.get("message"): if renderer.get("message"):
runs=renderer["message"].get("runs") runs = renderer["message"].get("runs")
if runs: if runs:
for r in runs: for r in runs:
if r: if r:
if r.get('emoji'): if r.get('emoji'):
message += r['emoji'].get('shortcuts',[''])[0] message += r['emoji'].get('shortcuts', [''])[0]
else: else:
message += r.get('text','') message += r.get('text', '')
return message return message
def get_badges(self,renderer): def get_badges(self, renderer):
isVerified = False isVerified = False
isChatOwner = False isChatOwner = False
isChatSponsor = False isChatSponsor = False
isChatModerator = False isChatModerator = False
badges=renderer.get("authorBadges") badges = renderer.get("authorBadges")
if badges: if badges:
for badge in badges: for badge in badges:
author_type = badge["liveChatAuthorBadgeRenderer"]["accessibility"]["accessibilityData"]["label"] author_type = badge["liveChatAuthorBadgeRenderer"]["accessibility"]["accessibilityData"]["label"]
if author_type == '確認済み': if author_type == 'VERIFIED' or author_type == '確認済み':
isVerified = True isVerified = True
if author_type == '所有者': if author_type == 'OWNER' or author_type == '所有者':
isChatOwner = True isChatOwner = True
if 'メンバー' in author_type: if 'メンバー' in author_type or 'MEMBER' in author_type:
isChatSponsor = True isChatSponsor = True
if author_type == 'モデレーター': if author_type == 'MODERATOR' or author_type == 'モデレーター':
isChatModerator = True isChatModerator = True
return isVerified, isChatOwner, isChatSponsor, isChatModerator return isVerified, isChatOwner, isChatSponsor, isChatModerator
def get_id(self): def get_id(self):
return self.renderer.get('id') return self.renderer.get('id')
def get_publishedat(self,timestamp): def get_publishedat(self, timestamp):
dt = datetime.datetime.fromtimestamp(int(timestamp)/1000000) dt = datetime.fromtimestamp(int(timestamp) / 1000000)
return dt.astimezone(pytz.utc).isoformat( return dt.astimezone(TZ_UTC).isoformat(
timespec='milliseconds').replace('+00:00','Z') timespec='milliseconds').replace('+00:00', 'Z')

View File

@@ -1,12 +1,12 @@
''' '''
YouTubeスーパーチャットで使用される通貨の記号とレート検索用の略号の Table of symbols for the currencies used in YouTube Superchat.
対応表
Key Key
YouTubeスーパーチャットで使用される通貨の記号 Currency symbols used in YouTube Super Chat
(アルファベットで終わる場合、0xA0(&npsp)が付く) If it ends with an alphabet, it will be followed by 0xA0(&npsp).
Value: Value:
fxtext: 3文字の通貨略称 fxtext: ISO 4217 currency code
jptest: 日本語テキスト jptest: japanese text
''' '''
symbols = { symbols = {
"$": {"fxtext": "USD", "jptext": "米・ドル"}, "$": {"fxtext": "USD", "jptext": "米・ドル"},
@@ -33,5 +33,6 @@ symbols = {
"ARS\xa0": {"fxtext": "ARS", "jptext": "アルゼンチン・ペソ"}, "ARS\xa0": {"fxtext": "ARS", "jptext": "アルゼンチン・ペソ"},
"CLP\xa0": {"fxtext": "CLP", "jptext": "チリ・ペソ"}, "CLP\xa0": {"fxtext": "CLP", "jptext": "チリ・ペソ"},
"NOK\xa0": {"fxtext": "NOK", "jptext": "ノルウェー・クローネ"}, "NOK\xa0": {"fxtext": "NOK", "jptext": "ノルウェー・クローネ"},
"BAM\xa0": {"fxtext": "BAM", "jptext": "ボスニア・兌換マルカ"} "BAM\xa0": {"fxtext": "BAM", "jptext": "ボスニア・兌換マルカ"},
} "SGD\xa0": {"fxtext": "SGD", "jptext": "シンガポール・ドル"}
}

View File

@@ -1,4 +1,6 @@
from .base import BaseRenderer from .base import BaseRenderer
class LiveChatLegacyPaidMessageRenderer(BaseRenderer): class LiveChatLegacyPaidMessageRenderer(BaseRenderer):
def __init__(self, item): def __init__(self, item):
super().__init__(item, "newSponsorEvent") super().__init__(item, "newSponsorEvent")
@@ -8,36 +10,33 @@ class LiveChatLegacyPaidMessageRenderer(BaseRenderer):
message = self.get_message(self.renderer) message = self.get_message(self.renderer)
return { return {
"type" : self.chattype, "type": self.chattype,
"liveChatId" : "", "liveChatId": "",
"authorChannelId" : self.renderer.get("authorExternalChannelId"), "authorChannelId": self.renderer.get("authorExternalChannelId"),
"publishedAt" : self.get_publishedat(self.renderer.get("timestampUsec",0)), "publishedAt": self.get_publishedat(self.renderer.get("timestampUsec", 0)),
"hasDisplayContent" : True, "hasDisplayContent": True,
"displayMessage" : message, "displayMessage": message,
} }
def get_authordetails(self): def get_authordetails(self):
authorExternalChannelId = self.renderer.get("authorExternalChannelId") authorExternalChannelId = self.renderer.get("authorExternalChannelId")
#parse subscriber type # parse subscriber type
isVerified, isChatOwner, _, isChatModerator = ( isVerified, isChatOwner, _, isChatModerator = (
self.get_badges(self.renderer) self.get_badges(self.renderer)
) )
return { return {
"channelId" : authorExternalChannelId, "channelId": authorExternalChannelId,
"channelUrl" : "http://www.youtube.com/channel/"+authorExternalChannelId, "channelUrl": "http://www.youtube.com/channel/" + authorExternalChannelId,
"displayName" : self.renderer["authorName"]["simpleText"], "displayName": self.renderer["authorName"]["simpleText"],
"profileImageUrl" : self.renderer["authorPhoto"]["thumbnails"][1]["url"] , "profileImageUrl": self.renderer["authorPhoto"]["thumbnails"][1]["url"],
"isVerified" : isVerified, "isVerified": isVerified,
"isChatOwner" : isChatOwner, "isChatOwner": isChatOwner,
"isChatSponsor" : True, "isChatSponsor": True,
"isChatModerator" : isChatModerator "isChatModerator": isChatModerator
} }
def get_message(self, renderer):
def get_message(self,renderer):
message = (renderer["eventText"]["runs"][0]["text"] message = (renderer["eventText"]["runs"][0]["text"]
)+' / '+(renderer["detailText"]["simpleText"]) ) + ' / ' + (renderer["detailText"]["simpleText"])
return message return message

View File

@@ -0,0 +1,40 @@
from .base import BaseRenderer
class LiveChatMembershipItemRenderer(BaseRenderer):
def __init__(self, item):
super().__init__(item, "newSponsorEvent")
def get_snippet(self):
message = self.get_message(self.renderer)
return {
"type": self.chattype,
"liveChatId": "",
"authorChannelId": self.renderer.get("authorExternalChannelId"),
"publishedAt": self.get_publishedat(self.renderer.get("timestampUsec", 0)),
"hasDisplayContent": True,
"displayMessage": message,
}
def get_authordetails(self):
authorExternalChannelId = self.renderer.get("authorExternalChannelId")
# parse subscriber type
isVerified, isChatOwner, _, isChatModerator = (
self.get_badges(self.renderer)
)
return {
"channelId": authorExternalChannelId,
"channelUrl": "http://www.youtube.com/channel/" + authorExternalChannelId,
"displayName": self.renderer["authorName"]["simpleText"],
"profileImageUrl": self.renderer["authorPhoto"]["thumbnails"][1]["url"],
"isVerified": isVerified,
"isChatOwner": isChatOwner,
"isChatSponsor": True,
"isChatModerator": isChatModerator
}
def get_message(self, renderer):
message = ''.join([mes.get("text", "")
for mes in renderer["headerSubtext"]["runs"]])
return message, [message]

View File

@@ -3,6 +3,7 @@ from . import currency
from .base import BaseRenderer from .base import BaseRenderer
superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$") superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$")
class LiveChatPaidMessageRenderer(BaseRenderer): class LiveChatPaidMessageRenderer(BaseRenderer):
def __init__(self, item): def __init__(self, item):
super().__init__(item, "superChatEvent") super().__init__(item, "superChatEvent")
@@ -10,32 +11,32 @@ class LiveChatPaidMessageRenderer(BaseRenderer):
def get_snippet(self): def get_snippet(self):
authorName = self.renderer["authorName"]["simpleText"] authorName = self.renderer["authorName"]["simpleText"]
message = self.get_message(self.renderer) message = self.get_message(self.renderer)
amountDisplayString, symbol, amountMicros =( amountDisplayString, symbol, amountMicros = (
self.get_amountdata(self.renderer) self.get_amountdata(self.renderer)
) )
return { return {
"type" : self.chattype, "type": self.chattype,
"liveChatId" : "", "liveChatId": "",
"authorChannelId" : self.renderer.get("authorExternalChannelId"), "authorChannelId": self.renderer.get("authorExternalChannelId"),
"publishedAt" : self.get_publishedat(self.renderer.get("timestampUsec",0)), "publishedAt": self.get_publishedat(self.renderer.get("timestampUsec", 0)),
"hasDisplayContent" : True, "hasDisplayContent": True,
"displayMessage" : amountDisplayString+" from "+authorName+': \"'+ message+'\"', "displayMessage": amountDisplayString + " from " + authorName + ': \"' + message + '\"',
"superChatDetails" : { "superChatDetails": {
"amountMicros" : amountMicros, "amountMicros": amountMicros,
"currency" : currency.symbols[symbol]["fxtext"] if currency.symbols.get(symbol) else symbol, "currency": currency.symbols[symbol]["fxtext"] if currency.symbols.get(symbol) else symbol,
"amountDisplayString" : amountDisplayString, "amountDisplayString": amountDisplayString,
"tier" : 0, "tier": 0,
"backgroundColor" : self.renderer.get("bodyBackgroundColor", 0) "backgroundColor": self.renderer.get("bodyBackgroundColor", 0)
} }
} }
def get_amountdata(self,renderer): def get_amountdata(self, renderer):
amountDisplayString = renderer["purchaseAmountText"]["simpleText"] amountDisplayString = renderer["purchaseAmountText"]["simpleText"]
m = superchat_regex.search(amountDisplayString) m = superchat_regex.search(amountDisplayString)
if m: if m:
symbol = m.group(1) symbol = m.group(1)
amountMicros = int(float(m.group(2).replace(',',''))*1000000) amountMicros = int(float(m.group(2).replace(',', '')) * 1000000)
else: else:
symbol = "" symbol = ""
amountMicros = 0 amountMicros = 0
return amountDisplayString, symbol, amountMicros return amountDisplayString, symbol, amountMicros

View File

@@ -3,46 +3,45 @@ from . import currency
from .base import BaseRenderer from .base import BaseRenderer
superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$") superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$")
class LiveChatPaidStickerRenderer(BaseRenderer): class LiveChatPaidStickerRenderer(BaseRenderer):
def __init__(self, item): def __init__(self, item):
super().__init__(item, "superStickerEvent") super().__init__(item, "superStickerEvent")
def get_snippet(self): def get_snippet(self):
authorName = self.renderer["authorName"]["simpleText"] authorName = self.renderer["authorName"]["simpleText"]
amountDisplayString, symbol, amountMicros =( amountDisplayString, symbol, amountMicros = (
self.get_amountdata(self.renderer) self.get_amountdata(self.renderer)
) )
return { return {
"type" : self.chattype, "type": self.chattype,
"liveChatId" : "", "liveChatId": "",
"authorChannelId" : self.renderer.get("authorExternalChannelId"), "authorChannelId": self.renderer.get("authorExternalChannelId"),
"publishedAt" : self.get_publishedat(self.renderer.get("timestampUsec",0)), "publishedAt": self.get_publishedat(self.renderer.get("timestampUsec", 0)),
"hasDisplayContent" : True, "hasDisplayContent": True,
"displayMessage" : "Super Sticker " + amountDisplayString + " from "+authorName, "displayMessage": "Super Sticker " + amountDisplayString + " from " + authorName,
"superStickerDetails" : { "superStickerDetails": {
"superStickerMetaData" : { "superStickerMetaData": {
"stickerId": "", "stickerId": "",
"altText": "", "altText": "",
"language": "" "language": ""
}, },
"amountMicros" : amountMicros, "amountMicros": amountMicros,
"currency" : currency.symbols[symbol]["fxtext"] if currency.symbols.get(symbol) else symbol, "currency": currency.symbols[symbol]["fxtext"] if currency.symbols.get(symbol) else symbol,
"amountDisplayString" : amountDisplayString, "amountDisplayString": amountDisplayString,
"tier" : 0, "tier": 0,
"backgroundColor" : self.renderer.get("bodyBackgroundColor", 0) "backgroundColor": self.renderer.get("bodyBackgroundColor", 0)
} }
} }
def get_amountdata(self,renderer): def get_amountdata(self, renderer):
amountDisplayString = renderer["purchaseAmountText"]["simpleText"] amountDisplayString = renderer["purchaseAmountText"]["simpleText"]
m = superchat_regex.search(amountDisplayString) m = superchat_regex.search(amountDisplayString)
if m: if m:
symbol = m.group(1) symbol = m.group(1)
amountMicros = int(float(m.group(2).replace(',',''))*1000000) amountMicros = int(float(m.group(2).replace(',', '')) * 1000000)
else: else:
symbol = "" symbol = ""
amountMicros = 0 amountMicros = 0
return amountDisplayString, symbol, amountMicros return amountDisplayString, symbol, amountMicros

View File

@@ -1,4 +1,6 @@
from .base import BaseRenderer from .base import BaseRenderer
class LiveChatTextMessageRenderer(BaseRenderer): class LiveChatTextMessageRenderer(BaseRenderer):
def __init__(self, item): def __init__(self, item):
super().__init__(item, "textMessageEvent") super().__init__(item, "textMessageEvent")

View File

@@ -0,0 +1,11 @@
import json
from .renderer.base import Author
from .renderer.paidmessage import Colors
from .renderer.paidsticker import Colors2
class CustomEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Author) or isinstance(obj, Colors) or isinstance(obj, Colors2):
return vars(obj)
return json.JSONEncoder.default(self, obj)

View File

@@ -1,29 +1,132 @@
import asyncio import asyncio
import json
import time import time
from .custom_encoder import CustomEncoder
from .renderer.textmessage import LiveChatTextMessageRenderer from .renderer.textmessage import LiveChatTextMessageRenderer
from .renderer.paidmessage import LiveChatPaidMessageRenderer from .renderer.paidmessage import LiveChatPaidMessageRenderer
from .renderer.paidsticker import LiveChatPaidStickerRenderer from .renderer.paidsticker import LiveChatPaidStickerRenderer
from .renderer.legacypaid import LiveChatLegacyPaidMessageRenderer from .renderer.legacypaid import LiveChatLegacyPaidMessageRenderer
from .renderer.membership import LiveChatMembershipItemRenderer
from .. chat_processor import ChatProcessor
from ... import config
logger = config.logger(__name__)
class Chat:
def json(self) -> str:
return json.dumps(vars(self), ensure_ascii=False, cls=CustomEncoder)
class Chatdata: class Chatdata:
def __init__(self,chatlist:list, timeout:float):
def __init__(self, chatlist: list, timeout: float, abs_diff):
self.items = chatlist self.items = chatlist
self.interval = timeout self.interval = timeout
self.abs_diff = abs_diff
self.itemcount = 0
def tick(self): def tick(self):
if self.interval == 0: '''DEPRECATE
Use sync_items()
'''
if len(self.items) < 1:
time.sleep(1) time.sleep(1)
return return
time.sleep(self.interval/len(self.items)) if self.itemcount == 0:
self.starttime = time.time()
if len(self.items) == 1:
total_itemcount = 1
else:
total_itemcount = len(self.items) - 1
next_chattime = (self.items[0].timestamp + (self.items[-1].timestamp - self.items[0].timestamp) / total_itemcount * self.itemcount) / 1000
tobe_disptime = self.abs_diff + next_chattime
wait_sec = tobe_disptime - time.time()
self.itemcount += 1
if wait_sec < 0:
wait_sec = 0
time.sleep(wait_sec)
async def tick_async(self): async def tick_async(self):
if self.interval == 0: '''DEPRECATE
Use async_items()
'''
if len(self.items) < 1:
await asyncio.sleep(1) await asyncio.sleep(1)
return return
await asyncio.sleep(self.interval/len(self.items)) if self.itemcount == 0:
self.starttime = time.time()
if len(self.items) == 1:
total_itemcount = 1
else:
total_itemcount = len(self.items) - 1
next_chattime = (self.items[0].timestamp + (self.items[-1].timestamp - self.items[0].timestamp) / total_itemcount * self.itemcount) / 1000
tobe_disptime = self.abs_diff + next_chattime
wait_sec = tobe_disptime - time.time()
self.itemcount += 1
if wait_sec < 0:
wait_sec = 0
await asyncio.sleep(wait_sec)
def sync_items(self):
starttime = time.time()
if len(self.items) > 0:
last_chattime = self.items[-1].timestamp / 1000
tobe_disptime = self.abs_diff + last_chattime
wait_total_sec = max(tobe_disptime - time.time(), 0)
if len(self.items) > 1:
wait_sec = wait_total_sec / len(self.items)
elif len(self.items) == 1:
wait_sec = 0
for c in self.items:
if wait_sec < 0:
wait_sec = 0
time.sleep(wait_sec)
yield c
stop_interval = time.time() - starttime
if stop_interval < 1:
time.sleep(1 - stop_interval)
async def async_items(self):
starttime = time.time()
if len(self.items) > 0:
last_chattime = self.items[-1].timestamp / 1000
tobe_disptime = self.abs_diff + last_chattime
wait_total_sec = max(tobe_disptime - time.time(), 0)
if len(self.items) > 1:
wait_sec = wait_total_sec / len(self.items)
elif len(self.items) == 1:
wait_sec = 0
for c in self.items:
if wait_sec < 0:
wait_sec = 0
await asyncio.sleep(wait_sec)
yield c
stop_interval = time.time() - starttime
if stop_interval < 1:
await asyncio.sleep(1 - stop_interval)
def json(self) -> str:
return ''.join(("[", ','.join((a.json() for a in self.items)), "]"))
class DefaultProcessor(ChatProcessor):
def __init__(self):
self.first = True
self.abs_diff = 0
self.renderers = {
"liveChatTextMessageRenderer": LiveChatTextMessageRenderer(),
"liveChatPaidMessageRenderer": LiveChatPaidMessageRenderer(),
"liveChatPaidStickerRenderer": LiveChatPaidStickerRenderer(),
"liveChatLegacyPaidMessageRenderer": LiveChatLegacyPaidMessageRenderer(),
"liveChatMembershipItemRenderer": LiveChatMembershipItemRenderer()
}
class DefaultProcessor:
def process(self, chat_components: list): def process(self, chat_components: list):
chatlist = [] chatlist = []
@@ -31,49 +134,46 @@ class DefaultProcessor:
if chat_components: if chat_components:
for component in chat_components: for component in chat_components:
if component is None:
continue
timeout += component.get('timeout', 0) timeout += component.get('timeout', 0)
chatdata = component.get('chatdata') chatdata = component.get('chatdata') # if from Extractor, chatdata is generator.
if chatdata is None:
if chatdata is None: continue continue
for action in chatdata: for action in chatdata:
if action is None: continue if action is None:
if action.get('addChatItemAction') is None: continue continue
if action['addChatItemAction'].get('item') is None: continue if action.get('addChatItemAction') is None:
continue
chat = self.parse(action) item = action['addChatItemAction'].get('item')
if item is None:
continue
chat = self._parse(item)
if chat: if chat:
chatlist.append(chat) chatlist.append(chat)
return Chatdata(chatlist, float(timeout))
if self.first and chatlist:
self.abs_diff = time.time() - chatlist[0].timestamp / 1000
self.first = False
def parse(self, sitem): chatdata = Chatdata(chatlist, float(timeout), self.abs_diff)
action = sitem.get("addChatItemAction") return chatdata
if action:
item = action.get("item") def _parse(self, item):
if item is None: return None
try: try:
renderer = self.get_renderer(item) key = list(item.keys())[0]
if renderer == None: renderer = self.renderers.get(key)
if renderer is None:
return None return None
renderer.setitem(item.get(key), Chat())
renderer.settype()
renderer.get_snippet() renderer.get_snippet()
renderer.get_authordetails() renderer.get_authordetails()
except (KeyError,TypeError,AttributeError) as e: rendered_chatobj = renderer.get_chatobj()
print(f"------{str(type(e))}-{str(e)}----------") renderer.clear()
print(sitem) except (KeyError, TypeError) as e:
logger.error(f"{str(type(e))}-{str(e)} item:{str(item)}")
return None return None
return renderer
return rendered_chatobj
def get_renderer(self, item):
if item.get("liveChatTextMessageRenderer"):
renderer = LiveChatTextMessageRenderer(item)
elif item.get("liveChatPaidMessageRenderer"):
renderer = LiveChatPaidMessageRenderer(item)
elif item.get( "liveChatPaidStickerRenderer"):
renderer = LiveChatPaidStickerRenderer(item)
elif item.get("liveChatLegacyPaidMessageRenderer"):
renderer = LiveChatLegacyPaidMessageRenderer(item)
else:
renderer = None
return renderer

View File

@@ -1,85 +1,101 @@
from datetime import datetime from datetime import datetime
class Author: class Author:
pass pass
class BaseRenderer: class BaseRenderer:
def __init__(self, item, chattype): def setitem(self, item, chat):
self.renderer = list(item.values())[0] self.item = item
self.chattype = chattype self.chat = chat
self.author = Author() self.chat.author = Author()
def settype(self):
pass
def get_snippet(self): def get_snippet(self):
self.type = self.chattype self.chat.id = self.item.get('id')
self.id = self.renderer.get('id') timestampUsec = int(self.item.get("timestampUsec", 0))
timestampUsec = int(self.renderer.get("timestampUsec",0)) self.chat.timestamp = int(timestampUsec / 1000)
self.timestamp = int(timestampUsec/1000) tst = self.item.get("timestampText")
tst = self.renderer.get("timestampText")
if tst: if tst:
self.timestampText = tst.get("simpleText") self.chat.elapsedTime = tst.get("simpleText")
else: else:
self.timestampText = "" self.chat.elapsedTime = ""
self.datetime = self.get_datetime(timestampUsec) self.chat.datetime = self.get_datetime(timestampUsec)
self.message = self.get_message(self.renderer) self.chat.message, self.chat.messageEx = self.get_message(self.item)
self.id = self.renderer.get('id') self.chat.id = self.item.get('id')
self.amountValue= 0.0 self.chat.amountValue = 0.0
self.amountString = "" self.chat.amountString = ""
self.currency= "" self.chat.currency = ""
self.bgColor = 0 self.chat.bgColor = 0
def get_authordetails(self): def get_authordetails(self):
self.author.badgeUrl = "" self.chat.author.badgeUrl = ""
(self.author.isVerified, (self.chat.author.isVerified,
self.author.isChatOwner, self.chat.author.isChatOwner,
self.author.isChatSponsor, self.chat.author.isChatSponsor,
self.author.isChatModerator) = ( self.chat.author.isChatModerator) = (
self.get_badges(self.renderer) self.get_badges(self.item)
) )
self.author.channelId = self.renderer.get("authorExternalChannelId") self.chat.author.channelId = self.item.get("authorExternalChannelId")
self.author.channelUrl = "http://www.youtube.com/channel/"+self.author.channelId self.chat.author.channelUrl = "http://www.youtube.com/channel/" + self.chat.author.channelId
self.author.name = self.renderer["authorName"]["simpleText"] self.chat.author.name = self.item["authorName"]["simpleText"]
self.author.imageUrl= self.renderer["authorPhoto"]["thumbnails"][1]["url"] self.chat.author.imageUrl = self.item["authorPhoto"]["thumbnails"][1]["url"]
def get_message(self, item):
def get_message(self,renderer):
message = '' message = ''
if renderer.get("message"): message_ex = []
runs=renderer["message"].get("runs") runs = item.get("message", {}).get("runs", {})
if runs: for r in runs:
for r in runs: if not hasattr(r, "get"):
if r: continue
if r.get('emoji'): if r.get('emoji'):
message += r['emoji'].get('shortcuts',[''])[0] message += r['emoji'].get('shortcuts', [''])[0]
else: message_ex.append({
message += r.get('text','') 'id': r['emoji'].get('emojiId').split('/')[-1],
return message 'txt': r['emoji'].get('shortcuts', [''])[0],
'url': r['emoji']['image']['thumbnails'][0].get('url')
})
else:
message += r.get('text', '')
message_ex.append(r.get('text', ''))
return message, message_ex
def get_badges(self,renderer): def get_badges(self, renderer):
self.chat.author.type = ''
isVerified = False isVerified = False
isChatOwner = False isChatOwner = False
isChatSponsor = False isChatSponsor = False
isChatModerator = False isChatModerator = False
badges=renderer.get("authorBadges") badges = renderer.get("authorBadges", {})
if badges: for badge in badges:
for badge in badges: if badge["liveChatAuthorBadgeRenderer"].get("icon"):
author_type = badge["liveChatAuthorBadgeRenderer"]["accessibility"]["accessibilityData"]["label"] author_type = badge["liveChatAuthorBadgeRenderer"]["icon"]["iconType"]
if author_type == '確認済み': self.chat.author.type = author_type
if author_type == 'VERIFIED':
isVerified = True isVerified = True
if author_type == '所有者': if author_type == 'OWNER':
isChatOwner = True isChatOwner = True
if 'メンバー' in author_type: if author_type == 'MODERATOR':
isChatSponsor = True
self.get_badgeurl(badge)
if author_type == 'モデレーター':
isChatModerator = True isChatModerator = True
if badge["liveChatAuthorBadgeRenderer"].get("customThumbnail"):
isChatSponsor = True
self.chat.author.type = 'MEMBER'
self.get_badgeurl(badge)
return isVerified, isChatOwner, isChatSponsor, isChatModerator return isVerified, isChatOwner, isChatSponsor, isChatModerator
def get_badgeurl(self,badge): def get_badgeurl(self, badge):
self.author.badgeUrl = badge["liveChatAuthorBadgeRenderer"]["customThumbnail"]["thumbnails"][0]["url"] self.chat.author.badgeUrl = badge["liveChatAuthorBadgeRenderer"]["customThumbnail"]["thumbnails"][0]["url"]
def get_datetime(self, timestamp):
dt = datetime.fromtimestamp(timestamp / 1000000)
return dt.strftime('%Y-%m-%d %H:%M:%S')
def get_chatobj(self):
return self.chat
def get_datetime(self,timestamp): def clear(self):
dt = datetime.fromtimestamp(timestamp/1000000) self.item = None
return dt.strftime('%Y-%m-%d %H:%M:%S') self.chat = None

View File

@@ -33,5 +33,6 @@ symbols = {
"ARS\xa0": {"fxtext": "ARS", "jptext": "アルゼンチン・ペソ"}, "ARS\xa0": {"fxtext": "ARS", "jptext": "アルゼンチン・ペソ"},
"CLP\xa0": {"fxtext": "CLP", "jptext": "チリ・ペソ"}, "CLP\xa0": {"fxtext": "CLP", "jptext": "チリ・ペソ"},
"NOK\xa0": {"fxtext": "NOK", "jptext": "ノルウェー・クローネ"}, "NOK\xa0": {"fxtext": "NOK", "jptext": "ノルウェー・クローネ"},
"BAM\xa0": {"fxtext": "BAM", "jptext": "ボスニア・兌換マルカ"} "BAM\xa0": {"fxtext": "BAM", "jptext": "ボスニア・兌換マルカ"},
} "SGD\xa0": {"fxtext": "SGD", "jptext": "シンガポール・ドル"}
}

View File

@@ -1,18 +1,15 @@
from .base import BaseRenderer from .base import BaseRenderer
class LiveChatLegacyPaidMessageRenderer(BaseRenderer):
def __init__(self, item):
super().__init__(item, "newSponsor")
class LiveChatLegacyPaidMessageRenderer(BaseRenderer):
def settype(self):
self.chat.type = "newSponsor"
def get_authordetails(self): def get_authordetails(self):
super().get_authordetails() super().get_authordetails()
self.author.isChatSponsor = True self.chat.author.isChatSponsor = True
def get_message(self,renderer):
message = (renderer["eventText"]["runs"][0]["text"]
)+' / '+(renderer["detailText"]["simpleText"])
return message
def get_message(self, item):
message = (item["eventText"]["runs"][0]["text"]
) + ' / ' + (item["detailText"]["simpleText"])
return message, [message]

View File

@@ -0,0 +1,18 @@
from .base import BaseRenderer
class LiveChatMembershipItemRenderer(BaseRenderer):
def settype(self):
self.chat.type = "newSponsor"
def get_authordetails(self):
super().get_authordetails()
self.chat.author.isChatSponsor = True
def get_message(self, item):
try:
message = ''.join([mes.get("text", "")
for mes in item["headerSubtext"]["runs"]])
except KeyError:
return "Welcome New Member!", ["Welcome New Member!"]
return message, [message]

View File

@@ -3,34 +3,45 @@ from . import currency
from .base import BaseRenderer from .base import BaseRenderer
superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$") superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$")
class LiveChatPaidMessageRenderer(BaseRenderer):
def __init__(self, item):
super().__init__(item, "superChat")
class Colors:
pass
class LiveChatPaidMessageRenderer(BaseRenderer):
def settype(self):
self.chat.type = "superChat"
def get_snippet(self): def get_snippet(self):
super().get_snippet() super().get_snippet()
amountDisplayString, symbol, amount = (
self.author.name = self.renderer["authorName"]["simpleText"] self.get_amountdata(self.item)
amountDisplayString, symbol, amount =(
self.get_amountdata(self.renderer)
) )
self.message = self.get_message(self.renderer) self.chat.amountValue = amount
self.amountValue= amount self.chat.amountString = amountDisplayString
self.amountString = amountDisplayString self.chat.currency = currency.symbols[symbol]["fxtext"] if currency.symbols.get(
self.currency= currency.symbols[symbol]["fxtext"] if currency.symbols.get(symbol) else symbol symbol) else symbol
self.bgColor= self.renderer.get("bodyBackgroundColor", 0) self.chat.bgColor = self.item.get("bodyBackgroundColor", 0)
self.chat.colors = self.get_colors()
def get_amountdata(self, item):
def get_amountdata(self,renderer): amountDisplayString = item["purchaseAmountText"]["simpleText"]
amountDisplayString = renderer["purchaseAmountText"]["simpleText"]
m = superchat_regex.search(amountDisplayString) m = superchat_regex.search(amountDisplayString)
if m: if m:
symbol = m.group(1) symbol = m.group(1)
amount = float(m.group(2).replace(',','')) amount = float(m.group(2).replace(',', ''))
else: else:
symbol = "" symbol = ""
amount = 0.0 amount = 0.0
return amountDisplayString, symbol, amount return amountDisplayString, symbol, amount
def get_colors(self):
item = self.item
colors = Colors()
colors.headerBackgroundColor = item.get("headerBackgroundColor", 0)
colors.headerTextColor = item.get("headerTextColor", 0)
colors.bodyBackgroundColor = item.get("bodyBackgroundColor", 0)
colors.bodyTextColor = item.get("bodyTextColor", 0)
colors.timestampColor = item.get("timestampColor", 0)
colors.authorNameTextColor = item.get("authorNameTextColor", 0)
return colors

View File

@@ -1,13 +1,47 @@
import re import re
from . import currency from . import currency
from .paidmessage import LiveChatPaidMessageRenderer from .base import BaseRenderer
superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$")
class LiveChatPaidStickerRenderer(LiveChatPaidMessageRenderer):
def __init__(self, item):
super().__init__(item, "superSticker")
class Colors2:
pass
class LiveChatPaidStickerRenderer(BaseRenderer):
def settype(self):
self.chat.type = "superSticker"
def get_snippet(self):
super().get_snippet()
amountDisplayString, symbol, amount = (
self.get_amountdata(self.item)
)
self.chat.amountValue = amount
self.chat.amountString = amountDisplayString
self.chat.currency = currency.symbols[symbol]["fxtext"] if currency.symbols.get(
symbol) else symbol
self.chat.bgColor = self.item.get("backgroundColor", 0)
self.chat.sticker = "".join(("https:",
self.item["sticker"]["thumbnails"][0]["url"]))
self.chat.colors = self.get_colors()
def get_amountdata(self, item):
amountDisplayString = item["purchaseAmountText"]["simpleText"]
m = superchat_regex.search(amountDisplayString)
if m:
symbol = m.group(1)
amount = float(m.group(2).replace(',', ''))
else:
symbol = ""
amount = 0.0
return amountDisplayString, symbol, amount
def get_colors(self):
item = self.item
colors = Colors2()
colors.moneyChipBackgroundColor = item.get("moneyChipBackgroundColor", 0)
colors.moneyChipTextColor = item.get("moneyChipTextColor", 0)
colors.backgroundColor = item.get("backgroundColor", 0)
colors.authorNameTextColor = item.get("authorNameTextColor", 0)
return colors

View File

@@ -1,4 +1,6 @@
from .base import BaseRenderer from .base import BaseRenderer
class LiveChatTextMessageRenderer(BaseRenderer): class LiveChatTextMessageRenderer(BaseRenderer):
def __init__(self, item): def settype(self):
super().__init__(item, "textMessage") self.chat.type = "textMessage"

View File

@@ -0,0 +1,10 @@
from .chat_processor import ChatProcessor
class DummyProcessor(ChatProcessor):
'''
Dummy processor just returns received chat_components directly.
'''
def process(self, chat_components: list):
return chat_components

View File

@@ -0,0 +1,170 @@
import httpx
import os
import re
import time
from base64 import standard_b64encode
from concurrent.futures import ThreadPoolExecutor
from .chat_processor import ChatProcessor
from .default.processor import DefaultProcessor
from ..exceptions import UnknownConnectionError
import tempfile
PATTERN = re.compile(r"(.*)\(([0-9]+)\)$")
fmt_headers = ['datetime', 'elapsed', 'authorName',
'message', 'superchat', 'type', 'authorChannel']
HEADER_HTML = '''
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
'''
TABLE_CSS = '''
table.css {
border-collapse: collapse;
}
table.css thead{
border-collapse: collapse;
border: 1px solid #000
}
table.css tr td{
padding: 0.3em;
border: 1px solid #000
}
table.css th{
padding: 0.3em;
border: 1px solid #000
}
'''
class HTMLArchiver(ChatProcessor):
'''
HTMLArchiver saves chat data as HTML table format.
'''
def __init__(self, save_path, callback=None):
super().__init__()
self.client = httpx.Client(http2=True)
self.save_path = self._checkpath(save_path)
self.processor = DefaultProcessor()
self.emoji_table = {} # dict for custom emojis. key: emoji_id, value: base64 encoded image binary.
self.callback = callback
self.executor = ThreadPoolExecutor(max_workers=10)
self.tmp_fp = tempfile.NamedTemporaryFile(mode="a", encoding="utf-8", delete=False)
self.tmp_filename = self.tmp_fp.name
self.counter = 0
def _checkpath(self, filepath):
splitter = os.path.splitext(os.path.basename(filepath))
body = splitter[0]
extention = splitter[1]
newpath = filepath
counter = 1
while os.path.exists(newpath):
match = re.search(PATTERN, body)
if match:
counter = int(match[2]) + 1
num_with_bracket = f'({str(counter)})'
body = f'{match[1]}{num_with_bracket}'
else:
body = f'{body}({str(counter)})'
newpath = os.path.join(os.path.dirname(filepath), body + extention)
return newpath
def process(self, chat_components: list):
"""
Returns
----------
dict :
save_path : str :
Actual save path of file.
total_lines : int :
Count of total lines written to the file.
"""
if chat_components is None or len(chat_components) == 0:
return self.save_path, self.counter
for c in self.processor.process(chat_components).items:
self.tmp_fp.write(
self._parse_html_line((
c.datetime,
c.elapsedTime,
c.author.name,
self._parse_message(c.messageEx),
c.amountString,
c.author.type,
c.author.channelId)
)
)
if self.callback:
self.callback(None, 1)
self.counter += 1
return self.save_path, self.counter
def _parse_html_line(self, raw_line):
return ''.join(('<tr>',
''.join(''.join(('<td>', cell, '</td>')) for cell in raw_line),
'</tr>\n'))
def _parse_table_header(self, raw_line):
return ''.join(('<thead><tr>',
''.join(''.join(('<th>', cell, '</th>')) for cell in raw_line),
'</tr></thead>\n'))
def _parse_message(self, message_items: list) -> str:
return ''.join(''.join(('<span class="', self._set_emoji_table(item), '" title="', item['txt'], '"></span>'))
if type(item) is dict else item
for item in message_items)
def _encode_img(self, url):
err = None
for _ in range(5):
try:
resp = self.client.get(url, timeout=30)
break
except httpx.HTTPError as e:
err = e
time.sleep(3)
else:
raise UnknownConnectionError(str(err))
return standard_b64encode(resp.content).decode()
def _set_emoji_table(self, item: dict):
emoji_id = ''.join(('Z', item['id'])) if 48 <= ord(item['id'][0]) <= 57 else item['id']
if emoji_id not in self.emoji_table:
self.emoji_table.setdefault(emoji_id, self.executor.submit(self._encode_img, item['url']))
return emoji_id
def _stylecode(self, name, code, width, height):
return ''.join((".", name, " { display: inline-block; background-image: url(data:image/png;base64,",
code, "); background-repeat: no-repeat; width: ",
str(width), "; height: ", str(height), ";}"))
def _create_styles(self):
return '\n'.join(('<style type="text/css">',
TABLE_CSS,
'\n'.join(self._stylecode(key, self.emoji_table[key].result(), 24, 24)
for key in self.emoji_table.keys()),
'</style>\n'))
def finalize(self):
if self.tmp_fp:
self.tmp_fp.flush()
self.tmp_fp = None
with open(self.save_path, mode='w', encoding='utf-8') as outfile:
# write header
outfile.writelines((
HEADER_HTML, self._create_styles(), '</head>\n',
'<body>\n', '<table class="css">\n',
self._parse_table_header(fmt_headers)))
# write body
fp = open(self.tmp_filename, mode="r", encoding="utf-8")
for line in fp:
outfile.write(line)
outfile.write('</table>\n</body>\n</html>')
fp.close()
os.remove(self.tmp_filename)

View File

@@ -1,13 +0,0 @@
import json
from .chat_processor import ChatProcessor
class JsonDisplayProcessor(ChatProcessor):
def process(self,chat_components: list):
if chat_components:
for component in chat_components:
chatdata = component.get('chatdata')
if chatdata:
for chat in chatdata:
print(json.dumps(chat,ensure_ascii=False)[:200])

View File

@@ -1,46 +0,0 @@
import json
import os
import datetime
from .chat_processor import ChatProcessor
class JsonfileArchiveProcessor(ChatProcessor):
def __init__(self,filepath):
super().__init__()
if os.path.exists(filepath):
print('filepath is already exists!: ')
print(' '+filepath)
newpath=os.path.dirname(filepath) + \
'/'+datetime.datetime.now() \
.strftime('%Y-%m-%d %H-%M-%S')+'.data'
print('created alternate filename:')
print(' '+newpath)
self.filepath = newpath
else:
print('filepath: '+filepath)
self.filepath = filepath
def process(self,chat_components: list):
if chat_components:
with open(self.filepath, mode='a', encoding = 'utf-8') as f:
for component in chat_components:
if component:
chatdata = component.get('chatdata')
for action in chatdata:
if action:
if action.get("addChatItemAction"):
if action["addChatItemAction"]["item"].get(
"liveChatViewerEngagementMessageRenderer"):
continue
s = json.dumps(action,ensure_ascii = False)
#print(s[:200])
f.writelines(s+'\n')
def _parsedir(self,_dir):
if _dir[-1]=='\\' or _dir[-1]=='/':
separator =''
else:
separator ='/'
os.makedirs(_dir + separator, exist_ok=True)
return _dir + separator

View File

@@ -0,0 +1,69 @@
import json
import os
import re
from .chat_processor import ChatProcessor
PATTERN = re.compile(r"(.*)\(([0-9]+)\)$")
class JsonfileArchiver(ChatProcessor):
"""
JsonfileArchiver saves chat data as text of JSON lines.
Parameter:
----------
save_path : str :
save path of file.If a file with the same name exists,
it is automatically saved under a different name
with suffix '(number)'
"""
def __init__(self, save_path):
super().__init__()
self.save_path = self._checkpath(save_path)
self.line_counter = 0
def process(self, chat_components: list):
"""
Returns
----------
dict :
save_path : str :
Actual save path of file.
total_lines : int :
count of total lines written to the file.
"""
if chat_components is None:
return
with open(self.save_path, mode='a', encoding='utf-8') as f:
for component in chat_components:
if component is None:
continue
chatdata = component.get('chatdata')
if chatdata is None:
continue
for action in chatdata:
if action is None:
continue
json_line = json.dumps(action, ensure_ascii=False)
f.writelines(json_line + '\n')
self.line_counter += 1
return {"save_path": self.save_path,
"total_lines": self.line_counter}
def _checkpath(self, filepath):
splitter = os.path.splitext(os.path.basename(filepath))
body = splitter[0]
extention = splitter[1]
newpath = filepath
counter = 0
while os.path.exists(newpath):
match = re.search(PATTERN, body)
if match:
counter = int(match[2]) + 1
num_with_bracket = f'({str(counter)})'
body = f'{match[1]}{num_with_bracket}'
else:
body = f'{body}({str(counter)})'
newpath = os.path.join(os.path.dirname(filepath), body + extention)
return newpath

View File

@@ -1,47 +1,49 @@
import json
import os
import traceback
import datetime
import time
from .chat_processor import ChatProcessor from .chat_processor import ChatProcessor
##version 2
class SimpleDisplayProcessor(ChatProcessor): class SimpleDisplayProcessor(ChatProcessor):
def process(self, chat_components: list): def process(self, chat_components: list):
chatlist = [] chatlist = []
timeout = 0 timeout = 0
if chat_components is None: if chat_components is None:
return {"timeout":timeout, "chatlist":chatlist} return {"timeout": timeout, "chatlist": chatlist}
for component in chat_components: for component in chat_components:
timeout += component.get('timeout', 0) timeout += component.get('timeout', 0)
chatdata = component.get('chatdata') chatdata = component.get('chatdata')
if chatdata is None:break
for action in chatdata:
if action is None:continue
if action.get('addChatItemAction') is None:continue
if action['addChatItemAction'].get('item') is None:continue
root = action['addChatItemAction']['item'].get('liveChatTextMessageRenderer') if chatdata is None:
break
for action in chatdata:
if action is None:
continue
if action.get('addChatItemAction') is None:
continue
if action['addChatItemAction'].get('item') is None:
continue
root = action['addChatItemAction']['item'].get(
'liveChatTextMessageRenderer')
if root: if root:
author_name = root['authorName']['simpleText'] author_name = root['authorName']['simpleText']
message = self._parse_message(root.get('message')) message = self._parse_message(root.get('message'))
purchase_amount_text = '' purchase_amount_text = ''
else: else:
root = ( action['addChatItemAction']['item'].get('liveChatPaidMessageRenderer') or root = (action['addChatItemAction']['item'].get('liveChatPaidMessageRenderer')
action['addChatItemAction']['item'].get('liveChatPaidStickerRenderer') ) or action['addChatItemAction']['item'].get('liveChatPaidStickerRenderer'))
if root: if root:
author_name = root['authorName']['simpleText'] author_name = root['authorName']['simpleText']
message = self._parse_message(root.get('message')) message = self._parse_message(root.get('message'))
purchase_amount_text = root['purchaseAmountText']['simpleText'] purchase_amount_text = root['purchaseAmountText']['simpleText']
else: else:
continue continue
chatlist.append(f'[{author_name}]: {message} {purchase_amount_text}') chatlist.append(
return {"timeout":timeout, "chatlist":chatlist} f'[{author_name}]: {message} {purchase_amount_text}')
return {"timeout": timeout, "chatlist": chatlist}
def _parse_message(self,message):
def _parse_message(self, message):
if message is None: if message is None:
return '' return ''
if message.get('simpleText'): if message.get('simpleText'):
@@ -51,11 +53,9 @@ class SimpleDisplayProcessor(ChatProcessor):
tmp = '' tmp = ''
for run in runs: for run in runs:
if run.get('emoji'): if run.get('emoji'):
tmp+=(run['emoji']['shortcuts'][0]) tmp += (run['emoji']['shortcuts'][0])
elif run.get('text'): elif run.get('text'):
tmp+=(run['text']) tmp += (run['text'])
return tmp return tmp
else: else:
return '' return ''

View File

View File

@@ -0,0 +1,203 @@
"""
speed_calculator.py
チャットの勢いを算出するChatProcessor
Calculate speed of chat.
"""
import time
from .. chat_processor import ChatProcessor
class RingQueue:
"""
リング型キュー
Attributes
----------
items : list
格納されているアイテムのリスト。
first_pos : int
キュー内の一番古いアイテムを示すリストのインデックス。
last_pos : int
キュー内の一番新しいアイテムを示すリストのインデックス。
mergin : boolean
キュー内に余裕があるか。キュー内のアイテム個数が、キューの最大個数未満であればTrue。
"""
def __init__(self, capacity):
"""
コンストラクタ
Parameter
----------
capacity:このキューに格納するアイテムの最大個数。
格納時に最大個数を超える場合は一番古いアイテムから
上書きする。
"""
if capacity <= 0:
raise ValueError
self.items = list()
self.capacity = capacity
self.first_pos = 0
self.last_pos = 0
self.mergin = True
def put(self, item):
"""
引数itemに指定されたアイテムをこのキューに格納する。
キューの最大個数を超える場合は、一番古いアイテムの位置に上書きする。
Parameter
----------
item:格納するアイテム
"""
if self.mergin:
self.items.append(item)
self.last_pos = len(self.items) - 1
if self.last_pos == self.capacity - 1:
self.mergin = False
return
self.last_pos += 1
if self.last_pos > self.capacity - 1:
self.last_pos = 0
self.items[self.last_pos] = item
self.first_pos += 1
if self.first_pos > self.capacity - 1:
self.first_pos = 0
def get(self):
"""
キュー内の一番古いアイテムへの参照を返す
(アイテムは削除しない)
Return
----------
キュー内の一番古いアイテムへの参照
"""
return self.items[self.first_pos]
def item_count(self):
return len(self.items)
class SpeedCalculator(ChatProcessor, RingQueue):
"""
チャットの勢いを計算する。
一定期間のチャットデータのうち、最初のチャットの投稿時刻と
最後のチャットの投稿時刻の差を、チャット数で割り返し
1分あたりの速度に換算する。
Parameter
----------
capacity : int
RingQueueに格納するチャット勢い算出用データの最大数
"""
def __init__(self, capacity=10):
super().__init__(capacity)
self.speed = 0
def process(self, chat_components: list):
chatdata = []
if chat_components:
for component in chat_components:
if component.get("chatdata"):
chatdata.extend(component.get("chatdata"))
self._put_chatdata(chatdata)
self.speed = self._calc_speed()
return self.speed
def _calc_speed(self):
"""
RingQueue内のチャット勢い算出用データリストを元に、
チャット速度を計算して返す
Return
---------------------------
チャット速度(1分間で換算したチャット数)
"""
try:
# キュー内の総チャット数
total = sum(item['chat_count'] for item in self.items)
# キュー内の最初と最後のチャットの時間差
duration = (self.items[self.last_pos]['endtime'] - self.items[self.first_pos]['starttime'])
if duration != 0:
return int(total * 60 / duration)
return 0
except IndexError:
return 0
def _put_chatdata(self, actions):
"""
チャットデータからタイムスタンプを読み取り、勢い測定用のデータを組み立て、
RingQueueに投入する。
200円以上のスパチャはtickerとmessageの2つのデータが生成されるが、
tickerの方は時刻データの場所が異なることを利用し、勢いの集計から除外している。
Parameter
---------
actions : List[dict]
チャットデータ(addChatItemAction) のリスト
"""
def _put_emptydata():
'''
チャットデータがない場合に空のデータをキューに投入する。
'''
timestamp_now = int(time.time())
self.put({
'chat_count': 0,
'starttime': int(timestamp_now),
'endtime': int(timestamp_now)
})
def _get_timestamp(action: dict):
"""
チャットデータから時刻データを取り出す。
"""
try:
item = action['addChatItemAction']['item']
timestamp = int(item[list(item.keys())[0]]['timestampUsec'])
except (KeyError, TypeError):
return None
return timestamp
if actions is None or len(actions) == 0:
_put_emptydata()
return
# actions内の時刻データを持つチャットデータの数
counter = 0
# actions内の最初のチャットデータの時刻
starttime = None
# actions内の最後のチャットデータの時刻
endtime = None
for action in actions:
# チャットデータからtimestampUsecを読み取る
gettime = _get_timestamp(action)
# 時刻のないデータだった場合は次の行のデータで読み取り試行
if gettime is None:
continue
# 最初に有効な時刻を持つデータのtimestampをstarttimeに設定
if starttime is None:
starttime = gettime
# 最後のtimestampを設定(途中で時刻のないデータの場合もあるので上書きしていく)
endtime = gettime
# チャットの数をインクリメント
counter += 1
# チャット速度用のデータをRingQueueに送る
if starttime is None or endtime is None:
_put_emptydata()
return
self.put({
'chat_count': counter,
'starttime': int(starttime / 1000000),
'endtime': int(endtime / 1000000)
})

View File

View File

@@ -0,0 +1,75 @@
import re
from pytchat.processors.chat_processor import ChatProcessor
superchat_regex = re.compile(r"^(\D*)(\d{1,3}(,\d{3})*(\.\d*)*\b)$")
items_paid = [
'addChatItemAction',
'item',
'liveChatPaidMessageRenderer'
]
items_sticker = [
'addChatItemAction',
'item',
'liveChatPaidStickerRenderer'
]
class SuperchatCalculator(ChatProcessor):
"""
Calculate the amount of SuperChat by currency.
"""
def __init__(self):
self.results = {}
def process(self, chat_components: list):
"""
Return
------------
results : dict :
List of amount by currency.
key: currency symbol, value: total amount.
"""
if chat_components is None:
return self.results
for component in chat_components:
chatdata = component.get('chatdata')
if chatdata is None:
continue
for action in chatdata:
renderer = self._get_item(action, items_paid) or \
self._get_item(action, items_sticker)
if renderer is None:
continue
symbol, amount = self._parse(renderer)
self.results.setdefault(symbol, 0)
self.results[symbol] += amount
return self.results
def _parse(self, renderer):
purchase_amount_text = renderer["purchaseAmountText"]["simpleText"]
m = superchat_regex.search(purchase_amount_text)
if m:
symbol = m.group(1)
amount = float(m.group(2).replace(',', ''))
else:
symbol = ""
amount = 0.0
return symbol, amount
def _get_item(self, dict_body, items: list):
for item in items:
if dict_body is None:
break
if isinstance(dict_body, dict):
dict_body = dict_body.get(item)
continue
if isinstance(item, int) and \
isinstance(dict_body, list) and \
len(dict_body) > item:
dict_body = dict_body[item]
continue
return None
return dict_body

View File

@@ -0,0 +1,67 @@
import csv
import os
import re
from .chat_processor import ChatProcessor
from .default.processor import DefaultProcessor
PATTERN = re.compile(r"(.*)\(([0-9]+)\)$")
fmt_headers = ['datetime', 'elapsed', 'authorName', 'message',
'superchatAmount', 'authorType', 'authorChannel']
class TSVArchiver(ChatProcessor):
'''
TsvArchiver saves chat data as Tab Separated Values format text.
'''
def __init__(self, save_path):
super().__init__()
self.save_path = self._checkpath(save_path)
with open(self.save_path, mode='a', encoding='utf-8') as f:
writer = csv.writer(f, delimiter='\t')
writer.writerow(fmt_headers)
self.processor = DefaultProcessor()
def _checkpath(self, filepath):
splitter = os.path.splitext(os.path.basename(filepath))
body = splitter[0]
extention = splitter[1]
newpath = filepath
counter = 0
while os.path.exists(newpath):
match = re.search(PATTERN, body)
if match:
counter = int(match[2]) + 1
num_with_bracket = f'({str(counter)})'
body = f'{match[1]}{num_with_bracket}'
else:
body = f'{body}({str(counter)})'
newpath = os.path.join(os.path.dirname(filepath), body + extention)
return newpath
def process(self, chat_components: list):
"""
Returns
----------
dict :
save_path : str :
Actual save path of file.
total_lines : int :
count of total lines written to the file.
"""
if chat_components is None or len(chat_components) == 0:
return
with open(self.save_path, mode='a', encoding='utf-8') as f:
writer = csv.writer(f, delimiter='\t')
chats = self.processor.process(chat_components).items
for c in chats:
writer.writerow([
c.datetime,
c.elapsedTime,
c.author.name,
c.message,
c.amountString,
c.author.type,
c.author.channelId
])

View File

@@ -1,15 +1,120 @@
import requests,json,datetime import datetime
import httpx
import json
import os
import re
from urllib.parse import quote
from .. import config from .. import config
from .. exceptions import InvalidVideoIdException
def download(url): PATTERN = re.compile(r"(.*)\(([0-9]+)\)$")
_session = requests.Session()
PATTERN_YTURL = re.compile(r"((?<=(v|V)/)|(?<=be/)|(?<=(\?|\&)v=)|(?<=embed/))([\w-]+)")
PATTERN_CHANNEL = re.compile(r"\\\"channelId\\\":\\\"(.{24})\\\"")
YT_VIDEO_ID_LENGTH = 11
CLIENT_VERSION = ''.join(("2.", (datetime.datetime.today() - datetime.timedelta(days=1)).strftime("%Y%m%d"), ".01.00"))
UA = config.headers["user-agent"]
def extract(url):
_session = httpx.Client(http2=True)
html = _session.get(url, headers=config.headers) html = _session.get(url, headers=config.headers)
with open(str(datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S') with open(str(datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S')
)+'test.json',mode ='w',encoding='utf-8') as f: ) + 'test.json', mode='w', encoding='utf-8') as f:
json.dump(html.json(),f,ensure_ascii=False) json.dump(html.json(), f, ensure_ascii=False)
def save(data,filename): def save(data, filename, extention) -> str:
with open(str(datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S') save_filename = filename + "_" + \
)+filename,mode ='w',encoding='utf-8') as f: (datetime.datetime.now().strftime('%Y-%m-%d %H-%M-%S')) + extention
with open(save_filename, mode='w', encoding='utf-8') as f:
f.writelines(data) f.writelines(data)
return save_filename
def checkpath(filepath):
splitter = os.path.splitext(os.path.basename(filepath))
body = splitter[0]
extention = splitter[1]
newpath = filepath
counter = 1
while os.path.exists(newpath):
match = re.search(PATTERN, body)
if match:
counter = int(match[2]) + 1
num_with_bracket = f'({str(counter)})'
body = f'{match[1]}{num_with_bracket}'
else:
body = f'{body}({str(counter)})'
newpath = os.path.join(os.path.dirname(filepath), body + extention)
return newpath
def get_param(continuation, replay=False, offsetms: int = 0, dat=''):
if offsetms < 0:
offsetms = 0
ret = {
"context": {
"client": {
"visitorData": dat,
"userAgent": UA,
"clientName": "WEB",
"clientVersion": CLIENT_VERSION,
},
},
"continuation": continuation,
}
if replay:
ret.setdefault("currentPlayerState", {
"playerOffsetMs": str(int(offsetms))})
return ret
def extract_video_id(url_or_id: str) -> str:
ret = ''
if '[' in url_or_id:
url_or_id = url_or_id.replace('[', '').replace(']', '')
if type(url_or_id) != str:
raise TypeError(f"{url_or_id}: URL or VideoID must be str, but {type(url_or_id)} is passed.")
if len(url_or_id) == YT_VIDEO_ID_LENGTH:
return url_or_id
match = re.search(PATTERN_YTURL, url_or_id)
if match is None:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
try:
ret = match.group(4)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
if ret is None or len(ret) != YT_VIDEO_ID_LENGTH:
raise InvalidVideoIdException(f"Invalid video id: {url_or_id}")
return ret
def get_channelid(client, video_id):
resp = client.get("https://www.youtube.com/embed/{}".format(quote(video_id)), headers=config.headers)
match = re.search(PATTERN_CHANNEL, resp.text)
if match is None:
raise InvalidVideoIdException(f"Cannot find channel id for video id:{video_id}. This video id seems to be invalid.")
try:
ret = match.group(1)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {video_id}")
return ret
async def get_channelid_async(client, video_id):
resp = await client.get("https://www.youtube.com/embed/{}".format(quote(video_id)), headers=config.headers)
match = re.search(PATTERN_CHANNEL, resp.text)
if match is None:
raise InvalidVideoIdException(f"Cannot find channel id for video id:{video_id}. This video id seems to be invalid.")
try:
ret = match.group(1)
except IndexError:
raise InvalidVideoIdException(f"Invalid video id: {video_id}")
return ret

View File

@@ -1,4 +1 @@
aiohttp httpx[http2]==0.16.1
pytz
requests
urllib3

View File

@@ -1,5 +1,2 @@
aioresponses pytest-mock
mock pytest-httpx
mocker
pytest
pytest-mock

View File

@@ -1,6 +1,5 @@
from setuptools import setup, find_packages, Command from setuptools import setup, find_packages, Command
from codecs import open from os import path, system, remove, rename, removedirs
from os import path, system
import re import re
package_name = "pytchat" package_name = "pytchat"
@@ -8,19 +7,27 @@ package_name = "pytchat"
root_dir = path.abspath(path.dirname(__file__)) root_dir = path.abspath(path.dirname(__file__))
def _requirements(): def _requirements():
return [name.rstrip() for name in open(path.join(root_dir, 'requirements.txt')).readlines()] return [name.rstrip()
for name in open(path.join(
root_dir, 'requirements.txt')).readlines()]
def _test_requirements(): def _test_requirements():
return [name.rstrip() for name in open(path.join(root_dir, 'requirements_test.txt')).readlines()] return [name.rstrip()
for name in open(path.join(
root_dir, 'requirements_test.txt')).readlines()]
with open(path.join(root_dir, package_name, '__init__.py')) as f: with open(path.join(root_dir, package_name, '__init__.py')) as f:
init_text = f.read() init_text = f.read()
version = re.search(r'__version__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1) version = re.search(
license = re.search(r'__license__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1) r'__version__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1)
author = re.search(r'__author__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1) license = re.search(
author_email = re.search(r'__author_email__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1) r'__license__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1)
url = re.search(r'__url__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1) author = re.search(
r'__author__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1)
author_email = re.search(
r'__author_email__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1)
url = re.search(
r'__url__\s*=\s*[\'\"](.+?)[\'\"]', init_text).group(1)
assert version assert version
assert license assert license
@@ -28,35 +35,33 @@ assert author
assert author_email assert author_email
assert url assert url
with open('README.md', encoding='utf-8') as f: with open('README.md', encoding='utf-8') as f:
long_description = f.read() long_description = f.read()
setup( setup(
name=package_name,
packages=find_packages(),
version=version,
url=url,
author=author, author=author,
author_email=author_email, author_email=author_email,
long_description=long_description,
long_description_content_type='text/markdown',
license=license,
install_requires=_requirements(),
tests_require=_test_requirements(),
description="a python library for fetching youtube live chat.",
classifiers=[ classifiers=[
'Natural Language :: Japanese', 'Natural Language :: Japanese',
'Development Status :: 4 - Beta', 'Development Status :: 4 - Beta',
'Programming Language :: Python', 'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'License :: OSI Approved :: MIT License', 'License :: OSI Approved :: MIT License',
], ],
description="a python library for fetching youtube live chat.",
install_requires=_requirements(),
keywords='youtube livechat asyncio', keywords='youtube livechat asyncio',
license=license,
long_description=long_description,
long_description_content_type='text/markdown',
name=package_name,
packages=find_packages(exclude=['*log.txt','*tests','*testrun']),
url=url,
version=version,
) )

View File

@@ -1,26 +0,0 @@
import pytest
from pytchat.parser.replay import Parser
import pytchat.config as config
import requests, json
from pytchat.paramgen import arcparam
def test_arcparam_0(mocker):
param = arcparam.get("01234567890")
assert "op2w0wRyGjxDZzhhRFFvTE1ERXlNelExTmpjNE9UQWFFLXFvM2JrQkRRb0xNREV5TXpRMU5qYzRPVEFnQVElM0QlM0QoATAAOABAAEgEUhwIABAAGAAgACoOc3RhdGljY2hlY2tzdW1AAFgDYAFoAXIECAEQAXgA" == param
def test_arcparam_1(mocker):
param = arcparam.get("01234567890", seektime = 100000)
assert "op2w0wR3GjxDZzhhRFFvTE1ERXlNelExTmpjNE9UQWFFLXFvM2JrQkRRb0xNREV5TXpRMU5qYzRPVEFnQVElM0QlM0QogNDbw_QCMAA4AEAASANSHAgAEAAYACAAKg5zdGF0aWNjaGVja3N1bUAAWANgAWgBcgQIARABeAA%3D" == param
def test_arcparam_2(mocker):
param = arcparam.get("SsjCnHOk-Sk")
url=f"https://www.youtube.com/live_chat_replay/get_live_chat_replay?continuation={param}&pbj=1"
resp = requests.Session().get(url,headers = config.headers)
jsn = json.loads(resp.text)
parser = Parser()
_ , chatdata = parser.parse(jsn)
test_id = chatdata[0]["addChatItemAction"]["item"]["liveChatTextMessageRenderer"]["id"]
print(test_id)
assert "CjoKGkNMYXBzZTdudHVVQ0Zjc0IxZ0FkTnFnQjVREhxDSnlBNHV2bnR1VUNGV0dnd2dvZDd3NE5aZy0w" == test_id

View File

@@ -0,0 +1,140 @@
from pytchat.processors.superchat.calculator import SuperchatCalculator
get_item = SuperchatCalculator()._get_item
dict_test = {
'root':{
'node0' : 'value0',
'node1' : 'value1',
'node2' : {
'node2-0' : 'value2-0'
},
'node3' : [
{'node3-0' : 'value3-0'},
{'node3-1' :
{'node3-1-0' : 'value3-1-0'}
}
],
'node4' : [],
'node5' : [
[
{'node5-1-0' : 'value5-1-0'},
{'node5-1-1' : 'value5-1-1'},
],
{'node5-0' : 'value5-0'},
]
}
}
items_test0 = [
'root',
'node1'
]
items_test_not_found0 = [
'root',
'other_data'
]
items_test_nest = [
'root',
'node2',
'node2-0'
]
items_test_list0 = [
'root',
'node3',
1,
'node3-1'
]
items_test_list1 = [
'root',
'node3',
1,
'node3-1',
'node3-1-0'
]
items_test_list2 = [
'root',
'node4',
None
]
items_test_list3 = [
'root',
'node4'
]
items_test_list_nest = [
'root',
'node5',
0,
1,
'node5-1-1'
]
items_test_list_nest_not_found1 = [
'root',
'node5',
0,
1,
'node5-1-1',
'nodez'
]
items_test_not_found1 = [
'root',
'node3',
2,
'node3-1',
'node3-1-0'
]
items_test_not_found2 = [
'root',
'node3',
2,
'node3-1',
'node3-1-0',
'nodex'
]
def test_get_items_0():
assert get_item(dict_test, items_test0) == 'value1'
def test_get_items_1():
assert get_item(dict_test, items_test_not_found0) is None
def test_get_items_2():
assert get_item(dict_test, items_test_nest) == 'value2-0'
def test_get_items_3():
assert get_item(
dict_test, items_test_list0) == {'node3-1-0' : 'value3-1-0'}
def test_get_items_4():
assert get_item(dict_test, items_test_list1) == 'value3-1-0'
def test_get_items_5():
assert get_item(dict_test, items_test_not_found1) == None
def test_get_items_6():
assert get_item(dict_test, items_test_not_found2) == None
def test_get_items_7():
assert get_item(dict_test, items_test_list2) == None
def test_get_items_8():
assert get_item(dict_test, items_test_list_nest) == 'value5-1-1'
def test_get_items_9():
assert get_item(dict_test, items_test_list_nest_not_found1) == None
def test_get_items_10():
assert get_item(dict_test, items_test_list3) == []

View File

@@ -0,0 +1,72 @@
import json
from pytchat.parser.live import Parser
from pytchat.processors.superchat.calculator import SuperchatCalculator
from pytchat.exceptions import ChatParseException
parse = SuperchatCalculator()._parse
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()
def load_chatdata(filepath):
parser = Parser(is_replay=True)
# print(json.loads(_open_file(filepath)))
contents = parser.get_contents(json.loads(_open_file(filepath)))[0]
return parser.parse(contents)[1]
def test_parse_1():
renderer = {"purchaseAmountText": {"simpleText": "¥2,000"}}
symbol, amount = parse(renderer)
assert symbol == ''
assert amount == 2000.0
def test_parse_2():
renderer = {"purchaseAmountText": {"simpleText": "ABC\x0a200"}}
symbol, amount = parse(renderer)
assert symbol == 'ABC\x0a'
assert amount == 200.0
def test_process_0():
"""
parse superchat data
"""
chat_component = {
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/superchat_0.json")
}
assert SuperchatCalculator().process([chat_component]) == {
'': 6800.0, '': 2.0}
def test_process_1():
"""
parse no superchat data
"""
chat_component = {
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/text_only.json")
}
assert SuperchatCalculator().process([chat_component]) == {}
def test_process_2():
"""
try to parse after replay end
"""
try:
chat_component = {
'video_id': '',
'timeout': 10,
'chatdata': load_chatdata(r"tests/testdata/calculator/replay_end.json")
}
assert False
SuperchatCalculator().process([chat_component])
except ChatParseException:
assert True

View File

@@ -1,18 +1,9 @@
import json import json
import pytest
import asyncio,aiohttp
from pytchat.parser.live import Parser from pytchat.parser.live import Parser
from pytchat.processors.compatible.processor import CompatibleProcessor from pytchat.processors.compatible.processor import CompatibleProcessor
from pytchat.exceptions import (
NoLivechatRendererException,NoYtinitialdataException,
ResponseContextError, NoContentsException)
from pytchat.processors.compatible.renderer.textmessage import LiveChatTextMessageRenderer parser = Parser(is_replay=False)
from pytchat.processors.compatible.renderer.paidmessage import LiveChatPaidMessageRenderer
from pytchat.processors.compatible.renderer.paidsticker import LiveChatPaidStickerRenderer
from pytchat.processors.compatible.renderer.legacypaid import LiveChatLegacyPaidMessageRenderer
parser = Parser()
def test_textmessage(mocker): def test_textmessage(mocker):
'''api互換processorのテスト通常テキストメッセージ''' '''api互換processorのテスト通常テキストメッセージ'''
@@ -20,36 +11,39 @@ def test_textmessage(mocker):
_json = _open_file("tests/testdata/compatible/textmessage.json") _json = _open_file("tests/testdata/compatible/textmessage.json")
_, chatdata = parser.parse(json.loads(_json)) _, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = { data = {
"video_id" : "", "video_id": "",
"timeout" : 7, "timeout": 7,
"chatdata" : chatdata "chatdata": chatdata
} }
ret = processor.process([data]) ret = processor.process([data])
assert ret["kind"]== "youtube#liveChatMessageListResponse" assert ret["kind"] == "youtube#liveChatMessageListResponse"
assert ret["pollingIntervalMillis"]==data["timeout"]*1000 assert ret["pollingIntervalMillis"] == data["timeout"] * 1000
assert ret.keys() == { assert ret.keys() == {
"kind", "etag", "pageInfo", "nextPageToken","pollingIntervalMillis","items" "kind", "etag", "pageInfo", "nextPageToken", "pollingIntervalMillis", "items"
} }
assert ret["pageInfo"].keys() == { assert ret["pageInfo"].keys() == {
"totalResults", "resultsPerPage" "totalResults", "resultsPerPage"
} }
assert ret["items"][0].keys() == { assert ret["items"][0].keys() == {
"kind", "etag", "id", "snippet", "authorDetails" "kind", "etag", "id", "snippet", "authorDetails"
} }
assert ret["items"][0]["snippet"].keys() == { assert ret["items"][0]["snippet"].keys() == {
'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage', 'textMessageDetails' 'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage',
'textMessageDetails'
} }
assert ret["items"][0]["authorDetails"].keys() == { assert ret["items"][0]["authorDetails"].keys() == {
'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor', 'isChatModerator' 'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor',
'isChatModerator'
} }
assert ret["items"][0]["snippet"]["textMessageDetails"].keys() == { assert ret["items"][0]["snippet"]["textMessageDetails"].keys() == {
'messageText' 'messageText'
} }
assert "LCC." in ret["items"][0]["id"] assert "LCC." in ret["items"][0]["id"]
assert ret["items"][0]["snippet"]["type"]=="textMessageEvent" assert ret["items"][0]["snippet"]["type"] == "textMessageEvent"
def test_newsponcer(mocker): def test_newsponcer(mocker):
'''api互換processorのテストメンバ新規登録''' '''api互換processorのテストメンバ新規登録'''
@@ -57,34 +51,72 @@ def test_newsponcer(mocker):
_json = _open_file("tests/testdata/compatible/newSponsor.json") _json = _open_file("tests/testdata/compatible/newSponsor.json")
_, chatdata = parser.parse(json.loads(_json)) _, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = { data = {
"video_id" : "", "video_id": "",
"timeout" : 7, "timeout": 7,
"chatdata" : chatdata "chatdata": chatdata
} }
ret = processor.process([data]) ret = processor.process([data])
assert ret["kind"]== "youtube#liveChatMessageListResponse" assert ret["kind"] == "youtube#liveChatMessageListResponse"
assert ret["pollingIntervalMillis"]==data["timeout"]*1000 assert ret["pollingIntervalMillis"] == data["timeout"] * 1000
assert ret.keys() == { assert ret.keys() == {
"kind", "etag", "pageInfo", "nextPageToken","pollingIntervalMillis","items" "kind", "etag", "pageInfo", "nextPageToken", "pollingIntervalMillis", "items"
} }
assert ret["pageInfo"].keys() == { assert ret["pageInfo"].keys() == {
"totalResults", "resultsPerPage" "totalResults", "resultsPerPage"
} }
assert ret["items"][0].keys() == { assert ret["items"][0].keys() == {
"kind", "etag", "id", "snippet","authorDetails" "kind", "etag", "id", "snippet", "authorDetails"
} }
assert ret["items"][0]["snippet"].keys() == { assert ret["items"][0]["snippet"].keys() == {
'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage' 'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage'
} }
assert ret["items"][0]["authorDetails"].keys() == { assert ret["items"][0]["authorDetails"].keys() == {
'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor', 'isChatModerator' 'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor',
'isChatModerator'
} }
assert "LCC." in ret["items"][0]["id"] assert "LCC." in ret["items"][0]["id"]
assert ret["items"][0]["snippet"]["type"]=="newSponsorEvent" assert ret["items"][0]["snippet"]["type"] == "newSponsorEvent"
def test_newsponcer_rev(mocker):
'''api互換processorのテストメンバ新規登録'''
processor = CompatibleProcessor()
_json = _open_file("tests/testdata/compatible/newSponsor_rev.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data])
assert ret["kind"] == "youtube#liveChatMessageListResponse"
assert ret["pollingIntervalMillis"] == data["timeout"] * 1000
assert ret.keys() == {
"kind", "etag", "pageInfo", "nextPageToken", "pollingIntervalMillis", "items"
}
assert ret["pageInfo"].keys() == {
"totalResults", "resultsPerPage"
}
assert ret["items"][0].keys() == {
"kind", "etag", "id", "snippet", "authorDetails"
}
assert ret["items"][0]["snippet"].keys() == {
'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage'
}
assert ret["items"][0]["authorDetails"].keys() == {
'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor',
'isChatModerator'
}
assert "LCC." in ret["items"][0]["id"]
assert ret["items"][0]["snippet"]["type"] == "newSponsorEvent"
def test_superchat(mocker): def test_superchat(mocker):
@@ -93,38 +125,56 @@ def test_superchat(mocker):
_json = _open_file("tests/testdata/compatible/superchat.json") _json = _open_file("tests/testdata/compatible/superchat.json")
_, chatdata = parser.parse(json.loads(_json)) _, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = { data = {
"video_id" : "", "video_id": "",
"timeout" : 7, "timeout": 7,
"chatdata" : chatdata "chatdata": chatdata
} }
ret = processor.process([data]) ret = processor.process([data])
assert ret["kind"]== "youtube#liveChatMessageListResponse" assert ret["kind"] == "youtube#liveChatMessageListResponse"
assert ret["pollingIntervalMillis"]==data["timeout"]*1000 assert ret["pollingIntervalMillis"] == data["timeout"] * 1000
assert ret.keys() == { assert ret.keys() == {
"kind", "etag", "pageInfo", "nextPageToken","pollingIntervalMillis","items" "kind", "etag", "pageInfo", "nextPageToken", "pollingIntervalMillis", "items"
} }
assert ret["pageInfo"].keys() == { assert ret["pageInfo"].keys() == {
"totalResults", "resultsPerPage" "totalResults", "resultsPerPage"
} }
assert ret["items"][0].keys() == { assert ret["items"][0].keys() == {
"kind", "etag", "id", "snippet", "authorDetails" "kind", "etag", "id", "snippet", "authorDetails"
} }
assert ret["items"][0]["snippet"].keys() == { assert ret["items"][0]["snippet"].keys() == {
'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage', 'superChatDetails' 'type', 'liveChatId', 'authorChannelId', 'publishedAt', 'hasDisplayContent', 'displayMessage',
'superChatDetails'
} }
assert ret["items"][0]["authorDetails"].keys() == { assert ret["items"][0]["authorDetails"].keys() == {
'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor', 'isChatModerator' 'channelId', 'channelUrl', 'displayName', 'profileImageUrl', 'isVerified', 'isChatOwner', 'isChatSponsor',
'isChatModerator'
} }
assert ret["items"][0]["snippet"]["superChatDetails"].keys() == { assert ret["items"][0]["snippet"]["superChatDetails"].keys() == {
'amountMicros', 'currency', 'amountDisplayString', 'tier', 'backgroundColor' 'amountMicros', 'currency', 'amountDisplayString', 'tier', 'backgroundColor'
} }
assert "LCC." in ret["items"][0]["id"] assert "LCC." in ret["items"][0]["id"]
assert ret["items"][0]["snippet"]["type"]=="superChatEvent" assert ret["items"][0]["snippet"]["type"] == "superChatEvent"
def test_unregistered_currency(mocker):
processor = CompatibleProcessor()
_json = _open_file("tests/testdata/unregistered_currency.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data])
assert ret["items"][0]["snippet"]["superChatDetails"]["currency"] == "[UNREGISTERD]"
def _open_file(path): def _open_file(path):
with open(path,mode ='r',encoding = 'utf-8') as f: with open(path, mode='r', encoding='utf-8') as f:
return f.read() return f.read()

View File

@@ -0,0 +1,227 @@
import json
from datetime import datetime
from pytchat.parser.live import Parser
from pytchat.processors.default.processor import DefaultProcessor
TEST_TIMETSTAMP = 1570678496000000
def get_local_datetime(timestamp):
dt = datetime.fromtimestamp(timestamp / 1000000)
return dt.strftime('%Y-%m-%d %H:%M:%S')
def test_textmessage(mocker):
'''text message'''
processor = DefaultProcessor()
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/textmessage.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.id == "dummy_id"
assert ret.message == "dummy_message"
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == ""
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is False
assert ret.author.isChatModerator is False
def test_textmessage_replay_member(mocker):
'''text message replay member'''
processor = DefaultProcessor()
parser = Parser(is_replay=True)
_json = _open_file("tests/testdata/default/replay_member_text.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.type == "textMessage"
assert ret.id == "dummy_id"
assert ret.message == "dummy_message"
assert ret.messageEx == ["dummy_message"]
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.elapsedTime == "1:23:45"
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == "https://yt3.ggpht.com/X=s16-c-k"
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is True
assert ret.author.isChatModerator is False
def test_superchat(mocker):
'''superchat'''
processor = DefaultProcessor()
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/superchat.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.type == "superChat"
assert ret.id == "dummy_id"
assert ret.message == "dummy_message"
assert ret.messageEx == ["dummy_message"]
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.elapsedTime == ""
assert ret.amountValue == 800
assert ret.amountString == "¥800"
assert ret.currency == "JPY"
assert ret.bgColor == 4280150454
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == ""
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is False
assert ret.author.isChatModerator is False
assert ret.colors.headerBackgroundColor == 4278239141
assert ret.colors.headerTextColor == 4278190080
assert ret.colors.bodyBackgroundColor == 4280150454
assert ret.colors.bodyTextColor == 4278190080
assert ret.colors.authorNameTextColor == 2315255808
assert ret.colors.timestampColor == 2147483648
def test_supersticker(mocker):
'''supersticker'''
processor = DefaultProcessor()
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/supersticker.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.type == "superSticker"
assert ret.id == "dummy_id"
assert ret.message == ""
assert ret.messageEx == []
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.elapsedTime == ""
assert ret.amountValue == 200
assert ret.amountString == "¥200"
assert ret.currency == "JPY"
assert ret.bgColor == 4278237396
assert ret.sticker == "https://lh3.googleusercontent.com/param_s=s72-rp"
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == ""
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is False
assert ret.author.isChatModerator is False
assert ret.colors.backgroundColor == 4278237396
assert ret.colors.moneyChipBackgroundColor == 4278248959
assert ret.colors.moneyChipTextColor == 4278190080
assert ret.colors.authorNameTextColor == 3003121664
def test_sponsor(mocker):
'''sponsor(membership)'''
processor = DefaultProcessor()
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/newSponsor_current.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.type == "newSponsor"
assert ret.id == "dummy_id"
assert ret.message == "新規メンバー"
assert ret.messageEx == ["新規メンバー"]
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.elapsedTime == ""
assert ret.bgColor == 0
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == "https://yt3.ggpht.com/X=s32-c-k"
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is True
assert ret.author.isChatModerator is False
def test_sponsor_legacy(mocker):
'''lagacy sponsor(membership)'''
processor = DefaultProcessor()
parser = Parser(is_replay=False)
_json = _open_file("tests/testdata/default/newSponsor_lagacy.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 7,
"chatdata": chatdata
}
ret = processor.process([data]).items[0]
assert ret.type == "newSponsor"
assert ret.id == "dummy_id"
assert ret.message == "新規メンバー / ようこそ、author_name"
assert ret.messageEx == ["新規メンバー / ようこそ、author_name"]
assert ret.timestamp == 1570678496000
assert ret.datetime == get_local_datetime(TEST_TIMETSTAMP)
assert ret.elapsedTime == ""
assert ret.bgColor == 0
assert ret.author.name == "author_name"
assert ret.author.channelId == "author_channel_id"
assert ret.author.channelUrl == "http://www.youtube.com/channel/author_channel_id"
assert ret.author.imageUrl == "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg"
assert ret.author.badgeUrl == ""
assert ret.author.isVerified is False
assert ret.author.isChatOwner is False
assert ret.author.isChatSponsor is True
assert ret.author.isChatModerator is False
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()

View File

@@ -0,0 +1,55 @@
from pytchat.util import extract_video_id
from pytchat.exceptions import InvalidVideoIdException
VALID_TEST_PATTERNS = (
("ABC_EFG_IJK", "ABC_EFG_IJK"),
("vid_test_be", "vid_test_be"),
("https://www.youtube.com/watch?v=123_456_789", "123_456_789"),
("https://www.youtube.com/watch?v=123_456_789&t=123s", "123_456_789"),
("www.youtube.com/watch?v=123_456_789", "123_456_789"),
("watch?v=123_456_789", "123_456_789"),
("youtube.com/watch?v=123_456_789", "123_456_789"),
("http://youtu.be/ABC_EFG_IJK", "ABC_EFG_IJK"),
("youtu.be/ABC_EFG_IJK", "ABC_EFG_IJK"),
("https://www.youtube.com/watch?v=ABC_EFG_IJK&list=XYZ_ABC_12345&start_radio=1&t=1", "ABC_EFG_IJK"),
("https://www.youtube.com/embed/ABC_EFG_IJK", "ABC_EFG_IJK"),
("www.youtube.com/embed/ABC_EFG_IJK", "ABC_EFG_IJK"),
("youtube.com/embed/ABC_EFG_IJK", "ABC_EFG_IJK")
)
INVALID_TEST_PATTERNS = (
("", ""),
("0123456789", "0123456789"), # less than 11 letters id
("more_than_11_letter_string", "more_than_11_letter_string"),
("https://www.youtube.com/watch?v=more_than_11_letter_string", "more_than_11_letter_string"),
("https://www.youtube.com/channel/123_456_789", "123_456_789"),
)
TYPEERROR_TEST_PATTERNS = (
(100, 100), # not string
(["123_456_789"], "123_456_789"), # not string
)
def test_extract_valid_pattern():
for pattern in VALID_TEST_PATTERNS:
ret = extract_video_id(pattern[0])
assert ret == pattern[1]
def test_extract_invalid_pattern():
for pattern in INVALID_TEST_PATTERNS:
try:
extract_video_id(pattern[0])
assert False
except InvalidVideoIdException:
assert True
def test_extract_typeerror_pattern():
for pattern in TYPEERROR_TEST_PATTERNS:
try:
extract_video_id(pattern[0])
assert False
except TypeError:
assert True

View File

@@ -0,0 +1,48 @@
import json
from pytchat.processors.jsonfile_archiver import JsonfileArchiver
from unittest.mock import patch, mock_open
from tests.testdata.jsonfile_archiver.chat_component import chat_component
def _open_file(path):
with open(path,mode ='r',encoding = 'utf-8') as f:
return f.read()
def test_checkpath(mocker):
processor = JsonfileArchiver("path")
mocker.patch('os.path.exists').side_effect = exists_file
'''Test no duplicate file.'''
assert processor._checkpath("z:/other.txt") == "z:/other.txt"
'''Test duplicate filename.
The case the name first renamed ('test.txt -> test(0).txt')
is also duplicated.
'''
assert processor._checkpath("z:/test.txt") == "z:/test(1).txt"
'''Test no extention file (duplicate).'''
assert processor._checkpath("z:/test") == "z:/test(0)"
def test_read_write():
'''Test read and write chatdata'''
mock = mock_open(read_data = "")
with patch('builtins.open',mock):
processor = JsonfileArchiver("path")
save_path = processor.process([chat_component])
fh = mock()
actuals = [args[0] for (args, kwargs) in fh.writelines.call_args_list]
'''write format is json dump string with 0x0A'''
to_be_written = [json.dumps(action, ensure_ascii=False)+'\n'
for action in chat_component["chatdata"]]
for i in range(len(actuals)):
assert actuals[i] == to_be_written[i]
assert save_path == {'save_path': 'path', 'total_lines': 7}
def exists_file(path):
if path == "z:/test.txt":
return True
if path == "z:/test(0).txt":
return True
if path == "z:/test":
return True

View File

@@ -1,53 +0,0 @@
import pytest
from pytchat.parser.live import Parser
import json
import asyncio,aiohttp
from aioresponses import aioresponses
from pytchat.core_async.livechat import LiveChatAsync
from pytchat.exceptions import (
NoLivechatRendererException,NoYtinitialdataException,
ResponseContextError,NoContentsException)
from pytchat.core_multithread.livechat import LiveChat
import unittest
from unittest import TestCase
def _open_file(path):
with open(path,mode ='r',encoding = 'utf-8') as f:
return f.read()
@aioresponses()
def test_Async(*mock):
vid=''
_text = _open_file('tests/testdata/paramgen_firstread.json')
_text = json.loads(_text)
mock[0].get(f"https://www.youtube.com/live_chat?v={vid}&is_popout=1", status=200, body=_text)
try:
chat = LiveChatAsync(video_id='')
assert chat.is_alive()
chat.terminate()
assert not chat.is_alive()
except ResponseContextError:
assert not chat.is_alive()
def test_MultiThread(mocker):
_text = _open_file('tests/testdata/paramgen_firstread.json')
_text = json.loads(_text)
responseMock = mocker.Mock()
responseMock.status_code = 200
responseMock.text = _text
mocker.patch('requests.Session.get').return_value = responseMock
try:
chat = LiveChatAsync(video_id='')
assert chat.is_alive()
chat.terminate()
assert not chat.is_alive()
except ResponseContextError:
chat.terminate()
assert not chat.is_alive()

View File

@@ -1,9 +0,0 @@
import pytest
from pytchat.paramgen import liveparam
def test_liveparam_0(mocker):
_ts1= 1546268400
param = liveparam._build("01234567890",
*([_ts1*1000000 for i in range(5)]))
test_param="0ofMyAPiARp8Q2c4S0RRb0xNREV5TXpRMU5qYzRPVEFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5TURFeU16UTFOamM0T1RBbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiAuNbVqsrfAjAAOABAAkorCAEQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgLjW1arK3wJYA1CAuNbVqsrfAliAuNbVqsrfAmgBggEECAEQAIgBAKABgLjW1arK3wI%3D"
assert test_param == param

View File

@@ -1,44 +1,34 @@
import pytest
from pytchat.parser.live import Parser from pytchat.parser.live import Parser
import json import json
import asyncio,aiohttp from pytchat.exceptions import NoContents
from aioresponses import aioresponses
from pytchat.exceptions import (
NoLivechatRendererException,NoYtinitialdataException, parser = Parser(is_replay=False)
ResponseContextError, NoContentsException)
def _open_file(path): def _open_file(path):
with open(path,mode ='r',encoding = 'utf-8') as f: with open(path, mode='r', encoding='utf-8') as f:
return f.read() return f.read()
parser = Parser()
@aioresponses()
def test_finishedlive(*mock): def test_finishedlive(*mock):
'''配信が終了した動画を正しく処理できるか'''
_text = _open_file('tests/testdata/finished_live.json') _text = _open_file('tests/testdata/finished_live.json')
_text = json.loads(_text) _text = json.loads(_text)
try: try:
parser.parse(_text) parser.parse(parser.get_contents(_text)[0])
assert False assert False
except NoContentsException: except NoContents:
assert True assert True
@aioresponses()
def test_parsejson(*mock):
'''jsonを正常にパースできるか'''
def test_parsejson(*mock):
_text = _open_file('tests/testdata/paramgen_firstread.json') _text = _open_file('tests/testdata/paramgen_firstread.json')
_text = json.loads(_text) _text = json.loads(_text)
try: try:
parser.parse(_text) s, _ = parser.parse(parser.get_contents(_text)[0])
jsn = _text assert s['timeoutMs'] == 5035
timeout = jsn["response"]["continuationContents"]["liveChatContinuation"]["continuations"][0]["timedContinuationData"]["timeoutMs"] assert s['continuation'] == "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%3D"
continuation = jsn["response"]["continuationContents"]["liveChatContinuation"]["continuations"][0]["timedContinuationData"]["continuation"] except Exception:
assert 5035 == timeout assert False
assert "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%3D" == continuation
except:
assert False

View File

@@ -0,0 +1,64 @@
import json
from pytchat.parser.live import Parser
from pytchat.processors.speed.calculator import SpeedCalculator
parser = Parser(is_replay=False)
def test_speed_1(mocker):
'''test speed calculation with normal json.
test json has 15 chatdata, duration is 30 seconds,
so the speed of chatdata is 30 chats/minute.
'''
processor = SpeedCalculator(capacity=30)
_json = _open_file("tests/testdata/speed/speedtest_normal.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,
"chatdata": chatdata
}
ret = processor.process([data])
assert 30 == ret
def test_speed_2(mocker):
'''test speed calculation with no valid chat data.
'''
processor = SpeedCalculator(capacity=30)
_json = _open_file("tests/testdata/speed/speedtest_undefined.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,
"chatdata": chatdata
}
ret = processor.process([data])
assert ret == 0
def test_speed_3(mocker):
'''test speed calculation with empty data.
'''
processor = SpeedCalculator(capacity=30)
_json = _open_file("tests/testdata/speed/speedtest_empty.json")
_, chatdata = parser.parse(parser.get_contents(json.loads(_json))[0])
data = {
"video_id": "",
"timeout": 10,
"chatdata": chatdata
}
ret = processor.process([data])
assert ret == 0
def _open_file(path):
with open(path, mode='r', encoding='utf-8') as f:
return f.read()

View File

@@ -0,0 +1,18 @@
{
"response": {
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"playerSeekContinuationData": {
"continuation": "___reload_continuation___"
}
}
]
}
}
}
}

3322
tests/testdata/calculator/superchat_0.json vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,87 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"replayChatItemAction": {
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [
{
"text": "dummy_message"
}
]
},
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"id": "dummy_id",
"timestampUsec": 0,
"authorExternalChannelId": "http://www.youtube.com/channel/author_channel_url",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
},
"timestampText": {
"simpleText": "0:00"
}
}
},
"clientId": "dummy_client_id"
}
}
],
"videoOffsetTimeMsec": "10000"
}
}
]
}
}
}

164
tests/testdata/chat.json vendored Normal file
View File

@@ -0,0 +1,164 @@
{
"timing": {
"info": {
"st": 164
}
},
"csn": "",
"response": {
"responseContext": {
"serviceTrackingParams": [{
"service": "CSI",
"params": [{
"key": "GetLiveChat_rid",
"value": ""
}, {
"key": "c",
"value": "WEB"
}, {
"key": "cver",
"value": "2.20191219.03.01"
}, {
"key": "yt_li",
"value": "0"
}]
}, {
"service": "GFEEDBACK",
"params": [{
"key": "e",
"value": ""
}, {
"key": "logged_in",
"value": "0"
}]
}, {
"service": "GUIDED_HELP",
"params": [{
"key": "logged_in",
"value": "0"
}]
}, {
"service": "ECATCHER",
"params": [{
"key": "client.name",
"value": "WEB"
}, {
"key": "client.version",
"value": "2.2"
}, {
"key": "innertube.build.changelist",
"value": "228"
}, {
"key": "innertube.build.experiments.source_version",
"value": "2858"
}, {
"key": "innertube.build.label",
"value": "youtube.ytfe.innertube_"
}, {
"key": "innertube.build.timestamp",
"value": "154"
}, {
"key": "innertube.build.variants.checksum",
"value": "e"
}, {
"key": "innertube.run.job",
"value": "ytfe-innertube-replica-only.ytfe"
}]
}],
"webResponseContextExtensionData": {
"ytConfigData": {
"csn": "ADw",
"visitorData": "%3D%3D"
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [{
"timedContinuationData": {
"timeoutMs": 10000,
"continuation": "continuation"
}
}],
"actions": [{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [{
"text": "message"
}]
},
"authorName": {
"simpleText": "authorName"
},
"authorPhoto": {
"thumbnails": [{
"url": "https://yt3.ggpht.com/photo.jpg",
"width": 32,
"height": 32
}, {
"url": "https://yt3.ggpht.com/photo.jpg",
"width": 64,
"height": 64
}]
},
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "params"
}
},
"id": "id",
"timestampUsec": "1576851922945411",
"authorBadges": [{
"liveChatAuthorBadgeRenderer": {
"customThumbnail": {
"thumbnails": [{
"url": "https://yt3.ggpht.com/photo.jpg"
}, {
"url": "https://yt3.ggpht.com/photo.jpg"
}]
},
"tooltip": "メンバー6 か月)",
"accessibility": {
"accessibilityData": {
"label": "メンバー6 か月)"
}
}
}
}],
"authorExternalChannelId": "UC",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
},
"clientId": "00000000000000000000"
}
}
]}
},
"xsrf_token": "xsrf_token",
"url": "/live_chat/get_live_chat?continuation=0",
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0",
"rootVe": 0
}
},
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0"
}
}
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,282 +1,281 @@
{ {
"xsrf_token": "QUFFLUhqbWVqWVRTUjhBcGRkNWR5Q2F3VWhxcTd0RkF1UXxBQ3Jtc0trMThFVGVVNTFnWmZucnkwNlJJMEZ2bndUS1I3b2dpLUtTNE92dEgwX3Y0MkNZU2NXdGY1QTMtX09BUGphYUpDQlc1dFhiTm9jLS1sQXVCNHpsdllqcm1id0t1RFBCM3A1b2o3OGt0Yjd6TE5wcmxBbjNyQktjc1lWZ3hjM1RuYk83YkQ0VVN3MGUybjAwSE90SS1YNkxvMUV5YVE=", "xsrf_token": "QUFFLUhqbWVqWVRTUjhBcGRkNWR5Q2F3VWhxcTd0RkF1UXxBQ3Jtc0trMThFVGVVNTFnWmZucnkwNlJJMEZ2bndUS1I3b2dpLUtTNE92dEgwX3Y0MkNZU2NXdGY1QTMtX09BUGphYUpDQlc1dFhiTm9jLS1sQXVCNHpsdllqcm1id0t1RFBCM3A1b2o3OGt0Yjd6TE5wcmxBbjNyQktjc1lWZ3hjM1RuYk83YkQ0VVN3MGUybjAwSE90SS1YNkxvMUV5YVE=",
"timing": { "timing": {
"info": { "info": {
"st": 148 "st": 148
} }
},
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
}, },
"endpoint": { "urlEndpoint": {
"commandMetadata": { "url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
"webCommandMetadata": { }
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D" },
} "url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
}, "csn": "n2STXd2iKZr2gAOt9qvgCg",
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D" "responseContext": {
"serviceTrackingParams": [
{
"service": "CSI",
"params": [
{
"key": "GetLiveChat_rid",
"value": "0x9290108c05344647"
},
{
"key": "c",
"value": "WEB"
},
{
"key": "cver",
"value": "2.20191001.04.00"
},
{
"key": "yt_li",
"value": "0"
}
]
},
{
"service": "GFEEDBACK",
"params": [
{
"key": "e",
"value": "23744176,23788875,23793834,23794620,23804281,23806159,23816483,23819244,23820768,23826780,23827354,23830392,23832125,23835020,23836965,23837741,23837772,23837993,23838235,23839362,23840155,23840217,23841118,23841454,23842630,23842662,23842883,23842986,23843289,23843534,23843767,23845644,9449243,9471239,9474360"
},
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "GUIDED_HELP",
"params": [
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "ECATCHER",
"params": [
{
"key": "client.name",
"value": "WEB"
},
{
"key": "client.version",
"value": "2.20191001"
},
{
"key": "innertube.build.changelist",
"value": "272006966"
},
{
"key": "innertube.build.experiments.source_version",
"value": "272166268"
},
{
"key": "innertube.build.label",
"value": "youtube.ytfe.innertube_20190930_5_RC0"
},
{
"key": "innertube.build.timestamp",
"value": "1569863426"
},
{
"key": "innertube.build.variants.checksum",
"value": "1a800c1a2396906f1cbb7f670d43b6f5"
},
{
"key": "innertube.run.job",
"value": "ytfe-innertube-replica-only.ytfe"
}
]
}
],
"webResponseContextExtensionData": {
"ytConfigData": {
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"visitorData": "CgtPQm1xTmtvNm1Tcyifyc3sBQ%3D%3D"
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"timedContinuationData": {
"timeoutMs": 8860,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCid_ejQpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DF-ovRpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABtNic0aT75AI%3D"
}
} }
}, ],
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D", "actions": [
"csn": "n2STXd2iKZr2gAOt9qvgCg", {
"response": { "addChatItemAction": {
"responseContext": { "item": {
"serviceTrackingParams": [ "liveChatPaidMessageRenderer": {
{ "id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"service": "CSI", "timestampUsec": "1569940638420061",
"params": [ "authorName": {
{ "simpleText": "九十九 万"
"key": "GetLiveChat_rid",
"value": "0x9290108c05344647"
},
{
"key": "c",
"value": "WEB"
},
{
"key": "cver",
"value": "2.20191001.04.00"
},
{
"key": "yt_li",
"value": "0"
}
]
}, },
{ "authorPhoto": {
"service": "GFEEDBACK", "thumbnails": [
"params": [
{
"key": "e",
"value": "23744176,23788875,23793834,23794620,23804281,23806159,23816483,23819244,23820768,23826780,23827354,23830392,23832125,23835020,23836965,23837741,23837772,23837993,23838235,23839362,23840155,23840217,23841118,23841454,23842630,23842662,23842883,23842986,23843289,23843534,23843767,23845644,9449243,9471239,9474360"
},
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "GUIDED_HELP",
"params": [
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "ECATCHER",
"params": [
{
"key": "client.name",
"value": "WEB"
},
{
"key": "client.version",
"value": "2.20191001"
},
{
"key": "innertube.build.changelist",
"value": "272006966"
},
{
"key": "innertube.build.experiments.source_version",
"value": "272166268"
},
{
"key": "innertube.build.label",
"value": "youtube.ytfe.innertube_20190930_5_RC0"
},
{
"key": "innertube.build.timestamp",
"value": "1569863426"
},
{
"key": "innertube.build.variants.checksum",
"value": "1a800c1a2396906f1cbb7f670d43b6f5"
},
{
"key": "innertube.run.job",
"value": "ytfe-innertube-replica-only.ytfe"
}
]
}
],
"webResponseContextExtensionData": {
"ytConfigData": {
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"visitorData": "CgtPQm1xTmtvNm1Tcyifyc3sBQ%3D%3D"
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{ {
"timedContinuationData": { "url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"timeoutMs": 8860, "width": 32,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCid_ejQpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DF-ovRpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABtNic0aT75AI%3D" "height": 32
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatPaidMessageRenderer": {
"id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"timestampUsec": "1569940638420061",
"authorName": {
"simpleText": "九十九 万"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"purchaseAmountText": {
"simpleText": "¥846"
},
"message": {
"runs": [
{
"text": "ボルガ博士お許しください代"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMDV1V21oTmNXc3RMVkZEUmxGSlQxbEJiMlJ5V0d0RmMyY1FBQm80Q2cwS0N6Z3dUM3AwU21oMU56YzRLaWNLR0ZWRFNYbDBUbU52ZWpSd1YzcFlaa3hrWVRCRWIxVk1VUklMT0RCUGVuUkthSFUzTnpnZ0FpZ0JNaG9LR0ZWRGIxWjBRMDE2WnpKMmJHVkJWbk5oTjJ0M01EVkJRUSUzRCUzRA=="
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
}, },
{ {
"addLiveChatTickerItemAction": { "url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"item": { "width": 64,
"liveChatTickerPaidMessageItemRenderer": { "height": 64
"id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"amount": {
"simpleText": "¥846"
},
"amountTextColor": 4278190080,
"startBackgroundColor": 4280150454,
"endBackgroundColor": 4278239141,
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"durationSec": 120,
"showItemEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"showLiveChatItemEndpoint": {
"renderer": {
"liveChatPaidMessageRenderer": {
"id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"timestampUsec": "1569940638420061",
"authorName": {
"simpleText": "九十九 万"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"purchaseAmountText": {
"simpleText": "¥846"
},
"message": {
"runs": [
{
"text": "ボルガ博士お許しください代"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMDV1V21oTmNXc3RMVkZEUmxGSlQxbEJiMlJ5V0d0RmMyY1FBQm80Q2cwS0N6Z3dUM3AwU21oMU56YzRLaWNLR0ZWRFNYbDBUbU52ZWpSd1YzcFlaa3hrWVRCRWIxVk1VUklMT0RCUGVuUkthSFUzTnpnZ0FpZ0JNaG9LR0ZWRGIxWjBRMDE2WnpKMmJHVkJWbk5oTjJ0M01EVkJRUSUzRCUzRA=="
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
},
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"fullDurationSec": 120
}
},
"durationSec": "120"
}
} }
] ]
},
"purchaseAmountText": {
"simpleText": "¥846"
},
"message": {
"runs": [
{
"text": "ボルガ博士お許しください代"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMDV1V21oTmNXc3RMVkZEUmxGSlQxbEJiMlJ5V0d0RmMyY1FBQm80Q2cwS0N6Z3dUM3AwU21oMU56YzRLaWNLR0ZWRFNYbDBUbU52ZWpSd1YzcFlaa3hrWVRCRWIxVk1VUklMT0RCUGVuUkthSFUzTnpnZ0FpZ0JNaG9LR0ZWRGIxWjBRMDE2WnpKMmJHVkJWbk5oTjJ0M01EVkJRUSUzRCUzRA=="
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
} }
}
},
{
"addLiveChatTickerItemAction": {
"item": {
"liveChatTickerPaidMessageItemRenderer": {
"id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"amount": {
"simpleText": "¥846"
},
"amountTextColor": 4278190080,
"startBackgroundColor": 4280150454,
"endBackgroundColor": 4278239141,
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"durationSec": 120,
"showItemEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"showLiveChatItemEndpoint": {
"renderer": {
"liveChatPaidMessageRenderer": {
"id": "ChwKGkNOblpoTXFrLS1RQ0ZRSU9ZQW9kclhrRXNn",
"timestampUsec": "1569940638420061",
"authorName": {
"simpleText": "九十九 万"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-FoBvzoKhbZs/AAAAAAAAAAI/AAAAAAAAAAA/oyb-UCYFbfA/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"purchaseAmountText": {
"simpleText": "¥846"
},
"message": {
"runs": [
{
"text": "ボルガ博士お許しください代"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMDV1V21oTmNXc3RMVkZEUmxGSlQxbEJiMlJ5V0d0RmMyY1FBQm80Q2cwS0N6Z3dUM3AwU21oMU56YzRLaWNLR0ZWRFNYbDBUbU52ZWpSd1YzcFlaa3hrWVRCRWIxVk1VUklMT0RCUGVuUkthSFUzTnpnZ0FpZ0JNaG9LR0ZWRGIxWjBRMDE2WnpKMmJHVkJWbk5oTjJ0M01EVkJRUSUzRCUzRA=="
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
},
"authorExternalChannelId": "UCoVtCMzg2vleAVsa7kw05AA",
"fullDurationSec": 120
}
},
"durationSec": "120"
}
} }
]
} }
} }
}

View File

@@ -1,197 +1,195 @@
{ {
"xsrf_token": "QUFFLUhqbWVqWVRTUjhBcGRkNWR5Q2F3VWhxcTd0RkF1UXxBQ3Jtc0trMThFVGVVNTFnWmZucnkwNlJJMEZ2bndUS1I3b2dpLUtTNE92dEgwX3Y0MkNZU2NXdGY1QTMtX09BUGphYUpDQlc1dFhiTm9jLS1sQXVCNHpsdllqcm1id0t1RFBCM3A1b2o3OGt0Yjd6TE5wcmxBbjNyQktjc1lWZ3hjM1RuYk83YkQ0VVN3MGUybjAwSE90SS1YNkxvMUV5YVE=", "xsrf_token": "QUFFLUhqbWVqWVRTUjhBcGRkNWR5Q2F3VWhxcTd0RkF1UXxBQ3Jtc0trMThFVGVVNTFnWmZucnkwNlJJMEZ2bndUS1I3b2dpLUtTNE92dEgwX3Y0MkNZU2NXdGY1QTMtX09BUGphYUpDQlc1dFhiTm9jLS1sQXVCNHpsdllqcm1id0t1RFBCM3A1b2o3OGt0Yjd6TE5wcmxBbjNyQktjc1lWZ3hjM1RuYk83YkQ0VVN3MGUybjAwSE90SS1YNkxvMUV5YVE=",
"timing": { "timing": {
"info": { "info": {
"st": 148 "st": 148
}
},
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"response": {
"responseContext": {
"serviceTrackingParams": [
{
"service": "CSI",
"params": [
{
"key": "GetLiveChat_rid",
"value": "0x9290108c05344647"
},
{
"key": "c",
"value": "WEB"
},
{
"key": "cver",
"value": "2.20191001.04.00"
},
{
"key": "yt_li",
"value": "0"
}
]
},
{
"service": "GFEEDBACK",
"params": [
{
"key": "e",
"value": "23744176,23788875,23793834,23794620,23804281,23806159,23816483,23819244,23820768,23826780,23827354,23830392,23832125,23835020,23836965,23837741,23837772,23837993,23838235,23839362,23840155,23840217,23841118,23841454,23842630,23842662,23842883,23842986,23843289,23843534,23843767,23845644,9449243,9471239,9474360"
},
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "GUIDED_HELP",
"params": [
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "ECATCHER",
"params": [
{
"key": "client.name",
"value": "WEB"
},
{
"key": "client.version",
"value": "2.20191001"
},
{
"key": "innertube.build.changelist",
"value": "272006966"
},
{
"key": "innertube.build.experiments.source_version",
"value": "272166268"
},
{
"key": "innertube.build.label",
"value": "youtube.ytfe.innertube_20190930_5_RC0"
},
{
"key": "innertube.build.timestamp",
"value": "1569863426"
},
{
"key": "innertube.build.variants.checksum",
"value": "1a800c1a2396906f1cbb7f670d43b6f5"
},
{
"key": "innertube.run.job",
"value": "ytfe-innertube-replica-only.ytfe"
}
]
}
],
"webResponseContextExtensionData": {
"ytConfigData": {
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"visitorData": "CgtPQm1xTmtvNm1Tcyifyc3sBQ%3D%3D"
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"timedContinuationData": {
"timeoutMs": 8860,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCid_ejQpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DF-ovRpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABtNic0aT75AI%3D"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatPaidStickerRenderer": {
"id": "ChwKGkNQX2Qzb2pUcU9VQ0ZRdnVXQW9kaTNJS3NB",
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMUJmWkROdmFsUnhUMVZEUmxGMmRWZEJiMlJwTTBsTGMwRVFBQm80Q2cwS0N6VlVUSE42U0hNd2QxYzBLaWNLR0ZWRFdGSnNTVXN6UTNkZlZFcEpVVU0xYTFOS1NsRk5aeElMTlZSTWMzcEljekIzVnpRZ0FpZ0JNaG9LR0ZWRFRHOXJPVWQ0WVRGYU5rWTVWV3d5WVV0MlRFWkdadyUzRCUzRA=="
}
},
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
},
"timestampUsec": "1571499325098699",
"authorPhoto": {
"thumbnails": [
{
"url": "https: //yt3.ggpht.com/-xRQVNtDSO3w/AAAAAAAAAAI/AAAAAAAAAAA/Is9D9D7wwAE/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-xRQVNtDSO3w/AAAAAAAAAAI/AAAAAAAAAAA/Is9D9D7wwAE/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"authorName": {
"simpleText": "りお"
},
"authorExternalChannelId": "UCLok9Gxa1Z6F9Ul2aKvLFFg",
"sticker": {
"thumbnails": [
{
"url": "//lh3.googleusercontent.com/1aIk6vlk4gZ2ytc42j3WcIHYtWFWo2uVWVqbHFuxiGHO4XwyAS0u8vuu6VkiX5eR6uy9mfAupyP786_TbP0=s72-rwa",
"width": 72,
"height": 72
},
{
"url": "//lh3.googleusercontent.com/1aIk6vlk4gZ2ytc42j3WcIHYtWFWo2uVWVqbHFuxiGHO4XwyAS0u8vuu6VkiX5eR6uy9mfAupyP786_TbP0=s144-rwa",
"width": 144,
"height": 144
}
],
"accessibility": {
"accessibilityData": {
"label": "気付いてもらえるように人差し指を上げたり下げたりしている柴犬"
}
}
},
"moneyChipBackgroundColor": 4278248959,
"moneyChipTextColor": 4278190080,
"purchaseAmountText": {
"simpleText": "¥200"
},
"stickerDisplayWidth": 72,
"stickerDisplayHeight": 72,
"backgroundColor": 4278237396,
"authorNameTextColor": 3003121664
}
}
}
}
]
}
}
} }
} },
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D"
}
},
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCja2bHMpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DXxuTMpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABxM3xzKT75AI%253D",
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"responseContext": {
"serviceTrackingParams": [
{
"service": "CSI",
"params": [
{
"key": "GetLiveChat_rid",
"value": "0x9290108c05344647"
},
{
"key": "c",
"value": "WEB"
},
{
"key": "cver",
"value": "2.20191001.04.00"
},
{
"key": "yt_li",
"value": "0"
}
]
},
{
"service": "GFEEDBACK",
"params": [
{
"key": "e",
"value": "23744176,23788875,23793834,23794620,23804281,23806159,23816483,23819244,23820768,23826780,23827354,23830392,23832125,23835020,23836965,23837741,23837772,23837993,23838235,23839362,23840155,23840217,23841118,23841454,23842630,23842662,23842883,23842986,23843289,23843534,23843767,23845644,9449243,9471239,9474360"
},
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "GUIDED_HELP",
"params": [
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "ECATCHER",
"params": [
{
"key": "client.name",
"value": "WEB"
},
{
"key": "client.version",
"value": "2.20191001"
},
{
"key": "innertube.build.changelist",
"value": "272006966"
},
{
"key": "innertube.build.experiments.source_version",
"value": "272166268"
},
{
"key": "innertube.build.label",
"value": "youtube.ytfe.innertube_20190930_5_RC0"
},
{
"key": "innertube.build.timestamp",
"value": "1569863426"
},
{
"key": "innertube.build.variants.checksum",
"value": "1a800c1a2396906f1cbb7f670d43b6f5"
},
{
"key": "innertube.run.job",
"value": "ytfe-innertube-replica-only.ytfe"
}
]
}
],
"webResponseContextExtensionData": {
"ytConfigData": {
"csn": "n2STXd2iKZr2gAOt9qvgCg",
"visitorData": "CgtPQm1xTmtvNm1Tcyifyc3sBQ%3D%3D"
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"timedContinuationData": {
"timeoutMs": 8860,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xPREJQZW5SS2FIVTNOemdhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5T0RCUGVuUkthSFUzTnpnbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCid_ejQpPvkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQwJKPoqT75AJYA1DF-ovRpPvkAliL_fvZoPvkAmgBggEECAEQAIgBAKABtNic0aT75AI%3D"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatPaidStickerRenderer": {
"id": "ChwKGkNQX2Qzb2pUcU9VQ0ZRdnVXQW9kaTNJS3NB",
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2g0S0hBb2FRMUJmWkROdmFsUnhUMVZEUmxGMmRWZEJiMlJwTTBsTGMwRVFBQm80Q2cwS0N6VlVUSE42U0hNd2QxYzBLaWNLR0ZWRFdGSnNTVXN6UTNkZlZFcEpVVU0xYTFOS1NsRk5aeElMTlZSTWMzcEljekIzVnpRZ0FpZ0JNaG9LR0ZWRFRHOXJPVWQ0WVRGYU5rWTVWV3d5WVV0MlRFWkdadyUzRCUzRA=="
}
},
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
},
"timestampUsec": "1571499325098699",
"authorPhoto": {
"thumbnails": [
{
"url": "https: //yt3.ggpht.com/-xRQVNtDSO3w/AAAAAAAAAAI/AAAAAAAAAAA/Is9D9D7wwAE/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-xRQVNtDSO3w/AAAAAAAAAAI/AAAAAAAAAAA/Is9D9D7wwAE/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"authorName": {
"simpleText": "りお"
},
"authorExternalChannelId": "UCLok9Gxa1Z6F9Ul2aKvLFFg",
"sticker": {
"thumbnails": [
{
"url": "//lh3.googleusercontent.com/1aIk6vlk4gZ2ytc42j3WcIHYtWFWo2uVWVqbHFuxiGHO4XwyAS0u8vuu6VkiX5eR6uy9mfAupyP786_TbP0=s72-rwa",
"width": 72,
"height": 72
},
{
"url": "//lh3.googleusercontent.com/1aIk6vlk4gZ2ytc42j3WcIHYtWFWo2uVWVqbHFuxiGHO4XwyAS0u8vuu6VkiX5eR6uy9mfAupyP786_TbP0=s144-rwa",
"width": 144,
"height": 144
}
],
"accessibility": {
"accessibilityData": {
"label": "気付いてもらえるように人差し指を上げたり下げたりしている柴犬"
}
}
},
"moneyChipBackgroundColor": 4278248959,
"moneyChipTextColor": 4278190080,
"purchaseAmountText": {
"simpleText": "¥200"
},
"stickerDisplayWidth": 72,
"stickerDisplayHeight": 72,
"backgroundColor": 4278237396,
"authorNameTextColor": 3003121664
}
}
}
}
]
}
}
}

View File

@@ -1,177 +1,175 @@
{ {
"response": { "responseContext": {
"responseContext": { "serviceTrackingParams": [
"serviceTrackingParams": [ {
{ "service": "CSI",
"service": "CSI", "params": [
"params": [ {
{ "key": "GetLiveChat_rid",
"key": "GetLiveChat_rid", "value": "0x3eff0db28fc39bbe"
"value": "0x3eff0db28fc39bbe" },
}, {
{ "key": "c",
"key": "c", "value": "WEB"
"value": "WEB" },
}, {
{ "key": "cver",
"key": "cver", "value": "2.20190920.05.01"
"value": "2.20190920.05.01" },
}, {
{ "key": "yt_li",
"key": "yt_li", "value": "0"
"value": "0" }
} ]
] },
}, {
{ "service": "GFEEDBACK",
"service": "GFEEDBACK", "params": [
"params": [ {
{ "key": "e",
"key": "e", "value": "23744176,23748146,23788851,23788875,23793834,23804281,23807353,23808952,23828082,23828243,23829333,23832544,23834418,23834656,23835020,23836434,23836965,23837742,23837772,23837993,23838301,23838576,23838576,23838742,23839360,23840216,23841655,23842986,23843288,23843533,23843743,23844780,24630231,9425362,9449243,9466592,9469037,9471235,9474358"
"value": "23744176,23748146,23788851,23788875,23793834,23804281,23807353,23808952,23828082,23828243,23829333,23832544,23834418,23834656,23835020,23836434,23836965,23837742,23837772,23837993,23838301,23838576,23838576,23838742,23839360,23840216,23841655,23842986,23843288,23843533,23843743,23844780,24630231,9425362,9449243,9466592,9469037,9471235,9474358" },
}, {
{ "key": "logged_in",
"key": "logged_in", "value": "0"
"value": "0" }
} ]
] },
}, {
{ "service": "GUIDED_HELP",
"service": "GUIDED_HELP", "params": [
"params": [ {
{ "key": "logged_in",
"key": "logged_in", "value": "0"
"value": "0" }
} ]
] },
}, {
{ "service": "ECATCHER",
"service": "ECATCHER", "params": [
"params": [ {
{ "key": "client.name",
"key": "client.name", "value": "WEB"
"value": "WEB" },
}, {
{ "key": "client.version",
"key": "client.version", "value": "2.20190920"
"value": "2.20190920" },
}, {
{ "key": "innertube.build.changelist",
"key": "innertube.build.changelist", "value": "270169303"
"value": "270169303" },
}, {
{ "key": "innertube.build.experiments.source_version",
"key": "innertube.build.experiments.source_version", "value": "270377311"
"value": "270377311" },
}, {
{ "key": "innertube.build.label",
"key": "innertube.build.label", "value": "youtube.ytfe.innertube_20190919_5_RC1"
"value": "youtube.ytfe.innertube_20190919_5_RC1" },
}, {
{ "key": "innertube.build.timestamp",
"key": "innertube.build.timestamp", "value": "1568942548"
"value": "1568942548" },
}, {
{ "key": "innertube.build.variants.checksum",
"key": "innertube.build.variants.checksum", "value": "392d499f55b5e2c240adde58886a8143"
"value": "392d499f55b5e2c240adde58886a8143" },
}, {
{ "key": "innertube.run.job",
"key": "innertube.run.job", "value": "ytfe-innertube-replica-only.ytfe"
"value": "ytfe-innertube-replica-only.ytfe" }
} ]
] }
} ],
], "webResponseContextExtensionData": {
"webResponseContextExtensionData": { "ytConfigData": {
"ytConfigData": { "csn": "n96GXabRGouFlQTigY2YDg",
"csn": "n96GXabRGouFlQTigY2YDg", "visitorData": "CgtKUldQeGJJRXhkcyifvZvsBQ%3D%3D"
"visitorData": "CgtKUldQeGJJRXhkcyifvZvsBQ%3D%3D" }
}
}
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"timedContinuationData": {
"timeoutMs": 5041,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCj7hLmSs-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1DsuuGSs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKAB7KDVk7Pj5AI%3D"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [
{
"text": "text"
}
]
},
"authorName": {
"simpleText": "name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-8sLtPu1Hyw0/AAAAAAAAAAI/AAAAAAAAAAA/a_52bWnC0-s/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-8sLtPu1Hyw0/AAAAAAAAAAI/AAAAAAAAAAA/a_52bWnC0-s/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2pzS09Rb2FRMHRQTkhRMVEzbzBMVkZEUmxweGRrUlJiMlJZV0d0QlVHY1NHME5PYm5OdlVESm1OQzFSUTBaUmRsTlhRVzlrVVZGclJUTlJNeEFBR2pnS0RRb0xhelJMZDBsWFQwdERkalFxSndvWVZVTnZTWEZ1TVZvMWFYaERXbmRqTUVSWFNqZHlTME5uRWd0ck5FdDNTVmRQUzBOMk5DQUNLQUV5R2dvWVZVTnlOVGxXVlY5amRtWnlkVEF0YW1GeWNtUk1NMDEz"
}
},
"id": "CjkKGkNLTzR0NUN6NC1RQ0ZacXZEUW9kWFhrQVBnEhtDTm5zb1AyZjQtUUNGUXZTV0FvZFFRa0UzUTM%3D",
"timestampUsec": "1569119896722467",
"authorExternalChannelId": "UCr59VU_cvfru0-jarrdL3Mw",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
},
"clientId": "CNnsoP2f4-QCFQvSWAodQQkE3Q3"
}
}
]
}
}
},
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D"
}
},
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D"
}
},
"csn": "n96GXabRGouFlQTigY2YDg",
"xsrf_token": "QUFFLUhqbHNNWTF3NFJqc2h3cGE1NE9FWGdaWk5mRlVhUXxBQ3Jtc0tuTWhZNFcyTW1iZnA3ZnFTYUFudVFEUVE0cnFEOVBGcEU1MEh0Zlh4bll1amVmRl9OMkxZV3pKV1ZSbExBeDctTl95NGtBVnJZdlNxeS1KdWVNempEN2N6MHhaU1laV3hnVkZPeHp1OHVDTGVFSGUyOGduT0szbDV5N05LYUZTdzdoTDRwV1VJWndaVjdQVGRjNWVpR0YwUXgtZXc=",
"url": "\/live_chat\/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D",
"timing": {
"info": {
"st": 81
}
} }
} },
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"timedContinuationData": {
"timeoutMs": 5041,
"continuation": "0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCj7hLmSs-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1DsuuGSs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKAB7KDVk7Pj5AI%3D"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [
{
"text": "text"
}
]
},
"authorName": {
"simpleText": "name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/-8sLtPu1Hyw0/AAAAAAAAAAI/AAAAAAAAAAA/a_52bWnC0-s/s32-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/-8sLtPu1Hyw0/AAAAAAAAAAI/AAAAAAAAAAA/a_52bWnC0-s/s64-c-k-no-mo-rj-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "Q2pzS09Rb2FRMHRQTkhRMVEzbzBMVkZEUmxweGRrUlJiMlJZV0d0QlVHY1NHME5PYm5OdlVESm1OQzFSUTBaUmRsTlhRVzlrVVZGclJUTlJNeEFBR2pnS0RRb0xhelJMZDBsWFQwdERkalFxSndvWVZVTnZTWEZ1TVZvMWFYaERXbmRqTUVSWFNqZHlTME5uRWd0ck5FdDNTVmRQUzBOMk5DQUNLQUV5R2dvWVZVTnlOVGxXVlY5amRtWnlkVEF0YW1GeWNtUk1NMDEz"
}
},
"id": "CjkKGkNLTzR0NUN6NC1RQ0ZacXZEUW9kWFhrQVBnEhtDTm5zb1AyZjQtUUNGUXZTV0FvZFFRa0UzUTM%3D",
"timestampUsec": "1569119896722467",
"authorExternalChannelId": "UCr59VU_cvfru0-jarrdL3Mw",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
},
"clientId": "CNnsoP2f4-QCFQvSWAodQQkE3Q3"
}
}
]
}
},
"endpoint": {
"commandMetadata": {
"webCommandMetadata": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D"
}
},
"urlEndpoint": {
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D"
}
},
"csn": "n96GXabRGouFlQTigY2YDg",
"xsrf_token": "QUFFLUhqbHNNWTF3NFJqc2h3cGE1NE9FWGdaWk5mRlVhUXxBQ3Jtc0tuTWhZNFcyTW1iZnA3ZnFTYUFudVFEUVE0cnFEOVBGcEU1MEh0Zlh4bll1amVmRl9OMkxZV3pKV1ZSbExBeDctTl95NGtBVnJZdlNxeS1KdWVNempEN2N6MHhaU1laV3hnVkZPeHp1OHVDTGVFSGUyOGduT0szbDV5N05LYUZTdzdoTDRwV1VJWndaVjdQVGRjNWVpR0YwUXgtZXc=",
"url": "/live_chat/get_live_chat?continuation=0ofMyAPiARp8Q2c4S0RRb0xhelJMZDBsWFQwdERkalFhUTZxNXdiMEJQUW83YUhSMGNITTZMeTkzZDNjdWVXOTFkSFZpWlM1amIyMHZiR2wyWlY5amFHRjBQM1k5YXpSTGQwbFhUMHREZGpRbWFYTmZjRzl3YjNWMFBURWdBZyUzRCUzRCiPz5-Os-PkAjAAOABAAUorCAAQABgAIAAqDnN0YXRpY2NoZWNrc3VtOgBAAEoCCAFQgJqXjrPj5AJYA1CRwciOs-PkAli3pNq1k-PkAmgBggEECAEQAIgBAKABjbfnjrPj5AI%253D",
"timing": {
"info": {
"st": 81
}
}
}

View File

@@ -0,0 +1,98 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatMembershipItemRenderer": {
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"authorExternalChannelId": "author_channel_id",
"headerSubtext": {
"runs": [
{
"text": "新規メンバー"
}
]
},
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"authorBadges": [
{
"liveChatAuthorBadgeRenderer": {
"customThumbnail": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/X=s32-c-k"
},
{
"url": "https://yt3.ggpht.com/X=s64-c-k"
}
]
},
"tooltip": "新規メンバー",
"accessibility": {
"accessibilityData": {
"label": "新規メンバー"
}
}
}
}
],
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
}
]
}
}
}

View File

@@ -0,0 +1,80 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatLegacyPaidMessageRenderer": {
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"eventText": {
"runs": [
{
"text": "新規メンバー"
}
]
},
"detailText": {
"simpleText": "ようこそ、author_name"
},
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"authorExternalChannelId": "author_channel_id",
"contextMenuEndpoint": {
"clickTrackingParams": "___clickTrackingParams___",
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
}
]
}
}
}

View File

@@ -0,0 +1,110 @@
{
"responseContext": {
"webResponseContextExtensionData": "data"
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"liveChatReplayContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"replayChatItemAction": {
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [
{
"text": "dummy_message"
}
]
},
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"contextMenuEndpoint": {
"clickTrackingParams": "___clickTrackingParams___",
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"authorBadges": [
{
"liveChatAuthorBadgeRenderer": {
"customThumbnail": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/X=s16-c-k"
},
{
"url": "https://yt3.ggpht.com/X=s32-c-k"
}
]
},
"tooltip": "メンバー1 か月)",
"accessibility": {
"accessibilityData": {
"label": "メンバー1 か月)"
}
}
}
}
],
"authorExternalChannelId": "author_channel_id",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
},
"timestampText": {
"simpleText": "1:23:45"
}
}
},
"clientId": "dummy_client_id"
}
}
],
"videoOffsetTimeMsec": "5025120"
}
}
]
}
}
}

182
tests/testdata/default/superchat.json vendored Normal file
View File

@@ -0,0 +1,182 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatPaidMessageRenderer": {
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"purchaseAmountText": {
"simpleText": "¥800"
},
"message": {
"runs": [
{
"text": "dummy_message"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "author_channel_id",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
},
{
"addLiveChatTickerItemAction": {
"item": {
"liveChatTickerPaidMessageItemRenderer": {
"id": "dummy_id",
"amount": {
"simpleText": "¥846"
},
"amountTextColor": 4278190080,
"startBackgroundColor": 4280150454,
"endBackgroundColor": 4278239141,
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"durationSec": 120,
"showItemEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"showLiveChatItemEndpoint": {
"renderer": {
"liveChatPaidMessageRenderer": {
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"purchaseAmountText": {
"simpleText": "¥846"
},
"message": {
"runs": [
{
"text": "dummy_message"
}
]
},
"headerBackgroundColor": 4278239141,
"headerTextColor": 4278190080,
"bodyBackgroundColor": 4280150454,
"bodyTextColor": 4278190080,
"authorExternalChannelId": "author_channel_id",
"authorNameTextColor": 2315255808,
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"timestampColor": 2147483648,
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
}
}
},
"authorExternalChannelId": "http://www.youtube.com/channel/author_channel_url",
"fullDurationSec": 120
}
},
"durationSec": "120"
}
}
]
}
}
}

View File

@@ -0,0 +1,97 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatPaidStickerRenderer": {
"id": "dummy_id",
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
},
"timestampUsec": 1570678496000000,
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"authorName": {
"simpleText": "author_name"
},
"authorExternalChannelId": "author_channel_id",
"sticker": {
"thumbnails": [
{
"url": "//lh3.googleusercontent.com/param_s=s72-rp",
"width": 72,
"height": 72
},
{
"url": "//lh3.googleusercontent.com/param_s=s144-rp",
"width": 144,
"height": 144
}
],
"accessibility": {
"accessibilityData": {
"label": "___sticker_label___"
}
}
},
"moneyChipBackgroundColor": 4278248959,
"moneyChipTextColor": 4278190080,
"purchaseAmountText": {
"simpleText": "¥200"
},
"stickerDisplayWidth": 72,
"stickerDisplayHeight": 72,
"backgroundColor": 4278237396,
"authorNameTextColor": 3003121664
}
}
}
}
]
}
}
}

77
tests/testdata/default/textmessage.json vendored Normal file
View File

@@ -0,0 +1,77 @@
{
"responseContext": {
"webResponseContextExtensionData": ""
},
"continuationContents": {
"liveChatContinuation": {
"continuations": [
{
"invalidationContinuationData": {
"invalidationId": {
"objectSource": 1000,
"objectId": "___objectId___",
"topic": "chat~00000000000~0000000",
"subscribeToGcmTopics": true,
"protoCreationTimestampMs": "1577804400000"
},
"timeoutMs": 10000,
"continuation": "___continuation___"
}
}
],
"actions": [
{
"addChatItemAction": {
"item": {
"liveChatTextMessageRenderer": {
"message": {
"runs": [
{
"text": "dummy_message"
}
]
},
"authorName": {
"simpleText": "author_name"
},
"authorPhoto": {
"thumbnails": [
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s32-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 32,
"height": 32
},
{
"url": "https://yt3.ggpht.com/------------/AAAAAAAAAAA/AAAAAAAAAAA/xxxxxxxxxxxx/s64-x-x-xx-xx-xx-c0xffffff/photo.jpg",
"width": 64,
"height": 64
}
]
},
"contextMenuEndpoint": {
"commandMetadata": {
"webCommandMetadata": {
"ignoreNavigation": true
}
},
"liveChatItemContextMenuEndpoint": {
"params": "___params___"
}
},
"id": "dummy_id",
"timestampUsec": 1570678496000000,
"authorExternalChannelId": "author_channel_id",
"contextMenuAccessibility": {
"accessibilityData": {
"label": "コメントの操作"
}
}
}
},
"clientId": "dummy_client_id"
}
}
]
}
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

3076
tests/testdata/fetch_patch/pt0-0.json vendored Normal file

File diff suppressed because it is too large Load Diff

3076
tests/testdata/fetch_patch/pt0-1.json vendored Normal file

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More