Compare commits

...

45 Commits

Author SHA1 Message Date
24455ee327 Revert changes related to YouTube Data API integration and threading for message sending. Restore chat scraping functionality. 2025-11-03 12:26:42 +01:00
1ef278293c Updated with CODE OSS from CachyHelit 2025-10-31 11:46:38 +01:00
01249cb73b Docs: Added Matt North's YouTube link to Contributors 2025-10-30 22:48:39 +01:00
c9390f1583 Docs: Added License, Contributors, and API Auth clarification 2025-10-30 22:32:22 +01:00
3845f15405 Merge branch 'main' of https://gitea.ramforth.net/ramforth/youtube-chat-webhook-v2 2025-10-30 22:14:54 +01:00
7f3f65f0bf Working state: Reinstated terminal title fix 2025-10-30 22:01:05 +01:00
0279a41939 Update documentation with Phase 3 Action 2 findings 2025-10-30 20:33:24 +01:00
3b5100999a Update documentation with Phase 3 Action 1 findings and pytchat fork discussion 2025-10-30 20:30:03 +01:00
59912bd4e6 Update documentation with Phase 3 Action 1 findings 2025-10-30 20:28:39 +01:00
4d02a548b8 Update documentation with Phase 2 Action 3 findings 2025-10-30 20:24:15 +01:00
a35d23c598 Update documentation with Phase 2 Action 2 findings 2025-10-30 20:17:39 +01:00
0d45d295c3 Update DEVELOPMENT_PLAN.md with Phase 2 Action 1 findings 2025-10-30 20:14:16 +01:00
c4fc7e1680 Update DEVELOPMENT_PLAN.md with Phase 1.5 Display Enhancements 2025-10-30 20:11:08 +01:00
566d26791e Implement reserved colors logic for user_colors.json 2025-10-30 20:03:58 +01:00
d86aad9d82 Add annotations to pytchat_listener.py for clarity 2025-10-30 19:59:59 +01:00
7b4134f351 Fix rich markup interpretation in message_text using Text.from_markup() 2025-10-30 19:57:03 +01:00
061276618e Fix rich markup rendering for message_text 2025-10-30 19:53:50 +01:00
bda9164f7c Fix rich markup rendering for username style 2025-10-30 19:48:46 +01:00
7052a80afe Fix multi-line background fill using Text.stylize 2025-10-30 19:46:23 +01:00
93986e0681 Fix multi-line background fill using Text.assemble and Text.pad_to_width 2025-10-30 19:44:25 +01:00
3d68a8efc7 Fix multi-line background fill and add user_colors.json error handling 2025-10-30 19:38:07 +01:00
6c68edb9fe Fix multi-line background fill for rich display 2025-10-30 19:34:04 +01:00
ffe51afd37 Update documentation with current status and add installation instructions 2025-10-30 19:28:57 +01:00
865330db76 Implement persistent unique colors for chatters 2025-10-30 19:26:29 +01:00
eede9a3560 Fix rich display padding by using .maximum attribute 2025-10-30 19:21:29 +01:00
cceb082631 Implement alternating full-width background for chat messages 2025-10-30 19:14:51 +01:00
99ed27a8b7 Integrate cleaner colour_emoji function into pytchat_listener.py 2025-10-30 19:09:24 +01:00
c2d033e2d7 Fix SyntaxError in emoji_pattern regex (correct multi-line raw string with re.VERBOSE) 2025-10-30 19:04:14 +01:00
8f686e6466 Fix SyntaxError in emoji_pattern regex in pytchat_listener.py (correct multi-line string) 2025-10-30 19:01:43 +01:00
f17ed7cdd5 Fix SyntaxError in emoji_pattern regex in pytchat_listener.py 2025-10-30 18:57:10 +01:00
f7bfa63562 Integrate rich display logic and logging into pytchat_listener.py 2025-10-30 18:54:57 +01:00
3b5ec1fb47 Add PROGRESS_LOG.md 2025-10-30 18:51:41 +01:00
bfcdcb3271 Update DEVELOPMENT_PLAN.md with pytchat internal analysis findings and Gitea fork info 2025-10-30 18:50:30 +01:00
2c7db98112 Update DEVELOPMENT_PLAN.md with pytchat internal analysis findings 2025-10-30 18:46:25 +01:00
2ada77840d Update README.md and DEVELOPMENT_PLAN.md with pytchat status 2025-10-30 18:21:29 +01:00
6f0483f51b Implement 5-second display for video ID in pytchat_listener.py 2025-10-30 18:18:39 +01:00
cecc52f52b Add pytchat_listener.py for basic chat fetching 2025-10-30 18:12:53 +01:00
956d5d7c6f Update DEVELOPMENT_PLAN.md with internal API findings and pytchat focus 2025-10-30 18:06:17 +01:00
a00c58989c Update DEVELOPMENT_PLAN.md with internal API findings 2025-10-30 18:00:56 +01:00
b0443a9c4a Update README.md and DEVELOPMENT_PLAN.md with revised research plan 2025-10-30 17:33:27 +01:00
60bf3290d3 Add DEVELOPMENT_PLAN.md and update README.md 2025-10-30 17:21:57 +01:00
765821d26b Update README.md with gRPC implementation plan 2025-10-30 17:17:10 +01:00
9899b04dc0 Update README.md with research plan 2025-10-30 17:03:16 +01:00
d4c83fb908 Resolve merge conflict in README.md 2025-10-29 13:53:51 +01:00
dbb5a3ca77 Initial commit: Webhook Listener Project Plan 2025-10-29 13:39:37 +01:00
6 changed files with 497 additions and 1 deletions

122
DEVELOPMENT_PLAN.md Normal file
View File

@@ -0,0 +1,122 @@
# YouTube Chat Listener (Version 2) - Revised Research & Exploration Plan
This document outlines the revised plan for exploring sustainable, quota-friendly, compliant, open-source, and Linux-compatible methods for monitoring YouTube Live Chat. Our previous assumption of a public gRPC `liveChatMessages.streamList` endpoint was incorrect. This plan focuses on finding alternative solutions that do not rely on continuous, quota-limited API polling or requesting quota increases.
## Project Goal
To identify and, if feasible, implement a sustainable, quota-friendly, compliant, open-source, and Linux-compatible method for receiving real-time YouTube Live Chat messages, processing them, and displaying them in the terminal with rich formatting. This goal explicitly rules out relying on YouTube Data API v3 quota increases.
## Phase 1: Deep Dive into YouTube's Web Client Communication
* **Objective:** Understand how YouTube's official web client obtains live chat data to identify potential internal APIs, WebSocket connections, or other event-driven mechanisms.
* **Actions:**
1. **Network Traffic Analysis:** Use browser developer tools (e.g., Chrome DevTools, Firefox Developer Tools) to inspect network traffic when viewing a live stream's chat. Look for WebSocket connections, XHR requests, or other non-standard API calls related to chat messages.
2. **Identify Internal APIs:** Analyze the payloads and endpoints of any discovered internal APIs.
3. **Protocol Analysis:** If WebSockets are found, attempt to understand the communication protocol.
4. **Tooling:** Consider using tools like `mitmproxy` for more in-depth network traffic interception and analysis on a Linux system.
* **Expected Outcome:** A detailed understanding of YouTube's internal live chat data acquisition methods.
### Findings: Internal `POST /youtubei/v1/live_chat/get_live_chat` API and `pytchat`
Our network analysis revealed that YouTube's web client uses a `POST` request to `https://www.youtube.com/youtubei/v1/live_chat/get_live_chat` for receiving live chat messages. This is a continuation-based polling mechanism, not a gRPC stream or a simple public API call.
**Request Payload Key Elements:**
* `context`: Contains extensive client-specific information (e.g., `userAgent`, `clientName`, `clientVersion`, `visitorData`, `timeZone`, `browserName`, `browserVersion`). Mimicking this accurately is crucial.
* `continuation`: A string token that dictates which set of messages to fetch next. This token is received in the previous response.
**Response Payload Key Elements:**
* `continuationContents.liveChatContinuation.continuations`: Contains an `invalidationContinuationData` object with the *new* `continuation` token for the subsequent request.
* `actions`: An array containing `addChatItemAction` objects, each representing a chat message.
* `addChatItemAction.item.liveChatTextMessageRenderer`: Contains the message details.
* `message.runs[0].text`: The actual chat message content.
* `authorName.simpleText`: The author's display name.
* `authorPhoto.thumbnails`: URLs for the author's profile picture.
* `timestampUsec`: Timestamp of the message.
* `authorExternalChannelId`: The author's channel ID.
**Comprehensive Summary of `pytchat`'s Internal Mechanisms:**
`pytchat` leverages this internal `POST /youtubei/v1/live_chat/get_live_chat` API. It does not use explicit OAuth or API keys for live chat fetching. Instead, it relies on:
* **Mimicking Browser Headers:** Uses specific `User-Agent` strings.
* **`visitorData`:** A key session identifier extracted from the `responseContext` of a previous YouTube API call and included in the `context` object of subsequent requests. `pytchat` maintains a session by passing this `visitorData` back and forth.
* **`clientVersion`:** Dynamically generated to match a recent YouTube web client version.
* **Cookies:** The `httpx.Client` (used internally) handles cookies automatically, which are essential for maintaining a YouTube session.
* **Continuation Token:** A complex, encoded parameter generated using a custom Protocol Buffers-like encoding and various timestamps. This token is extracted from the `metadata` of the previous response.
* **Channel ID Discovery:** Performs lightweight scraping of YouTube's `embed` or `m.youtube.com` pages to extract the `channelId` using regular expressions.
**Implications (Reconfirmed):**
* **Fragility:** The reliance on dynamically generated `clientVersion`, extracted `visitorData`, regex-based scraping for `channelId`, and the undocumented internal API structure makes `pytchat` highly susceptible to breaking if YouTube changes its web client or internal API structure.
* **Compliance:** The lightweight scraping for `channelId` and the use of undocumented internal APIs are clear violations of YouTube's Terms of Service. This is a **critical risk** that could lead to account-level sanctions or blocking.
### Next Research Steps for Phase 1 (Focus on `pytchat` for experimental implementation):
1. **`pytchat` Installation and Basic Usage:**
* **Action:** Install the `pytchat` library.
* **Action:** Create a basic Python script to fetch and display chat using `pytchat` for a given live stream ID.
* **Status:** **COMPLETED.** The `pytchat_listener.py` script is working as expected.
2. **`pytchat` Internal Mechanism Analysis:**
* **Action:** Investigate how `pytchat` manages session/authentication internally (e.g., does it require a logged-in browser session, or does it generate necessary headers?).
* **Action:** Understand how `pytchat` handles the `continuation` token and polling.
* **Status:** **COMPLETED.** Analysis of `pytchat` source code (`api.py`, `core/__init__.py`, `core/pytchat.py`, `config/__init__.py`, `paramgen/liveparam.py`, `paramgen/enc.py`, `util/__init__.py`, `parser/live.py`) has provided a comprehensive understanding of its internal mechanisms. The `pytchat` repository has been mirrored to `https://gitea.ramforth.net/ramforth/pytchat-fork` for easier access.
3. **Integration with `rich` Display:**
* **Action:** Adapt the existing `rich` display logic from `main.py` to consume messages received from `pytchat`.
* **Status:** **COMPLETED.** The `pytchat_listener.py` script now includes full-width alternating backgrounds (with ongoing minor issue), emoji coloring, and persistent unique user colors.
## Phase 1.5: Display Enhancements (from To-Do-List)
* **Objective:** Implement additional display enhancements based on user feedback and the `To-Do-List.md`.
* **Actions:**
1. **Fix Multi-line Background Wrap:** Resolve the issue where alternating grey backgrounds do not fill 100% of the terminal width for multi-line messages.
2. **Add Animations (ref. Kitty terminal):** Investigate and implement subtle animations for new messages or other events.
3. **Set Terminal Title:** Dynamically set the terminal title to display relevant information (e.g., video ID, live status).
4. **Notification on New Message:** Implement a notification system for new messages, with a toggle to enable/disable it.
## Phase 2: Re-exploration of YouTube Data API v3 (Creative Use)
* **Objective:** Explore creative, highly optimized uses of the existing REST API that might offer better sustainability, even if not truly event-driven.
* **Actions:**
1. **Live Chat Replay API:** Investigate the `liveChatMessages.list` endpoint when used for *replays*. Does it have different quota characteristics or offer a more complete historical view that could be adapted for near real-time (e.g., fetching a larger batch less frequently)?
* **Findings:** `liveChatMessages.list` costs 5 quota points per request, regardless of whether it's for live or replay chat. Frequent polling (e.g., 1 request/second) will exhaust the 10,000 daily quota in about 33 minutes. The method is not designed for efficiently replaying extensive past chat history. There's no indication of different or more lenient quota characteristics for replay usage. This approach does not offer a sustainable, quota-friendly solution for continuous monitoring.
* **Status:** **COMPLETED.** Conclusion: Not a sustainable solution for continuous monitoring.
2. **Minimal `part` Parameters:** Re-confirm the absolute minimum `part` parameters required for `liveChatMessages.list` to reduce quota cost per call.
* **Findings:** The minimal `part` parameters to retrieve essential chat message information (author's name, message content, and author's unique ID for persistent colors) are `snippet,authorDetails`. This will incur a cost of 5 quota points per request.
* **Status:** **COMPLETED.**
3. **Intelligent Polling Refinement:** Explore advanced adaptive polling strategies beyond `pollingIntervalMillis`, potentially incorporating machine learning to predict chat activity and adjust polling frequency.
* **Findings:** While intelligent polling is a valuable concept for API management, it does not offer a viable path to a sustainable, quota-friendly solution for *continuous, real-time YouTube Live Chat using the official API*. Its application to `pytchat` is also not directly beneficial as `pytchat` already adapts its polling based on YouTube's internal signals.
* **Status:** **COMPLETED.** Conclusion: Not a primary solution for continuous chat fetching using the official API; not directly beneficial for `pytchat`.
## Phase 3: Community Solutions and Open-Source Projects
* **Objective:** Identify and analyze existing open-source projects that have successfully tackled sustainable YouTube Live Chat monitoring.
* **Actions:**
1. **GitHub/GitLab Search (Targeted `taizan-hokuto`):** Search for projects related to `pytchat` mentions by `taizan-hokuto` (original author).
* **Findings:** The original `pytchat` repository on GitHub (`https://github.com/taizan-hokuto/pytchat`) is publicly archived and no longer maintained by the author. No new active forks or related projects by the original author were immediately identified through this targeted search. However, the existence of our own fork (`https://gitea.ramforth.net/ramforth/pytchat-fork`) provides a controlled environment for potential maintenance and adaptation of the `pytchat`-based approach.
* **Status:** **COMPLETED.** Conclusion: Confirmed `pytchat`'s archived status; no direct new leads from `taizan-hokuto`. Our fork offers a path for maintenance.
2. **GitHub/GitLab Search (General):** Search for projects related to "YouTube Live Chat bot," "YouTube Live Chat client," "YouTube Live Chat API alternative," focusing on Python and Linux compatibility.
* **Findings:** Several promising projects were identified:
* **PyLivestream** (https://github.com/scivision/PyLivestream)
* **youtube-live-chat-client** (https://github.com/xenova/youtube-live-chat-client)
* **youtube-chat-downloader** (https://github.com/xenova/youtube-chat-downloader)
* **youtube-live-chat-bot** (https://github.com/xenova/youtube-live-chat-bot)
* **Status:** **COMPLETED.**
3. **Project Analysis:** For promising projects, analyze their source code to understand their data acquisition methods, quota management, and compliance strategies.
4. **Community Forums:** Explore discussions on platforms like Reddit (r/youtube, r/livestreamfails, r/programming), Stack Overflow, and relevant developer forums for insights into unofficial methods or workarounds.
## Phase 4: Re-evaluation of Third-Party Services (Event-Driven Focus)
* **Objective:** Re-examine third-party services, not for raw chat feeds, but for *any* form of event-driven notifications for specific chat events.
* **Actions:**
1. **Specific Event Triggers:** Investigate if services like StreamElements, Streamlabs, or others offer webhooks for specific, high-value chat events (e.g., Super Chats, new members, specific keywords) that could be consumed.
2. **Chat Relay Services:** Search for services that act as a "chat relay" for YouTube Live, potentially offering a more accessible API or WebSocket for consumption.
---
**Prioritization:** All research will prioritize **open-source and Linux-compatible solutions**. Compliance with YouTube's Terms of Service remains a critical factor.
**Next Steps:** The findings from this revised research will be compiled into a structured document to inform the design and implementation of a robust YouTube Live Chat monitoring solution.
## Phase 5: Advanced Features
* **Objective:** Explore and implement advanced functionalities to enhance chat interaction and moderation.
* **Potential Ideas:**
1. **LLM for Message Moderation:** Integrate a Large Language Model (LLM) to spot-check incoming messages for potential bad actors, inappropriate content, or spam. This would involve sending chat messages to the LLM for analysis and taking automated or semi-automated actions based on its output.

74
INSTALLATION.md Normal file
View File

@@ -0,0 +1,74 @@
# Installation and Usage Instructions
This document provides detailed instructions for setting up and running the `pytchat_listener.py` script for monitoring YouTube Live Chat.
## Prerequisites
* Python 3.8+
* `pip` (Python package installer)
* `git` (for cloning the repository)
## Installation Steps
1. **Clone the Repository:**
Open your terminal and clone the project repository from Gitea:
```bash
git clone https://gitea.ramforth.net/ramforth/youtube-chat-webhook-v2.git
cd youtube-chat-webhook-v2
```
2. **Create and Activate a Virtual Environment:**
It is highly recommended to use a Python virtual environment to manage project dependencies.
```bash
python3 -m venv venv
```
* **For Bash/Zsh:**
```bash
source venv/bin/activate
```
* **For Fish Shell:**
```bash
source venv/bin/activate.fish
```
3. **Install Dependencies:**
With your virtual environment activated, install the required Python packages:
```bash
pip install pytchat rich
```
## Usage
1. **Run the Script:**
Ensure your virtual environment is activated, then run the `pytchat_listener.py` script:
```bash
python pytchat_listener.py
```
2. **Enter Video ID:**
The script will prompt you to "Enter the YouTube Live Stream Video ID:". Paste the 11-character ID of the live stream you wish to monitor and press Enter.
3. **Observe Chat:**
The script will then start fetching and displaying live chat messages in your terminal.
4. **Save Chat Log:**
To exit the script gracefully, press `Ctrl+C`. You will then be prompted if you want to save the chat log.
## User Colors Persistence (`user_colors.json`)
The script assigns unique, persistent colors to chatters. This mapping is stored in a file named `user_colors.json` in the project directory.
* If this file does not exist, it will be created automatically.
* If it exists, the script will load the previously assigned colors.
* You can manually edit this file to change a user's color or reset the mapping.
## YouTube API Authentication (Optional)
For the core functionality of fetching live chat using `pytchat`, explicit YouTube Data API v3 authentication (e.g., API keys or OAuth 2.0 credentials) is **not required**. `pytchat` leverages an internal YouTube API endpoint that does not necessitate these credentials for public live chat streams.
However, if you plan to extend this project to interact with other YouTube Data API v3 services (e.g., managing videos, accessing private data, or performing actions that require user authorization), you will need to set up API credentials. You can find detailed instructions on how to obtain and configure these credentials in the official Google Cloud documentation:
* [Google Cloud Console - Credentials](https://console.cloud.google.com/apis/credentials)
* [YouTube Data API v3 Overview](https://developers.google.com/youtube/v3)
For Python projects, you would typically use libraries like `google-auth-oauthlib` and `google-api-python-client` to handle authentication and API interactions.

76
PROGRESS_LOG.md Normal file
View File

@@ -0,0 +1,76 @@
# YouTube Chat Listener (Version 2) - Project Progress Log
This document chronicles the key steps, findings, and decisions made during the development and research phases of the YouTube Chat Listener (Version 2) project.
## Initial State
* Existing `youtube_chat_terminal` project used YouTube Data API v3 polling.
* Concerns about API quota exhaustion led to the conception of `youtube-chat-webhook-v2`.
## Research Phase - Initial Plan (gRPC Misconception)
* **Goal:** Find a quota-friendly, compliant solution.
* **Initial Hypothesis:** `liveChatMessages.streamList` (gRPC) was believed to be the ideal solution.
* **Action:** A detailed "Deep Research Plan" was outlined in `README.md` and `DEVELOPMENT_PLAN.md`.
* **Finding:** Subsequent research (via `google_web_search`) revealed that `liveChatMessages.streamList` (gRPC) is **not publicly available** for the YouTube Data API v3. This was a critical correction to the initial understanding.
## Research Phase - Revised Plan (Focus on Sustainability & `pytchat`)
* **Goal:** Identify a sustainable, quota-friendly, compliant, open-source, Linux-compatible method for real-time YouTube Live Chat, explicitly ruling out API quota increases.
* **Revised Strategy:** Shifted focus to:
* Deep dive into YouTube's web client communication.
* Re-exploration of YouTube Data API v3 (creative use).
* Community solutions and open-source projects.
* Re-evaluation of third-party services.
* **Action:** `README.md` and `DEVELOPMENT_PLAN.md` were updated to reflect this revised plan.
## Experimental Implementation - `pytchat` Exploration
* **Objective:** Experiment with `pytchat` as a direct (but risky) solution for zero-quota live chat fetching.
* **Action:** Installed `pytchat` and created `pytchat_listener.py` for basic chat fetching and display.
* **Status:** `pytchat_listener.py` is working as expected.
* **Internal Mechanism Analysis of `pytchat`:**
* **Endpoint:** `POST https://www.youtube.com/youtubei/v1/live_chat/get_live_chat` (internal, undocumented API).
* **Authentication/Session:** Relies on mimicking browser headers (`User-Agent`), `visitorData` (extracted from previous responses), dynamically generated `clientVersion`, and `httpx.Client`'s automatic cookie handling.
* **Continuation Token:** Complex, encoded parameter generated using custom Protocol Buffers-like encoding and timestamps.
* **Channel ID Discovery:** Performs lightweight scraping of YouTube's `embed` or `m.youtube.com` pages using regex.
* **Implications:** Highly fragile (subject to breaking), **critical compliance risk** (violates YouTube's Terms of Service).
* **Action:** `DEVELOPMENT_PLAN.md` was updated with these findings.
## Phase 2: Re-exploration of YouTube Data API v3 (Creative Use)
### Action 1: Live Chat Replay API
* **Investigation:** Explored `liveChatMessages.list` for replays to assess quota characteristics and suitability for near real-time.
* **Findings:** `liveChatMessages.list` costs 5 quota points per request, regardless of live or replay. Frequent polling exhausts the 10,000 daily quota quickly (approx. 33 mins at 1 req/sec). Not designed for efficient extensive chat history replay. No special quota for replay usage.
* **Conclusion:** Not a sustainable, quota-friendly solution for continuous monitoring.
### Action 2: Minimal `part` Parameters
* **Investigation:** Re-confirmed the absolute minimum `part` parameters for `liveChatMessages.list` to reduce quota cost.
* **Findings:** The minimal `part` parameters to retrieve essential chat message information (author's name, message content, and author's unique ID for persistent colors) are `snippet,authorDetails`. This will incur a cost of 5 quota points per request.
* **Conclusion:** While minimal parameters are identified, the base cost of 5 quota points per request still makes continuous polling unsustainable for the project's goal.
### Action 3: Intelligent Polling Refinement
* **Investigation:** Explored advanced adaptive polling strategies beyond `pollingIntervalMillis`, potentially incorporating machine learning to predict chat activity and adjust polling frequency.
* **Findings:** While intelligent polling is a valuable concept for API management, it does not offer a viable path to a sustainable, quota-friendly solution for *continuous, real-time YouTube Live Chat using the official API*. Its application to `pytchat` is also not directly beneficial as `pytchat` already adapts its polling based on YouTube's internal signals.
* **Conclusion:** Not a primary solution for continuous chat fetching using the official API; not directly beneficial for `pytchat`.
## Phase 3: Community Solutions and Open-Source Projects
### Action 1: GitHub/GitLab Search (Targeted `taizan-hokuto`)
* **Investigation:** Searched for projects related to `pytchat` mentions by `taizan-hokuto` (original author).
* **Findings:** The original `pytchat` repository on GitHub (`https://github.com/taizan-hokuto/pytchat`) is publicly archived and no longer maintained by the author. No new active forks or related projects by the original author were immediately identified through this targeted search. However, the existence of our own fork (`https://gitea.ramforth.net/ramforth/pytchat-fork`) provides a controlled environment for potential maintenance and adaptation of the `pytchat`-based approach.
* **Conclusion:** Confirmed `pytchat`'s archived status; no direct new leads from `taizan-hokuto`. Our fork offers a path for maintenance.
### Action 2: GitHub/GitLab Search (General)
* **Investigation:** Performed a general search for projects related to "YouTube Live Chat bot," "YouTube Live Chat client," "YouTube Live Chat API alternative," focusing on Python and Linux compatibility.
* **Findings:** Several promising projects were identified:
* **PyLivestream** (https://github.com/scivision/PyLivestream)
* **youtube-live-chat-client** (https://github.com/xenova/youtube-live-chat-client)
* **youtube-chat-downloader** (https://github.com/xenova/youtube-chat-downloader)
* **youtube-live-chat-bot** (https://github.com/xenova/youtube-live-chat-bot)
* **Conclusion:** These projects provide good starting points for further analysis of alternative solutions.
## Next Steps
* Proceed with "Project Analysis" as outlined in `DEVELOPMENT_PLAN.md`.
* Continue with other phases of the revised research plan, keeping the compliance and fragility risks of `pytchat` in mind.

View File

@@ -1,2 +1,44 @@
# youtube-chat-webhook-v2
# YouTube Chat Listener (Version 2) - Sustainable Live Chat Exploration
This project aims to create a robust, quota-friendly, open-source, and Linux-compatible solution for monitoring YouTube Live Chat. Our initial research indicated a gRPC endpoint, but further investigation revealed this is not publicly available. Therefore, this project is now focused on a deeper exploration of alternative, sustainable methods to acquire live chat data without relying on continuous, quota-limited API polling or requesting quota increases.
## Project Goal
To identify and, if feasible, implement a sustainable, quota-friendly, compliant, open-source, and Linux-compatible method for receiving real-time YouTube Live Chat messages, processing them, and displaying them in the terminal with rich formatting. This goal explicitly rules out relying on YouTube Data API v3 quota increases.
## Detailed Implementation Plan
For a detailed breakdown of the revised research and exploration steps, please refer to the [DEVELOPMENT_PLAN.md](DEVELOPMENT_PLAN.md) file.
## Current Status
The `pytchat_listener.py` script has been successfully implemented and tested for basic chat fetching and display. Key features include:
* **Full-width alternating backgrounds** for readability.
* **Emoji coloring** for standard emojis.
* **Persistent unique colors** for chatters across sessions.
## Installation and Usage
For detailed installation and usage instructions, please refer to the [INSTALLATION.md](INSTALLATION.md) file.
## Dependencies
* `pytchat`
* `rich`
* `google-auth-oauthlib` (for potential initial API calls or authentication research)
* `google-api-python-client` (for potential initial API calls or authentication research)
## Future Enhancements
* Interactive message sending.
* More advanced terminal UI (e.g., `prompt_toolkit` for input).
* Web overlay integration.
## License
This project is licensed under the MIT License.
## Contributors
* [Matt North](https://www.youtube.com/channel/UCaSAujYjLliDCkwODH2z5vg)

167
pytchat_listener.py Normal file
View File

@@ -0,0 +1,167 @@
import pytchat
import sys
import os
import time
import re
import json
from datetime import datetime
from rich.console import Console
from rich.style import Style
from rich.text import Text
console = Console()
# Define the emoji pattern and coloring function
emoji_pattern = re.compile(
r'('
r' \U0001F600-\U0001F64F' # emoticons
r' \U0001F300-\U0001F5FF' # symbols & pictographs
r' \U0001F680-\U0001F6FF' # transport & map symbols
r' \U0001F1E0-\U0001F1FF' # flags (iOS)
r' \u2702-\u27B0' # Dingbats
r' \u24C2-\u2B55' # Enclosed characters
r' \U0001F900-\U0001F9FF' # Supplemental Symbols & Pictographs
r' \u200D' # ZWJ
r' \uFE0F' # VS-16
r')+', flags=re.UNICODE)
def colour_emoji(txt):
return emoji_pattern.sub(r'[magenta]\1[/magenta]', txt)
# --- User Color Management ---
USER_COLORS_FILE = "user_colors.json"
# A palette of distinct colors for usernames
COLOR_PALETTE = [
"#FF0000", "#00FF00", "#0000FF", "#FFFF00", "#00FFFF", "#FF00FF",
"#FFA500", "#800080", "#008000", "#FFC0CB", "#808000", "#008080",
"#C0C0C0", "#800000", "#000080", "#FFD700", "#ADFF2F", "#FF69B4"
]
user_color_map = {}
def load_user_colors():
global user_color_map
if os.path.exists(USER_COLORS_FILE):
try:
with open(USER_COLORS_FILE, 'r') as f:
user_color_map = json.load(f)
except json.JSONDecodeError:
console.print(f"[red]Error loading {USER_COLORS_FILE}. File might be corrupted. Starting with empty colors.[/red]")
user_color_map = {}
def save_user_colors():
with open(USER_COLORS_FILE, 'w') as f:
json.dump(user_color_map, f, indent=4)
def get_user_color(author_id):
global user_color_map
if author_id not in user_color_map:
# Determine which colors are already in use by existing users
used_colors = set(user_color_map.values())
# Find available colors from the palette that are not yet used
available_colors = [color for color in COLOR_PALETTE if color not in used_colors]
if available_colors: # If there are unused colors in the palette
next_color_index = len(user_color_map) % len(available_colors)
user_color_map[author_id] = available_colors[next_color_index]
else: # If all colors in the palette are already used, cycle through the palette
next_color_index = len(user_color_map) % len(COLOR_PALETTE)
user_color_map[author_id] = COLOR_PALETTE[next_color_index]
console.print("[yellow]Warning: All colors in palette are reserved. Cycling through existing colors.[/yellow]")
save_user_colors() # Save immediately after assigning a new color
return user_color_map[author_id]
# --- Main Script Logic ---
def main():
# Set terminal title
sys.stdout.write("\033]0;YouTube Live Chat\007")
sys.stdout.flush()
# Clear the terminal screen
os.system('clear')
video_id = input("Enter the YouTube Live Stream Video ID: ")
# Setup chat logging
log_dir = "chat_logs"
os.makedirs(log_dir, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
log_filename = os.path.join(log_dir, f"chat_{video_id}_{timestamp}.log")
log_file = open(log_filename, "w", encoding="utf-8")
console.print(f"[green]Chat will be logged to {log_filename}[/green]")
# Load existing user colors from user_colors.json
load_user_colors()
try:
livechat = pytchat.create(video_id=video_id)
console.print(f"[green]Listening to live chat for video ID: {video_id}[/green]")
console.print(f"[green]Video ID accepted: {video_id}[/green]")
# Wait for 5 seconds, then clear the screen
time.sleep(5)
os.system('clear')
message_count = 0
while livechat.is_alive():
for c in livechat.get().sync_items():
author_display_name = c.author.name
# pytchat provides author.channelId, which is perfect for unique identification
author_channel_id = c.author.channelId
message_text = c.message
# Process text-based emotes (e.g., :face-purple-sweating:)
message_text = re.sub(r'(:[a-zA-Z0-9_-]+:)', r'[blue]\1[/blue]', message_text)
# Apply emoji coloring using the new function
message_text = colour_emoji(message_text)
# Get persistent color for the user
user_color = get_user_color(author_channel_id)
username_style = Style(color=user_color, bold=True)
# Create a Text object for the username and apply style directly
username_text = Text(f"{author_display_name}: ", style=username_style)
# Create a Text object for the message, interpreting rich markup
# This allows rich markup (e.g., [bold]) within the message text itself to be rendered.
message_text_rich = Text.from_markup(message_text)
# Assemble the full message
full_message_text = Text.assemble(username_text, message_text_rich)
# Alternate background styles
background_style = Style(bgcolor="#2B2B2B") if message_count % 2 == 0 else Style(bgcolor="#3A3A3A")
# Apply the background style to the entire Text object
# This ensures the background color extends across the full width of the terminal
# for all lines of the message, even if it wraps.
full_message_text.stylize(background_style)
# Print the message, letting rich handle wrapping and filling the width
# The 'width' parameter ensures the background fills the terminal width.
# 'overflow="crop"' prevents text from wrapping beyond the terminal width.
console.print(full_message_text, width=console.width, overflow="crop")
log_file.write(f"{datetime.now().strftime("%H:%M:%S")} {full_message_text.plain}\n")
message_count += 1
except Exception as e:
console.print(f"[red]An error occurred: {e}[/red]")
finally:
global _stop_input_thread
_stop_input_thread = True # Signal the input thread to stop
# input_thread.join() # Optionally wait for the input thread to finish, but daemon=True makes it optional
log_file.close()
save_log = input(f"\nDo you want to save the chat log to {log_filename}? (y/n): ").lower()
if save_log != 'y' and save_log != 'yes':
os.remove(log_filename)
console.print(f"[red]Chat log not saved. {log_filename} deleted.[/red]")
else:
console.print(f"[green]Chat log saved to {log_filename}[/green]")
if __name__ == '__main__':
main()

15
user_colors.json Normal file
View File

@@ -0,0 +1,15 @@
{
"UCR12saiy07zwHngHGRPElPA": "#FF8000",
"UCUN25adHEN3RsJv5zG5q-NA": "#6600CC",
"UCnQp1K5xb4BwkQcgGWLDgXg": "#CCCC00",
"UCHKOdAXNoqWigv68Akl2tfQ": "#FFFF00",
"UCzGyNkYAsiz44Y2j_Sow-Cg": "#FF00FF",
"UCB-9oxvzRaKE7nXTjlxGVuw": "#800080",
"UCFEBDP1nhQvSfCM-jwzRYYg": "#FFC0CB",
"UCqWPyox1YcGyLvd5diOHQzQ": "#008080",
"UChYWkaRhR6QjpPg89QEH9dw": "#800000",
"UCM_70tWxaeehPw_muaS8ViA": "#FFD700",
"UCaSAujYjLliDCkwODH2z5vg": "#FF69B4",
"UCZ5WJG7fitlL0lru5yW4Nbw": "#00FF00",
"UCQy5619DyyQ0wktqB_RnXzw": "#FFA500"
}