ramforth/youtube-chat-webhook-v2
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Update README.md and DEVELOPMENT_PLAN.md with revised research plan

Browse Source
This commit is contained in:
Ramforth
2025-10-30 17:33:27 +01:00
parent 60bf3290d3
commit b0443a9c4a
2 changed files with 42 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
DEVELOPMENT_PLAN.md
View File

@@ -1,47 +1,46 @@
# YouTube Chat Webhook Listener (Version 2) - gRPC Implementation Plan # YouTube Chat Listener (Version 2) - Revised Research & Exploration Plan
This document outlines the detailed plan for implementing a robust, quota-friendly, open-source, and Linux-compatible solution for monitoring YouTube Live Chat by leveraging the recommended `liveChatMessages.streamList` gRPC endpoint. This document outlines the revised plan for exploring sustainable, quota-friendly, compliant, open-source, and Linux-compatible methods for monitoring YouTube Live Chat. Our previous assumption of a public gRPC `liveChatMessages.streamList` endpoint was incorrect. This plan focuses on finding alternative solutions that do not rely on continuous, quota-limited API polling or requesting quota increases.
## Phase 1: gRPC Client Setup ## Project Goal
1. **Install Dependencies:** To identify and, if feasible, implement a sustainable, quota-friendly, compliant, open-source, and Linux-compatible method for receiving real-time YouTube Live Chat messages, processing them, and displaying them in the terminal with rich formatting. This goal explicitly rules out relying on YouTube Data API v3 quota increases.
* **Action:** Install `grpcio` and `grpcio-tools` Python packages.
2. **Obtain `.proto` File:** ## Phase 1: Deep Dive into YouTube's Web Client Communication
* **Action:** Locate and download the official Protocol Buffers (`.proto`) file that defines the `liveChatMessages.streamList` service.
3. **Generate Python Client Code:** * **Objective:** Understand how YouTube's official web client obtains live chat data to identify potential internal APIs, WebSocket connections, or other event-driven mechanisms.
* **Action:** Use `grpc_tools.protoc` to generate the Python client-side libraries from the `.proto` file. * **Actions:**
1. **Network Traffic Analysis:** Use browser developer tools (e.g., Chrome DevTools, Firefox Developer Tools) to inspect network traffic when viewing a live stream's chat. Look for WebSocket connections, XHR requests, or other non-standard API calls related to chat messages.
2. **Identify Internal APIs:** Analyze the payloads and endpoints of any discovered internal APIs.
3. **Protocol Analysis:** If WebSockets are found, attempt to understand the communication protocol.
4. **Tooling:** Consider using tools like `mitmproxy` for more in-depth network traffic interception and analysis on a Linux system.
* **Expected Outcome:** A detailed understanding of YouTube's internal live chat data acquisition methods.
4. **Develop gRPC Client:** ## Phase 2: Re-exploration of YouTube Data API v3 (Creative Use)
* **Action:** Write a Python script to:
* Establish a secure gRPC channel to the YouTube API endpoint.
* Create a client stub for the `liveChatMessages.streamList` service.
* Initiate the `StreamList` request to begin receiving messages.
* Implement a loop to continuously process messages as they are pushed from the server.
## Phase 2: Integration and Enhancements * **Objective:** Explore creative, highly optimized uses of the existing REST API that might offer better sustainability, even if not truly event-driven.
* **Actions:**
1. **Live Chat Replay API:** Investigate the `liveChatMessages.list` endpoint when used for *replays*. Does it have different quota characteristics or offer a more complete historical view that could be adapted for near real-time (e.g., fetching a larger batch less frequently)?
2. **Minimal `part` Parameters:** Re-confirm the absolute minimum `part` parameters required for `liveChatMessages.list` to reduce quota cost per call.
3. **Intelligent Polling Refinement:** Explore advanced adaptive polling strategies beyond `pollingIntervalMillis`, potentially incorporating machine learning to predict chat activity and adjust polling frequency.
1. **Integrate with Display Logic:** ## Phase 3: Community Solutions and Open-Source Projects
* **Action:** Adapt the existing `rich` display logic from `main.py` to consume messages received from the gRPC client instead of the polling mechanism.
2. **Error Handling & Resilience:** * **Objective:** Identify and analyze existing open-source projects that have successfully tackled sustainable YouTube Live Chat monitoring.
* **Action:** Implement robust error handling for gRPC connections, including automatic reconnection logic. * **Actions:**
* **Action:** Utilize `nextPageToken` (if provided by the gRPC stream) to resume receiving messages from where the connection was interrupted, preventing data loss. 1. **GitHub/GitLab Search:** Search for projects related to "YouTube Live Chat bot," "YouTube Live Chat client," "YouTube Live Chat API alternative," focusing on Python and Linux compatibility.
2. **Project Analysis:** For promising projects, analyze their source code to understand their data acquisition methods, quota management, and compliance strategies.
3. **Community Forums:** Explore discussions on platforms like Reddit (r/youtube, r/livestreamfails, r/programming), Stack Overflow, and relevant developer forums for insights into unofficial methods or workarounds.
3. **Configuration:** ## Phase 4: Re-evaluation of Third-Party Services (Event-Driven Focus)
* **Action:** Externalize API keys, default video ID, and display preferences into a configuration file (e.g., `config.ini` or `config.json`).
4. **Enhance Display Features:** * **Objective:** Re-examine third-party services, not for raw chat feeds, but for *any* form of event-driven notifications for specific chat events.
* **Action:** Implement a more sophisticated system to assign consistent, unique colors to each user. * **Actions:**
* **Action:** Improve emote rendering and potentially integrate with external emote services (e.g., BTTV, FrankerFaceZ) if feasible and compliant. 1. **Specific Event Triggers:** Investigate if services like StreamElements, Streamlabs, or others offer webhooks for specific, high-value chat events (e.g., Super Chats, new members, specific keywords) that could be consumed.
* **Action:** Add options for message filtering (e.g., by user, keywords, message type). 2. **Chat Relay Services:** Search for services that act as a "chat relay" for YouTube Live, potentially offering a more accessible API or WebSocket for consumption.
## Phase 3: Testing and Documentation ---
1. **Unit and Integration Tests:** **Prioritization:** All research will prioritize **open-source and Linux-compatible solutions**. Compliance with YouTube's Terms of Service remains a critical factor.
* **Action:** Write comprehensive unit tests for the gRPC client, message processing, and display logic.
* **Action:** Develop integration tests to ensure the end-to-end flow works correctly.
2. **Update Documentation:** **Next Steps:** The findings from this revised research will be compiled into a structured document to inform the design and implementation of a robust YouTube Live Chat monitoring solution.
* **Action:** Update the project's `README.md` with detailed usage instructions, setup guides, and explanations of the gRPC implementation.

22
README.md
View File

@@ -1,25 +1,23 @@
# YouTube Chat Webhook Listener (Version 2) - gRPC Implementation # YouTube Chat Listener (Version 2) - Sustainable Live Chat Exploration
This project aims to create a robust, quota-friendly, open-source, and Linux-compatible solution for monitoring YouTube Live Chat by implementing the recommended `liveChatMessages.streamList` gRPC endpoint. This approach will provide an event-driven, server-push model for receiving live chat messages, effectively eliminating the limitations and quota consumption associated with continuous API polling. This project aims to create a robust, quota-friendly, open-source, and Linux-compatible solution for monitoring YouTube Live Chat. Our initial research indicated a gRPC endpoint, but further investigation revealed this is not publicly available. Therefore, this project is now focused on a deeper exploration of alternative, sustainable methods to acquire live chat data without relying on continuous, quota-limited API polling or requesting quota increases.
## Project Goal ## Project Goal
To build a Python application that leverages the `liveChatMessages.streamList` gRPC endpoint to receive real-time YouTube Live Chat messages, process them, and display them in the terminal with rich formatting. This will ensure a highly efficient and compliant method for sustained live stream monitoring. To identify and, if feasible, implement a sustainable, quota-friendly, compliant, open-source, and Linux-compatible method for receiving real-time YouTube Live Chat messages, processing them, and displaying them in the terminal with rich formatting. This goal explicitly rules out relying on YouTube Data API v3 quota increases.
## Detailed Implementation Plan ## Revised Research & Exploration Plan
For a detailed breakdown of the implementation steps, please refer to the [DEVELOPMENT_PLAN.md](DEVELOPMENT_PLAN.md) file. For a detailed breakdown of the revised research and exploration steps, please refer to the [DEVELOPMENT_PLAN.md](DEVELOPMENT_PLAN.md) file.
## Dependencies ## Dependencies
* `grpcio` * `google-auth-oauthlib` (for potential initial API calls or authentication research)
* `grpcio-tools` * `google-api-python-client` (for potential initial API calls or authentication research)
* `google-auth-oauthlib` * `rich` (for terminal display)
* `google-api-python-client`
* `rich`
## Future Enhancements ## Future Enhancements
* Interactive message sending via gRPC (if supported by the API). * Interactive message sending.
* More advanced terminal UI (e.g., `prompt_toolkit` for input). * More advanced terminal UI (e.g., `prompt_toolkit` for input).
* Web overlay integration. * Web overlay integration.