67 lines
5.5 KiB
Markdown
67 lines
5.5 KiB
Markdown
# YouTube Chat Listener (Version 2) - Project Progress Log
|
|
|
|
This document chronicles the key steps, findings, and decisions made during the development and research phases of the YouTube Chat Listener (Version 2) project.
|
|
|
|
## Initial State
|
|
|
|
* Existing `youtube_chat_terminal` project used YouTube Data API v3 polling.
|
|
* Concerns about API quota exhaustion led to the conception of `youtube-chat-webhook-v2`.
|
|
|
|
## Research Phase - Initial Plan (gRPC Misconception)
|
|
|
|
* **Goal:** Find a quota-friendly, compliant solution.
|
|
* **Initial Hypothesis:** `liveChatMessages.streamList` (gRPC) was believed to be the ideal solution.
|
|
* **Action:** A detailed "Deep Research Plan" was outlined in `README.md` and `DEVELOPMENT_PLAN.md`.
|
|
* **Finding:** Subsequent research (via `google_web_search`) revealed that `liveChatMessages.streamList` (gRPC) is **not publicly available** for the YouTube Data API v3. This was a critical correction to the initial understanding.
|
|
|
|
## Research Phase - Revised Plan (Focus on Sustainability & `pytchat`)
|
|
|
|
* **Goal:** Identify a sustainable, quota-friendly, compliant, open-source, Linux-compatible method for real-time YouTube Live Chat, explicitly ruling out API quota increases.
|
|
* **Revised Strategy:** Shifted focus to:
|
|
* Deep dive into YouTube's web client communication.
|
|
* Re-exploration of YouTube Data API v3 (creative use).
|
|
* Community solutions and open-source projects.
|
|
* Re-evaluation of third-party services.
|
|
* **Action:** `README.md` and `DEVELOPMENT_PLAN.md` were updated to reflect this revised plan.
|
|
|
|
## Experimental Implementation - `pytchat` Exploration
|
|
|
|
* **Objective:** Experiment with `pytchat` as a direct (but risky) solution for zero-quota live chat fetching.
|
|
* **Action:** Installed `pytchat` and created `pytchat_listener.py` for basic chat fetching and display.
|
|
* **Status:** `pytchat_listener.py` is working as expected.
|
|
* **Internal Mechanism Analysis of `pytchat`:**
|
|
* **Endpoint:** `POST https://www.youtube.com/youtubei/v1/live_chat/get_live_chat` (internal, undocumented API).
|
|
* **Authentication/Session:** Relies on mimicking browser headers (`User-Agent`), `visitorData` (extracted from previous responses), dynamically generated `clientVersion`, and `httpx.Client`'s automatic cookie handling.
|
|
* **Continuation Token:** Complex, encoded parameter generated using custom Protocol Buffers-like encoding and timestamps.
|
|
* **Channel ID Discovery:** Performs lightweight scraping of YouTube's `embed` or `m.youtube.com` pages using regex.
|
|
* **Implications:** Highly fragile (subject to breaking), **critical compliance risk** (violates YouTube's Terms of Service).
|
|
* **Action:** `DEVELOPMENT_PLAN.md` was updated with these findings.
|
|
|
|
## Phase 2: Re-exploration of YouTube Data API v3 (Creative Use)
|
|
|
|
### Action 1: Live Chat Replay API
|
|
* **Investigation:** Explored `liveChatMessages.list` for replays to assess quota characteristics and suitability for near real-time.
|
|
* **Findings:** `liveChatMessages.list` costs 5 quota points per request, regardless of live or replay. Frequent polling exhausts the 10,000 daily quota quickly (approx. 33 mins at 1 req/sec). Not designed for efficient extensive chat history replay. No special quota for replay usage.
|
|
* **Conclusion:** Not a sustainable, quota-friendly solution for continuous monitoring.
|
|
|
|
### Action 2: Minimal `part` Parameters
|
|
* **Investigation:** Re-confirmed the absolute minimum `part` parameters for `liveChatMessages.list` to reduce quota cost.
|
|
* **Findings:** The minimal `part` parameters to retrieve essential chat message information (author's name, message content, and author's unique ID for persistent colors) are `snippet,authorDetails`. This will incur a cost of 5 quota points per request.
|
|
* **Conclusion:** While minimal parameters are identified, the base cost of 5 quota points per request still makes continuous polling unsustainable for the project's goal.
|
|
|
|
### Action 3: Intelligent Polling Refinement
|
|
* **Investigation:** Explored advanced adaptive polling strategies beyond `pollingIntervalMillis`, potentially incorporating machine learning to predict chat activity and adjust polling frequency.
|
|
* **Findings:** While intelligent polling is a valuable concept for API management, it does not offer a viable path to a sustainable, quota-friendly solution for *continuous, real-time YouTube Live Chat using the official API*. Its application to `pytchat` is also not directly beneficial as `pytchat` already adapts its polling based on YouTube's internal signals.
|
|
* **Conclusion:** Not a primary solution for continuous chat fetching using the official API; not directly beneficial for `pytchat`.
|
|
|
|
## Phase 3: Community Solutions and Open-Source Projects
|
|
|
|
### Action 1: GitHub/GitLab Search (Targeted `taizan-hokuto`)
|
|
* **Investigation:** Searched for projects related to `pytchat` mentions by `taizan-hokuto` (original author).
|
|
* **Findings:** The original `pytchat` repository on GitHub (`https://github.com/taizan-hokuto/pytchat`) is publicly archived and no longer maintained by the author. No new active forks or related projects by the original author were immediately identified through this targeted search.
|
|
* **Conclusion:** Confirmed `pytchat`'s archived status; no direct new leads from `taizan-hokuto`.
|
|
|
|
## Next Steps
|
|
|
|
* Proceed with "GitHub/GitLab Search (General)" as outlined in `DEVELOPMENT_PLAN.md`.
|
|
* Continue with other phases of the revised research plan, keeping the compliance and fragility risks of `pytchat` in mind. |