Don't veto PDFCrawlerStrategy placeholder responses

[SohamKukreti]

Summary

PDFCrawlerStrategy returns stub html (real content comes from PDFContentScrapingStrategy), which the anti-bot checks misread as a block — every PDF crawl failed with Blocked by anti-bot protection: Near-empty content (33 bytes) with HTTP 200 even though extraction succeeded, breaking the documented PDF workflow. The phantom "blocked" verdict also burned every configured retry/proxy attempt and pointlessly invoked fallback_fetch_function on crawls that had already succeeded.

Fix: add AsyncCrawlResponse.placeholder_html (default False) so a crawler strategy can declare its html is a stand-in — mirroring the existing raw:-URL and downloaded_files exemptions — and skip both anti-bot call sites for flagged responses. The is_blocked() heuristic itself is unchanged; responses without the flag behave exactly as before.

Known limitation (disclosed): pairing PDFCrawlerStrategy with a non-PDF scraping strategy now yields the placeholder text as content with success=True; previously it produced a misleading anti-bot failure (wrong, but loud). Documented in the PDFCrawlerStrategy docstring.

Fixes #2135

List of files changed and why

• crawl4ai/models.py — add placeholder_html: bool = False to AsyncCrawlResponse so crawler strategies can declare stub html; default False keeps every existing strategy unchanged.
• crawl4ai/processors/pdf/__init__.py — PDFCrawlerStrategy.crawl() sets placeholder_html=True on its stub response; added a class docstring stating it must be paired with PDFContentScrapingStrategy.
• crawl4ai/async_webcrawler.py — skip the anti-bot classification in the attempt loop and the final anti-bot veto when the response declares placeholder_html (2 guard lines, alongside the existing raw:/downloaded_files exemptions).
• tests/test_placeholder_html_antibot.py — new regression tests (network-free, generated local PDF fixture; integration tests skip cleanly via pytest.importorskip("pypdf") when the [pdf] extra is absent).

How Has This Been Tested?

• New regression tests (4): the documented pairing succeeds on a local file:// PDF with text extracted; no retry burn or fallback invocation with max_retries=2 and a spy fallback_fetch_function (attempts == 1, resolved_by == "direct"); placeholder_html defaults to False; is_blocked() still flags near-empty html (heuristic unchanged).
• Fail-on-revert verified: with the fix reverted, 3 of the 4 tests fail — reproducing the exact original error (Blocked by anti-bot protection: Near-empty content (33 bytes)) and showing pre-fix code exhausting all retry attempts and invoking the fallback.
• Adversarial pass (16 targeted tests, all green): model serialization round-trip and type validation; cache write/replay; arun_many over multiple PDFs; failures stay loud (HTML bytes through the PDF pipeline, password-protected PDFs); normal browser crawls unchanged; a genuinely near-empty HTML page is still vetoed (the exemption is opt-in, not a bypass); the raw:// exemption still works.
• Full regression suite: tests/regression/ — 309 passed, 0 regressions (1 pre-existing failure from the missing optional transformers extra, unrelated).
• Manual: the exact example from docs/md_v2/advanced/pdf-parsing.md (the issue's repro) now returns success=True with full extracted markdown.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added/updated unit tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes