Audible scraper service

Node.js HTTP service that uses Puppeteer to load Audible product (PDP) and public listener profile pages, then parses HTML with JSDOM — similar in spirit to bookreverb/scraper-service for Amazon.

Important: Automated access to Audible may conflict with their Terms of Use. Use for internal tooling or with appropriate legal review.

What it does

Endpoint	Description
GET /health	Liveness; no auth
POST /scrape/pdp	Body: { "asin", "market": "us" | "uk" } — audiobook product metadata
POST /scrape/listener	Body: { "audibleUserId", "asinCandidates": [], "market" } — parse reviews on a listener page
POST /scrape/audiobook-profile	Body: { "asin", "audibleUserId", "market" } — PDP + listener in one call; matches the review for asin
POST /scrape/review-match-ocr	Body: { "asin", "market", "screenshotUrl" } — PDP reviews + OCR match (wrong profile recovery)

If AUTH_TOKEN is set in the environment, send Authorization: Bearer <token> on scrape routes.

audibleUserId format

Accepts either the raw id or the full prefix: values like amzn1.account.XXXX are normalized (see BookReverb app/api/audible/scrape/route.ts).

Local development

cp .env.example .env
npm install
npm run dev

In another terminal:

# Listener profile + PDP (needs correct listener id for a match)
npm run scrape -- --asin <AUDIBLE_BOOK_ASIN> --user <LISTENER_USER_ID> --market us

If the listener id is wrong but you have a screenshot of the review (public URL), use OCR + book PDP reviews. Quote the URL in zsh/bash — characters like & and ? are special unless the string is quoted:

npm run scrape -- --asin <ASIN> --market us --screenshot 'https://firebasestorage.googleapis.com/...?alt=media&token=...'

(OCR mode does not use --user; run it on one line so the whole command goes to npm run scrape.)

Optional: SCRAPE_BASE=http://127.0.0.1:8787 (default), --base to override.

• Set HEADLESS=false in .env to watch Chromium locally.
• Set DEBUG_SAVE_HTML=./test/fixtures/debug to dump HTML for debugging parsers.

Tests

npm test

Parser tests use saved HTML strings (no live Audible). Set AUDIBLE_LIVE_TEST=1 only if you add optional integration tests later.

Docker (parity with production)

docker compose up --build

Service listens on 8787 (mapped to container 8080).

Railway deployment

1. Create a new service from this directory (Dockerfile included).
2. Set environment variables:
   • PORT — Railway usually injects this (defaults to 8080 in the image).
   • PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium — already set in Dockerfile.
   • PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true — already set in Dockerfile.
   • Optional: AUTH_TOKEN, HEADLESS=true.
3. Memory: allocate at least ~1 GB for Chromium; keep scrape concurrency low (single browser, sequential navigations).
4. Browserless alternative: Run a Browserless (or self-hosted) container on Railway and point Puppeteer at browserWSEndpoint instead of launch() if you outgrow in-process Chrome.

Integrating with BookReverb (optional)

From Cloud Functions or Next.js, call POST /scrape/pdp or /scrape/audiobook-profile with the same JSON bodies, using your deployed base URL and Authorization if configured. Merge returned fields into Firestore similarly to how scrapeBookAll.ts consumes the Amazon scraper service — no code changes in BookReverb are required unless you want a direct wire-up.

Environment variables

See .env.example.

OCR fallback (wrong listener profile)

When the user saved the wrong Audible profile but submitted a screenshot of their review, call:

POST /scrape/review-match-ocr

Body: { "asin", "market": "us" | "uk", "screenshotUrl": "<public image URL>", "minConfidence": 0.9 }

The service scrolls the book PDP, parses reviews that link to listener profiles, runs Tesseract OCR on the screenshot, and picks the review whose text best matches the OCR (default 0.9 Dice similarity, including sliding windows for partial screenshots). Response includes listenerProfileId to store on the user.

Environment: OCR_MATCH_MIN_CONFIDENCE (optional), REVIEWS_SEE_MORE_MAX (default 25). Docker image installs tesseract-ocr for native OCR support.

Server logs for /scrape/review-match-ocr are prefixed with [pdp-reviews] (Puppeteer: URL, sort, “See more”, listener link count) and [review-match-ocr] (PDP title, OCR length/preview, parsed rows, match result). Watch the terminal where npm run dev runs.

BookReverb Cloud Function

Set on the scrape audiobook review function (or project env):

• AUDIBLE_SCRAPER_SERVICE_URL — base URL of this service (no trailing slash required)
• AUDIBLE_SCRAPER_AUTH_TOKEN — optional; must match AUTH_TOKEN here if set

When profile scraping fails for all domains and profile id candidates, the function calls /scrape/review-match-ocr if the submission has audibleReviewScreenshotUrl / audible_review_screenshot_url. On success it sets users/{userId}.audibleId to the resolved id and audibleIds: arrayUnion that id, then retries profile scrape for full star ratings when possible.