01SOURCE
Supplier pages
Caple WooCommerce product and accessory pages.
Reliable selectors expose SKU, price, gallery images, features, accessories and related items.→SUPPLIER DATA / REVIEW ARCHITECTURE
Caple Scrape Review
A staged catalogue pipeline that turns supplier product pages into structured, reviewable and verified website data without publishing uncertain information automatically.
01 / SYSTEM FLOWHow information
How information
moves through it.
The architecture separates collection, judgement and delivery so each stage can be inspected and rerun independently.
02AUTOMATION
Scrape + validate
The scraper normalises the source into one catalogue shape.
scrapeCapleSample.mjs uses shared validation before anything moves into the later stages.→03BOUNDARY
JSON contract
caple-sample-products.json becomes the canonical snapshot.
Every later step consumes the same reviewed snapshot instead of scraping the supplier again.→04CONTROL
Human review
The Next.js review route makes errors visible.
Products, prices, images, features and product relationships can be checked together before import.→05STORAGE
R2 + Supabase
Images are converted to WebP and data is mapped relationally.
R2 holds web media; Supabase stores products, images, specifications and ordered product links.→06DELIVERY
Verify + render
Exact verification gates the production website.
The verifier checks rows and ordering; product pages then read the catalogue through Supabase RPCs.02 / RESPONSIBILITIESClear boundaries.
Clear boundaries.
Smaller failure areas.
Collection layer
Responsible only for reading and normalising supplier content.
- WooCommerce HTML
- Category index
- Rate-limited requests
- Shared validation
Review boundary
Keeps automation and publication deliberately separate.
- Canonical JSON
- Caple review UI
- Feature audit
- Small proof batches
Delivery layer
Moves approved assets and relationships into production-ready storage.
- Sharp / WebP
- Cloudflare R2
- Supabase tables
- Catalogue RPCs
03 / DESIGN DECISIONSWhy it is built
Why it is built
this way.
Review before import
A scraper can be syntactically correct while still misunderstanding a product. The visual review surface catches those semantic errors.
One reusable snapshot
Image processing, importing and verification share the same JSON contract, avoiding different stages seeing different supplier data.
Idempotent, small batches
The pipeline can be safely rerun, but independent verification and small batches remain the safety net because the workflow is not one large transaction.
MORE WORK