← Back to portfolio BUSINESS INTELLIGENCE / REVIEW ARCHITECTURE
Lead Audit Review A research pipeline that discovers suitable businesses, audits public website signals, develops evidence-backed opportunity hypotheses and keeps a person in control of every outreach decision.
01 / LEAD GENERATOR
Current From search terms and manually supplied URLs through crawling, evidence, scoring and human-reviewed outreach. The branches show where sources split, combine and become the next system record.
Search terms DuckDuckGo / Bing HTML search Manual URL CSV Directly supplied businesses Discovery candidates
FIRST, REMOVE THE NOISE A search result is not yet a useful lead. Search engines return directories, social profiles, repeated websites and businesses outside the brief. This check removes those distractions first, because every later answer is only useful if it belongs to the right company.
URL qualification, exclusions and domain deduplication
ONE BUSINESS, ONE STORY Everything now has somewhere to belong. Once a website passes the first check, the business receives one record. Pages, findings, scores and later conversations all attach to it, so the system can build one continuous picture instead of a pile of disconnected facts.
Lead records OPTIONAL Related-company expansion
THE RESEARCH STARTS The record tells us who. The crawler begins to learn how they work. It visits the public website just as a visitor would and brings back the pages needed for the audit. If the site cannot be reached, that failure stays with this one record instead of stopping research into every other business.
HTTP crawler HOMEPAGE DISCOVERY
sitemap.xml robots.txt Homepage links Page classification and sampling
FOLLOW THE USEFUL PATHS The system looks where a real opportunity would show itself. A homepage rarely tells the whole story. Contact, booking, product, trade and careers pages reveal how customers take action and how the business operates. Reading these next makes the audit focused rather than simply large.
Targeted page fetches Contact · product · trade · service · careers · booking · case study · location
TURN PAGES INTO SOMETHING READABLE A webpage has to be stripped back before it can be understood consistently. Menus, headings, forms and written content arrive mixed together as page code. This step separates the useful words and structures, giving the next stage clean material to inspect without changing what the website actually said.
BeautifulSoup HTML and text inspection
ASK THE SAME QUESTIONS EVERY TIME Clean page content still needs a fair way to be judged. This is a shared list of clear checks: can a visitor book, is there a useful contact route, are important pages missing, and does the site show signs of repetitive work? Using the same checks for every business makes the findings explainable and comparable.
Deterministic signal dictionary
KEEP THE PROOF WITH THE FINDING Each answer becomes evidence someone can check. A finding keeps the page it came from, the relevant words and an explanation of why it may matter commercially. The scoring stages can now use it, while a person can still trace any conclusion back to what was actually visible.
Evidence records URL + page type + issue + snippet + severity + commercial explanation Rule scores Business profile Company-size rules Combined commercial assessment
BRING THE THREE VIEWS BACK TOGETHER A weakness only matters when it connects to a real business problem. The rule scores, the type of business and its likely size are considered together here. That turns separate findings into a practical idea about where time may be lost, where customers may drop away, and what could be improved first.
Recommendations and bottleneck hypotheses
DECIDE WHAT DESERVES ATTENTION FIRST The score organises the work; it does not make the decision. Businesses with clearer problems, stronger evidence and a better chance of receiving value rise in the queue. The result helps a reviewer spend time wisely, but it never gives the system permission to contact anyone.
Opportunity priority score
MAKE ONE COMPLETE HANDOVER The research is gathered into a single finished result. Discovery details, evidence, scores and recommendations have travelled through different parts of the system. This step brings the important pieces back together so the same business can be reviewed, exported or reopened without rebuilding its story.
AuditResult Flattened output row
LEAVE A RECORD OF THE RUN The result should still make sense after the automation has stopped. Each run writes ordinary files that can be opened, compared and kept. This creates a clear boundary between research and review: the interface reads a saved result rather than silently changing the evidence in the background.
Run outputs CSV + XLSX + timestamped JSON
THE HUMAN CHECKPOINT This is where automation gives the work back to a person. The reviewer sees the website, the supporting evidence, the reasoning and any drafted message together. They can reject a weak lead, add context the crawler could not know, or approve the next action. Nothing is sent simply because a score was high.
Local Next.js review UI Notes / rejection SMTP sending IMAP reply check JSON state files
Follow-up reminders 02 / TOOLS BEHIND THE SYSTEM What powers each part. Each tool has a narrow job. Together they move the work from public research to a decision a person can understand and control.
01 DISCOVER
Find and reach the right websites The first group discovers businesses, checks their addresses and visits public pages without letting one slow website hold up the full run.
Python Runs the research pipeline and keeps each business isolated. DuckDuckGo + Bing Provide the first set of public discovery results. Requests Fetches pages with timeouts, rate limits and safe retries. Playwright Optional fallback for websites that need a real browser to appear. 02 UNDERSTAND
Read pages and form evidence The page code is reduced to useful content, then checked in the same consistent way for every business.
Beautiful Soup Separates useful text, links and page structure from the HTML. Python rules Applies visible, repeatable checks instead of hidden AI judgement. Evidence records Keep the source page and supporting words beside every finding. 03 PRESERVE
Preserve each research run The completed audit is written into ordinary, inspectable formats before it reaches the review interface.
Pandas Shapes results into consistent rows for comparison and export. OpenPyXL Creates the Excel workbook used for wider review. JSON + CSV Keep a timestamped source record that can be reopened later. 04 REVIEW
Review and manage the next action The final group makes the research understandable to a person and records what happens after they make a decision.
Next.js + React Power the local workspace where evidence and live websites meet. SMTP Sends a message only after it has been reviewed and approved. IMAP Checks for replies and connects them to the original business. Local JSON state Stores notes, rejections and follow-up reminders. 03 / WHY THESE CHOICES
Why these choices 01 Evidence before messaging Every recommendation is tied back to observable public pages so a reviewer can challenge the conclusion before contacting anyone.
02 One bad site cannot stop a run Audits are isolated per company. Failures are logged while successful businesses continue into the output artifacts.
03 Human-controlled outreach Scoring prioritises attention; it does not grant permission to send. The review UI keeps drafts, notes, replies and follow-ups visible.
