Scraping React, Vue & Angular SPAs | Web Scraping API React

Introduction

If you've ever pointed a plain HTTP client at a React app and gotten back an empty , you've already hit the core failure mode: modern SPAs ship a minimal HTML shell, then the real page appears only after JavaScript runs, async API requests resolve, and client-side routing settles.

In this guide, you'll build a repeatable playbook for scraping dynamic content without turning your scraper into a flaky mess.

You'll learn when to scrape rendered DOM vs. when to capture network data, how to make Puppeteer waits reliable on client-side navigation, and how Browserless can simplify production scraping with a simple API (/scrape) and more advanced workflows in BrowserQL.

Why SPAs break traditional scrapers

A lot of web scraping needs still start with "fetch HTML and parse." That works on server-rendered web pages where the server returns the content you want.

SPAs flip that around:

The initial response is usually an HTML shell plus script tags
The app boots, hydrates, and then fetches data via XHR/fetch/GraphQL
The UI updates in multiple passes as data arrives and components re-render

So when you just grab the HTML, you're often grabbing the pre-render state. You can confirm this in DevTools with the following quick two-step process:

In the Network tab, open the initial document response and view "Response"
Compare it to the Elements panel after the page finishes loading

If those don't match, you're dealing with JavaScript rendering, not a parsing problem.

How React, Vue, and Angular actually render content

You don't need framework trivia to scrape reliably. You need a mental model that predicts timing and data flow.

Most SPAs follow this sequence:

Initial shell: Server returns minimal HTML, often a single root container
JS boot: Scripts load, attach event handlers, initialize state
**Route resolution: **The client router decides which view to show
Data fetch: The app fires API requests (REST, GraphQL, RPC-ish calls)
Component render: The UI appears, then updates as more data arrives
Background churn: Polling, analytics, lazy-loaded chunks, token refresh

Angular-heavy sites can be extra misleading because lazy loading and guards can delay route completion even when navigation events have fired. In practice, "route changed" does not mean that the content is ready to extract.

The takeaway: your scraper needs an explicit "done" signal. "DOMContentLoaded" and "load" are rarely enough, and "network idle" can lie if the app keeps background requests open.

Pick your strategy: scrape the DOM vs. scrape the data (network/API)

Here's a decision tree that holds up in production.

Hybrid approaches are common:

Render once to establish auth/state (cookies, local storage tokens)
Then switch to capturing API requests for the bulk data collection

This is how a lot of "enterprise scraping" ends up working: use the browser to get into the right state, then treat the app like an API client.

Recommended tooling for SPA scraping: Playwright/Puppeteer and Browserless

For SPAs, you need a real browser engine to run scripts. Puppeteer and Playwright are the usual tools because they can drive headless browsers, evaluate custom JavaScript, intercept requests, and wait on UI signals.

Where teams get stuck is everything around the code:

Browser crashes and memory spikes
Concurrency and queueing
Proxy rotation and session reuse
Debuggability when a scrape gets flaky

Browserless is the layer that smooths the process: hosted browsers and HTTP APIs so you can scale without managing your own browser fleet. The REST endpoints cover common workflows, and BrowserQL gives you a higher-level way to define extraction and structured output.

A simple split that works well:

**Local dev: **direct Puppeteer/Playwright against your machine
Production: Browserless for reliability, scaling, and repeatable request configuration across endpoints like /scrape (the Scrape API) and /unblock

Puppeteer scrape SPA: waiting correctly on React, Vue, and Angular

The question that matters is: when is it done?

In SPAs, navigation events can fire before the content is useful, and route transitions often don't reload the page. You need waits that reflect your extraction target.

goto() waitUntil tradeoffs, and why "network idle" can mislead

Puppeteer's page.goto(url, { waitUntil }) gives you lifecycle waits:

domcontentloaded is quick, but often too early for dynamic content
load waits for subresources, still not a guarantee for SPA data
networkidle2 / networkidle0 is useful, but misleading on apps with polling, websockets, or long-lived requests

A lot of flaky automation comes from over-trusting networkidle2. If an app keeps one analytics request open, you'll hang. If it fires late XHR after becoming "idle," you'll extract too early.

Prefer deterministic waits

A deterministic wait is one where you know what success looks like.

waitForSelector: E.g., the results list exists
waitForFunction: E.g., the list has at least 20 items
waitForEvent: E.g., the app dispatched a ready event

Browserless exposes these same concepts across REST endpoints via shared request configuration, including waitForEvent, waitForFunction, waitForSelector, and waitForTimeout.

Here's a baseline Puppeteer pattern:

If you control the app (internal scraping), the cleanest option is an app-defined ready event. The Browserless request configuration docs call out waitForEvent specifically for custom events, which is ideal when you can add a single document.dispatchEvent(new Event("scrapeReady")) after your UI is stable.

A solid selector strategy for component-driven UIs

Component-driven apps love generating classnames that are meaningless outside the build pipeline. If you anchor extraction on .css-13a7k9, you'll be rewriting scrapers weekly.

Here are some basic rules of thumb to follow:

Prefer stable attributes: data-testid, data-test, data-cy, data-qa
Prefer semantic selectors: role, aria-label, aria-describedby
Use visible text sparingly: It breaks under localization and copy changes
Avoid deep selector chains: They're fragile under layout tweaks

A practical fallback strategy for maintainable data extraction:

Store multiple selectors per field (primary plus backup)
Add selector health checks:
- Count (how many matches)Null rate (how often a field is missing)Diff alerts (selector matched different nodes than yesterday)
Count (how many matches)
Null rate (how often a field is missing)
Diff alerts (selector matched different nodes than yesterday)

Taking these steps turns silent failures into observable changes, which is what you want in production scraping.

Handling common SPA patterns

Infinite scroll, "load more", virtualized lists, modals – this is where scrapers get flaky because the DOM lies.

Infinite scroll: scroll-and-detect loop

"Load more" buttons: click until disabled

Virtualized lists: scroll to force DOM creation

With virtualization, not all items exist in the DOM at once. You can't assume querySelectorAll() returns everything.

One approach is to do the following:

Scroll in increments
Extract visible rows each time
De-dupe by an ID (SKU, URL, data-key)

Modals, tabs, accordions

Scrape the state you can see. If content is behind UI, you need to reveal it.

Click the tab
Wait for a selector inside the panel
Extract, then move on

In Browserless, you can speed up repeated interaction workflows by blocking unnecessary assets (images, fonts) via request configuration like rejectResourceTypes and rejectRequestPattern.

Quick wins with the Browserless Scrape API

If you want a simple API that renders the page and returns structured JSON for selectors, Browserless's /scrape endpoint is designed for that.

The shape is intentionally small: every request includes a url and an elements array containing CSS selectors.

Start with the minimal request:

The response includes per-selector results with fields like text and html.

For SPAs, the reliability comes from adding explicit waits and navigation behavior:

gotoOptions.waitUntil and gotoOptions.timeout
waitForSelector / waitForFunction / waitForEvent for "content ready"

Here's an example, where you wait for a results container before extracting:

If you're doing web scraping for e-commerce stores, this pattern gets you to "first batch of cards loaded" fast, without writing const puppeteer boilerplate for every project.

Capturing XHR/fetch/GraphQL responses instead of the DOM

Rendered DOM is downstream. It changes when designers refactor CSS, rewrap elements, or ship a new component library.

API responses tend to be more stable. When you can capture the underlying JSON, you get:

Fewer brittle selectors
Cleaner structured data
Faster extraction per page
Easier diffs when fields change

Here's a practical workflow to manage this:

Open DevTools on the public website
Trigger the UI state that loads the data
Filter Network by "Fetch/XHR"
Find the request that returns the list/detail JSON
Reproduce it or wait for it during browser automation

In Puppeteer, waiting for a specific response looks like this:

That's your "search data" source of truth. You can still use DOM extraction for presentation-only fields, but your core data collection becomes resilient.

A good timing toolbox uses multiple wait types, each for a different job:

Lifecycle waits are good for initial boot (gotoOptions.waitUntil)
Selector waits are good for when the UI exists
Function waits are good for when data is present
Event waits are best when the app can tell you it's ready

Browserless surfaces these waits as shared request configuration across REST APIs, so you can apply the same mental model whether you're calling /content, /scrape, or /unblock.

One detail that's worth keeping straight:

Lifecycle events are browser events (DOMContentLoaded, load)
waitForEvent is for custom application events, not lifecycle events

So if you want "wait for route settled," your best bet is usually a selector/function wait, unless you can instrument the app.

Authentication and state

Authenticated scraping is where stateless requests fall apart. A stateless request is a one-off call that doesn't remember anything. SPAs often need state: cookies, CSRF tokens, refresh tokens, or local storage values that get rotated.

Here's a practical approach to overcome this challenge:

Use a real login flow when needed (and plan for MFA)
Reuse sessions so you're not logging in on every run
Treat tokens as expiring and build refresh handling

Browserless supports session continuity via reconnects in BrowserQL: You can start a session, then reuse it via a reconnect endpoint, instead of launching a blank browser every time.

On the product side, Browserless also positions cookies and reconnects as a way to keep browsers alive and reuse state across runs, reducing repeated logins and bot checks.

Making extraction maintainable

If you're scraping lists (product cards, search results, directories), you don't want a pile of ad-hoc selectors scattered through your source code. You want schema-like output: arrays of items with named fields.

BrowserQL's mapSelector is built for this. It lets you "map" over a repeated selector and extract fields per item, including nesting another mapSelector when the page structure is hierarchical.

A pattern you can adapt to a React app listing page:

Now your scraped data is already structured data, not strings you have to re-parse later.

Scaling in production

Once you run this on real traffic, the problems stop being "how do I parse HTML" and start being:

Timeouts that cascade
Partial failures you didn't notice
Rate limiting and anti bot measures
Noisy pages that load 200 unnecessary resources

These are some tactics that work:

Block non-essential resources when safe:
- Images, fonts, media, trackersAd networks and analytics endpoints
Images, fonts, media, trackers
Ad networks and analytics endpoints
Set explicit timeouts at every layer
Add retries with backoff for transient failures
Keep partial results when you can, but log what failed

Browserless calls out the knobs you'll use a lot in production scraping:

rejectResourceTypes and rejectRequestPattern to reduce noise
bestAttempt to continue when async waits fail, so you can still inspect partial output
Request-level timeouts (including a timeout query parameter in examples) plus gotoOptions.timeout for navigation

That's the difference between a scraper that dies on one bad page and a system that can make progress and tell you what broke.

Avoiding blocks

Some sites will block automated traffic with fingerprinting, proxies, and CAPTCHA reality checks. You're not invisible; you're just trying to look less obviously broken than a default headless browser with no session history.

A realistic plan usually includes:

Session hygiene: Reuse cookies, avoid "fresh browser every request"
Slower interactions: Don't click faster than a human can render
**Proxy rotation **when you need it, not by default
Rate limiting: Keep request volume sustainable

Browserless provides an Unblock API positioned for bot detection and CAPTCHA scenarios, with optional proxy support and the ability to return cookies or a browserWSEndpoint so you can continue automation in Puppeteer or Playwright.

If you need a clean session, to take over from, here's a good flow:

Call /unblock (optionally with &proxy=residential)
Receive cookies and/or a websocket endpoint
Continue your custom automation

Browserless also recommends BrowserQL for advanced detection and CAPTCHA solving via a solve mutation, with built-in stealth capabilities.

You can also find practical tactics like rotating user agents and storing cookies to make returning sessions look more natural in the Browserless 2025 Puppeteer guide.

Debugging playbook for flaky SPA scrapes

When scraping goes wrong, you want artifacts, not guesses.

A checklist that saves time:

Save a screenshot on failure (or on every Nth run)
Save the rendered HTML for the failed URL
Log selector counts (0 matches is a signal)
Record failed URLs and keep a small reproduction set in CI
Capture key network requests (status codes, response bodies when safe)

If you're using Browserless REST APIs, bestAttempt: true can be a useful debugging mode because you still get partial output even when a wait fails, which lets you inspect what was actually rendered.

End-to-end recipe examples: React vs. Vue vs. Angular

These aren't"framework differences as much as places where timing and state can bite you.

Recipe 1: React listing page (DOM extraction, selector readiness)

**Readiness signal: **waitForSelector on a stable results container
**Strategy: **DOM extraction for card fields
Brittle point: Classnames change
Fallback: Use data-testid and multiple selectors per field

If you're doing it via /scrape, keep it boring: explicit selector waits, then extract.

Recipe 2: Vue search (network capture, structured JSON)

**Readiness signal: **waitForResponse on a /search JSON endpoint
Strategy: Parse API response, then optionally enrich from DOM
Brittle point: Endpoint signature changes or adds auth headers
**Fallback: **Render once, capture the request headers, then replay API calls

This is where doing API requests first pays off. Your scraped data is already JSON.

Recipe 3: Angular dashboard (authenticated, session reuse)

Readiness signal: Custom event (best) or waitForFunction that checks a global store state
**Strategy: **Establish auth once, then reuse the session
**Brittle point: **Token refresh mid-run
Fallback: Reconnect a live session or re-inject cookies

If you're doing this repeatedly, session reuse and cookies matter more than any single wait condition.

Legal, ethical, and ToS considerations for SPA scraping

This isn't legal advice, but you do want a sustainable posture:

Read and respect each site's Terms of Service and robots guidance where applicable
Avoid collecting sensitive data or anything you can't justify as a business need
Use rate limiting and backoff to avoid harming the site
Identify yourself where appropriate (especially for B2B data collection)
Keep a paper trail: what you collect, why, retention, and access controls

If your scraper needs stealth tricks to function at all, that's a signal to reassess the approach or get permission.

Conclusion

Scraping React, Vue, and Angular SPAs stops being painful once you treat rendering and readiness as first-class problems. Decide early whether you want DOM extraction or network capture, make your waits deterministic, and build your scraper like you'd build any production system: With timeouts, retries, observability, and a plan for bot detection.

If you want a production-grade way to render dynamic content and return structured data without running your own browser infrastructure, start with the Browserless Scrape API (/scrape) and shared request configuration for reliable waits. For more complex workflows, BrowserQL gives you structured extraction patterns like mapSelector, plus session management via reconnects.

Scraping React, Vue, and SPAs FAQs

Should you always render SPAs with a headless browser?

No, you shouldn't always render SPAs with a headless browser. If you can reliably capture stable API calls, you'll get faster, cleaner data extraction than parsing a rendered page.

What's the safest waitUntil for SPAs?

There isn't a safest waitUntil for SPAs. Use a lifecycle wait for boot, then a deterministic selector/function wait for "content is ready."

How do you keep selector-based scrapes from breaking weekly?

Use stable hooks (data-testid), keep fallbacks, and run selector health checks so UI changes show up as alerts, not silent nulls.

Do you need proxy rotation for every scrape?

You don't usually need proxy rotation for every scrape. Start with sane rate limiting and session reuse. Add proxies when blocks actually show up.

What's the fastest way to get structured output without writing a lot of Puppeteer code?

Use a scraping API that returns structured JSON from selectors, or use a mapping approach like BrowserQL mapSelector for repeated UI.

What are the best tools for scraping single-page apps?

For single-page apps, the best tool is usually a real browser automation library (Playwright or Puppeteer) because it can execute JavaScript rendering, follow client-side routing, and wait on UI signals instead of guessing based on raw HTML. That's what turns an empty shell into actual web pages you can extract data from with selectors or by observing API requests.

Once you're moving beyond a one-off script, the tooling that tends to win is "Playwright or Puppeteer for control" plus an infrastructure layer that makes production reliable. Browserless gives you that as a web scraping api: /scrape is the simple API for "render + selector output," shared request configuration lets you standardize waits/timeouts across endpoints, and BrowserQL helps you return structured data (especially list pages) without writing a ton of glue code.

How to scrape JavaScript-heavy SPA pages

Treat it as a timing problem, not a parsing problem: load the page in a browser context, then wait for a deterministic readiness signal before you extract.

In practice, that means doing an initial navigation wait (often domcontentloaded), then waiting for a selector/function/event that proves the dynamic content is present (for example, "results list exists" or "at least 20 cards rendered"), because SPA navigation can "finish" long before the UI is useful.

From there, pick the extraction path: if the app makes stable JSON API calls, capture those responses and parse the data directly; if the data only exists after interactions, extract from the rendered DOM. With Browserless, you can do both styles while keeping the config consistent: add gotoOptions plus waitForSelector/waitForFunction for reliable waits, block noisy resources with rejectResourceTypes, and use bestAttempt when you want partial scraped data instead of a hard failure while debugging.

Which scraper should I use for React and Vue sites?

If your React app or Vue site calls clean JSON endpoints, you'll usually get the most durable setup by scraping the data (network/API) instead of the DOM – the UI can change weekly, but the underlying API requests often stay stable. You still use a browser when you need state (cookies, auth, local storage), but your data extraction comes from responses rather than brittle CSS selectors.

When you do need rendered DOM extraction, Browserless is a strong default for production: use /scrape for get a structured JSON from specified selectors, and switch to BrowserQL when you want maintainable, list-shaped output via mapSelector (think product cards, search results, backend directory listings).

If you're hitting anti-bot measures, Browserless's /unblock flow can get you through bot detection and return cookies or a browserWSEndpoint – and it's designed to pair with proxy rotation (for example, residential) when that's actually necessary.

Scraping React, Vue and Angular SPAs: An In-Depth Technical Guide