Scrape BMO Canada ETF Listing

Purpose

Retrieve the Canadian BMO ETF listing and emit ETF cards and their detail-page URLs in a scraper-friendly structure, while distinguishing delayed client rendering, consent overlays, and Akamai blocking from an empty catalogue.

When to Use

Use for any task requiring the current ETF catalogue on BMO's Canadian site, including identifying ETF detail links or diagnosing scraper failures such as detached frames or container memory exhaustion. Route the browser through a Canadian proxy when CA geolocation or an Akamai block is possible.

Workflow

1. With a Canadian proxy configured when needed, navigate directly to https://bmogam.com/ca-en/products/exchange-traded-funds/find-an-etf/ using domcontentloaded; do not visit the homepage or assume the listing is in an iframe.
2. If present, click #onetrust-accept-btn-handler to dismiss the OneTrust cookie banner. If a visible div[role="dialog"] contains a link whose text includes AGREE AND PROCEED, click that attestation link, then wait briefly for the client-rendered listing.
3. Run this self-contained extractor on the current top-level page. Re-query all DOM nodes inside the same evaluation; do not retain frame or element handles across navigation, and avoid serializing the complete DOM or card HTML.

(() => {
  const clean = (s) => (s || "").replace(/\\s+/g, " ").trim();
  const cards = [...document.querySelectorAll(".card.overview")];
  const etfs = cards.map((card, index) => {
    const link =
      card.querySelector(".card-row a[href]") ||
      card.querySelector('a[href*="/ca-en/products/exchange-traded-fund/"]');
    return {
      index,
      name: clean(link?.textContent || card.querySelector("h2,h3,h4")?.textContent),
      href: link?.href || null,
      text: clean(card.textContent),
      htmlClass: card.className,
    };
  });
  const bodyText = document.body?.innerText || "";
  return {
    url: location.href,
    title: document.title,
    readyState: document.readyState,
    cardCount: cards.length,
    etfs,
    hasListing: cards.length > 0,
    hasAkamaiOrBlockText:
      /access denied|akamai|request unsuccessful|temporarily unavailable/i.test(
        bodyText,
      ),
    consentVisible: !!document.querySelector(
      '#onetrust-banner-sdk, div[role="dialog"]',
    ),
    iframeCount: document.querySelectorAll("iframe").length,
    bodyTextLength: bodyText.length,
  };
})();

4. If cardCount is zero but readyState is complete and no block text is present, wait briefly and rerun the extractor once. If block text is present, retry through the Canadian proxy rather than guessing alternate URLs. If the scraper reports a detached frame, discard the stale page/frame reference and repeat the direct top-level navigation; do not loop operations against the detached handle.

Site-Specific Gotchas

• The canonical listing path is /ca-en/products/exchange-traded-funds/find-an-etf/; the listing is client-rendered, so domcontentloaded alone may precede the cards.
• OneTrust cookies and a separate website-attestation dialog can obscure the cards. The cookie control is #onetrust-accept-btn-handler; the attestation action is identified by dialog-link text containing AGREE AND PROCEED.
• ETF cards use .card.overview; the detail link is normally the anchor under .card-row. Detail URLs use the slugged form /ca-en/products/exchange-traded-fund/{etf-slug}/ (singular fund), so extract the actual href rather than constructing a slug.
• Akamai may require a Canadian-origin proxy. A page that loads without cards is not evidence that the catalogue is empty.
• Detached-frame loops are an automation-lifecycle problem, not evidence that the ETF listing is unavailable: use the current top-level document and reacquire DOM nodes after navigation.
• Large whole-page snapshots or outerHTML/full-card serialization can cause container memory pressure. Extract only the required card fields and compact diagnostics; do not use an LLM page scraper for this structured listing.
• The extractor reads only the current page DOM and does not fetch linked fund-facts or PDF resources.

Expected Output

An object containing the listing URL, document title, readyState, cardCount, an etfs array of {index, name, href, text, htmlClass}, and compact flags/diagnostics for listing presence, blocking text, consent UI, iframe count, and body text size.