Fetch DuckDuckGo's server-rendered HTML search results directly from JavaScript and extract structured result links without interacting with the search form. Multiple related queries can be fetched in one page evaluation for fact-checking or triangulation.
Use Cases
Use when the caller needs DuckDuckGo search results and can execute JavaScript in a page on duckduckgo.com. This is useful for validating several phrasings of a historical, scientific, or factual query while avoiding the interactive homepage search view.
Automation Flow
- Navigate to
https://duckduckgo.com/to establish a same-origin browser context. - In one page evaluation, fetch
https://duckduckgo.com/html/?q={query}(URL-encode{query}), parse the response withDOMParser, and run this extractor:
(async () => {
const queries = ["{query}"];
const out = {};
for (const query of queries) {
try {
const response = await fetch(`/html/?q=${encodeURIComponent(query)}`, {
credentials: "same-origin"
});
const html = await response.text();
const doc = new DOMParser().parseFromString(html, "text/html");
out[query] = {
ok: response.ok,
status: response.status,
contentType: response.headers.get("content-type"),
htmlLength: html.length,
resultCount: doc.querySelectorAll(".result").length,
results: [...doc.querySelectorAll(".result")].map((card) => {
const a = card.querySelector(".result__a");
const snippet = card.querySelector(".result__snippet");
if (!a) return null;
const rawHref = a.getAttribute("href") || "";
let href = a.href;
try {
const u = new URL(rawHref, location.origin);
const redirected = u.searchParams.get("uddg");
if (redirected) href = redirected;
} catch (_) {}
return {
title: a.textContent.trim(),
href,
snippet: snippet ? snippet.textContent.trim() : ""
};
}).filter(Boolean)
};
} catch (error) {
out[query] = { error: String(error) };
}
}
return out;
})()For a single search, set queries to one value. For corroboration, replace it with an array of distinct phrasings or terminology variants and fetch all of them in the same evaluation; optionally delay between requests if many queries are needed.
Possible Friction Points
- The scrapeable server-rendered endpoint is
/html/?q={query}, not the interactive homepage search view. - Keep the fetch same-origin and use
credentials: "same-origin"; a page on another origin may be blocked by CORS or behave differently. - Parse the fetched string with
DOMParser; the fetched response is HTML rather than JSON. - Result cards use
.result, primary links use.result__a, and snippets commonly use.result__snippet. Treat these as site-specific selectors that may change. - DuckDuckGo may wrap result destinations in an
uddgquery parameter.URLSearchParams.get("uddg")already percent-decodes it, so use the value as-is; a seconddecodeURIComponentcorrupts destinations that legitimately contain percent-encoded characters and throws on a bare%. - Check
ok,status, andcontentTypebefore relying on extracted results; an HTML error or consent page can otherwise look like a valid search response. - When verifying a factual claim, use several targeted query variants and compare independent result titles, snippets, and destination pages; search-result snippets alone are not proof of the claim.