Extract GitHub Trending Repositories

Site github.comTask extract-github-trending-repositoriesVersion v17Updated Sep 16, 2026Category browser-automation

Retrieve GitHub Trending repositories or topic-taxonomy listings with filters, pagination checks, ranking controls, repository metadata, missing-language values, and exact empty-state diagnostics. This skill was captured from a live agent session on github.com and is published here as a reusable recipe for agents.

NoteSelectors and URL schemes drift as sites change. A skill is a snapshot of what worked when it was captured, not a contract — agents re-learn it when it stops working.

Retrieve GitHub Trending repositories or GitHub topic-taxonomy listings. Return repository identity, descriptions, primary languages including null values, stars, forks, page-relative rank, visible ranking controls, retained filters, pagination state, duplicate-page status, and exact empty-state or pagination-failure diagnostics.

Use Cases

  • Collect current daily, weekly, or monthly GitHub Trending repositories.
  • Filter Trending results by programming-language or spoken-language code.
  • Verify whether a later Trending page contains repositories different from page 1.
  • Extract unique repositories from a GitHub topic taxonomy page.
  • Detect topic sort controls and follow topic-list pagination.

Automation Flow

  1. For Trending, build https://github.com/trending/{language}?since={since}&page={page}; append spoken_language_code={spoken_language_code} when requested. For an unfiltered language view, use https://github.com/trending?since={since}&page={page}. Preserve the supplied language-path casing and all query parameters when reporting the final URL.
  2. For a topic taxonomy, navigate directly to https://github.com/topics/{topic}?page={page}. If pagination is exposed only through Load more, read form.js-ajax-pagination input[name="page"]; direct topic pages 1 through 4 have returned distinct 20-card listings, and page 3 has exposed next_page=4.
  3. In one BQL call per loaded page, navigate directly, wait for DOM content, and evaluate the matching extractor. If the first navigation exposes about:blank, repeat the complete URL before evaluating.
  4. For Trending, evaluate:
(()=>{const clean=s=>(s||'').replace(/\s+/g,' ').trim();const metric=value=>{const m=clean(value).match(/[\d,.]+\s*[KMB]?/i);if(!m)return null;const raw=m[0].replace(/,/g,'').replace(/\s/g,'');const n=parseFloat(raw);if(Number.isNaN(n))return null;if(/m$/i.test(raw))return Math.round(n*1e6);if(/k$/i.test(raw))return Math.round(n*1e3);if(/b$/i.test(raw))return Math.round(n*1e9);return Math.round(n)};const url=location.href;const qs=new URL(url).searchParams;const requestedPage=Number(qs.get('page')||'1');const rows=[...document.querySelectorAll('article.Box-row')];const repositories=rows.map((row,index)=>{const link=row.querySelector('h2 a[href^="/"]');const href=link?.getAttribute('href')||'';const parts=href.split('/').filter(Boolean);const periodText=[...row.querySelectorAll('span')].map(e=>clean(e.textContent)).find(t=>/stars?\s+(today|this week|this month)/i.test(t))||null;const periodMatch=periodText?.match(/([\d,.]+(?:\s*[KMB])?)\s+stars?\s+(today|this week|this month)/i)||null;const starLink=row.querySelector('a[href$="/stargazers"]');const forkLink=row.querySelector('a[href$="/forks"]');return{rank:index+1,owner:parts[0]||null,repository:parts[1]||null,url:href?new URL(href,location.origin).href:null,description:clean(row.querySelector('p')?.textContent)||null,language:clean(row.querySelector('[itemprop="programmingLanguage"]')?.textContent)||null,language_color:row.querySelector('.repo-language-color')?.getAttribute('style')?.match(/#[0-9A-Fa-f]{6}/)?.[0]||null,stars_total:metric(starLink?.textContent),forks_total:metric(forkLink?.textContent),stars_period:periodMatch?metric(periodMatch[1]):null,period_label:periodMatch?`stars ${periodMatch[2].toLowerCase()}`:null,built_by:[...row.querySelectorAll('img[alt^="@"]').values()].map(img=>clean(img.alt.slice(1))).filter(Boolean)}});const emptyState=[...document.querySelectorAll('h1,h2,h3,p,[role="status"]')].map(e=>clean(e.textContent)).find(e=>/there\s+(?:aren't|are not)\s+any trending repositories|don['’]?t have any trending repositories|no trending repositories(?: found)?|no repositories found|nothing trending/i.test(e))||null;const diagnostics=[...document.querySelectorAll('[role="alert"],.flash,.js-flash-alert')].map(e=>clean(e.textContent)).filter(Boolean).slice(0,20);const body=clean(document.body?.innerText);const blocked=!repositories.length&&!emptyState&&/sign in|captcha|rate limit|access denied|not found|error/i.test(body);const filters={language:location.pathname.match(/^\/trending\/([^/?]+)/)?.[1]||null,since:qs.get('since'),page:qs.get('page'),spoken_language_code:qs.get('spoken_language_code')};return{ok:!blocked&&(repositories.length>0||Boolean(emptyState)),url,title:document.title,requested_page:requestedPage,count:repositories.length,repositories,empty_state:emptyState,filters,filter_retained:Boolean(filters.since&&filters.page&&(filters.language||location.pathname==='/trending')),diagnostics};})()
  1. To test pagination or duplicate-page status, run the same extractor on page 1 and the requested page using identical filters. Compare repositories[].url; if both pages have zero rows and the same explicit empty_state, return pagination_available:false rather than duplicate repositories. If a later nonempty page repeats page 1 URLs, return pagination_failure:true, page_ignored:true, and preserve the requested page result.
  2. For a topic page, evaluate:
(async()=>{const clean=s=>(s||'').replace(/\s+/g,' ').trim();const metric=s=>{const m=clean(s).match(/[\d,.]+\s*[KMB]?/i);if(!m)return null;const raw=m[0].replace(/,/g,'').replace(/\s/g,'');const n=parseFloat(raw);if(Number.isNaN(n))return null;return /m$/i.test(raw)?Math.round(n*1e6):/k$/i.test(raw)?Math.round(n*1e3):/b$/i.test(raw)?Math.round(n*1e9):Math.round(n)};const page=Number(new URL(location.href).searchParams.get('page')||'1');const cards=[...document.querySelectorAll('article')].filter(a=>a.querySelector('h3 a[href^="/"]'));const sortControls=[...document.querySelectorAll('summary[role="button"],button,a,[role="tab"],select option')].map(x=>clean(x.textContent)).filter(Boolean).filter(t=>/most stars|recently updated|most forks|sort|ranking/i.test(t));const topicSortOrRanking=sortControls[0]||null;const topicSortOptions=[...new Set(sortControls)];const urls=cards.map(a=>{const x=[...a.querySelectorAll('h3 a[href^="/"]')].find(x=>/^\/[\w.-]+\/[\w.-]+$/.test(x.getAttribute('href')||''));return x?'https://github.com'+x.getAttribute('href'):null}).filter(Boolean);const forks=await Promise.all(urls.map(async url=>{try{const d=new DOMParser().parseFromString(await fetch(url,{credentials:'same-origin'}).then(r=>r.text()),'text/html');return metric(d.querySelector('a[href$="/forks"]')?.textContent)}catch(e){return null}}));const seen=new Set();const repos=cards.map((a,i)=>{const x=[...a.querySelectorAll('h3 a[href^="/"]')].find(x=>/^\/[\w.-]+\/[\w.-]+$/.test(x.getAttribute('href')||''));const href=x?.getAttribute('href')||'';const p=href.split('/').filter(Boolean);const url=href?'https://github.com'+href:null;if(!url||seen.has(url))return null;seen.add(url);const language=clean(a.querySelector('[itemprop="programmingLanguage"]')?.textContent)||null;return{rank:i+1,owner:p[0]||null,repo:p[1]||null,url,description:clean(a.querySelector('p')?.textContent)||null,language,missing_language:language===null,stars:metric(a.querySelector('span.js-social-count')?.textContent||a.querySelector('a[href$="/stargazers"]')?.textContent||a.querySelector('[aria-label*="starred this repository"]')?.getAttribute('title')),forks:forks[i]??null}}).filter(Boolean);const nextPage=document.querySelector('form.js-ajax-pagination input[name="page"]')?.value||null;const loadMore=clean(document.querySelector('button.ajax-pagination-btn')?.textContent)||null;return{title:document.title,url:location.href,topic:location.pathname.split('/').filter(Boolean)[1]||null,page,count:repos.length,unique_repository_cards:repos.length,missing_language_count:repos.filter(r=>r.missing_language).length,topic_sort_or_ranking:topicSortOrRanking,topic_sort_options:topicSortOptions,repos,next_page:nextPage,load_more:loadMore,pagination:{requested_page:page,next_page:nextPage,load_more:loadMore}}})()
  1. For topic collections, load pages 1 through the requested maximum, compare repos[].url, discard duplicates, retain page-relative rank, preserve requested and loaded pages, and return at most limit unique repositories. Mark a page page_ignored:true and record it in pagination diagnostics when all URLs were already seen.
  2. Return only the first requested rows after stitching topic pages. For filtered Trending empty states, use count:0 rather than treating the result as navigation failure.

Params

ParamWhat it doesExample value
sinceSelects the Trending perioddaily, weekly, or monthly
languageRestricts Trending by programming-language path; preserve supplied casingCOBOL
spoken_language_codeOptionally filters Trending by spoken-language codezh
pageRequests a Trending or topic listing page3
validate_paginationLoads page 1 and the requested Trending page to compare empty states or repository URLstrue
topicSelects a GitHub topic taxonomy pagelinux
limitMaximum repositories returned after page stitching30
max_pageLast topic page to load and stitch4

Possible Friction Points

TriggerAction
Initial direct navigation exposes about:blankRepeat the complete direct URL, wait for domcontentloaded, then evaluate.
A filtered Trending page returns zero rows and GitHub's empty-state messageReturn count:0, the exact extracted empty_state, final URL, and retained filters.
Adjacent filtered page validation is requestedNavigate directly to the adjacent page value with identical filters and run the count/empty-state evaluator.
Pages 1 and the requested filtered page return the same explicit empty stateReport pagination_available:false, preserve both zero-count page results and all filters, and do not infer a duplicate repository page.
A later filtered page repeats page 1 URLsSet pagination_failure:true and page_ignored:true; do not duplicate repositories.
A Trending page has no pagination navigation in the rendered documentUse the direct page URL, compare repository URLs with page 1, and report it unavailable if unchanged.
Extraction runs while the target page is in a background tabActivate the target tab before evaluating the DOM.
Topic listing exposes Load more instead of a normal next linkRead form.js-ajax-pagination input[name="page"], then navigate directly to /topics/{topic}?page={page}.
Topic page exposes a visible Sort controlReturn its visible label and all matching visible options; page 3 has shown Sort: Most stars.
Topic cards expose stars in abbreviated span.js-social-count elementsRead that selector before falling back to stargazer links or aria labels.
Topic cards omit fork countsFetch each extracted repository URL same-origin and read a[href$="/forks"]; return null when unavailable.
Topic cards omit a primary languageReturn language:null, missing_language:true, and include it in missing_language_count; do not infer it.
No topic sort control or ranking label is visibleReturn topic_sort_or_ranking:null and preserve positional card rank.
Final URL or extracted filters omit a requested parameterReload the complete direct URL and verify the resulting URL before extraction.
article.Box-row returns no Trending repositories after navigationWait for DOM content and rerun the evaluator on the same URL; return count:0 and the extracted empty state.
GitHub changes Trending row or topic-card selectorsInspect one rendered row and update the evaluator's article, heading, language, metric, or pagination selectors.
Built-by extraction returns a syntax errorUse img[alt^="@"] and map each image's alt without the leading @.
Topic page 2 exposes next_page=3 and a Load more controlUse the returned next-page value to construct the next direct topic URL; page 3 has returned a distinct listing and next_page=4 has been observed.
Topic page 4 is requestedNavigate directly to ?page=4; it has returned a distinct 20-card listing.
Topic URL is beyond the available range, such as ?page=9999Treat GitHub's 404 Page not found as an out-of-range pagination diagnostic, not as an empty repository listing.
Topic page page=0 is requestedTreat it as a normal listing response; GitHub has returned topic cards rather than an invalid-page error.
Topic language-filter menu does not open through selector clicksSkip the menu and use the direct topic-page URL and extractor.
Spoken-language-filtered pages 1 and 5 both show GitHub's explicit empty statePreserve since, page, and spoken_language_code; report unavailable pagination rather than treating page 5 as a duplicate listing.
A requested out-of-range Trending page returns GitHub's Page not found responseRecord the 404 as an out-of-range pagination diagnostic and do not classify it as an empty repository result.
GitHub preserves an uppercase language path such as /trending/PythonKeep the exact final URL and report the retained path casing instead of normalizing it.
Trending page=0 is acceptedTreat it as a valid requested page, preserve all filters, and report its extracted result and empty state normally.

Call it

GET https://production-sfo.browserless.io/skills?token=TOKEN-HERE&domain=github.com&task=extract-github-trending-repositories