Enumerate CKAN Datasets by Theme

Purpose

Enumerate all data.ontario.ca CKAN packages relevant to a thematic request such as environmental enforcement, compliance, spills, inspections, orders, pollution, or penalties. Use the CKAN API rather than the human-facing catalog and deduplicate results returned by overlapping keyword searches.

When to Use

Use when the caller needs a catalog-wide or organization-scoped inventory of datasets matching several related concepts, rather than metadata for one known package slug. This is especially useful when relevant dataset titles do not all contain the same exact phrase.

Workflow

1. Construct a direct CKAN search URL using a Solr-style OR expression and pagination: https://data.ontario.ca/api/3/action/package_search?q={url-encoded-keywords}&rows=100&start={offset} For environmental enforcement/compliance/spills, use a generalized expression such as environmental OR enforcement OR compliance OR spill OR spills OR inspection OR inspections OR pollution OR penalty OR penalties OR orders, but retain any additional caller-supplied terms.
2. If an organization scope is required, first navigate to: https://data.ontario.ca/api/3/action/organization_list?all_fields=true&limit=200 and resolve the organization by its returned name or title; then add fq=organization:{organization-name} to package_search. Do not guess the organization slug.
3. In one browser call per page, navigate directly to the search URL with waitUntil: "domcontentloaded", then run this evaluate() extractor on the loaded API response. Continue with start={offset+100} while start + returnedCount < count; merge pages by name or id.

   (() => {
     const text = document.body.innerText;
     try {
       const payload = JSON.parse(text);
       const result = payload.result || {};
       const rows = Array.isArray(result.results) ? result.results : [];
       return {
         success: payload.success === true,
         count: Number.isFinite(result.count) ? result.count : rows.length,
         start: Number.isFinite(result.start) ? result.start : 0,
         returnedCount: rows.length,
         nextStart: rows.length
           ? (Number.isFinite(result.start) ? result.start : 0) + rows.length
           : null,
         datasets: rows.map((d) => ({
           id: d.id ?? null,
           name: d.name ?? null,
           title: d.title ?? null,
           notes: d.notes ?? null,
           organization: d.organization
             ? {
                 id: d.organization.id ?? null,
                 name: d.organization.name ?? null,
                 title: d.organization.title ?? null,
               }
             : null,
           tags: Array.isArray(d.tags)
             ? d.tags.map((t) => ({
                 name: t.name ?? null,
                 display_name: t.display_name ?? null,
               }))
             : [],
           metadata_modified: d.metadata_modified ?? null,
           update_frequency: d.update_frequency ?? null,
           access_level: d.access_level ?? null,
           url:
             d.url ?? (d.name ? "https://data.ontario.ca/dataset/" + d.name : null),
         })),
       };
     } catch (e) {
       return {
         success: false,
         count: 0,
         start: 0,
         returnedCount: 0,
         nextStart: null,
         datasets: [],
         error: "NOT_JSON",
       };
     }
   })();

4. For maximum recall, run separate searches for important individual terms when the combined OR query appears incomplete, union the returned datasets, and deduplicate by CKAN id (falling back to name). Preserve the search term(s) used as an optional caller-side matchedBy annotation.
5. If a selected dataset needs validation or resource details, pass its returned name to the existing verify-ckan-package-metadata skill and call package_show directly.

Site-Specific Gotchas

• Ontario CKAN search is available at /api/3/action/package_search; q searches package metadata and rows/start control pagination.
• Search results are relevance-ranked and overlapping keyword searches can return duplicates; use a union of queries and deduplicate by opaque package id or slug name.
• A single keyword search is not a reliable definition of “all” thematic datasets: relevant packages may use terms such as orders, directors' orders, occurrences, approvals, inspections, or penalties instead of spills or compliance.
• Organization identifiers are opaque CKAN names. Resolve them from organization_list before using fq=organization:{name}; similar organization titles may have different slugs.
• The API response is JSON rendered in document.body.innerText, with packages under result.results and the total under result.count.
• A successful response may contain zero results; treat success: false and malformed JSON as API errors rather than an empty catalog.
• Dataset pages and resource URLs are not needed for enumeration; use the returned package slug or the separate package_show endpoint only when resource inventory is requested.

Expected Output

A deduplicated array of matching dataset records, each containing its CKAN id, name, title, notes, organization, tags, metadata timestamp, update frequency, access level, and dataset URL, together with the API success status and total/pagination information. For incomplete or failed pages, include the returned error instead of silently treating them as complete.