本文へ移動
cccskills
無料GitHub で公開

arcgis-to-portaljs

Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute.

インストール方法を見る

含まれるファイル(3)

  • SKILL.md7.8 KB
  • references/reference.md5.4 KB
  • references/sync-and-cutover.md14.6 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

ArcGIS Hub → PortalJS

Overview

Migrate an entire ArcGIS Hub open-data site into a PortalJS Arc portal in one pass. Every Hub site is machine-readable — a DCAT-US catalog at /data.json, with every dataset backed by an ArcGIS REST FeatureService — so migration is a harvest → export → convert → publish → verify pipeline that runs almost fully automated on the operator's machine, no server-side compute. The tooling is the reusable arcgis-to-portaljs migrator: input is one Hub URL, output is a ready-to-deploy PortalJS catalog plus a parity report.

The skill is an orchestrator: it reuses the DCAT-US harvest from portaljs-migrate, the ogr2ogr/tippecanoe/duckdb dual-tier conversion from portaljs-add-geo, and the bulk Git-LFS → R2 push from portaljs-migrate. Its novel parts are the FeatureService REST export loop (paged features, not just a link) and the source-vs-derived parity report.

Prerequisites

  • A scaffolded PortalJS portal whose template ships components/MapPreview.tsx and components/GeoQuery.tsx (PR #1647 or later). Run portaljs-new-portal first if none.
  • Native CLIs: GDAL (ogr2ogr, ogrinfo), tippecanoe, duckdb (with spatial), and jq. macOS: brew install gdal tippecanoe duckdb jq; Debian/Ubuntu: apt-get install gdal-bin duckdb jq plus tippecanoe (apt or build from source); Windows via WSL. The skill hard-stops with the install hint if any is missing.
  • Arc credentials for the Git-LFS → R2 push (the token portaljs-deploy resolves), or an OSS self-hosted Giftless.

Instructions

The canonical, full step-by-step workflow is .claude/commands/arcgis-to-portaljs.md — the single source of truth. Read and follow it when executing. Summary:

  1. Gather input — Hub URL, portal directory, project slug, optional flags (--limit, --only, --dry-run, --namespace-mode). Interview if missing; never dead-end.
  2. Check native tools (ogr2ogr, tippecanoe, duckdb + spatial, jq). Any missing → print the per-OS install and stop.
  3. Validate the portal directory and confirm the geo showcase components exist.
  4. Harvest the Hub /data.json (reuse the portaljs-migrate DCAT-US map) and classify each item: vector (FeatureService), table, or non-data (web map / 3D / imagery → skipped). Under --namespace-mode owner, resolve namespaces through a publisher-normalization table with title-prefix fallback for broken {{source}} publishers (multi-publisher Hubs ship dirty publisher labels). Dedup near-duplicate hosted-view layers — but only after a mandatory live record-count check on BOTH twins: equal ⇒ dedup (keep the source layer, log the pair); different ⇒ keep both as distinct datasets. Consolidate per-year dataset series into one year-partitioned Parquet with legacy per-year view entries. Enrich from the AGOL item: sanitized metadata (license/description/dates), cleaned display title (cleanTitle — raw title still drives the slug), category (item categories → meaningful theme → keyword mapping), and a thumbnail snapshot into public/thumbnails/.
  5. Export each vector layer through the ArcGIS REST query API with resultOffset paging (f=geojson, outSR=4326); fall back to keyset paging on transfer limits; accept a customer File Geodatabase dump for very large layers.
  6. Convert each layer to the dual tier via the portaljs-add-geo recipe (PMTiles + GeoParquet); tabular items to Parquet. Preserve the native-CRS original.
  7. Publish — bulk Git-LFS track + one push to R2 through Giftless, then append dual-tier datasets.json entries (upsert on (namespace, slug)).
  8. Write arcgis-parity-report.md — record count, extent, attribute schema, and geometry validity, source vs derived, per dataset, plus the migrated/skipped/failed accounting.
  9. Report the inventory, migrated datasets, R2 push, and parity summary.

Output

  • Created: data/<namespace>/<slug>.pmtiles, .parquet, and the original per vector dataset (all LFS-tracked → R2); Parquet + original per table; arcgis-parity-report.md.
  • Modified: datasets.json (one dual-tier entry per vector dataset, one resource entry per table); .gitattributes (LFS tracking).
  • Verified: the parity report compares each derived artifact to the live FeatureService.
  • Result: /@<namespace>/<slug> renders <MapPreview> + <GeoQuery> for each vector dataset with no page edits; the catalog lists everything migrated.

Error Handling

SymptomCauseFix
MISSING_INPUTNo Hub URL providedPass the site root (e.g. https://hub-lewisville.opendata.arcgis.com) and retry.
MISSING_TOOLSogr2ogr/tippecanoe/duckdb/jq (or duckdb spatial) absentPrint the per-OS install line and stop; re-run after installing.
NOT_A_PORTALTarget dir has no datasets.json / geo componentsRun portaljs-new-portal first, then re-run.
HARVEST_FAILED/data.json unreachable or not DCAT-USConfirm the site is an ArcGIS Hub and the feed loads in a browser.
EXPORT_FAILEDOne FeatureService layer errored or hit a hard transfer capLogged and skipped; try keyset paging or a customer FGDB dump for that layer.
LFS_PUSH_FAILEDMissing/expired Arc token or unset lfs.urlRe-mint the JWT (see portaljs-deploy); confirm git config lfs.url.

Examples

Example 1 — Migrate a City Hub (Lewisville)

/arcgis-to-portaljs https://hub-lewisville.opendata.arcgis.com slug=lewisville

Example 2 — Dry-run inventory + plan only (no writes)

/arcgis-to-portaljs https://streamwaterdata.co.uk --dry-run

Example 3 — Migrate a subset, one namespace per publisher

/arcgis-to-portaljs https://streamwaterdata.co.uk --only sewer-catchments,water-boundaries --namespace-mode owner

Resources

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

Add a chart (line, bar, area, pie, or scatter) to a dataset's showcase in a PortalJS portal. Installs recharts, writes a reusable Chart component, and renders it in the showcase Views section. Use when visualizing a dataset already registered in datasets.json.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough. Use when registering a new dataset in a scaffolded portal.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

Make a PortalJS portal harvestable by national/EU/US open-data portals — emit standards-compliant DCAT catalog feeds (DCAT 2/3, DCAT-AP, DCAT-US, national profiles) in JSON-LD, Turtle, and RDF/XML at build, with autodiscovery and per-profile conformance checking. Use when a portal needs to be harvested by data.europa.eu, data.gov, or a national open-data catalog.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

Auto-ingest a geospatial file (GeoJSON, Shapefile, GeoPackage, KML/KMZ, FlatGeobuf, CSV-with-geometry) into a PortalJS portal on the user's own machine, with no server. Normalizes CRS to EPSG:4326, derives a PMTiles render tier and a GeoParquet query tier, pushes all three artifacts to Cloudflare R2 via Git LFS, and appends one dual-tier datasets.json entry the showcase auto-renders. Use when the source is a vector geo format that needs a map or spatial-query view.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

Render a GeoJSON dataset on an interactive Leaflet map in the Views section of a dataset's showcase. Installs react-leaflet and a Map component, then renders the map for the chosen dataset. Use when a dataset's data is geographic and a map view is needed alongside the showcase's default metadata and download.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

Add another file (resource) to an EXISTING dataset in a PortalJS portal — a data dictionary, methodology, or an additional data file. Turns a single-file dataset into a multi-resource one; the showcase renders a section per resource. Use when a dataset needs a second file, such as a data dictionary, methodology doc, or an additional period's data.

日本語の概要は準備中です。原文の説明を表示しています。

datopian/portaljs2,3622026年10月8日 更新

datopian のスキルをすべて見る

このスキルの問題を報告する