GeoAI Object Detection with ArcGIS

What problem does this solve?

Imagery has stopped being the constraint. Entities in the UAE hold high-resolution satellite coverage, run drone flights over corridors and sites, and have LiDAR from mobile mapping or aerial surveys. What has not scaled is turning that imagery into features. Counting palm trees, outlining building footprints, finding solar panels, locating manholes and street furniture, mapping informal storage yards — these are still done by analysts drawing polygons, and the backlog grows faster than the team. By the time a layer is complete it is describing the city as it was two years ago.

Deep learning is the obvious answer and also the most commonly mis-scoped project in GIS. The demonstrations look effortless: a pretrained model, a few clicks in ArcGIS Pro, buildings appearing across a scene. Real projects run into the parts the demonstration skipped. Somebody has to label several hundred to several thousand examples of the target class, consistently, in the same imagery conditions as the production data. Somebody has to decide what counts as an object — is a covered car park a building? Somebody has to buy or borrow a GPU. And somebody has to accept that a detection is a suggestion until a human confirms it.

The third problem is what happens after inference. A shapefile of ninety thousand detections with confidence scores is not an asset register. Without deduplication against existing features, a review workflow, an owner for rejected detections and a route into the maintenance or planning system, the output becomes another layer nobody trusts. Models also drift: a model trained on winter imagery at 10 cm resolution behaves differently on summer imagery at 30 cm, and nobody notices until the counts look odd.

How the solution works

ArcGIS provides the whole chain in one place. Training samples are labelled in ArcGIS Pro against the same imagery the model will see in production, exported as image chips in a standard format, and used to train an object-detection, classification or segmentation model with the arcgis.learn module — either from scratch, by fine-tuning a pretrained backbone, or by starting from an Esri pretrained model from ArcGIS Living Atlas and adapting it to local conditions. Trained models are packaged as a deep learning package (.dlpk) and run with the Image Analyst tools: Detect Objects Using Deep Learning, Classify Pixels Using Deep Learning and Classify Objects Using Deep Learning. Point-cloud classification for LiDAR follows the same pattern with the 3D Analyst tools.

Because the tools are standard geoprocessing tools, inference fits the automation you already have: a Python toolbox or scheduled ArcPy job runs it over a tile index on a GPU workstation, or ArcGIS Image Server distributes it across raster analytics nodes for city-scale coverage. Swedish Technology builds the parts around the model that decide whether the project succeeds — a labelling standard and quality checks on the training data, a held-out test area that the model never sees during training, a written accuracy report with precision, recall and IoU per class, post-processing to deduplicate and snap detections to your schema, a review application where an inspector accepts or corrects each detection, and a feedback loop that turns corrections into the next training round. Segmentation and change detection use the same pipeline: instead of drawing boxes the model produces per-pixel classes, and comparing two dates produces a change layer that flags new construction, encroachment or vegetation loss for review.

1. 1
   Input Define the target classes precisely and collect the source data: satellite or aerial imagery, drone orthomosaics, oblique or vehicle-mounted camera frames, or LiDAR point clouds, with their resolution, capture dates and coordinate system recorded.
2. 2
   Capture Label training samples in ArcGIS Pro against representative areas — different densities, lighting, seasons and sensors — and export them as image chips with the Export Training Data For Deep Learning tool. A separate area is held back untouched as the test set.
3. 3
   Processing Train or fine-tune the model with arcgis.learn on a GPU, tune the learning rate and augmentation, and measure precision, recall and IoU per class on the held-out area rather than on the training data.
4. 4
   Integration Package the model as a .dlpk and run inference from ArcGIS Pro, a scheduled ArcPy job or ArcGIS Image Server raster analytics; post-process detections — non-maximum suppression, confidence threshold, minimum size, deduplication against existing features and snapping to the target schema.
5. 5
   Action Detections land in a review feature class where an inspector accepts, edits or rejects each one; accepted features move into the production layer and can raise a work order, a planning case or an inspection task in the downstream system.
6. 6
   Reporting Counts, area, change between dates and model performance are reported to dashboards; rejected detections and corrections are fed back into the next training round, and each production run records the model version used.

Reference architecture

Treat the model as one replaceable component inside a data pipeline that you can audit, rerun and improve.

Deployment options: Training and inference run on-premise on GPU hardware (a single workstation-class GPU for pilots, server GPUs for production) or in a UAE-region private cloud. No imagery has to leave the environment; models, weights and the Python environment can be installed from an internal mirror for air-gapped sites.

Key capabilities

Integrations

Detections only create value when they reach the system that acts on them, so the pipeline ends in your geodatabase and your operational systems, not in an export folder.

Industry use cases

Implementation approach

1. 1
   Scoping (1 week) Define the classes precisely with examples and edge cases, confirm the imagery available and its resolution, and agree what accuracy would make the output useful in the downstream process.
2. 2
   Data assessment Review imagery quality, coverage, capture dates and existing digitised features; identify whether existing features can seed the training set and how much labelling is genuinely needed.
3. 3
   Proof of concept (3–6 weeks) Label a representative sample, train a first model, and measure precision, recall and IoU on a held-out area. The deliverable is a written accuracy report including failure examples, not a demonstration.
4. 4
   Decision point Go/no-go based on measured accuracy against the agreed threshold. Some classes are not viable at the available resolution, and saying so early is cheaper than proving it in production.
5. 5
   Scale-up Expand training data to cover the conditions found across the full area, retrain, and set up inference on the tile index with the chosen execution surface (workstation or Image Server).
6. 6
   Review workflow Build the review application, deduplication rules and the route into the geodatabase and downstream systems; define who owns rejected detections.
7. 7
   Production run & validation First full run with a sampled QA check on accepted detections, run logs and a recorded model version for every output feature.
8. 8
   Hand-over & retraining plan Training data, model files, code, accuracy report and a documented schedule for retraining when imagery sources, seasons or sensors change.

Security & deployment

GeoAI is normally deployed entirely inside the entity's network because the imagery itself is the sensitive asset. Training and inference run on on-premise GPU hardware or in a UAE-region private cloud, with model weights, Python packages and GPU drivers installed from an internal mirror where the environment is air-gapped. Access to imagery, detections and review applications follows the portal's role-based permissions, so an inspector sees only their area. Every inference run records the model version, parameters, input tiles and operator, and every accepted detection keeps a link to that run, which matters when a detection later supports an enforcement or spending decision. Model files and training data are delivered to the client and stored in their repository, not held by the vendor.

Limitations & prerequisites

• Accuracy is bounded by imagery resolution: an object that is three pixels across cannot be reliably detected regardless of the model, so resolution requirements must be fixed before any modelling starts.
• Training data is the main cost and schedule risk — several hundred to several thousand consistently labelled examples per class, covering the range of conditions in the production area.
• Pretrained models built on other regions usually underperform on GCC imagery until fine-tuned on local samples; published accuracy figures do not transfer.
• Detections are suggestions with confidence scores, not verified facts; a human review step is required wherever the output drives enforcement, spending or safety decisions.
• Models drift when the sensor, resolution, season or processing chain changes; retraining and periodic accuracy re-measurement must be budgeted, not treated as a one-off project.
• GPU hardware is required for practical training and for large-area inference; CPU-only inference is possible but too slow for city-scale work.
• Some classes remain genuinely hard — objects under canopy or shadow, partially occluded features, and classes whose definition varies between departments.
• Change detection is sensitive to differences in capture angle, lighting and orthorectification between dates; without consistent inputs it produces false change.

Manual digitising vs GeoAI vs field survey

GeoAI is not a replacement for surveying — it changes where human effort is spent.

The usual production pattern is GeoAI for coverage, human review for confirmation, and field survey targeted only at what the first two flagged as uncertain or important.

FAQ

Related topics & solutions

• Road Inspection & Pavement Condition Intelligence in GIS
• ArcGIS Python Toolbox & ArcPy Automation
• ArcGIS Experience Builder Custom Widgets
• ArcGIS Pro Add-in Development
• ArcGIS – IBM Maximo Integration
• AI Products Portfolio
• RASM ÔÇô Digital Twin
• Smart Inspection