BRKMYR/ AI Builder Portfolio/ SAR VLM
SAR VLM
A natural-language console over synthetic-aperture radar satellite imagery. Ask "how many vessels?" and get a number, a confidence, and the raw scores backing it.
Problem
Radar is on every satellite. Almost nobody can read it.
NewSpace SAR constellations image the whole earth daily at sub-meter resolution, day or night, through clouds. But radar backscatter doesn't look like a photograph. It looks like static with structure. The people who care about the data (port managers, defense analysts, commodity desks, insurers) don't want to look at it. They want to ask questions.
Two failure modes are equally bad: a specialist takes days to answer, or a generic vision-language model hallucinates a plausible-sounding count that isn't grounded in the physics of what the sensor can actually see.
Solution
Turn the question into a classification run. Refuse it when the physics can't answer.
Questions are parsed into (task, target) intents. The scene is tiled and passed through SARCLIP (a SAR-native CLIP variant); each tile gets a score against a positive and a negative prompt. A capability gate checks whether the requested target is resolvable at the scene's band and pixel size, and refuses the query if not. Positive tiles are grouped into clusters, and the count is the number of clusters. There is no LLM in the answer path, so the number can only come from the encoder scores.
Every query and response is persisted to a local SQLite audit log with the full score vector, the threshold, and the prompt strings. Nothing about an answer is opaque.
Live capture · supported query
Scene Viewer
Response
capability: ok · target supported at 0.5 m/px
tiles: 25 total, 15 positive at threshold 0.55
clusters: 2 (scores 0.94, 0.86)
Live capture · refused query
Scene Viewer
Response
large_vessel, port_activity.
reason: aircraft need < 3 m/px; scene is 10 m/px
answer: none emitted
confidence: 0.000
Success criteria
When we'd call this "working."
- Never emit a number for a query the imaging physics cannot support. This is the whole point of the system.
- Every answer is backed by a per-tile score vector, a threshold, and a prompt string that the user can inspect.
- Latency under 100 ms per query on CPU. No GPU required for the demo.
- Every query and response persisted to a local audit log. Nothing lost.
- Runs on a laptop with no cloud dependency.
Metrics, KPIs, evals
Numbers from the two captures above.
| KPI | Value | What it means |
|---|---|---|
| Capability-gate precision | 1.00 | The gate refused the aircraft query on a 10 m/px scene. It did not refuse the vessel query on the 0.5 m/px scene. This is the KPI the whole system is designed around. |
| Answer confidence | 0.887 | Vessel query on S0101. Composite of cluster scores and positive-tile ratio. Human-readable rendering: 9 of 10 confidence segments filled. |
| Latency P50 | 51 ms / 28 ms | Supported query (full inference) vs refused query (short-circuit before inference). CPU only. |
| Audit completeness | 4 / 4 queries | Every response, including refusals, persisted to SQLite with the full payload. See the audit log JSON below. |
| Determinism | byte-identical | Scene generator seeded with a fixed SeedSequence. Two runs produce the same PNGs and the same scores. |
| Test coverage | 47 tests | Backend, generator, audit store, and frontend all green. Ruff clean. |
- Real SARCLIP weights end-to-end. The demo runs the deterministic
syntheticmodel variant; the SARCLIP swap-in is a config toggle. - Real satellite scenes. Five synthetic scenes ship in the demo; the DLR/ESA loader is spec'd but not wired.
- Multi-target intents (e.g. "count vessels and tanks"). Single-target only in this version.
Raw artefacts
Every element on this page comes from a real API call captured during the demo run: scenes.json · query_S0101_vessels.json · query_S0104_aircraft.json · audit.json · health.json