Illumination-Aware Active Perception for Spacecraft Inspection

Summary

Abstract Summary

Harsh and rapidly varying orbital illumination introduces failure modes during spacecraft inspection. Existing planners rely on line-of-sight inspection metrics which perform occlusion checks against a known target and conflate geometric visibility with task-relevant information. These metrics overlook illumination effects including eclipse, self-shadowing, and specular reflection from spacecraft materials and tend to declare components as inspected despite the corresponding images lacking sufficient information. We propose an inspection-planning framework that replaces geometric visibility with a learned measure of camera information derived from a classifier variational autoencoder (VAE) that operates directly on the captured image. The encoder model is trained to classify spacecraft surface components from rendered patches, and its total latent variance is used to learn the varying illumination of said components, replacing binary visibility checks with a continuous, image-grounded quantity. We evaluate the approach by implementing a Guidance, Navigation, and Control architecture in a photorealistic physics simulator, and vary lighting and viewing geometries with multiple orbits. We show that, although all approaches successfully maximize their own objective, our approach results in better illumination: a mean brightness of $0.57$, with $81.6\%$ of observations above $0.25$, compared with $0.02$ ($3.0\%$) for geometric planning with an illumination band and $0.04$ ($6.0\%$) for purely geometric planning.

Key Points

Harsh and rapidly varying orbital illumination introduces failure modes during spacecraft inspection
Existing planners rely on line-of-sight inspection metrics which perform occlusion checks against a known target and conflate geometric visibility with task-relevant information.
These metrics overlook illumination effects including eclipse, self-shadowing, and specular reflection from spacecraft materials and tend to declare components as inspected despite the corresponding images lacking sufficient information.
We propose an inspection-planning framework that replaces geometric visibility with a learned measure of camera information derived from a classifier variational autoencoder (VAE) that operates directly on the captured image.
The encoder model is trained to classify spacecraft surface components from rendered patches, and its total latent variance is used to learn the varying illumination of said components, replacing binary visibility checks with a continuous, image-grounded quantity.

Problem Statement

The inspection task is formulated as a Markov Decision Process (MDP) [ 27 ] as $M={S,A,T,R,\gamma}$ where $S$ corresponds to the state space, $A$ to the action space , $T$ is the transition kernel and $R$ is the reward function. No discount factor is used in determining the reward. We use k subscript for the points of interest (POI) index.

State Space : The complete state $s$ is a combination of the physical state $x$ and inspection $z$: $s=[x;z]$.

Problem Statement — Illumination-Aware Active Perception for Spacecraft Inspection — Fig. 1 : Active-perception inspection conducted by the chaser to gain maximum information under illumination constraints. Top: chaser spacecraft on the right, target spacecraft on the left. Bottom: chaser’s camera view of target spacecraft. Green dots are Points of Interest (POI) successfully inspected.

Guidance, Navigation, and Control (GN&C) System Architecture

The GN&C system architecture is shown in (Fig. 2 ): navigation, discrete planning, continuous planning, attitude control, thrust control, and inspection metric and are discussed in the section below as part of a closed loop model.

Navigation The navigation model produces the perfect physical state $x=[q_{c};\dot{q}_{c};q_{t};\dot{q}_{t}]$ and all quantities are directly read from the simulator with zero measurement noise for chaser and target. Given perfect physical state we realize a hierarchical planning algorithm. Discrete Planning.

Guidance, Navigation, and Control (GN&C) System Architecture — Illumination-Aware Active Perception for Spacecraft Inspection — Fig. 2 : Closed-loop inspection architecture. Only the inspection metric (shown in green) differs between the three models; the decision layer, trajectory layer, and controllers are common to all three.

Methods

The complete GNC stack is implemented in NVIDIA Isaac Sim with the Lunar Gateway as a USD asset; the chaser carries a single forward-facing camera rendering $256\times 256$ RGB frames through the photorealistic ray-traced pipeline.

All three inspection probabilities share the planner machinery, PRO pool, SCP transfer, attitude QP, controllers, chaser, target, and environment, differing only in the detection probability $\nu_{k}(x)$ of the inspection transition in Eq. (1).

Methods — Illumination-Aware Active Perception for Spacecraft Inspection — Fig. 3 : Variational Auto Encoder Architecture adapted from [ 16 ] . Blocks in green represent the loss functions used in the model.

Experiments

A quantitative and qualitative analysis are presented to test the three different inspection probabilities. The experimental set up in IsaacSim used open source Nasa models [ 23 ] of the Lunar Gateway for the target and a general satellite model adapted from CloudSat models for the chaser along with an Earth, Moon and Sun module.

A quantitative analysis is presented to test the three detection probability functions $\nu_{1}$, $\nu_{2}$ and $\nu_{3}$ under identical initial conditions.

Experiments — Illumination-Aware Active Perception for Spacecraft Inspection — Figure from the paper. Source: Illumination-Aware Active Perception for Spacecraft Inspection.

Sources and Demonstrations

Figures are reproduced from Valluri et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.