FlashPose: Range and intensity image-based terrain and vehicle relative pose estimation algorithm

This suite of technologies includes a method, algorithms, and computer processing techniques to provide for image photometric correction and resolution enhancement at video rates (30 frames per second). This 3D (2D spatial and range) resolution enhancement uses the spatial and range information contained in each image frame, in conjunction with a sequence of overlapping or persistent images, to simultaneously enhance the spatial resolution and range and photometric accuracies. In other words, the technologies allows for generating an elevation (3D) map of a targeted area (e.g., terrain) with much enhanced resolution by blending consecutive camera image frames. The degree of image resolution enhancement increases with the number of acquired frames.

The novel Vision-based Approach and Landing System (VALS) provides Advanced Air Mobility (AAM) aircraft with an Alternative Position, Navigation, and Timing (APNT) solution for approach and landing without relying on GPS. VALS operates on multiple images obtained by the aircraft’s video camera as the aircraft performs its descent. In this system, a feature detection technique such as Hough circles and Harris corner detection is used to detect which portions of the image may have landmark features. These image areas are compared with a stored list of known landmarks to determine which features correspond to the known landmarks. The world coordinates of the best matched image landmarks are inputted into a Coplanar Pose from Orthography and Scaling with Iterations (COPOSIT) module to estimate the camera position relative to the landmark points, which yields an estimate of the position and orientation of the aircraft. The estimated aircraft position and orientation are fed into an extended Kalman filter to further refine the estimation of aircraft position, velocity, and orientation. Thus, the aircraft’s position, velocity, and orientation are determined without the use of GPS data or signals. Future work includes feeding the vision-based navigation data into the aircraft’s flight control system to facilitate aircraft landing.

Aerial planetary exploration spacecraft require lightweight, compact, and low power sensing systems to enable successful landing operations. The Ocellus 3D lidar meets those criteria as well as being able to withstand harsh planetary environments. Further, the new tool is based on space-qualified components and lidar technology previously developed at NASA Goddard (i.e., the Kodiak 3D lidar) as shown in the figure below. The Ocellus 3D lidar quickly scans a near infrared laser across a planetary surface, receives that signal, and translates it into a 3D point cloud. Using a laser source, fast scanning MEMS (micro-electromechanical system)-based mirrors, and NASA-developed processing electronics, the 3D point clouds are created and converted into elevations and images onboard the craft. At ~2 km altitudes, Ocellus acts as an altimeter and at altitudes below 200 m the tool produces images and terrain maps. The produced high resolution (centimeter-scale) elevations are used by the spacecraft to assess safe landing sites. The Ocellus 3D lidar is applicable to planetary and lunar exploration by unmanned or crewed aerial vehicles and may be adapted for assisting in-space servicing, assembly, and manufacturing operations. Beyond exploratory space missions, the new compact 3D lidar may be used for aerial navigation in the defense or commercial space sectors. The Ocellus 3D lidar is available for patent licensing.

The DLC platform is composed of three key components: a NASA-designed field programmable gate array (FPGA) board, a NASA-designed multiprocessor on-a-chip (MPSoC) board, and a proprietary datapath that links the boards to available inputs and outputs to enable high-bandwidth data collection and processing. The inertial measurement unit (IMU), camera, Navigation Doppler Lidar (NDL), and Hazard Detection Lidar (HDL) navigation sensors (depicted in the diagram below) are connected to the DLC’s FPGA board. The datapath on this board consists of high-speed serial interfaces for each sensor, which accept the sensor data as input and converts the output to an AXI stream format. The sensor streams are multiplexed into an AXI stream which is then formatted for input to a XAUI high speed serial interface. This interface sends the data to the MPSoC Board, where it is converted back from the XAUI format to a combined AXI stream, and demultiplexed back into individual sensor AXI streams. These AXI streams are then inputted into respective DMA interfaces that provide an interface to the DDRAM on the MPSoC board. This architecture enables real-time high-bandwidth data collection and processing by preserving the MPSoC’s full ability. This sensor datapath architecture may have other potential applications in aerospace and defense, transportation (e.g., autonomous driving), medical, research, and automation/control markets where it could serve as a key component in a high-performance computing platform and/or critical embedded system for integrating, processing, and analyzing large volumes of data in real-time.

While real-time positioning computed by standard GPS service is adequate for some onboard applications, inherent position discontinuities are not acceptable for high-precision instrument applications, such as view-period prediction and maneuver planning, both of which are computations that require a continuous prediction of the spacecraft state. Real-time positioning also requires simultaneous measurements from four GPS satellites, a mission-limiting factor that must be considered. GEONS processes data from standard GPS receivers, onboard communication equipment, and/or attitude sensors, producing accurate absolute and relative navigation solutions in real time. Other functions, including onboard maneuver control and relative navigation for keeping formations are also supported by GEONS. All information is quickly available to scientists when it is included in the down-linked telemetry stream. GEONS provides high-quality solutions with fewer than four visible GPS space vehicles by employing an extended Kalman filter (EKF) augmented with physically representative models for gravity, atmospheric drag, solar radiation pressure, clock bias, and drift to provide accurate state estimation and a realistic state error covariance. GEONS incorporates the information from all past measurements—carefully balanced with GEONS’ data of the physical models governing these measurements—to produce an optimal estimate of the user spacecraft’s orbit. GEONS’ high-fidelity state dynamics model reduces sensitivity to measurement errors and provides high-accuracy velocity estimates, permitting accurate state prediction during signal outages or degraded coverage. Autonomous navigation reduces total mission cost by eliminating the need for routine, ground-based orbit determination and special tracking services, and it is required for advanced mission concepts, such as satellite formation flying. GEONS was designed for autonomous operation within the very limited resources of an onboard computer. Autonomous initialization and enhanced fault detection capabilities are implemented using instantaneous geometric GPS solutions. By incorporating information from past measurements, GEONS provides highly accurate orbit estimates even with one visible GPS space vehicle—and even during signal outages or degraded coverage. This unprecedented accuracy and reliability reduces navigation errors and does it autonomously with minimal onboard computer resources. Additionally, GEONS processes Doppler measurement data from onboard attitude sensors. These different types of measurements are all incorporated into the GEONS software, producing a navigation system capable of handling multiple orbit regimes and navigation subsystems, while requiring no additional hardware. Fusion of these data types increases system stability and reliability and enables graceful degradation in the event that a component fails during orbit.