By Thomas Guillemain & Thomas Porchez, Teledyne e2v
Of the 6500+ operational satellites currently orbiting the planet, at least 1000 of them are engaged in some form of Earth observation (EO) work. The imagery being obtained from such activities can serve a multitude of different purposes, many of which will have either ecological or socio-political benefits. With advances being made in relation to the imaging technology involved, the level of detail that can now be obtained is continually improving. Consequently, the scope of applications that can be undertaken is expanding and the quality of the results is being raised. Unfortunately, this is creating problems elsewhere though – as the resolution of images increases and the quantity of data grows, communication bottlenecks are beginning to appear. In some cases, data coming from constellations of hundreds of satellites may need to be dealt with – so, once again, this is problematic, as there will be too much material to examine, unless it is categorised beforehand and the excess filtered out.
In the past, it was possible for relatively small quantities of data to be transmitted directly back to Earth and then processed through dedicated data centres. The migration from such data centre infrastructure to cloud based platforms, alongside the evolution of the sensor technology employed (with more sophisticated, higher resolution devices being specified), have meant that downlinks are simply no longer able to cope, as they cannot scale with the growing data demands. Consequently, a whole new approach is needed. Perhaps one that leverages advances in Artificial Intelligence (AI) in space applications.
Bringing the processing aspect closer to the source, in a similar way to the edge computing arrangements that are starting to be implemented in terrestrial communication networks, will make it possible for several key advantages to be derived. Firstly, it will no longer be necessary to sort through all the images on the ground, as only the ones of value will be transmitted. Secondly, it will overcome the acute bandwidth limitations that are now starting to be witnessed (as just outlined). Thirdly, responsiveness to an arising situation will be quicker. This could be important when dealing with different forms of natural or human-made disasters, as it will be possible for them to be identified quicker, so that emergency services and aid organisations may be informed within a shorter time frame (and more lives might thereby be saved).
On-board processing
For the reasons just outlined, there is huge interest in moving EO work from a centralised architectural set up, to one that is edge-based. Rather than sending everything back, by having more analysis capabilities on the satellite itself, there is the prospect for acquired data to be interpreted there, then decisions made as to its relevance. This would put much less strain on a satellite’s downlink, and also mean that power is not consumed transmitting data that will not be of use.
If it is determined that the images obtained feature items that are of interest and deserve more in-depth analysis, or they show signs of something that needs to be reacted to with great urgency, then their transmission will clearly be justified. Conversely, if the material is shown to be inconsequential, then the need to transmit it is negated and bandwidth is not wasted. This can be achieved through the usage of Artificial Intelligence in space applications
Essential processor characteristics
Any type of semiconductor technology intended for use in space needs attributes that go way beyond what is expected for conventional application scenarios. Once hardware is in space cannot be repaired, so any damage or functional problems will jeopardise the mission. Constituent components need to exhibit resilience to the intense shocks and vibrational forces they will be exposed to at launch, as well as the temperature extremes that move from sun-facing to dark-side during their orbit.
They must also be rugged enough to withstand exposure to radiation too. Ions hitting a processor device can cause single event latch-ups (SELs) and single event upsets (SEUs). In addition, there is the total ionizing dosage (TID) to consider, as the lifespan of a device can be shortened because of this. So that it can be ensured that a specified processor will provide prolonged operation once it has been deployed into space, and the risk of functional errors are not witnessed, comprehensive radiation testing is mandated.
There are a few other points that should not be overlooked. Satellites have very little room in which to accommodate all their necessary electronics to enable Artificial Intelligence in space applications. They also have a limited power budget (based on what their photovoltaic cells can generate. Finally, the ‘New Space’ community does not generally have huge financial reserves at their disposal. The outlay on projects must be kept in check, so the devices selected need to have attractive pricing.
Use case example
Swiss headquartered space systems integrator Beyond Gravity is currently developing a high-performance processor platform that will enable real-time data processing via Artificial Intelligence in space applications on satellites conducting Low Earth Orbit (LEO) observation tasks. The Lynx platform needs to have superior computational capabilities, while at the same time not placing too heavy demands on the available power budget. It also has to be robust enough to support long-term operation and Artificial Intelligence in space applications.
Based on the different aspects already outlined, the company needed a radiation tolerant processing solution upon which sophisticated AI algorithms could be run. This had to be done while drawing only minimal power, not occupying excessive board real estate, or having too high a price.
Consultations with the staff at Teledyne e2v proved successful, leading to one of the company’s processor solutions being chosen. By taking commercial-off-the-shelf (COTS) processing technology, then applying extensive screening in order to select the best performing units, Teledyne e2v is able to offer processors that are more cost-effective than custom-built solutions.
Figure 1: The LS1046-Space multi-core radiation-tolerant processor from Teledyne e2v
Designed to cope with the challenging application environment that space represents, but still capable of running at speeds of up to 1.8GHz, Teledyne e2v’s LS1046-Space is becoming the go-to solution for on-board processing on satellites and enabling Artificial Intelligence in space applications. It is the most powerful space-qualified processor currently on the market – delivering more than an order of magnitude better performance than competing solutions.
Thanks to its multi-core processor architecture, comprised of four 64-bit Arm® Cortex® A72 cores, it can deliver 30k DMIPs of processing performance. Other functionality incorporated into this device includes a highly effective DDR4 memory controller with embedded 8-bit error corrected code (ECC) to mitigate the threat of data corruption, as well as a 2MB L2 cache that attends to all of its processor cores.
This processor is supplied in a 780-ball BGA package and has a 23 x 23 mm footprint that means only minimal board real estate is taken up. To enable integration into a broad variety of different system designs, this processor also incorporates an extensive array of interfaces. These include 10 Gbit Ethernet, PCIe Gen 3.0, SPI and I2C. 72-bit bus width (where 64 bits are dedicated to data and a further 8 bits are assigned to ECC). In addition to its processing capabilities, the LS1046-Space has exceptional robustness, with both NASA Level 1 and ESA ECSS Class 1 qualifications, with a 55 ⁰C to 125 ⁰C operating temperature range.
Accompanying the processor is Teledyne e2v’s DDR4T04G72M, which is a 4GB radiation tolerant DDR4 memory that uses a multi-chip package (MCP) arrangement to significantly raise density levels. Both the memory and the processor devices have passed 100 krad TID testing, meaning they have extended operational longevity. They have also attained 60 MeV.cm²/mg radiation tolerance in relation to both single event latch-ups (SELs) and single event upsets (SEUs), so functional integrity is assured.
The software aspect
To complement this space-grade radiation-tolerant hardware, Teledyne partner Klepsydra has developed embedded software for the Beyond Gravity Lynx product. This software has been highly optimised for resource-constrained applications as well as Artificial Intelligence in space applications. Thanks to proprietary parallelisation technology, it can handle complex AI workflows while only drawing minimal power, and avoiding the loss of data that could otherwise lead to mission failures. By running Klepsydra’s software stack on the LS1046-Space processor, a 50% reduction in CPU load can be realised. This is combined with a trebling of the overall processing capability, plus a dramatic reduction in latency. The performance of the software was measured using AI to identify points of interest within captured images, and for cloud detection. The latter algorithm is a very important area of research, as it can determine if the cloud coverage is too high to make the captured images worth sending back.
Among the numerous Earth observation applications that can be addressed via this technology will be the monitoring of deforestation or urbanisation, smart farming, cloud detection, as well as the recording glacial movements, the examining of flooding and wildfires, tracking military activity, etc. It could also potentially be used to provide an early warning system for life-endangering incidents – such as tsunamis, for example.
Conclusion
By applying edge-based computing principles and Artificial Intelligence in space applications to equipment that has been deployed into space will mean that issues relating to downlink bandwidth limitations can be mitigated, with only the data that has real value needing to be transmitted. Leveraging the power of Artificial Intelligence in space applications to carry out processing at source will lead to a much more efficient workflow, and enable better informed decision-making.
Though for space-based processors reliability was always the priority over performance, today both are needed simultaneously. Engineering innovations, such as the ones described above, are bringing the processing capabilities of terrestrial systems to space applications. Via the collaboration being undertaken between Teledyne e2v, Klepsydra and Beyond Gravity, it will be possible for a new generation of satellites and spacecraft to be developed. These will feature the processing power needed to run complex AI algorithms, and this will result in the higher levels of autonomy needed to make decisions about imaging data being captured, so that operations can be conducted more efficiently and without putting excessive strain on the bandwidth capacity or the power reserves.