Developing Vision Systems with Dissimilar Sensors: Integrating Image, Radar, and Time-of-Flight Sensors in Embedded Applications
13 May 2020
Drones, intelligent cars and augmented or virtual reality (AR/VR) headsets all use multiple image sensors, often of different types, to capture data about their operating environment. To supply the image data the system needs, each sensor requires a connection to the system’s application processor (AP), which presents design challenges for embedded engineers. First, APs have a finite number of I/O ports available for connecting with sensors, so I/O ports must be carefully allocated to ensure all discrete components requiring a connection to the AP have one. Secondly, drone and AR/VR headsets have small form factors and use batteries for power, so components used in these applications must be as small and power efficient as possible.
One solution to the AP’s shortage of I/O ports is the use of Virtual Channels, as defined in the MIPI Camera Serial Interface-2 (CSI-2) specification, which can consolidate up to 16 different sensor streams into a single stream that can then be sent to the AP over just one I/O port. The hardware platform of choice for a Virtual Channel implementation is the field programmable gate array (FPGA). Alternative hardware platforms take a long time to design, and may not have the low power performance needed for applications like drones or AR/VR headsets. Some would argue that FPGAs have too large a footprint and consume too much power to be a feasible platform for Virtual Channel support. But advances in semiconductor design and manufacturing are enabling a new generation of smaller, more power efficient FPGAs.
The CrossLink™ family of FPGAs from Lattice Semiconductor provides the right combination of performance, size and power consumption for video bridge applications utilizing Virtual Channels. They offer two 4-lane MIPI D-PHY transceivers operating at up to 6 Gbps per PHY and a form factor as small as 6 mm2. They support up to 15 programmable source synchronous differential I/O pairs such as MIPID-PHY, LVDS, sub-LVDS, and even single ended parallel CMOS, yet consume less than 100 mW in many applications. The CrossLink FPGA family supports sleep mode to reduce power usage in standby. Lattice also provides a comprehensive software IP library to help customers more quickly implement different types of bridging solutions.
Virtual Channels enabled by the MIPI Camera Serial Interface-2 (CSI-2) specification help embedded engineers consolidate multiple sensor data streams over a single I/O port, reducing overall design footprint and power consumption for applications using large numbers of images sensors. By virtue of their reprogrammability, performance and size, low power FPGAs like Lattice Semiconductor’s CrossLink family let customers add support for Virtual Channels to their device designs quickly and easily.
To read the full white paper, please visit http://www.latticesemi.com/en/Products/FPGAandCPLD/CrossLink