CameraLink on the Zynq based Tulipp Platform

A CameraLink to HDMI with Sobel filter demo showing the possibilities for hardware acceleration and low-power on Tulipp platform

Similar projects worth following
This demo project is based on a stand-alone EMC² Development Platform PCIe/104 OneBank™ board featuring a Zynq 7015 SoC.
The board connects to an FMC-CAMERALINK daughter board in order to grab frames from a CameraLink compatible camera. Such frames are stored into DDR memory, processed in some way (in our demo, by a Sobel filter) and then output to an HDMI compatible screen.
This demo shows how most of the most computing intensive tasks are implemented, where possible, in hardware, both using standard or custom IP cores and by using High Level Synthesis tools.

One first fundamental step was to develop a custom IP core in order to interface with a CameraLink camera. In our demo we used a Sentech STC-CLC83A camera, which has a resolution of 1024x768, 30 fps, 8 bits per pixel, Bayer color scheme, BASE only configuration.

For the video output section, we used an existing simple IP for HDMI output, which we just modified slightly to adapt to our case. HDMI output signals interface with the ADV7511 chip, which needs to be configured by software on startup. HDMI is available through Sundance External Interface Connector add-on board. The output settings fit XGA standard: 1024x768, 60 fps, RGB.

Hardware Design

In this section we'll summarize the steps taken in developing the hardware firmware. Our project is based on Xilinx's SDx 2017.1 design suite.

FC500 CameraLink IP core

As first step we needed to develop an IP core in order to interface with our CameraLink camera. This is the overall scheme of such IP:

Data serialization is a fundamental step in order to get the 28 bit word from four serial data lines with serial factor 7x. Video signal conversion generate active video, hsync and vsync in a format compatible with other standard Xilinx IP core.

UART interace is fundamental in order to communicate with the camera and initializate it from software.

Here is the IP core web page:

Overall architecture

The following figure shows a simplified view of our hardware design, developed using Vivado 2017.1:

The orange coloured IP is the FC500 IP core discussed previously. Our design also includes the Zynq Hard Processing system, not shown here for space reasons.

The FC500 IP core output video data (8 bit per pixel), Active Video, Horizontal Sync and Vertical Sync signals. The Video-to-AXI4-Stream-Converter converts such signals to AXI Streaming Bus signals, which are compatible with several AXI based video processing IPs. We used then a Color Filter Array IP to convert the Bayer color mosaic into a standard RGB signal (24 bits per pixel). This RGB signal is itself converted to YUV 4:2:2 format (16 bits per pixel) in order to facilitate processing based mainly on luminance component. In such format frames are stored in DDR memory by the VDMA. For this demo we used a triple buffering mode.

In the output section (lower part of the figure) we find some 'inverse' operations with respect to the input, plus a Video Timing Controller, which generates the correct sync signals for the chosen output format (XGA 1024x768, 60 fps), and an HDMI output IP, which makes just some signal adjustments and DDR modulation for the ADV7511.

There are three main clock domains:

  • CameraLink clock: this is an external and therefore asynchronous clock which needs to be synchronized using IDELAY and ISERDES components plus an FSM. For our camera, its frequency is 30 MHz.
  • HDMI output clock: its frequency matches the one required for XGA pixel clock (explained previously) and it's 65 MHz.
  • AXI Bus clock: we chose its frequency to be 130 MHz, which is more than 100 MHz (cosidered as minimum requirement) and its the exact double of HDMI clock frequency. This setting helped meet clock domain crossing timing constraints.

Other secondary clocks are for example a 50 MHz used for the CameraLink UART interface and a shifted version of the HDMI clock, which is the one actually fowarded externally to the ADV7511.

Software Design

Using Xilinx SDK 2017.1 (subset of SDSoC) we created a Board Support Package for our hardware project. This can be used to generate bare-metal application projects.

Then we created a Hardware/Software platform to be used for SDSoC high level synthesis projects. We used the SDSoC Platform Utility to generate a platform for bare-metal (standalone) oriented application projects. A custom platform TCL script must be supplied, beside the hardware project and BSP files.

When a new platfrom is correctly generated it can be...

Read more »

  • 1 × EMC2 Development Platform from Sundance
  • 1 × Trenz Electronic TE0715 (Z-7015)
  • 1 × Sundance External Interface Connector add-on board
  • 1 × FMC-CAMERALINK module from Alpha Data
  • 1 × SenTech STC-CLC83A camera

  • 1
    Running the demo

    This instructions will help you setup and run the demo as it is by booting from an SD card.

    1. Hardware configuration

    1.1 Hardware requirement

    • All things listed in Components section
    • A HDMI compatible screen
    • HDMI cable
    • Power supply
    • CamerLink cable
    • miniUSB to USB cable
    • Power/IO connector for the camera
    • a computer

    1.2 Hardware setup

    1. Connect the power cable to the EMC² board but don’t switch the power on for now.
    2. Connect the USB cable between the EMC² board and the computer.
    3. Connect the screen to the HDMI output on the SEIC add-on board.
    4. Connect the camera to the FMC-CAMERALINK daughter board (Channel 2) using the CameraLink cable.
    5. Connect the Power/IO connector to the camera

    Figure 1: EMC2 board setup

    Figure 2: Camera setup

    2. Software configuration

    2.1 Software setup

    • Copy the BOOT.BIN file from the project folder "sdsoc_app2\Release\sd_card" to the root folder of the first partition of a micro SD card. Such partition must be formatted as FAT32.
    • Insert the card into the micro SD socket on the EMC2 board and power on the board.
    • Open a serial terminal application (we tested it with Putty) and use the following settings:
      • Baud rate: 115200
      • Data bits: 8
      • Stop bits: 1
      • Parity bits: None
      • Flow control: None

    • After the terminal is connected, reset the board and you will see a menu with 4 choices:

    • Make your choice and you should see the demo on the screen. If no signal appear, try to reset the board and make your choice again.

    The console terminal will show the current frame-per-second-value:

View all instructions

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates