• Sidequest: overclocking imx477 camera

    Monstrofil05/31/2024 at 17:47 0 comments

    I'm writing this primarily for my future self, who may one day decide to return to the IMX477 driver implementation to add features like HDR support, 4 MIPI lanes, or 4K 60fps mode.

    The issue is that the RPI implementation of the driver is locked at a maximum of 10 fps in 4056x3040 mode. While this may be suitable for the RPI with its slow processors and ISP, which can only handle 1080p images, it is insufficient for the RK3588.


    Read more »

  • Drivers

    Monstrofil09/03/2023 at 21:55 0 comments

    First thing that you need in order to work with mipi camera on software level is kernel driver. Linux uses v4l2 layer in order to provide API for userspace applications. RockChip extends base driver requirements with couple vendor-specific requirements.


    - bunch of specific v4l2 callbacks, such as enum_bus_code, enum_frame_size, s_stream

    Unfortunately, rockchip did not include imx477 driver with their linux 5.10 kernel,but luckily RPi has open source driver used on their boards.

    Let's add couple missing parts.

    Read more »

  • Overview

    Monstrofil08/07/2023 at 18:36 0 comments

    I have been exploring the possibility of creating a 360 camera for street motion capture for a while. It all started with an article I came across on the internet about Surround360 - a solution that was designed and released as open-source by the Facebook team a few years ago. Unfortunately, the solution turned out to be far from budget-friendly: by the most optimistic estimates, assembling the setup would cost around $30,000.

    The Facebook design includes certain extravagances, which, if eliminated, could significantly reduce the cost of the construction. For instance, the use of 14 cameras in a circular arrangement is justified for capturing depth information and subsequent processing, but it is entirely excessive for capturing urban street scenes.

    Surround360 camera design.

    Surround360 camera design.

    Using the ready-made GS3-U3-41c6c-c module ($1,500/unit) simplifies the assembly and configuration of the camera, but it really hits the wallet hard.

    On the other hand, when aiming for cost reduction, it's crucial to maintain camera synchronization. Even a slight delay of a few milliseconds can impact the quality of the stitched panorama.

    Achieving perfect synchronization is possible with Global Shutter cameras, but such cameras are much more expensive than the conventional Rolling Shutter ones, and their resolution is lower. Fortunately, Sony offers a solution for synchronizing N Rolling Shutter cameras in their sensors. It's available, for example, for IMX415 or IMX477 sensors, and it's precisely the latter sensor that will be used in this project.

    Read more »