WEX Deals: Use code 10CAN to save 10% on select Canon lenses

Sony To Release 'Intelligent Vision' Sensors with AI Processing Functionality

Sony has announced that it is releasing two new sensors featuring AI processing functionality, the first of their kind, expanding opportunities to develop AI-equipped cameras.

|  Industry News
 Add Comment

sony image sensor

Sony has announced the upcoming release of two new sensors, which they say are the first Intelligent Vision sensors in the world to feature AI processing functionality.

AI, or Artificial Intelligence, is set to be the next step forward with modern camera technology and Sony claims that including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, which, when using cloud services, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs.

"These products expand the opportunities to develop AI-equipped cameras, enabling a diverse range of applications in the retail and industrial equipment industries and contributing to building optimal systems that link with the cloud." - Sony

The new sensor products feature a stacked configuration consisting of a pixel chip and logic chip. They are the world’s first image sensor to be equipped with AI image analysis and processing functionality on the logic chip. The signal acquired by the pixel chip is processed via AI on the sensor, eliminating the need for high-performance processors or external memory, enabling the development of edge AI systems.

The sensor outputs metadata (semantic information belonging to image data) instead of image information, making for reduced data volume and addressing privacy concerns. Moreover, the AI capability makes it possible to deliver diverse functionality for versatile applications, such as real-time object tracking with high-speed AI processing. Different AI models can also be chosen by rewriting internal memory in accordance with user requirements or the conditions of the location where the system is being used.

AI sensor output

AI Intelligent vision sensors key features:

  • World's first image sensor equipped with AI processing functionality - The pixel chip is back-illuminated and has approximately 12.3 effective megapixels for capturing information across a wide angle of view. In addition to the conventional image sensor operation circuit, the logic chip is equipped with Sony's original DSP (Digital Signal Processor) dedicated to AI signal processing, and memory for the AI model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for edge AI systems. 
  • Metadata output - Signals acquired by the pixel chip are run through an ISP (Image Signal Processor) and AI processing is done in the process stage on the logic chip, and the extracted information is output as metadata, reducing the amount of data handled. Ensuring that image information is not output helps to reduce security risks and address privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, including ISP format output images (YUV/RGB) and ROI (Region of Interest) specific area extract images.
  • High-speed AI processing - When a video is recorded using a conventional image sensor, it is necessary to send data for each individual output image frame for AI processing, resulting in increased data transmission and making it difficult to deliver real-time performance. The new sensor products from Sony perform ISP processing and high-speed AI processing on the logic chip, completing the entire process in a single video frame. This design makes it possible to deliver high-precision, real-time tracking of objects while recording video.
  • Selectable AI model - Users can write the AI models of their choice to the embedded memory and can rewrite and update it according to its requirements or the conditions of the location where the system is being used. For example, when multiple cameras employing this product are installed in a retail location, a single type of camera can be used with versatility across different locations, circumstances, times or purposes. It can count the number of visitors entering a store, or detect stock shortages. 

The new sensor is capable of 4K image recording at 60fps and features the Bayer colour filter array.

 

Support this site by making a Donation, purchasing Plus Membership, or shopping with one of our affiliates: Amazon UK, Amazon US, Amazon CA, ebay UK, WEX

It doesn't cost you anything extra when you use these links, but it does support the site, helping keep ePHOTOzine free to use, thank you.

Other articles you might find interesting...

Fujifilm Has Released Software To Turn Your X-Series And GFX...
LEE Direct Online Shop Is Now Open Again For UK Customers
Nikon UK Service Centre To Reopen
Affinity Want You To Learn For Free In May
Free Photography Webinars You Can Watch From Home
Stay At Home With Olympus - Free 1-2-1 Sessions, 'How To' Ph...
Nikon Wins Red Dot Award For Product Design
Canon Launch Canon Connected Free-To-Access Content Hub

Comments


JackAllTog Plus
11 6.0k 58 United Kingdom
14 May 2020 1:34PM
An automated security guard. a 12Mp Robocop.

Neat idea on the image compression for video streaming. I'd assume they pair it with local buffering for periods where the scene is fast moving.

Join ePHOTOzine for free and remove these adverts.

Sign In

You must be a member to leave a comment.

ePHOTOzine, the web's friendliest photography community.

Join For Free

Upload photos, chat with photographers, win prizes and much more.