A Camera or a Computer? AI and Machine Learning in Consumer Markets
A long-forecast surge in the number of products based on artificial intelligence (AI) and machine learning (ML) technologies is beginning to reach mainstream consumer markets.
It is true that research and development teams have found that, in some applications such as autonomous driving, the innate skill and judgement of a human is difficult, or perhaps even impossible, for a machine to learn. But while in some areas the hype around AI has run ahead of the reality, with less fanfare a number of real products based on ML capabilities are beginning to gain widespread interest from consumers. For instance, intelligent vision-based security and home monitoring systems have great potential: analyst firm Strategy Analytics forecasts growth in the home security camera market of more than 50% in the years between 2019 and 2023, from a market value of $8bn to $13bn.
The development of intelligent cameras is possible because one of the functions best suited to ML technology is image and scene recognition. Intelligence in home vision systems can be used to:
- Detect when an elderly or vulnerable person has fallen to the ground and is potentially injured
- Monitor that the breathing of a sleeping baby is normal
- Recognise the face of the resident of a home (in the case of a smart doorbell) or a pet (for instance in a smart cat flap), and automatically allow them to enter
- Detect suspicious or unrecognised activity outside the home and trigger an intruder alarm
These new intelligent vision systems for the home, based on advanced image signal processors (ISPs), are in effect function-specific computers. The latest products in this category have adopted computer-like architectures which depend for low latency, highly responsive operation on fast DRAM system memory to store the application code running on the ISP.
This article examines the factors affecting the choice of DRAM technology in the emerging category of home vision systems, and describes how DRAM manufacturers are responding to the specific demands of ISP-based architectures.
System Memory for a Full-blown Computer-camera in AI-inspired Home Visition Systems
The latest intelligent cameras for home monitoring use a neural network to perform image or scene recognition. These vision systems include an inference engine: this is a trained software algorithm which has ‘learned’ to recognise types of images or scenes by viewing and analysing thousands or millions of labelled images contained in a training dataset.
Fig. 1: the NetGear Arlo, one of the early examples of an AI-based home security camera. (image credit: Scott Lewis under Creative Commons licence)
Running an inference engine is a computationally heavy workload, so some of the latest home vision systems are based on a powerful integrated ISP with an AI inference engine from chipset manufacturers such as Ambarella. These ISPs typically feature a powerful Arm® Cortex®-A series applications processor core.
Another popular way to implement home security camera designs, an approach known as ‘light AI’, can use smaller, less powerful chipsets from manufacturers such as Omnivision, Kneron and NXP Semiconductors. While Omnivision and NXP are already globally famous suppliers of advanced sensor and processor chips, Kneron is fast emerging as a leader in AI technology – its KL520 was listed by EE Times as one of the top ten chipsets for edge AI applications.
So these AI-inspired home vision systems are effectively computer-cameras – a very different kind of device from an old-style closed-circuit TV (CCTV) camera or a security camera which simply records time-stamped video footage of a scene (see Figure 1). And so the new generation of home security cameras adopts a computer-style architecture, complete with fast DRAM memory large enough to store the complex inference engine and other application code (see Figure 2).
Fig. 2: the typical system architecture of a home security camera includes DRAM system memory supporting the ISP (Ambarella CV25S as example)
But unlike a tablet, smartphone or other general-purpose computer, the home vision system has specific image and inferencing functions for which the components and the system architecture can be optimised. And this means that the DRAM provision is limited to the needs of the image signal processing function, rather than of general computing tasks.
As a result, home security camera manufacturers are finding that they can benefit from the use of specialised low- or mid-density LPDRAM, rather than the high-density 4GB, 8GB or 16GB DRAM arrays typically found in smartphones or laptop computers.
Fig. 3: Kneron KL520 SoC on demo board. The KL520 Neural Processor Unit (NPU) was listed by EE Times as one of the top ten chipsets for edge AI applications
Low Power and Long Lifetime of DRAM
This substantially affects the manufacturer’s choice of DRAM type, and of the provider of this DRAM.
In high-end computers and tablets, OEMs need to maximise DRAM memory capacity and bandwidth and minimise its cost, which drives them to use the latest DRAM technology fabricated on a leading-edge process.
In home security and monitoring cameras, by contrast, the emphasis is on:
- Low power consumption – for instance, the design of a smart video doorbell must extend the run-time of its single-use alkaline batteries to minimise the frequency of battery replacement.
- Long lifetime – the economics of the home vision system market require that a successful design is on the market for potentially three years or more. Suppliers of the components in these designs therefore need to make a longevity commitment to avoid the need for the OEM to periodically redesign the hardware.
- Reliability – a home vision system might be installed in a fixed location and carefully positioned to capture the desired scene. This means that repair and replacement will require a technician to visit the installation – a much more expensive operation than shipping a failed laptop computer to a central repair laboratory. So customers and OEMs will place a high value on the reliability both of individual components and of the entire system.
This then determines the key factors which affect the OEM’s evaluation of DRAM products and suppliers. And the approach of specialty memory manufacturers will reveal a marked contrast with that of the large DRAM manufacturers fabricating on a leading-edge process and focussed on the PC and server markets.
A specialty supplier of DRAM will provide a broad range of products for applications which can operate with legacy DRAM technology, rather than leading-edge products. Winbond, for instance, supplies DRAM technology such as DDR, DDR2 and DDR3 standard versions of SDRAM, as well as Low-Power DDR, LPDDR2, LPDDR3, LPDDR4 and LPDDR4x mobile DRAM for power-conscious designs.
The mainstream DRAM manufacturers’ fabrication processes are also optimised for memory density and die cost. Manufacturers which choose not to follow the leading edge have the freedom to refine legacy fabrication processes to better meet the needs of customers in industrial, automotive, medical and consumer markets. In Winbond’s case, the process implemented at its own mobile DRAM wafer fabrication plant in Taiwan is configured to minimise the devices’ power consumption, helping Winbond’s standard DRAM products to offer better power performance than competitors’ standard products.
OEMs operating in the home vision systems market also need to rely on the longevity of their component choices. Here, too, a specialty memory supplier’s operation is configured to meet the market requirement. Winbond offers a standard 10-year longevity guarantee on its products from the date of introduction to the market.
Finally, while mainstream high-density DRAM features interfaces compatible with the x86 microprocessor architecture used in PCs and servers, specialty DRAM can be adapted to provide for easier integration with other types of processors, and particularly with Arm architecture devices.
To ease system integration for makers of home vision systems, Winbond has worked with ISP chipset manufacturers to support the reference design boards which provide the springboard for many security camera design projects. Software and hardware compatibility makes system integration easier for users of Ambarella ISPs in particular.
Specialty DRAM Supports Rapid Development of New, More Advanced Vision Systems
As AI and machine learning technologies advance, the requirement for greater signal processing throughput and faster memory to support the ISP will grow. Specialty DRAM roadmaps should include provision for the introduction of new standard technologies, such as LPDDR4/4x, to provide increased bandwidth.
Backed by comprehensive documentation and proven, high-quality in-house fabrication, Winbond’s DRAM and Flash memory products provide OEMs with the confidence to build new, more intelligent capabilities into home monitoring and security vision systems.
By Jacky Tseng, DRAM Marketing Manager, Winbond Electronics Corporation
A Department who works at the DRAM Marketing for Winbond Electronics Corporation. Mr. Tseng holds two Master degrees in Photonics of Feng Chu University and EMBA of National Taiwan University of Science and Technology. In 2011, he joined Winbond and sequentially served as Assistant Technology Manager, Technology Manager and Manager, and was responsible for LPDRAM new product planning and promotion for DRAM product lineup. Prior to joining Winbond, he served as Senior Engineer and PM in Hon-Hai, He has over 10 years of experience in memory related field.