This article refers to the address: http://
This paper proposes a practical digital wireless real-time video communication and shooting system. The system combines high speed and low power consumption, and can transmit real-time QQVGA video signals and 1.3M pixel image data in the 2.4 GHz ISM band. It has good scalability and broad application prospects.
Key words real-time QQVGA shooting system wireless video nRF24L01 DSP
Introduction <br> Currently, wireless RF chips are mostly used to transmit control signals or still images. The data throughput is not large, and under normal circumstances, the characteristics of the high data transmission rate of the radio frequency chip are not fully utilized. With the advancement of semiconductor manufacturing technology, there has been an RF chip with a Mbps-class air data transmission rate and low power consumption, enabling continuous transmission of multiple images into a town. In addition, wireless communication technology is developing rapidly, people are no longer satisfied with the wireless transmission of audio and images, and the requirements for wireless video are beginning to appear. Therefore, the video system using high-speed radio frequency chip will be a hot spot of digital wireless communication in the future.
For digital video (actually containing multiple images per second), various display standards have been developed internationally, such as QCIF (176×144), QQVGA (160×120), QVGA (320×240), CTF (352). ×288), VGA (640×480), SXGA (1 280×1 024), and the like. At present, wireless transmission of high-resolution video is technically unrealistic and unnecessary, because high resolution means a large area of ​​display material, usually only required to achieve a certain display area in a portable wireless communication device, so Low-resolution QCIF and QQVGA are the most popular among camera phones, digital cameras, PDAs and Other devices.
The wireless system is a video communication system based on a high-rate radio frequency chip, and has an instant shooting function; in video (continuous) mode, the video data is transmitted by using a resolution of QQVGA, which is 160×120 (@13fps), which can basically reach Live video of the day. In practical applications, the user can view the scene near the transmitting end (far away) in the LCD of the receiving end. When you see the scene of interest, press the button for a while to get an image of 1.3M pixels, which is convenient and practical. In order to simplify the design, the system only uses 8-bit color depth and RGB data format, and does not use CCD camera chip, nor does it use FPGA chip for logic control, which saves cost.
The following focuses on the wireless video communication system with TMS320VC5402 DSP as the control core. It describes the interface design of DSP and camera chip, DSP and RF chip in detail, analyzes the key points in the design, and finally gives some DSP assembly code.
1 The composition and working principle of wireless real-time video system
1.1 Introduction to OV9640 Camera Chip
   OV9640 is a high performance CM0S image sensor chip introduced by OmniVision Corporation of the United States. Supports 1.3 megapixel image capture and multiple resolutions, including l280×960, VGA, QQVGA, CIF, QCIF, etc. and various data output formats such as Raw RGB, YUV (4:2:2), YcbCr (4: 2:2), etc.; support 8-bit or 16-bit data output; program it through SC-CB interface, can realize various basic functions of image processing, such as exposure control, white balance, color saturation, gamma control, etc.; Low voltage requirements for embedded mobile devices.
1.2 nRF24L01 RF chip introduction nRF24L01 is Norwegian Nordic's single-chip wireless GFSK transceiver chip, working in the 2.4-2.5 GHz ISM band, the wireless transmission rate is up to 2Mbps, and the MCU uses SPI interface for control and data transmission. Compared to its predecessor, the nRF2401, the nRF24LOl offers superior performance and lower power consumption. It can support up to 6 data channels, and each channel supports Enhanced ShockBurst (ESB) technology, with automatic bad (AACK) and automatic retransmission (ART) functions, reducing the burden on the MCU and reducing the loss of wireless data. The packet rate improves the efficiency of two-way transmission. When the ESB is turned on, the nRF24L0l will automatically switch to the receiving mode after waiting for the packet to wait for the response from the other party. Automatic retransmission will be implemented according to the setting of the register.
1.3 System hardware circuit The hardware circuit consists of two parts: the transmit (acquisition) end and the receive (storage display) end. The high-speed RF chip realizes the wireless link in the 2.4 GHz band. The transmitter uses TMS320VC5402 DSP as the control core, OV9610 camera chip as the video (or image) acquisition front end, AT29LV1024 Flash ROM as the DSP bootloader memory chip, K4S161622H IMB capacity SDRAM as the program running space and video data buffer, the video data finally passes The RF chip nRF24LO1 is transmitted; the hardware structure of the receiving end is basically the same as that of the transmitting end, and the front end OV9640 is changed to the LCD display of the back end. The overall structural block diagram of the entire system is shown in Figure 1.
1.4 System working principle and process
1.4.1 The sender works in the video stream mode. The sender uses the DSP as the core control chip. The DSP is powered on and initialized. The code in the Flash ROM is loaded into the SDRAM through the BootLoader to realize high-speed operation of the system to speed up the processing speed of the data, and the HPI interface is set to general-purpose I/O. Then, the nRF24L0l is set to the transmission mode through the McBSPO buffer serial port, and the data packet containing the predetermined address is sent out to occupy the detection receiving end, and the nRF24L01 automatically switches to the mode of waiting for the response signal. If there is a correct receiving end (address matching), the nRF24L01 notifies the DSP through the INTO interrupt, causes the DSP to re-set the nRF24L01 to the transmitting mode, and immediately initializes the OV9640, implements the SCCB bus through the McBSP1 buffered serial port, starts the camera and sets For continuous frame mode. The resolution at this time is standard QQVGA, which is 160×120 (@8bit). Finally, the DSP converts the 8-bit parallel data obtained from D[7:O] into a serial format, and sends it to nRF24L01 through SDRAM buffer and McBSP0. Send video data out. If the correct receiver is not detected (no INTO interrupt occurs), the DSP will wait for INTO to occur or until the user turns off the power.
1.4.2 Transmitter works in shooting mode During video streaming, nRF24LOl can simultaneously monitor the air signal and automatically answer. If you receive a photo notification from the receiving end (press the button), set the OV9640 to the standard shooting mode with a resolution of 1280×960 (@8bit). Then, the DSP sets the nRF24L01 to the transmission mode and transmits the frame data at this time. After the image data transmission is completed and the acknowledgment signal is received, the system will return to the video stream mode. If the reception is unsuccessful, the automatic retransmission function of nRF24LO1 will ensure the integrity of the data transmission.
1.4.3 Workflow at the receiving end The power-on initialization of the receiving end is basically the same as that of the transmitting end, but nRF2dL01 (according to the predetermined address) is set to the receiving mode to receive the detection signal. After the matching address is detected, the auto-answer function of the nRF24L0l sends an acknowledgement signal to the sender to confirm the receipt of the signal. At this time, the two handshaking is successful. Next, the DSP is notified by the INTO interrupt so that the DSP re-sets the nRF24LO1 to the receive mode to receive the continuous video stream from the sender, and turns on the LCD module to prepare to display the video. Finally, the DSP buffers the video stream through SDRAM and sends it to the LCD display (if the other back-end modules such as LCD are parallel interfaces, the data needs to be converted into parallel data format). At this point, the system has been able to achieve real-time video data wireless transmission, the resolution of the real-time video stream is QQVGAl60 × 120 (@13fps).
During the display of the video stream, if the user presses the camera button, an INTl interrupt notification DSP is generated, and the DSP sets the nRF24LO1 to the transmission mode and transmits a photo notification signal. After receiving the response signal, the nRP24L0l returns to the receiving mode to receive the image data. After the receiving is completed, an acknowledgment signal is automatically sent to indicate that the image data is successfully received. Finally, the DSP passes the image data to the backend module for processing. This is the end. The system realizes the wireless shooting function of the image, and the photographing effect is 1280×960 (@1.3 million pixels), which basically satisfies the photographing requirements.
The workflow of the receiving and transmitting ends of the video system is shown in Figure 2.
2 DSP interface design
2.1 DSP and OV9640 interface design The chip is controlled by OmniVision's self-developed SCCB bus, using a three-wire connection. SCCB_E is the serial port enable/disable signal line, and SIO_C and SIO_D are the serial port clock line and data line respectively. Only the OV9640 is controlled in the system without knowing its status. It is connected by the DSP's McBSPO port. HD3 is used to open the serial port, and BCLKX0 and HDX0 are used to send the clock signal and control data respectively. The specific hardware connection is shown in Figure 3.
It is worth noting that:
1 When using the DSP's HPI interface to connect to the OV9640, in order to obtain signals such as field frequency, horizontal frequency and 8-bit video output, the HPIENA pin of the DSP must be pulled down so that HD[7:0] can be configured as a general-purpose I/O interface. Realize the start-stop or working mode control of the OV9640.
2 Set the output size to 1280×960 or QQVGA by modifying the register COMC of the OV9640.
3 Modify the register COMH, set the output format to 8-bit RGB format, and set it to Master mode.
4FREX is the enable/disable signal for continuous frame mode, but is disabled by default. The HD4 pin of the DSP should be configured to pull its level high, then use the HD3 to set the EXPTSB pin high for continuous frame data output; set FREX low to return to (single frame) shooting mode.
5 Calculate the data transmission rate. The chip outputs parallel 8-bit image signals and field-frequency and horizontal-frequency sync signals. The size of the l-frame image (160×120) in the video is:
160 × 120 × 8 = 153,600 bits / frame If the maximum transmission speed of nRF24L01 is 2 Mbps, 2000000/153600 ≈ 13 frames per second can be transmitted. The system can transmit video signals at 160 × 120 (@13 fps).
6 parallel conversion of output data.
The OV9640 outputs 8-bit parallel data, while the nRF24L01 is a serial interface, requiring parallel data to be serialized. The 8-bit parallel data format is HREF, B11, G2l, B22, G12, ..., HREF, G21, R22, G23, R24. By programming the timing of the transmitter and receiver, the PCLK, HREF, and VSYNC signals obtained from HD[2:0] can be omitted without being counted in the wireless transmission. Therefore, when serializing, the 8-bit data read in every time from D[7:O] is put into the SDRAM buffer in order from high to low, and then transmitted to nRF24LO1 for transmission.
2.2 DSP and nRF24L01 interface design
2.2.1 Design Points The DSP uses McBSPl to connect directly to the chip. The CSN is the SPI chip select pin, which is active low. It is connected to the CSN pin by the XF pin of the DSP. The CE is the transceiver mode select pin, and the HD7 is used for high and low level control. The wiring of other pins is shown in Figure 3. Need to pay attention when programming:
1 Before sending an instruction to the nRF24L01 through the SPI, the CSN must be made to have a high-to-low transition, that is, after each instruction is executed, the CSN must be set high to continue transmitting the next instruction.
The SPI of 2nRF24L01 latches data for the falling edge, so McBSP1 should be configured as "delayed edge with delay".
The 3IRQ pin is active low, and each time an interrupt generated to the DSP must be written with an "l" to clear.
4 If the sender needs to receive the response, the data channel O should be configured to receive the response signal, and the receiving address (RX_ADDR_PO) should be consistent with the sending address (TX_ADDR).
5 chips must go through Standby mode to enter TX or RX mode, so when switching between TX and RX modes, CE should be pulled low to enter Standby mode.
6 Write register instructions can only be executed in Powerdown or Standby mode, so CE should be pulled low before modifying register values.
The AACK and ART functions are not enabled in 2.2.2 and 2.2.3 below 7. Because the system runs in continuous video streaming mode, only high data transmission rate is required to meet real-time performance, and error correction retransmission is not required. However, in the shooting mode, these two functions should be turned on to ensure the integrity of the image data.
2.2.2 ESB sends data
   1 Set the configuration bit PRIM_RX low;
2 keep CSN low, send the address (TX_ADDR) and data (TX_PLD) to the receiving end;
3 Set CE to high and enable data transmission;
4 After the data is sent, the TX_DS interrupt is generated.
5CE is set low to enter Standby mode.
2.2.3 ESB receiving data
   1 configuration bit PRIM_RX is set high, CE is set high, then 130μs, nRF24L01 starts to monitor the air signal;
2 RX_DR generates an interrupt after receiving a valid data packet;
The RX_P_NO in the 3 status register records the received data channel;
4CE is set low to enter Standby mode;
The 5MCU gets the data through the SPI.
2.2.4 Part of the program example
(1) Write nRF24L01 register
(2) Read the nRF24L01 register
(3) Initialize nRF24L01 (shown down)
   The register access instruction is defined in (1) and (2), and the chip can be initialized with a "write" instruction.
(4) Sending data Assume that the data width has been set to 32 bytes in (3), then W_TX_PAYLOAD. Macro tx_start_byte
(5) Receiving data Also assume that the data width has been set to 32 bytes in (3), then R_RX_PAYLOAD. Macro rx_start_byte
Conclusion The DSP-based wireless communication system uses high-speed and low-power radio frequency chip nRF24L01, high-performance TMS320VC5402 digital signal processing chip and multi-function camera chip OV9640 in hardware to enable powerful video communication. And optimize the design at the software level to reduce latency and reduce power consumption. The scalability of the system is very strong. For example, the audio codec chip can be added to realize the synchronous transmission of wireless audio and video; the video compression chip can be added to increase the frame rate; and two (or more) wireless transceivers are used at the same time. The chip can achieve full duplex, which can improve the air data transmission rate; use high-resolution camera chip to obtain better video and image effects, but it also improves the difficulty of hardware connection and software design.
In short, wireless video and wireless image communication are the key development directions in the future wireless field. This design realizes real-time QQVGA video transmission at a certain frame rate, and has certain reference value for the design of similar systems in the future.
Jingkesai Electric Co., Ltd. , http://www.hobaoelec.com