Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.


  1. Advanced Patent Search
Publication numberCN101849416 B
Publication typeGrant
Application numberCN 200880112669
PCT numberPCT/IB2008/001155
Publication date24 Jul 2013
Filing date22 Jan 2008
Priority date14 Sep 2007
Also published asCA2699498A1, CN101849416A, EP2193660A2, US20110038408, WO2009034424A2, WO2009034424A3
Publication number200880112669.2, CN 101849416 B, CN 101849416B, CN 200880112669, CN-B-101849416, CN101849416 B, CN101849416B, CN200880112669, CN200880112669.2, PCT/2008/1155, PCT/IB/2008/001155, PCT/IB/2008/01155, PCT/IB/8/001155, PCT/IB/8/01155, PCT/IB2008/001155, PCT/IB2008/01155, PCT/IB2008001155, PCT/IB200801155, PCT/IB8/001155, PCT/IB8/01155, PCT/IB8001155, PCT/IB801155
Export CitationBiBTeX, EndNote, RefMan
External Links: SIPO, Espacenet
Method and system for processing of images
CN 101849416 B
Abstract  translated from Chinese
可从多个不同来源获得多个图像流。 A plurality of image streams can be obtained from many different sources. 首先缩减图像的颜色深度,随后组合流以形成具有已知格式并且位深度与缩减的位流的位深度之和相同的单一流。 First, reduced color depth of the image, then combined to form a stream having a known format and bit depth of the depth of cut of the same bit stream and single stream. 从而,可作为单一流处理多个流。 Thus, can be used as a single class to handle multiple streams. 在处理之后,通过应用相反的重新排序过程,再次分离所述多个流。 After processing, by applying the reverse reordering procedure, again separating the plurality of streams.
Claims(36)  translated from Chinese
1.一种处理表示按帧排列的像素的图像数据的方法,包括: 处理两个或者更多个图像数据流,以降低表示所述像素的数据的位深度,从而产生位深度缩减的流; 将位深度缩减的流组合成单一流,所述单一流具有等于或大于所述位深度缩减的流的位深度之和的位深度,其中将位深度缩减的流组合成单一流包括组合构成来自位深度缩减的流中的数据帧的像素的位,从而形成单一流中的单一数据帧的像素的位; 以已知格式传送所述单一流;和将所述单一流转换回两个或更多个图像数据流。 1. A method of processing image data representing the pixels are arranged in frames, comprising: two or more processing image data stream, in order to reduce the bit depth of the pixel representation data, resulting in reduced flow bit depth; The bit depth reduced streams into a single stream, the single-bit depth equal to or greater than the leading bit and the bit depth of the depth of the reduced flow, which will reduce the bit depth of streams into a single stream comprising a combination thereof from bit pixel data bit depth reduced stream of frames, so as to form a single stream in a single bit of data frames of pixels; in a known format for transmitting said single stream; and said two or more single stream back a plurality of image data streams.
2.按照权利要求1所述的方法,其中将位深度缩减的流组合成单一流包括通过位的级联,组合构成来自缩减位流中的数据帧的像素的位,从而形成单一流中的单一数据帧的像素的位。 2. The method according to claim 1 or claim 2, wherein the combination of bit depth reduction of flow into a single stream comprising by-bit cascade, combinations thereof to reduce the data bits from the bit stream of a frame of pixels, to form a single stream of single bit pixel data frame.
3.按照权利要求1或2所述的方法,其中按照控制机制组合所述流。 3. The method according to claim 1 or claim 2, wherein the composition according to the flow control mechanisms.
4.按照权利要求3所述的方法,其中所述控制机制包括关于来自缩减位流的位进入所述单一流中的何处的指令。 4. A method according to claim 3, wherein, wherein said control mechanism comprises instructions on where to cut bits from the bit stream into the single stream of.
5.按照权利要求3所述的方法,其中所述控制机制是调色板组合图。 5. A method according to claim 3, wherein, wherein said control mechanism is a combination of FIG palette.
6.按照权利要求3所述的方法,其中所述控制机制包括加密密钥。 6. The method according to claim 3, wherein, wherein said control mechanism comprises an encryption key.
7.按照权利要求3所述的方法,其中所述控制机制是查找表。 7. A method according to claim 3, wherein, wherein said control mechanism is a lookup table.
8.按照权利要求3所述的方法,其中所述控制机制包括与待处理的图像帧的数目、待组合的图像数据流的数目、和图像像素`在它们之中的位置有关的信息。 8. A method according to claim 3, wherein, wherein said control mechanism comprises a number of image frames to be processed, information about the number of images to be combined data stream, and image pixels' position among them.
9.按照权利要求3所述的方法,还包括将位深度缩减的流至少部分地转换回它们的原始位深度。 9. A method according to claim 3, wherein, further comprising reducing the bit depth of the flow at least partially converted back to their original bit depth.
10.按照权利要求5所述的方法,其中作为包括调色板组合图的数据文件来处理所述单一流。 10. A method according to claim 5, wherein the combination of the color palette as including graphic data file to process the single stream.
11.按照权利要求3所述的方法,其中所述位深度之和是由所需硬件支持的标准位深度。 11. A method according to claim 3, wherein, wherein the bit depth of the bit depth and is a standard supported by the required hardware.
12.按照权利要求3所述的方法,其中所述位深度之和为24位或32位。 12. A method according to claim 3, wherein, wherein the sum of the bit depth of 24-bit or 32-bit.
13.按照权利要求3所述的方法,其中组合位深度缩减的流中彼此对应的图像帧。 13. A method according to claim 3, wherein, wherein the combination of bit depth reduction stream corresponding to each other image frame.
14.按照权利要求13所述的方法,其中所述图像帧的对应在于像素是在相同的时刻或者按已知时差捕捉的。 14. A method according to claim 13, wherein the corresponding pixel of the image frame that is at the same time or in a known time difference captured.
15.按照权利要求3所述的方法,其中待处理的流捕获自多于一个的图像源。 15. A method according to claim 3, wherein, wherein the capture stream to be treated since more than one image source.
16.按照权利要求3所述的方法,其中按相同格式捕获图像数据流。 16. A method according to claim 3, wherein, wherein the captured image data stream in the same format.
17.按照权利要求3所述的方法,其中以不同格式捕获图像数据流。 17. A method according to claim 3, wherein, wherein the captured image data in a different format stream.
18.按照权利要求3所述的方法,还包括使用填充位形成位深度等于或大于位深度缩减的流的位深度之和的单一流。 18. A method according to claim 3, wherein, further comprising the use of stuffing bits forming a bit depth equal to or greater than the bit depth of the bit depth and the reduced flow of a single stream.
19.按照权利要求3所述的方法,其中所述单一流的图像格式等于输入流的最大图像格式。 19. The method according to claim 3, wherein, wherein the single-class image format image format is equal to the maximum input stream.
20.按照权利要求3所述的方法,其中可按照第二格式处理第一格式的单一流。 20. A method according to claim 3 or claim, wherein a single stream of the first format can be processed according to the second format.
21.按照权利要求3所述的方法,其中所述图像是视频图像。 21. A method according to claim 3 or claim, wherein the image is a video image.
22.按照权利要求3所述的方法,其中图像数据被实时处理。 22. A method according to claim 3, wherein, wherein the image data is processed in real time.
23.一种处理表示按帧排列的像素的图像数据的系统,包括: 用于处理两个或者更多个图像数据流,以降低表示所述像素的数据的位深度,从而产生位深度缩减的流的装置; 用于将位深度缩减的流组合成单一流的装置,所述单一流具有等于或大于所述位深度缩减的流的位深度之和的位深度,其中用于将位深度缩减的流组合成单一流的装置包括组合构成来自位深度缩减的流中的数据帧的像素的位,从而形成单一流中的单一数据帧的像素的位的装置; 用于以已知格式传送所述单一流的装置;和用于将所述单一流转换回两个或更多个图像数据流的装置。 23. A method of processing image data representing a system of pixels arranged in frames, comprising: means for processing the two or more image data stream, in order to reduce the bit depth of the pixel representation data, thereby generating the reduced bit depth stream means; means for reducing the bit depth of streams into a single stream apparatus, wherein said single stream is used to reduce the bit having a bit depth equal to or greater than the depth of the bit depth and the bit depth reduced stream, combined into a single stream flow apparatus comprises a combination of bits constituting the pixel bit depth reduction from the data stream of the frame, so as to form a single stream of bits means a single frame of pixel data; means for transmitting the format known referred to a single class of device; and for the single stream back two or more image data stream devices.
24.按照权利要求23所述的系统,其中用于将位深度缩减的流组合成单一流的装置包括用于通过位的级联,组合构成来自缩减位流中的数据帧的像素的位,从而形成单一流中的单一数据帧的像素的位的装置。 24. The system of claim 23, wherein the combination of bit depth for the reduced stream into a single stream by means comprises means for bit cascade, combinations thereof to reduce the data bits from the bit stream of pixels in a frame, thereby forming bit means a single stream single data frame pixel.
25.按照权利要求23所述的系统,具有用于提供关于来自缩减位流的位进入所述单一流中何处的控制机制的装置。 25. The system of claim 23, wherein, having regard for providing reduced bit stream from entering the control mechanism where a single stream apparatus.
26.按照权利要求25所述的系统,其中所述控制机制是调色板组合图。 26. The system of claim 25, wherein said control mechanism is a combination of the palette FIG.
27.按照权利要求25所述的系统,其中所述控制机制包括加密密钥。 27. The system of claim 25, wherein said control mechanism comprises an encryption key.
28.按照权利要求25所述的系统,其中所述控制机制是查找表。 28. The system of claim 25, wherein said control mechanism is a lookup table.
29.按照权利要求25-28中任一项所述的系统,其中所述控制机制包括与待处理的图像帧的数目、待组合的图像数据流的数目、和图像像素在它们之中的位置有关的信息。 29. The system of claim any one of claims 25-28, wherein the control mechanism comprises a number of image frames to be processed, the number of image data to be combined stream, and the image pixel position among them related information.
30.按照权利要求25所述的系统,其中所述位深度之和是由所需硬件支持的标准位深度。 30. The system of claim 25, wherein the bit depth of the bit depth and is a standard supported by the required hardware.
31.按照权利要求25所述的系统,其中所述位深度之和为24位或32位。 31. The system of claim 25, wherein the sum of the bit depth of 24-bit or 32-bit.
32.按照权利要求25所述的系统,其中组合位深度缩减的流中彼此对应的图像帧。 32. The system of claim 25, wherein the combination of bit depth reduction stream of image frames correspond to each other.
33.按照权利要求32所述的系统,其中所述图像帧的对应在于像素是在相同的时刻或者按已知时差捕捉的。 33. The system of claim 32, wherein the corresponding pixel of the image frame that is at the same time or in a known time difference captured.
34.按照权利要求25所述的系统,还包括用于使用填充位形成位深度等于或大于位深度缩减的流的位深度之和的单一流的装置。 34. The system of claim 25, further comprising means for the use of padding bits is equal to or greater than the bit depth of the formation of a single stream of bit depth of the apparatus and the reduction of the flow of bit depth.
35.一种处理表示按帧排列的像素的图像数据以便传输的编码器,包括: 用于处理两个或者更多个图像数据流,以降低表示所述像素的数据的位深度,从而产生位深度缩减的流的装置; 用于将所述位深度缩减的流组合成单一流的装置,所述单一流具有等于或大于所述位深度缩减的流的位深度之和的位深度,其中用于将位深度缩减的流组合成单一流的装置包括组合构成来自位深度缩减的流中的数据帧的像素的位,从而形成单一流中的单一数据帧的像素的位的装置; 用于以已知格式传送所述单一流以便传输的装置。 35. A method of processing image data representing the pixels are arranged in frames for transmission in the encoder comprising: means for processing image data of two or more streams, to reduce the bit depth representation of the pixel data to produce bit The depth of the reduced flow means; means for combining said stream of reduced bit depth into a single stream of apparatus, the single stream having a bit depth equal to or greater than the bit-depth reduction of the depth of the stream of bits and wherein with The bit depth in the reduced streams into a single stream of bits constituting the apparatus comprises a combination of a pixel bit depth reduction from the data stream of the frame, so as to form a single stream of bits means a single frame of pixel data; for to Known transfer format for transmission of the single-class devices.
36.一种用于与权利要求35的编码器一起使用的解码器,包含用于将权利要求35的编码器生成的所述单一流转换回权利要求35的编码器所处理的所述两个或更多个图像数据流的装置。 The encoder 35 36. A decoder for use in conjunction with claim 35 comprising an encoder for generating a single stream back to claim 35, the encoder processing Claim two means one or more image data stream.
Description  translated from Chinese

图像处理方法和系统 Image processing method and system

技术领域 FIELD

[0001] 本发明涉及图像的处理,更具体地说,涉及多个图像数据流的处理。 [0001] The present invention relates to image processing and, more particularly, relates to processing a plurality of image data streams.


[0002] 在许多应用中,捕捉多个图像,并且在观看之前,需要处理这多个图像,比如压缩、传送和保存。 [0002] In many applications, a plurality of images captured, and prior to viewing, this requires a plurality of image processing, such as compression, transfer and preservation.

[0003] 例如,为了监视生产线,摄像系统可包括各自产生图像流的多个摄像机。 [0003] For example, in order to monitor the production line, the imaging system may comprise a plurality of cameras each generate image stream. 另外,在许多360视频应用中,摄像机可包括各自产生图像流的两个鱼眼镜头和/或一个变焦镜头。 Additionally, in many video applications 360, the camera may comprise each generate two fisheye and / or a zoom image stream. 鱼眼镜头具有广角视场,存在许多变体。 Fisheye lens has a wide angle field of view, there are many variants. 典型的鱼眼镜头可用180半球全圆形成图像。 A typical 180 fisheye available hemisphere full circle to form an image. 从而,两个鱼眼镜头可背靠背地布置,以捕捉整个环境。 Thus, the two fisheye lens can be arranged back to back, in order to capture the entire environment. 变焦镜头可放大环境的所选区域,以更详细地显示所选区域。 Zoom lens to zoom environment selected area to selected area shown in more detail.

[0004] 从而可以产生多个图像数据流,这些图像数据流可以是相同或不同的格式。 [0004] thereby generating a plurality of image data streams, the image data streams may be the same or different formats. 例如,变焦镜头捕捉的图像可以是高清晰格式。 For example, the zoom lens to capture the image may be a high-definition format. HD分辨率视频的特征在于其宽幅(通常16: 9长宽比)及其高图像清晰度(与通常的帧大小为720X576像素的标准视频清晰度(SD)格式相比,通常的帧大小为1920 X 1080像素和1280 X 720像素)。 HD resolution video is characterized by its width (typically 16: 9 aspect ratio) and high image clarity (compared to the normal frame size of 720X576 pixels of the standard video definition (SD) format, the normal frame size for 1920 X 1080 pixels and 1280 X 720 pixels). 相反,由安装在适当摄像机上的鱼眼镜头捕捉的图像可以是超高清晰度(XHD)分辨率图像。 On the contrary, by a fisheye lens mounted on the camera to capture the appropriate images can be high-definition (XHD) resolution images. 超高清晰度(XHD)格式获得与高清晰度(HD)格式视`频相比尺寸更大的图像。 High-definition (XHD) format obtained with high definition (HD) format, as the `greater frequency compared to the size of the image. 在许多应用中这是合乎需要的,因为它提高了用户数字放大环境的能力。 In many applications it is desirable, because it increases the ability of the user digital zoom setting.

[0005] 每个图像通常具有受计算机和处理硬件支持的颜色深度。 [0005] Each image has a color depth is usually processed by computers and hardware support. 颜色深度描述用于表现位映射(bitmapped)图像或视频帧缓冲区中的单一像素的颜色的二进制位的数目,有时被称为每像素位数。 Color depth is described for representing bit-mapped (bitmapped) number of bits in an image or video frame buffer in a single pixel color, sometimes referred to as the number of bits per pixel. 较高的颜色深度呈现出范围更宽的不同颜色。 Exhibits higher color depth wider range of different colors.

[0006] 真彩色具有一千六百七十万种不同的颜色,并且模拟在真实世界中见到的许多颜色。 [0006] True Color has 16.7 million kinds of different colors, and simulate real-world saw many colors. 所产生颜色的范围接近人眼能够区分大多数摄影图像的颜色的水平。 Range of colors generated close to the human eye can distinguish the level of most photographic color image. 不过,当图像被处理,或者是黑白图像(它将真彩色局限于256灰度级)或者“纯粹”生成图像时,会出现 However, when the image is processed, or black and white images (which will be limited to 256 true color gray level) or "pure" generates an image, there will be

一些局限性。 Some limitations.

[0007] 通常,在目前的标准中,以24或32位颜色深度捕捉图像。 [0007] Typically, in the current standard, a 24 or 32-bit color depth image capture.

[0008] 24位真彩色使用8位来表示红色,8位来表示蓝色,和8位来表示绿色。 [0008] 24 true color using 8 to represent the red, 8 to represent the blue, and 8 to represent the green. 对这三种颜色中的每一种来说,这产生256种色度(shade)。 These three colors each, it produces 256 kinds of color (shade). 于是,可组合这些色度,从而形成总共16777216 种混合颜色(256X256X256)。 So, you can combine these chroma, creating a total of 16,777,216 kinds of mixed colors (256X256X256).

[0009] 32位颜色包含24位颜色和额外的8位,这额外的8位或者作为空的填充空间,或者表示α通道。 [0009] 32 color contains 24 color and additional 8, this additional eight or as filling empty space, or express α channel. 许多计算机在内部以32位为单位处理数据。 Many computer internally to 32 units processing data. 于是,使用32位颜色深度是合乎需要的,因为它可以优化速度。 Thus, using the 32-bit color depth is desirable, because it can optimize speed. 不过,这是以增大安装的视频存储器为代价的。 However, this is a video memory for increased installation cost.

[0010] HD或XHD流都具有已知的数字数据格式。 [0010] HD or XHD streams having known digital data format. 由标准位数(已知的颜色深度)表现的像素构成I和O的位流。 By the standard number of bits (color depth known) performance I and O of pixels constituting the bit stream. 在顺序扫描图像行的情况下,可使用逐行扫描,或者例如,在首先扫描奇数行,随后扫描偶数行的情况下,可使用隔行扫描。 In the case of line sequential scanning image, progressive scan may be used, or, for example, in the odd-numbered lines are scanned first, and then scanning the case of even-numbered lines, interlaced scanning can be used. 通常,每一行的扫描是从左到右。 Typically, each line is scanned from left to right. 通常存在至少一个报头,所述报头由指示在其之后的位流的有关信息的I和O构成。 Typically there is at least one header, the header indicator bit stream by subsequent thereto, information about the I and O constitute. 各种数字数据流格式,包括不同数目的报头都是可能的,并且为本领域的技术人员所知。 Various digital data stream format including a different number of headers are possible, and known to those of skill in the art. 为避免产生疑问,已知的数据格式是任何图像格式(例如,HD或XHD)的任何已知数字格式。 For the avoidance of doubt, a known data format is any image format (e.g., HD or XHD) any known digital format.

[0011] 图像数据流通常是MPEG 2和4兼容的。 [0011] image data streams are usually compatible with MPEG 2 and 4.

[0012] MPEG-2是由运动图像专家组为数字视频定义的标准。 [0012] MPEG-2 is a Moving Picture Experts Group standard definition digital video. 它规定封闭的视频位流的语法。 It provides syntax enclosed video bit stream. 另外,它规定用于对应视频流的后续编码和压缩的语义和方法。 In addition, it provides a method of semantic and follow the corresponding video stream encoded and compressed for. 不过,实现实际编码处理的方式由编码器设计决定。 However, the way to achieve the actual encoding process is determined by the encoder design. 于是,有利的是,所有MPEG-2兼容设备都是能共同使用的。 Thus, it is advantageous that all MPEG-2 compliant devices are able to be used together. 目前,MPEG-2标准被广泛接受。 Currently, MPEG-2 standard is widely accepted.

[0013] MPEG-2允许对从有限清晰度到全HDTV范围内的四种源格式,或者说“级别”编码,每一种源格式都具有多种位速率。 [0013] MPEG-2 allows for the limited resolution from the source format to four within the scope of the full HDTV, or "level" code, each source format are multi-bit rate. 另外,MPEG-2允许不同的“简档(profile)”。 In addition, MPEG-2 allows different "profiles (profile)". 每种简档提供一起构成编码系统的一批压缩工具。 Each profile provides coding system together constitute a group of compression tools. 不同的简档意味着可得到不同的一组压缩工具。 Different profiles available means a different set of compression tools.

[0014] 包含H.264压缩方案的MPEG-4标准处理覆盖低位速率和高位速率的较高压缩比。 [0014] contain H.264 MPEG-4 compression standard treatment program covering low rate and high rate of high compression ratio. 它兼容MPEG-2流,并开始成为未来的主流标准。 It is compatible with MPEG-2 stream and started to become mainstream in the future standards.

[0015] 存在许多符合规定的记录格式。 [0015] There are many compliance record format. 例如,HDV是常用的产生HD视频的记录格式。 For example, HDV is a common way to generate HD video recording format. 该格式与MPEG-2兼容,对流可以使用MPEG-2压缩。 This format is compatible with MPEG-2, convection can use MPEG-2 compression.

[0016] 来自MEPG-2视频编码器的输出被称为基本流(换句话说,数据或视频位流)。 [0016] The output from the MEPG-2 video encoder is called an elementary stream (in other words, data or video bitstream). 基本流只包含一种数据,并且是连续的。 An elementary stream contains only data, and is continuous. 在源结束之前,它们不会停止。 Before the end of the source, they will not stop. 基本流的准确格式将随编解码器或者流中携带的数据而变化。 The exact format of the elementary stream with the codec or the data carried in stream varies.

[0017] 连续的基本位流随后可被送入打包器,打包器把基本流分成具有一定数目字节的分组。 [0017] The substantially continuous bit stream may then be fed to the packer, the packer elementary stream into a packet having a certain number of bytes. 这些分组被称为打包基本流(PES)分组。 These packets are called Packetized Elementary Stream (PES) packet. PES通常包含来自单一编码器的仅一种有效载荷数据。 Typically contain only one PES payload data from a single encoder. 每个PES分组从包括唯一分组ID的分组报头开始。 Each PES packet includes a unique packet ID from the packet header starts. 报头数据还识别有效载荷的来源,以及排序和定时信息。 Header data also identify the source of the payload, and the sequencing and timing information.

[0018] 在MEPG标准内,建立于打包基本流之上的各种其它格式也是可能的。 [0018] In the MEPG standard, built on top of packetized elementary streams of various other formats are possible. 对于一些应用可以引入报头的分级结构。 For some applications may introduce a hierarchy header. 例如,位流可包括总序列头(overall sequence header)、一组图片报头、单个图片报头,和一部分的图片报头。 For example, the bit stream may include a total sequence header (overall sequence header), a group of pictures header, a single picture header, and a part of the picture header.

[0019] 在监视生产线的应用和许多360或其它视频应用中,理想的是同时查看在相同时刻拍摄的图像流。 [0019] In the production line 360 or application monitor and many other video applications, it is desirable to view an image stream captured at the same time. 这使用户能够查看真实的环境,例如显示生产线或者360图像,并且可选地显示指定时刻的放大的部分。 This enables the user to view the real-world environment, such as the display line, or 360 images, and optionally displaying an enlarged partial given moment. 另外,对许多应用来说,理想的是实时地查看图像流。 Further, for many applications, it is desirable to view images in real-time stream.

[0020] 我们已认识到理想的是以已知格式发送图像流数据,比如MPEG兼容格式的图像流,以致常用的MPEG兼容硬件可用于处理图像流。 [0020] We have recognized that the ideal is to transmit a known data format of the image stream, such as MPEG-compatible image stream format, so that the conventional MPEG compatible hardware can be used to process the image stream. 不过,我们还认识到在数据的传输和处理中,需要保持不同图像数据流之间的同步。 However, we also recognize that the transmission and processing of data, it is necessary to synchronize the data flow between the different images.


[0021] 在权利要求中限定了本发明。 [0021] The invention is defined in the claims.

[0022] 按照本发明,提供一种处理表示按帧排列的像素的图像数据的方法,包括:处理两个或者更多个图像数据流,以降低表示所述像素的数据的位深度,从而产生位深度缩减的流;将位深度缩减的流组合成单一流,所述单一流具有至少与所述位深度缩减的流的位深度之和相等的位深度;以已知格式传送所述单一流;和将所述单一流转换回两个或更多个图像数据流。 [0022] According to the present invention, there is provided a method of processing data representing pixels arranged in frames of image data, comprising: processing two or more image data stream, in order to reduce the bit depth representation of the pixel data, thereby generating Bit depth reduction of flow; reduce the bit depth of the stream are combined into a single class, a single stream bit-depth of at least the bit-depth reduction of flow and equal bit depth; known to transmit the single-class format ; and the single stream back into two or more image data stream.

[0023] 本发明的实施例的优点是能够同时处理,从而同时查看多个图像数据流。 [0023] advantages of embodiments of the present invention is capable of handling, thus simultaneously view a plurality of image data streams. 例如,如果通过通信链路分别传送两个流,一个流可能在结束时,来自一个流的数据在另一个流之前或之后到达,这从而会引起在显示器上同时显示数据方面的问题。 For example, if two streams are transmitted via the communications link, may be at the end of a stream, a stream of data from the stream before or after the arrival of another, which thereby causes the problem data is displayed on the display areas simultaneously. 通过组合两个或更多个图像数据流,并利用MPEG-2编码以诸如HD之类的格式将其呈现为单一流,本发明的实施例避免了该问题。 By combining two or more image streams, using MPEG-2 encoding format such as an HD or the like to be presented as a single stream, embodiments of the present invention avoid this problem. 可利用常规硬件传送和处理该单一流。 Using conventional hardware to transmit and process the single class. 由于数据被组合在一起形成单一流,因此保证了来自两个或更多个流的数据同步。 Since the data are grouped together to form a single stream, thereby ensuring the data from two or more streams of synchronization.

[0024] 因此,本发明的实施例的优点在于可保证在传输期间,表示两个或更多图像流的数据保持同步。 [0024] Accordingly, an advantage of embodiments of the present invention is that it can ensure that during transmission, the data indicate that two or more image streams synchronized. 这可保证来自一个源的各帧的像素按已知的时间差,或者在与来自另一个源的像素同时到达目的地。 This ensures that the pixels from a source of each frame in a known time difference, or from another source in the pixel while the destination. 例如,就捕捉时间来说,这些帧基本上对应,从而能够实现图像流的同时观看。 For example, the capture time, these frames substantially corresponds to the image-flow can be achieved simultaneously viewing. 对许多应用,包括监视生产线和最好实时地查看整个环境(例如,用多台摄像机捕捉)的各种360视频应用来说,这是有益的。 For many applications, including real-time monitoring of production lines and the best place to view the entire environment (for example, using multiple cameras to capture) a variety of 360 video applications, it is useful.

[0025] 本发明的附加好处在于通过缩减颜色深度,在数据的传输之前,带宽被降低。 [0025] Additional benefits of the present invention is that by reducing the color depth, the data prior to transmission, the bandwidth is reduced. 我们认识到缩减的颜色深度可足以满足许多应用,从而按照这种方式降低带宽是可接受的。 We recognize that a reduced color depth may be sufficient for many applications, so that in this way reduce the bandwidth is acceptable. 例如,对于从夜间摄像机拍摄的图像,只需要8位颜色深度(最大256色)。 For example, images from the cameras at night, and only 8-bit color depth (maximum 256 colors). 从而,把位深度从所捕捉的24位缩减到8位不会引起成问题的质量损失。 Thus, the bit depth of 24 from the captured reduced to 8 does not cause a problem of loss of quality.

[0026] 从而,图像数据流可被组合成已知格式的单一流。 [0026] Thereby, the image data stream may be combined into a single stream of a known format. 作为结果的流的长度不必比最长的输入流更长。 As a result of the length of the stream does not have to be longer than the longest input stream. 这有利于产生利用已知技术和硬件处理流,尤其是实时处理流的可能性。 This facilitates the use of known techniques and produce hardware processing flow, in particular the possibility of real-time processing flow. 与通过独立的通信链路传送多个流相比,仅处理一个流还简化传送流的硬件配置。 Compared by independent communication link to transmit multiple streams, handle only one stream also simplify the hardware configuration of the transport stream.

[0027] 尽管本发明有利于在希望同步的情况下,处理多个视频流,不过本发明也可用在希望作为单一流处理多个图像的各种各样的应用中。 [0027] While the present invention is beneficial in the case want to synchronize, processing a plurality of video streams, but may also be used in the present invention is desired as a single stream processing a plurality of images of a wide variety of applications.

[0028] 优选地,来自合并在一起的各个图像流的图像彼此对应,比如是同时从不同来源捕捉的。 [0028] Preferably, the image from the combined flow of each image corresponding to each other, for example, is simultaneously captured from different sources.

[0029] 通过使用加密密钥控制图像的合并和转换回,可使视频更安全。 [0029] By using the encryption key control images merge and convert it back, make video more secure. 作为替代地,可以使用查找表把合并的图像转换回它们的原始分离形式。 Alternatively, you can use a lookup table to convert the image merged back to their original isolated form.

附图说明 Brief Description

[0030] 下面参考附图,举例说明本发明的一个实施例,其中: [0030] Referring to the drawings, illustrate an embodiment of the present invention, wherein:

[0031] 图1是本发明的一个实施例的功能组件的示意概观; [0031] Figure 1 is a schematic functional components of an embodiment of the present invention, an overview;

[0032] 图2是实施例的编码器装置的示意图; [0032] FIG. 2 is a schematic view of an example of embodiment of the encoder apparatus;

[0033] 图3是本发明的一个实施例的可选第二级编码的示意图; [0033] FIG. 3 is a schematic view of an alternative embodiment of a second level encoded embodiment of the present invention;

[0034] 图4是图解说明该实施例的解码装置的示意图;以及 [0034] FIG. 4 is a schematic diagram illustrating an example of the decoding apparatus of the embodiment; and

[0035] 图5是图解说明缩减位流,并把缩减的位流组合成已知格式的单一位流的编码器过程的示意图。 [0035] FIG. 5 is a diagram of the reduced bit stream, and the combination of a reduced schematic view of the encoder to a known format, a single bit stream of the process stream.


[0036] 本发明的实施例使多个图像流可以被合并成单一图像流,并作为单一图像流被处理,随后被转换回分离的图像流。 [0036] Embodiments of the present invention enables a plurality of image streams can be combined into a single image stream, and the stream is processed as a single image, then the image is converted back to separate streams. 在下面的例子中,这些图像由作为视频源的分离图像源捕捉,不过这只是一个例子。 In the following example, the images from the video source as a source of separation image capture, but this is only one example.

[0037] 待描述的实施例具有三个分离的图像源,以及可选的第四图像源。 [0037] The embodiment to be described has three separate image source, and optionally a fourth image source. 所有视频源可以是实时的,或者是文件序列。 All video sources can be real-time, or the file sequence. 在本例中,图像源是监视生产线的摄像系统的一部分。 In the present embodiment, the image source is part of the monitoring camera system of the production line. 两台摄像机配有背对背布置以捕捉整个周围环境(360 )的超广角镜头,比如鱼眼镜头。 Back to back arrangement with two cameras to capture the entire surroundings (360 ) of the ultra wide-angle lens, such as fish-eye lens. 在本例中,这些摄像机捕捉超高清晰度(XHD)视频,为使用户能够有效地数字放大图像,XHD视频是所希望的。 In this case, these cameras capture high-definition (XHD) video, to enable users to efficiently digital zoom image, XHD video is desirable. 为了避免产生疑问,这里要指出的是XHD包含高于HD的任何清晰度。 For the avoidance of doubt, this should be noted that the XHD contain any resolution higher than HD. 这种情况下,对于每个图像帧,每个XHD源具有相同数目的像素,因为这两台摄像成像器是相同的,产生相同格式和长宽比的图像。 In this case, for each image frame, each XHD source having the same number of pixels, because these two imaging imager is identical, produce the same image format and aspect ratio.

[0038] 另外,存在配有变焦镜头的第三台摄像机,所述变焦镜头能够进一步放大环境。 [0038] In addition, the presence of a third with a zoom lens camera, the zoom lens can be further magnified environment. 在本例中,第三台摄像成像器产生高清晰度HD视频。 In this example, a third imaging imager producing a high definition HD video. 于是,和XHD图像帧一样,每个图像帧具有相同或不同数目的像素。 Thus, and XHD image frame, each image frame having the same or a different number of pixels. 所述摄像系统还可结合第四台HD摄像成像器。 The camera system also combines Radio 4 HD camera imager.

[0039] 应意识到该实施例并不局限于特定数目的视频源,下面说明的技术可以和图像源的许多其它组合一起使用。 [0039] will be appreciated that this embodiment is not limited to a specific number of video sources, techniques described below can be used with many other combinations of image sources. 特别地,尽管实施例尤其可用于处理不同图像格式的图像,不过并不局限于这样的图像。 In particular, although the embodiments are particularly useful for image processing different image format, but not limited to such images. 待处理的图像可以是相同的格式,或者各种不同的格式。 Image to be processed may be the same format, or a variety of different formats. 这些图像格式可以是标准格式或非标准格式。 These image formats can be standard or non-standard format format.

[0040] 图1表示具有三个图像源和可选的第四图像源1、2、3和4的实施本发明的装置的功能组件。 [0040] Figure 1 shows functional components of the three image sources, and an optional fourth embodiment of an image source 2, 3 and 4 of the present invention the apparatus. 捕捉的数据可以由处理器5处理,处理器5可分别配有内存6和/或存储容量 Data can be captured by the processor 5 processor, a processor with a memory 5, respectively 6 and / or storage capacity

7。 7. 图像流由装置8处理,装置8进行合并图像流的处理。 Image processing flow by means 8, merged image processing apparatus 8 streams. 处理器5、内存6、存储器7和装置8的功能组件可以在单一装置中实施。 Processor 5, memory 6, a memory 7 and functional components of apparatus 8 may be implemented in a single device. 在这样的安排中,图像源1、2、3可以是简单的图像捕捉装置,比如带有适当光学器件和驱动电子器件的CXD或CMOS传感器,处理器5、内存6和存储器7承担把原始图像数据变成图像流的处理。 In such an arrangement, the image source can be a simple 1,2,3 image capture device, such as the appropriate optics and drive electronics with the CXD or a CMOS sensor, a processor 5, memory 6 and the memory 7 to assume the original image processing the image data into the stream. 另一方面,图像源1、2、3本身可以是以XHD或HD格式产生图像流的图像摄像机,并且处理器5、内存6和存储器7要进行的处理成型较少。 On the other hand, the image source itself may be in the 1,2,3 or HD format XHD generate images from the camera image stream, and the processor 5, memory 6 and the memory 7 handle to be molded less.

[0041] 在编码器中,如图2中所示,存在来自具有24位颜色深度的三种视频流,即两个XHD视频流和一个HD视频流。 [0041] In the encoder, shown in Figure 2, there are three video streams having from 24-bit color depth, i.e. two XHD HD video stream and a video stream. 颜色深度缩减器12、13和14把每个图像流的颜色深度从24位减小到8-12位。 Color depth reducer 12, 13 and 14 for each color depth image stream is reduced from 24 to 8-12 bits. 即,每个像素现在用8-12位表示,可表现的颜色的数目减小。 That is, each pixel is now expressed by bits 8-12, the number of expressible colors is reduced. 例如,8位颜色深度提供的在任一时刻显示的颜色的最大数目为256。 For example, the maximum number of eight colors at any one time to provide the display of the color depth is 256.

[0042] 进行这种缩减的颜色深度缩减器在本领域中众所周知,例如使用取样和量化。 [0042] for such reduction of the color depth reduction is well known in the art, for example using sampling and quantization. 存在许多变体。 There are many variants. 例如,一种缩减颜色深度的简单技术包括把位组合在一起,以将数字0-65536表示成第一位,数字65536-131072由第二位表示,等等。 For example, a simple color depth reduction techniques include the combined bits to be represented as a first digital 0-65536, represented by the second digital 65536-131072, and the like.

[0043] 技术人员明白可能的缩减颜色位深度的技术有许多,比如通过用颜色查找表表示颜色,以致表现的颜色更少。 [0043] art will appreciate that the possible reduction in the color bit depth there are many techniques, such as indicated by the color of the color lookup table, so that the performance of color less. 这减小了色调的范围,不过在多数应用中不会造成问题。 This reduces the range of tones, but does not cause problems in most applications. 在用于传输的任何压缩技术之前,对原始像素数据进行缩减颜色位深度的处理。 Before any compression technique for transmission, the original color of the pixel data bit depth reduction processing.

[0044] 在本例中,每个图像流被缩减成相同的颜色深度。 [0044] In this example, each image stream is reduced to the same color depth. 不过,情况并非必需如此。 However, this is not necessary the case.

[0045] 8位或更大的颜色深度适合于/足以满足许多应用,包括监视生产线的摄像系统和许多360摄像应用。 [0045] 8 or greater color depth adapted / sufficient for many applications, including monitoring the production line 360 of the imaging system and many imaging applications. 应意识到颜色深度的其它缩减也适合于或足以满足各种其它应用。 Be appreciated that other color depth reduction is also suitable for a variety of other applications or sufficient.

[0046] 流合并器15把颜色深度减小的所述两个XHD视频流和所述HD视频流合并成总颜色深度为16-32位的单一视频流。 [0046] The two XHD video stream muxer 15 color depth decreases and the HD video streams are combined into a total color depth of a single video stream 16-32 bits. 在图2中,实现所述合并的处理器被称为XHD流合并器,因为这种情况下,合成的视频流的图像格式为XHD。 Image format video stream in Figure 2, to achieve the combined processor is called XHD muxer, because in this case, the synthesis of XHD. 合并的图像流具有已知的数字数据格式,颜色深度至少等于位深度缩减的图像流的位深度的总和。 The combined image stream having a known digital data format, color depth at least equal to the sum of the bit depth of the reduced bit depth image stream. 这种情况下,合并的图像流具有32位/像素的最大位深度。 In this case, the combined image stream has a maximum depth of 32 bits / pixel. 优选是标准的24或32位颜色深度。 Preferably a standard 24 or 32-bit color depth.

[0047] 合并图像流的许多组合都是可能的,在后面说明的图5中给出了一个例子。 [0047] The combined image stream many combinations are possible, described later in FIG. 5 shows an example. [0048] 在本例中,合并的图像流采用最大的输入流的格式大小-这种情况下是XHD。 [0048] In this example, the combined image stream using the format of the input stream of the largest size - in this case XHD. HD图像中的像素可被重新排列,以适合XHD图像格式。 HD pixel in the image can be rearranged to fit XHD image format. 为统一得到的流的颜色深度而需要的任何附加位可以是空填充空间。 Any additional bit streams unified color depth required may be obtained by filling empty space.

[0049] 为了得到组合流的希望的24位或32位颜色深度,可以合并均为8位的三个流(2XXHD和1XHD),以产生一个24位图像流。 [0049] In order to obtain the desired combination of stream 24 or 32-bit color depth, you can merge the three streams are eight (2XXHD and 1XHD), to produce a 24-bit image stream. 作为替代地,所述两个XHD流可具有12位并且所述HD流可具有8位,得到总共32位的颜色深度。 Alternatively, the two XHD stream may have 12 and the HD stream may have 8 to give a total of 32-bit color depth. 也可单独组合均为12位的两个XHD流,从而产生作为结果的24位的流。 Can also be a combination of two separate XHD are 12 streams, resulting as a result of the 24 streams. 例如,如果XHD流的长度长于HD流的长度,那么这是理想的。 For example, if the length is longer than the length of the HD XHD flow stream, then this is ideal. 在存在四个输入流(2XXHD和2XHD)的情况下,如果被缩减为8位颜色深度的话,那么所有流可被合并,从而产生作为结果的32位颜色深度的流。 In the case where there are four input streams (2XXHD and 2XHD), if is reduced to 8-bit color depth, then all streams can be combined to produce a flow as a result of 32-bit color depth.

[0050] 要意识到合并颜色深度缩减的流以产生已知的单一数字数据流的组合和可能性有很多。 [0050] To realize the color depth of the reduced flow merge to produce a single known combinations and possibilities of digital data streams are many. 特别地,对应于已知格式,产生合并的流的希望的总颜色深度的组合和可能性很多。 Many combinations and possibilities in particular, corresponds to a known format, the combined stream to produce the desired total color depth. 另外要意识到已知格式以及希望的颜色深度可以变化。 Also be aware of the known formats and the desired color depth can be changed.

[0051] 图5表示考虑实际的数字数据信息,合并三个图像源24、25和26的一种方式。 [0051] FIG. 5 shows consider the actual digital data, merge the three image sources 24, 25 and 26 in a way. 首先,在27、28和29,每个图像源具有报头和24位的数据帧(即,像素)。 Firstly, in 27, 28 and 29, each image source having a header and a data frame 24 (i.e., pixels). 在30、31和32,每个数据帧(像素)的位数被缩减为8位,如前所述。 At 30, 31 and 32, the number of bits in each data frame (pixels) is reduced to 8, as previously described. 在33,来自每个图像源的8位数据帧被级联,从而产生与一个24位“像素”对应的标准格式的24位数据字段。 33, 8 data frame from each image source is a cascade, resulting with a 24 "pixels" corresponding standard format 24 data fields. 这产生可按标准的已知格式处理的数字结构的数据,当然,24位“像素”并不代表实际图像。 This data structure generated according to the standard digital format known process, of course, 24 "pixels" do not represent the actual image. 如果处理器试图显示组合的单一流,那么它会以随机排列像素的方式显示图像。 If the processor attempts to display a combination of single-class, then it will be by way of a random arrangement of pixels to display images. 为了显示三个独立的图像流,如后所述,必须解构或解码所述单一流。 In order to display three separate image streams, as described later, the decoding must be deconstructed or single class.

[0052] 要意识到可按照各种其它方式合并位深度缩减的图像流,从而形成已知格式的单一流。 [0052] To realize the reduced bit depth can be combined in a variety of other ways the image stream to form a single class known formats. 例如,除了级联之外,可以从每个图像源的数据帧中取出交替的位,从而产生合并的24位数据帧。 For example, in addition to the cascade addition, the bit can be taken out alternately from the data source in each image frame, thereby generating the combined 24-bit data frames. 这样的方法适合于提高安全性。 Such methods suitable for improving security. [0053] 在本例中,如上所述,通过获得来自一个图像源的第一帧的第一像素,和来自第二个图像源的第一帧的第一像素,并把它们合并在一起(通过级联或以其它方式),可组合每个图像帧具有相同数目的像素的两个XHD流。 [0053] In the present embodiment, as described above, by obtaining an image from a first source pixel of the first frame, and from a second image source of the first pixel of the first frame, and merge them together ( Each image frame having the same number of two XHD stream of pixels or by a cascade), may be combined in other ways. 类似地,使来自一帧的第二像素与另一个图像源的第二像素组合,依次类推。 Similarly, the second source pixel with another image pixel from a combination of the second frame, and so on. 组合流的其它方法也是可能的,并且将被本领域的技术人员想到。 Other methods are also possible combined stream, and will occur to those skilled in the art.

[0054] 如果与XHD流相比,HD流的每图像帧的像素数较少,那么仍然可以使用上面说明的通过级联或以其它方式组合缩减的位像素的技术。 [0054] If compared with the XHD flow, fewer pixels per image frame HD stream, you can still use a combination of cascaded or otherwise reduce the bit pixel technique described above. 当在HD图像帧中不存在留待与XHD流像素组合的像素时,可以使用例如空填充空间。 When left with XHD flow binning of pixels do not exist in the HD picture frame, for example, filling the empty space.

[0055] 优选地,合并来自彼此对应的三个输入流的图像帧。 Three [0055] Preferably, the combined input from the image corresponding to each frame stream. 假定图像在作为单一流的后续传输期间始终保持同步,那么可保证来自一个源的帧的像素与来自另一个源的对应像素同时到达目的地。 Assuming that image as a single class during subsequent transmission always in sync, then you can guarantee the pixels from a source with the corresponding pixel frame from another source at the same time to reach the destination.

[0056] 例如,就监视生产线的摄像系统和许多360摄像应用来说,优选地,在相同时间捕捉的多个图像帧被合并成构成单一图像流的单一图像帧。 [0056] For example, to monitor the production line 360 of the imaging system and imaging in many applications, preferably at the same time to capture a plurality of image frames are merged into a single image frames constituting a single image stream. 这使用户能够例如按照捕捉图像流的时间点使图像流同步。 This enables users to, for example, capture time point of the image stream in accordance with the image to stream synchronization. 有利的是,这使得能够实时地同时查看多个视频源。 Advantageously, this makes it possible to simultaneously view multiple video sources in real time.

[0057] 通过使用相同的摄像机,并对摄像机中的数字信号处理器使用单一的时钟源,以致使摄像机真正同步,可在合并之前使图像流同步。 [0057] By using the same camera, and the camera's digital signal processor using a single clock source to cause real camera synchronization, you can make the image prior to the merger stream synchronization. 从而,在与来自另一源的第一帧的第一像素完全相同的时间,数字数据流会得到来自一个源的第一图像帧的第一像素。 Thus, in the first pixel from another source of the first frame exactly the same time, the digital data stream will be the first pixel from a first image frame source. 这会简化随后合并图像流的处理,因为数据位流已被同步。 This will simplify the subsequent processing of the combined image stream because the data bit stream has been synchronized.

[0058] 不过,更可能的是图像源并不完全同步,因为装置中的数字时钟可能完全不同。 [0058] However, the more likely that the image source is not fully synchronized because the digital clock device may be completely different. 在这种情况下,为了在合并之前使图像流同步,需要找出每个数据流内的报头,然后延迟数据流之一,直到使报头对齐为止。 In this case, prior to the merger in order to make the image stream synchronization, you need to find out the header of each data stream, and then delayed one data stream until the header so aligned. 从而使缩减位深度和把数据流组合在一起的所有后续数字处理完全同步。 Thereby reducing the bit depth and fully synchronized data streams together all subsequent digital processing.

[0059] 不过,应注意的是在这样的优选实施例中,并非必需使来自一个源的帧与在完全相同的时间从另一个源获得的帧合并。 [0059] However, it should be noted that in this preferred embodiment, is not necessary to make the frame from a source combined with the frame at the same time obtained from another source. 由于图像在作为单一流的后续传输期间保持同步,因此轻微的失准是可接受的。 Because images synchronized as a single class during subsequent transmission, so a slight misalignment is acceptable. 例如,可接受的是使来自一个源的帧与来自另一个源的就拍摄时间来说相差几个图像帧的帧合并。 For example, it is acceptable to make the frame from a source is merged with it from another source for photographing time difference of several image frames frame. TV摄像机一般具有50场/秒的图像帧速率。 TV camera 50 typically having a frame rate of image fields / second. 合并在一起的图像相隔几场或几帧是无关紧要的。 Merge several images separated by a few frames or is irrelevant. 只要系统和解码器知道,它能够保证图像在接收器端在正确时间被显示即可。 As long as the system and decoder knows that it can ensure that the image is displayed at the receiver end can be at the correct time.

[0060] 如上所述,表示来自图像源的每个像素的原始24位数据在位深度方面被缩减,与来自其它流的其它缩减的像素组合,随后被封装到选择的已知格式中。 [0060] As described above, represent the original 24-bit data for each pixel from the image source of the bit depth is reduced, in combination with other pixels from other stream reduced, then packaged into a known format choice. 通过对诸如16X16模式组之类的像素模式应用颜色缩减和合并,可使所得到的数据与例如MPEG-2或者其它压缩算法兼容。 Through such kind of 16X16 pixel mode Mode group of color reduction and consolidation applications, for example, allows data obtained MPEG-2 or other compatible compression algorithms.

[0061] 位深度缩减和合并方案可以是固定的或者自适应的。 [0061] Bit depth reduction and consolidation program can be fixed or adaptive. 如果所述方案是固定的,那么编码器和解码器都需要预先知道所述方案的安排。 If the program is fixed, then the encoder and decoder are required to know in advance arrangements for the program. 另一方面,如果所述方案是可变的或者自适应的,那么所选方案必须被记录,并从编码器传送给解码器。 On the other hand, if the program is variable or adaptive, then the selected program must be recorded, and transmitted from the encoder to the decoder. 编码方案可作为元数据被保存和传送,所述元数据可被称为“调色板组合图(palette combinationmap) ”。 Coding scheme can be saved as metadata and transport, the metadata can be called "Combination Chart palette (palette combinationmap)". 它包含解释像素是如何被缩减了位深度和如何被组合的信息。 It contains the pixel is explained how the bit depth is reduced and how the combined information. 例如,在图5中所示的方案中,调色板组合图包含查找表,所述查找表解释每个像素从24位被缩减到8位,随后按照第一像素,第二像素,第三像素的顺序,使3个像素中的每一个与来自另一个图像的一帧的对应像素级联。 For example, in the scheme shown in Figure 5, comprising the combination of Figure palette lookup table, the lookup table to explain each of the pixels is reduced from 24 to 8, then according to the first pixel, the second pixel, a third sequence of pixels, so that 3 pixels each with a corresponding pixel from another image of the cascade of one frame. 解码器可以使用该查找表或者说“密钥”来重新组装图像流。 The decoder can use the lookup table, or "key" to reassemble the image stream.

[0062] 所使用的方案可以一次设定后就固定,或者可以是自适应的,如上所述。 [0062] The scheme used can be set after a fixed, or may be adaptive, as mentioned above. 如果是自适应的,那么所述方案可以偶尔改变,例如每天一次,一天几次,或者可以更频繁地改变,t匕如随着所传送图像的性质的改变而改变。 If it is adaptive, then the scheme may occasionally change, e.g., once a day, several times a day, or may be changed more frequently, t dagger such as to change the nature of the transferred image is changed. 如果所述方案频繁地改变,那么将与图像流数据多路复用地传送调色板组合图,或者用独立通道发送调色板组合图。 If the scheme is changed frequently, it will be transmitted multiplexed palette combination map and image data stream, or send by independent channels palette combination chart. 当这种元数据较小时,应该不存在任何传输问题,从而不存在延迟风险。 When this meta data is small, there will not be any transmission problems, so there is no risk of delay. 不过,为了避免要是所述元数据未能到达解码器,从而解码器不能工作的可能性,当不存在从编码器到解码器的元数据传输时,可以使用默认的固定方案。 However, in order to avoid failure if the meta data arrives at the decoder, whereby the possibility of the decoder does not work when there is no metadata transmitted from the encoder to the decoder, and can use the default fixed scheme.

[0063] 优选地,在XHD流合并器15,保存各个流的颜色深度信息。 [0063] Preferably, the XHD muxer 15, save the color depth information for each stream. 该信息可保存在由XHD流合并器产生的调色板组合图中,其中颜色深度信息可被嵌入矩阵中。 This information can be stored in the palette combination chart by XHD flow generated by the merger, in which the color depth information can be embedded in the matrix. 该数据可被加密,以提闻安全性。 The data may be encrypted to mention smell security.

[0064] 优选地,还保存关于各个流的附加信息,以致合并的流可被解码。 [0064] Preferably, also holds additional information on each stream, so that the combined stream can be decoded. 这样的信息可包括初始图像/流的数目,各个流中图像像素的初始位置。 Such information may include the number of the initial image / stream, the initial position of each pixel in the image stream. 该数据可以矩阵的形式嵌入调色板组合图中,还可被加密,以便提高安全性。 The data may be in the form of a matrix embedding the palette composition figure, may also be encrypted to enhance security.

[0065] 现在,可以作为已知格式的单一流来处理初始图像流。 [0065] Now, as a single class of known formats to handle the initial image stream. 例如,如果合并的图像的格式大小是标准格式大小的话,这可利用常规硬件来实现。 For example, if the format of the combined image size is the size of the standard format, then it can be implemented using conventional hardware. 例如,目前就是这样,如果格式大小为HD的话。 For example, the current is such that if the size of the format for HD words. 该格式兼容MPEG-2和4。 This format is compatible with MPEG-2 and 4. 于是,如果待合并的输入流是HD格式,那么可以使用常规硬件。 Thus, if the input stream is to be merged HD format, you can use conventional hardware.

[0066] 不过,在本例中,所得到的图像的格式大小为XHD。 Format size of the image [0066] However, in this example, which was obtained as XHD. 目前,可利用MEPG压缩实现XHD分辨率视频的压缩、传送和存储,MPEG压缩产生造成传送和存储问题的巨大文件大小和带宽。 Currently, you can use compression to achieve MEPG XHD resolution video compression, transmission and storage, MPEG compression produces transmission and storage problems caused by the huge file size and bandwidth. 于是,为了使数据能够被实时压缩以便应用,需要强大的专用处理器和非常高速的网络。 Thus, in order to make real-time data can be compressed in order to apply, you need a powerful processor and a very dedicated high-speed network. 这些处理器和网络目前并不广泛可用,经济上也不可行。 These processors and networks are not widely available, not economically feasible.

[0067] 可以使用按照第二种格式处理以第一种格式获得的图像的方法和系统把组合流转换成低分辨率格式。 [0067] can be used in accordance with the second form method and system for processing images obtained in a first format of the combination stream into a low-resolution format. 例如,可以使用其中把像素分成16X16像素的“图案”,随后以HD格式传送的方法。 For example, the pixel in which the pixel is divided into 16X16 "patterning", and then transmitted to the HD format method. 这在图3中被表示成“Tetris”编码器和解码器。 This is expressed as "Tetris" encoder and decoder in FIG. 3. 这是一种把XHD数据转换成HD数据的编码方案,不过并不是本发明的实施例必不可少的。 This is a way to convert the data into a coding scheme XHD HD data, but the embodiment of the present invention is not essential. 可以使用其它转换方案,或者甚至可以使数据保持XHD格式。 You can use other conversion programs, or even to keep the data XHD format. 未来,硬件将允许传送和处理XHD,并且将不再需要图3中所示的可选的转换步骤。 The future, the hardware will allow transfer and processing XHD, and will not need to 3 shown in FIG optional conversion step. 如此,如果希望的话,可以使用常规的HD编解码器压缩和解压缩合并的数据。 So, if you want, you can use the regular HD codecs to compress and decompress data merge. 数据可以以压缩的形式被传送和/或保存。 Data can be transmitted in compressed form and / or save.

[0068] 图2表示用于把由流合并器15产生的XHD合并流转换成HD流的编码器16。 [0068] Figure 2 shows the flow for the XHD merger generated by the combined stream into the encoder 15 HD stream 16. 图像的有效画面部分被分成每个具有多个像素的图案。 Effective picture portion of the image is divided into a plurality of pixels each having a pattern. 这些图案被赋予坐标值,然后利用把图案重新排序的加密密钥被重新格式化成HD格式。 These patterns are assigned coordinate values, and then use the encryption key pattern reordering is reformatted into an HD format. 该加密密钥可由调色板组合图产生,不过情况并非必需如此。 The encryption key is generated by a combination chart palette, but this is not necessary the case.

[0069] 图3表示处理单一的合并流的例子的概述。 [0069] Figure 3 shows the processing overview of an example of a single combined stream. 图3表示把XHD格式重新格式化成HD格式的编码器、得到的HD流,和解码器。 Figure 3 shows the XHD reformatted into a format encoder HD format, the resulting HD stream, and a decoder. 通过在密钥的控制下,应用相反的重新排序过程,解码器把图像转换回XHD格式。 Under the control by the key, the application reverse reordering process, the decoder to convert the image back to XHD format. 这种情况下,密钥和HD流一起被发送。 In this case, the key and the HD stream is transmitted together. 该解码器在图4中还被表示成解码器17。 The decoder in Fig. 4 also shows a decoder 17.

[0070] 优选地,可以存储在调色板组合图中的输入流信息也与单一合并流一起被发送到如图4中所示的解码器。 Input flow information [0070] Preferably, the combination may be stored in the palette with the single figure of the combined stream is also sent to the decoder as shown in Figure 4.

[0071] 在XHD流分流器18,接收合并的单一流(在本例中为XHD的)。 [0071] In XHD stream splitter 18 receives the combined single class (in this case the XHD). 还接收调色板组合图,包括诸如输入流的数目、图像和图像像素在这些流内的位置之类的输入流信息。 Also receives palette Combination, comprising an input stream information such as the number of the input stream, the image and the image pixel location within these streams and the like. 利用该信息,合并的单一流被分离回在流合并器15合并的独立流,两个XHD流和一个HD流。 Using this information, the combined single stream is separated back in 15 separate merger muxer stream two XHD HD stream and a stream.

[0072] 这些分离的流随后可被发送给颜色深度转换器19、20和21。 [0072] These separate streams may then be transmitted to the color depth of the converters 19, 20 and 21. 分离流的颜色深度可被变回初始输入流9、10和11的颜色深度。 Color depth separated flow can be changed back to the original input stream, the color depth of 9, 10 and 11. 于是,把8-12位的每个缩减像素变回24-32位。 Thus, the 8-12 bits per pixel cut back 24-32 bits. 理想的是把位深度变回当前硬件所支持的标准位深度。 Ideally the bit depth change back to standard bit depth of the current hardware supports. 利用诸如GIF标准所使用的调色板之类的技术,实现这种功能的标准转换器在本领域中是已知的。 The use of such techniques as the palette GIF criteria used to achieve this functionality standard converters are known in the art of.

[0073] 要意识到由于在处理过程中使用量化和压缩,与输入流相比,来自颜色深度转换器19、20和21的输出流的色彩质量发生变化。 [0073] To realize the use in quantization and compression process, compared to the input stream, color quality from color depth converter output streams 19, 20 and 21 is changed. 不过,发明人意识到轻微的变化对人眼以及对许多应用,尤其是实时工作的那些应用来说并不明显,如此降低的质量是可接受的,所获得的优点胜过色彩质量的降低。 However, the inventors realized that minor changes to the human eye, and for many applications, especially those applications not apparent to work in real time, thus reducing the quality is acceptable, the advantages obtained than reduce the color quality.

[0074] 现在可在22处渲染输出流,和在23处显示输出流。 [0074] is now displayed in the output stream to render the output stream 22, and at 23. 例如,显示器可以是360视频播放器,在所述360视频播放器,最终用户可摇摄和放大3D世界。 For example, the display 360 may be a video player, the video player 360, the end user can pan and zoom 3D world.

[0075] 所描述的实施例具有能够以具有如下优点:作为已知格式的单一视频流处理、压缩、传送和/或存储可以是不同格式的多个视频流。 Embodiment [0075] In the described embodiment has the following advantages can be: a known format, a single video stream processing, compression, transmission and / or storage may be a plurality of video streams of different formats. 这简化了处理所需的硬件配置。 This simplifies the processing of the hardware configuration required. 按照这种方式组合流还意味单一的合并流的长度不必长于最长输入流的长度。 In this way also means that a single combined stream flows need not be longer than the combined length of the longest length of the input stream. 对视频流的存储和传送来说这是有益的。 Storage and transmission of video streams, this is useful. 另外,由于在传输之前,带宽被减小,因此该方法适合于实时应用。 Further, since prior to transmission, the bandwidth is reduced, so that the method is suitable for real-time applications. 该实施例还具有在传送期间,流保持同步(即,在传输期间,组合流的方式不发生变化)的优点。 This embodiment also has during the transfer, the stream remains synchronized (i.e., during transmission, mode combined stream does not change) advantages. 在该实施例中,流被组合得使得在拍摄时间上对应的帧被组合。 In this embodiment, the stream are combined so that the frame was captured at the time corresponding to be combined. 在所描述的应用中,这是特别有利的,因为它使得能够实时地同时查看整个环境。 In the application described, it is particularly advantageous, since it makes it possible to simultaneously view the entire environment in real time.

[0076] 本领域的技术人员会认识到本发明的应用例子只是用于举例说明,可按照许多其它方式利用本发明。 [0076] Those skilled in the art will recognize that the application examples of the present invention is for illustration only, many other ways in accordance with the present invention. 当某一过程要求多个视频源之间的同步在传输和处理期间保持固定时,本发明特别有益。 When synchronization and remains fixed during the processing and transmission of multiple video sources between the requirements of a process, the present invention is particularly advantageous. 可使图像帧同步,例如使得在相同时刻或者按已知时差捕捉的帧可以一起到达目的地。 Frame synchronization of the image, e.g., such that at the same time or in a known time difference captured frames can reach the destination together. 如果希望进行相关操作,例如在运动分析,立体3D或拼接中,同步可以是有利的。 If you want to perform related operations, such as in motion analysis, stereoscopic 3D or mosaic, the synchronization may be advantageous. 不过,本发明并不局限于这样的应用,可用在希望作为单一流处理多个流的许多应用中。 However, the present invention is not limited to such applications, can be used in many applications are desired as a single stream processing a plurality of streams.

[0077] 另外要意识到,待合并的视频流的数目和视频流的图像格式可变化。 [0077] Also be aware that the image format to be merged and the number of video streams video streams may vary. 缩减颜色深度并把流合并成已知格式的流的许多组合和方式都是可能的,并且会被本领域的技术人员想到。 Reduce the color depth and merged into the stream flows in many combinations and ways known formats are possible and will occur to those skilled in the art.

[0078] 对所述实施例的各种其它修改也是可能的,并且会被本领域的技术人员想到,而不脱离由附加权利要求限定的本发明`的范围。 [0078] Various other modifications to the described embodiments are possible, and will occur to those skilled in the art, without departing from the scope as defined by the appended claims of the present invention `.

Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
EP0954338A211 Jul 199710 Nov 1999Genzyme CorporationCationic amphiphile/dna complexes
EP1827024A123 Feb 200729 Aug 2007Sharp CorporationHigh dynamic range video coding
US2006/0197777 Title not available
US2006/0215875 Title not available
WO2001/72041A2 Title not available
WO2001/91454A2 Title not available
WO2004/045217A1 Title not available
Non-Patent Citations
1ERDEM A T ET AL.Compression of 10-bit video using the tools of MPEG-2.《SIGNAL PROCESSSING IMAGE COMMUNICATION》.1995,第7卷(第1期),
2MALAK M ET AL.An image database for low bandwidth communication links.《DATA COMPRESSION CONFERENCE,1991》.1991,
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
CN104519335A *30 Sep 201415 Apr 2015惟成科技有限公司Method, device and system for resizing original depth frame into resized depth frame
Legal Events
29 Sep 2010C06Publication
17 Nov 2010C10Request of examination as to substance
24 Jul 2013C14Granted