WO2014168499A1

WO2014168499A1 - Apparatus and operation method for visually impaired

Info

Publication number: WO2014168499A1
Application number: PCT/RS2013/000006
Authority: WO
Inventors: Darko TASOVAC; Dušan GRUJIĆ; Veljko MIHAJLOVIĆ; Pavle JOVANOVIĆ; Milan Savić
Original assignee: Novelic D.O.O.
Priority date: 2013-04-08
Filing date: 2013-04-08
Publication date: 2014-10-16

Abstract

A user wearable visual assistance solution is proposed. The apparatus and method of operation for providing specific solution for object detection and object behaviour detection using mm-wave radar approach, where apparatus is attached to the human head, and where the information about environment is presented to the human being by audio signal provision. Advantageously, the apparatus is embedded in spectacles-like shape, being provided to persons with visual impairments. The usage of the integrated mm-wave ISM Band 60 GHz front end, combined with the low cost printed high antenna with specific advantageously proposed radiation diagram, adequate baseband processing, and dedicated audio signal generation, is providing major solution advantage for people, by having light, non bulky, affordable help in daily life. The introduction of the "spectacles- like" support shape, affords the usage, without explicitly optically showing the purpose of the apparatus. Using proposed additional embodiments for apparatus with additional functionalities, and proposed operation method additional features like: pattern recognition, system training on typical obstacles, as well as acoustic guiding and audio navigation are proposed. Proposed invention addresses also features usable for applications for individuals being in darkness, dense fog and fog like environments like smoke. Thus this invention also explicitly addresses specific industrial, security and safety applications.

Description

Apparatus and Operation method for visually impaired

Technical Field

[0001] Aspects of the present invention relate to a user wearable visual

assistance system. The presented solution addresses supporting visually impaired people, or people acting in darkness, without using laser, video, infrared, acoustic, echoing or ultrasound assemblies.

Background Art

[0002] The different solutions for visually impaired individuals have been

introduced in the past. They may be classified in three major groups of the inventions and solutions:

1. Video systems

2. System with ultrasound detection

3. Laser based systems

4. Acoustic echo based systems with microphones and transducers

5. Navigation through GPS and cellular means

[0003] The US patent application US20120212593 "A user wearable visual

assistance systems" shows typical video processing approach where the cameras and object recognition is aimed. Similar video processing based approaches are described in several others applications. The basic approach is that one or more cameras are processing optical data in front of the person. Those systems are typically bulky, need a significant processing power, meaning also significant power supplies, and they are difficult to be realised in the way that the user may "hide" the nature of those HW, to the environment where he is moving around, meaning that the associated HW is recognisable as a special HW to other people without visual impairments. Also in case of night, processing in the infrared spectrum is needed, so object detection could not be easily achieved.

[0004] Systems with ultrasound detection for people with visual impairments are partly commercially available, have significant volume, weight, power consumption and high cost. The size of the ultrasound apparatus is large, which is related to the low frequency of the ultra sound and can be hardly used on the person's head. And if it would be used as such, this would be easily seen as "non typical" apparatus. In US6671226 "Ultrasonic path guidance for visually impaired" the ultrasound sensor is proposed to be attached on the front part body of the user (chest). The prior art device, the BRYTECH Sensory 6, is using ultrasound approach, with acoustic communication to the user. [0005] Laser based systems are proposed in the several US patents. In the early patent US3654477 Obstacle detection system for use by blind

compromising plural ranging channels mounted on the spectacles frames" pulsed light systems (coherent light like lasers) are proposed. In

US5487669 "Mobility aid for blind persons" narrow band laser beam with hand held device is proposed. In US6489605 "Device to aid the orientation of blind and partially sighted people", the device comprising long cane and additional device with laser measuring system is proposed. The laser- based systems in general require special hardware and significant power consumption. The usage of the lasers in the normal daily life may also be somehow restricted, due to hazard of eye damage for people standing in the front of the user. In general, they have punctual distance calculation, so that if the obstacle with non-punctual size is traced, the laser systems must scan the surface electrically or mechanically, which increase the total system complexity and respectively also cost.

[0006] In US20110307172 "Hand held navigation aid for individuals with visual impairment", optical and ultrasound system combined with GPS and further hand held systems are proposed.

[0007] In US4907136 Echo location system for visually impaired persons is

proposed, where transmitted and echoed sound pulses are used for object detection.

[0008] In US8 20521 Radar echolocator with audio output is proposed utilising two sound transducers, and wearable approaches on belt-like and garments.

[0009] In US7978120 radar based imaging systems are proposed as general approach. Those systems use large antenna areas, have very complicated signal processing, huge power consumptions, so they can not be driven by small battery systems and they are not intended to be placed on the human body and wearable utilisation. This type of imaging would be impractical for the single user and also the methods of interfacing to the individual are not feasible, despite the fact that it is a radar based solution operating at very high frequency, and as such would be interesting and favourable, by specific adaptation, to include as a system part in the innovative system solution.

[0010] Thus, there is a need to have a smart, low weight, small volume and

affordable solution, introducing new concepts for supporting and enhancing quality of life for the visually impaired individuals, as well as for special applications in darkness, dense fog and fog like environments like smoke and similar. Summary of Invention

[0011] The proposed invention is based on apparatus and related method of operations, which brings fundamental advantage compared to existing solution in supporting visually impaired people, or people acting in darkness, without using infrared assemblies. The inherent advantages of the proposed solution compared to the state of the art are:

Small size and light-weight, therefore having capability to be integrated in the typical "spectacles like" shape, on the user's head; Low cost, with bill of material which, for basic functionality may be in the range of 10-30$ in mass production. This would lead to retail price of under 100$. In the future with the typical semiconductor device cost as main driver for system cost, as well as assembly cost, will decrease. This makes this technical solution dramatically attractive compared to any of the state of the art solutions, and may offer to the all social classes of the visually impaired people worldwide the affordable solution for enhancing the quality of life.

Specific radiation pattern proposed by this invention has wide angle in elevation and narrow beam in azimuth. This allows the user to be instantly warned of the obstacle distance in front of the head, related to the specific narrow angle in azimuth, in the way that also obstacles appearing on the different heights are covered. These features cannot be realized by state of the art approaches. By mechanical movement of the head in specific direction, the user can easily„scan" the surrounding environment. This is allowed by the usage of the specific frequency band of the radio unit, being combined by the related proposed antenna solution. Due to the selected operation frequency in the ISM bands above 24 GHz, preferably at 60 GHz as well as in 122 GHz and 245 GHz, the smaller sizes of the antenna systems may be realized as well as the smaller weight of the complete proposed system.

Additionally, the proposed solution has inherent capability to provide the information about the relative speed of the object moving relative to the user, which may be very important for the user's orientation and behaviour and finally can save the user's life. This is done by combining specific signal processing and topology of the radio unit, that is capable to extract the information using Doppler principle. This feature would require much more complexity using state of the art solutions.

Additional processing power may be introduced so the system training on the typical objects scattering profiles being detected by the user, utilizes the proposed apparatus. This would allow the system to perform the pattern recognition and to enhance audio information content being sent to the user.

• Additional optional information about position of the user, coming from satellite based location systems, combined with apparatus generated information, and orientation of the user (miniature accelerometers and gyroscope sensors), may provide user with detailed navigation information, being communicated to the user by audio signals.

The basic building blocks of the proposed basic apparatus are: Radio Transmit and Receive unit being attached to the Tx and Rx antenna systems, signal processing unit, wired connection to the small

loudspeakers being placed in the human ears. Due to proposed method of operation, the radio unit radiates in the frequency range being above 24 GHz, preferable in 60 GHz, and higher frequency ISM bands. Those signals are transmitted over specific dedicated radiation pattern of the antenna. The radio signal is scattered from the possible obstacle, caught by the antenna system, received, down converted and processed by analogue and digital means. As a result of the processing, audio signals are generated and transmitted to the human ear. Audio signals are characterised as analogue signals in frequency spectrum being able to be percept by human ear, typically in the range 20-20kHz. Those audio signals are typically generated in the way that the specific frequency (pitch) is allocated for the static (non-moving) obstacle, where the distance to the obstacle is related to the signal magnitude (sound

loudness/volume). On the other hand, if the obstacle has moving projection trajectory and velocity component toward the user, changing of the frequency is applied, disclosing the increase of the frequency, related to the calculated speed of the obstacle, in the way that larger frequency is related to the larger speed component toward the user. If the obstacle is moving in his trajectory projection away related to the user, the respective frequency is lower than reference audio frequency related to non-moving obstacle. In the proposed invention plurality of the realisation options may be used, and transformation of the distance and velocity must not be transformed in the audio spectrum linearly. Advantageously the radio front end is integrated on single MMIC chip, antenna system is realised as a planar antenna with at least one row of the power of two radiation elements, ideally realised as wide band elements and planar feed, to have reduced assembly costs. The complete apparatus is advantageously realised in a spectacle-like form, providing mechanical stability, easily implemented wired connection to the ears, and not affecting human visual appearance at all. By adding additional functionalities like short range wireless functionality, satellite localisation, proposed radar information data may be combined by the location awareness so that comprehensive information may be extracted and communicated to the user by using audio means. The proposed method of operation may be extended to object recognition from existing data base, data base being updated by location awareness coming out from satellite location systems and data base being generated by the user, by the tracking and updating the objects in his daily life and behaviour patterns

Brief Description of Drawings

[0013] Fig. 1 presents advantageous embodiment of the proposed apparatus 100 being realised integrated in the spectacle frames

[0014] Fig. 2 qualitatively presents the radiation diagram of the integrated high gain antenna functionality 50, showing narrow angle diagram in azimuth and wide angle in elevation, being integrated in advantageously proposed embodiment

[0015] Fig. 3 presents functional blocks of the proposed apparatus 100,

showing functional blocks and additional optional functional blocks.

[0016] Fig. 4 presents proposed apparatus 100 being attached to the Human head, showing means of wireless and wired communication to the loudspeaker/headphone, with and without transmission through additional processing unit.

[0017] Fig. 5 presents possible antenna arrangements. The advantageous

realisation of the 60 GHz radiated elements with 8 ellipsoidal printed dipoles for Tx antenna and 8 printed dipoles for Rx antenna with planar feeding realisation option and proposed tapered symmetrical strip line matching structure is presented.

[0018] Fig. 6 presents typical advantageous arrangement of the functional blocks related to integrated MMIC functional block 90, of the apparatus 100.

[0019] Fig. 7 Method of Operation Classes regarding Apparatus 1 00 Features Description of Embodiments

[0020] Proposed apparatus 100 is presented in Fig. 1. It is shown in Fig. 3 that 100 comprises of Transmit functionality 10, Receive functionality 20, Antenna functionality 50, Signal processing functionality 30, System control functionality 40, means of wired connection functionality 60 and support structure functionality 70. The apparatus 100 is attached to the Human head 200 with loudspeakers being attached to the human ear 2 0, like in Fig. 4. The apparatus 100 comprises also of: HMI

Functionality 41 as well as integrated ACCU functionality 42, providing power supply for electric circuitries. Optionally one or more additional functionalities like; wireless connection functionality 80, accelerometers and gyroscope sensor functionality 44 and Satellite position functionality 43 may be by this invention included in the apparatus 100. Tx High gain antenna functionality 50 is connected to the Transmitter chain, precisely to the PA (Power amplifier) and it radiates radio waves by specific advantageously proposed radiation diagram. Rx High gain antenna functionality 55 is connected to the Receive chain, precisely to the LNA (low noise amplifier), receiving the radio waves coming from scattering from dedicated object. The supporting structure 70, of the proposed apparatus 100 has advantageously shape of the spectacle frame, like shown in Fig. 1. By using new advance technologies it is possible to use only one high gain antenna for both Rx and Tx activities, utilising chip integrated circulator functionalities. This would allow increased number of radiated elements on the same physical size, providing narrower beam in the azimuth, and respectively more azimuth special accuracy. Ideally on chip "circulator functionality" can be realised by the specific narrowband coupling structure providing isolation typically in the 20 dB ratio, with 1-3 dB insertion losses, which would be for specific applications sufficiently good for the system performance. In that case the "circulator functionality" would need to be integrated in the mm-wave integrated functionality 90, together with other sub-blocks shown in the Fig. 6.

[0021] All functionalities are realised by the plurality of the approaches and

technologies and by the plurality of the integration level. All of the functionalities may be functionally integrated in the existing semiconductor hardware, or physically integrated in the new HW like System on Chip (integrated IC on silicon).

[0022] Transmit and Receive functionality 10 and 20 are realised by the plurality of the semiconductor technology, and plurality of the integration level. They form the radar's front end and operate using the frequency range above 24 GHz. The proposal in this application to use the frequencies above 24 GHz is related to the reasonable small antenna array system (High gain antenna functionality 50), due to the fact that specific high gain antenna diagram is required to be achieved, and according to this invention, antenna requires many radiation elements. This antenna would need the specific number of the radiation elements, at least 8 elements in row. This required dedicated size of the system with additional feed networks, must be incorporated in the proposed support structure. Ideally, the support structure 70 should be acceptably small and light weight so it could be attached on the human head, and ideally the shape of the support structure would need to be non destructive, meaning that user moving around with the support structure would not be straight recognized as a person having something non usual or strange on his head. The supporting structure 70 can be realised by the plurality of the used materials and by the plurality of the shapes. On the other side the radar type of the operation is required, which may address at least 3 meter distance (Short range radar operation) and to detect object velocities (Doppler type of radar operation). Taking into account this typical basic operation requirements, available semiconductor technologies, restrictions in the battery driven power supply (limited size and weight of the battery as well as thermal dissipation) relative smaller power levels would be required, which combined with the antenna gain should provide enough S/N ratio for dedicated radar operation. Due to non-professional usage of the frequency bands, typically ISM Bands are proposed for this invention addressing, 24GHz, 60 GHz, 122GHz, 245GHz and higher frequencies. Detection range depends on the radiation power, which depends on the technology, frequency, package and connection technology used. Hence, in order to provide the affordable solution, trade offs need to be made. In this invention we are advantageously proposing the usage of the 60 GHz ISM Band. This frequency band allows the integration of the Transmit and Receive functionality on mm-wave MMIC, like proposed in the Fig 3.

In the Fig 5 advantageous arrangement of the 2 X 8 radiating elements, for example potentially in the 60 GHz frequency range is presented in upper part of the figure. It may be observed that one side of the antenna is acting as Tx antenna and other side is acting as Rx antenna having two separate connections to the integrated mm-wave functionality 90. The distance between the antenna dipoles on left and right size is shown as a same, and typically between 0.8 and 1 wavelength of the main frequency. The same distance between elements on left and right size and in phase feeding, leads to linear antenna array with typically side lobes of -13dB. By changing the distance between the elements, feeding phase and signal level the optimisation of the side lobes and radiation pattern in general may be achieved. For the system perspective, it is essentially important that the parasitic coupling from Tx Antenna to Rx Antenna is as small as possible, so that specific different techniques may be proposed to reduce this coupling. Following strategies are proposed alone or in the

combination:

The surface of the substrate where the antenna Tx and Rx parts are printed has a gap between upper part of the substrate and place where the integrated MM-wave functionality is attached to the substrate. The distance between the dipoles from one side and other side is N ^* Wavelength + Wavelength/4 to theoretically reduce the coupling between them

Between RX and TX antenna parts there is a printed metal strip on both sides of the substrate, approximately thick as Wavelength /4 and approximately having the distance to the left and right side of the dipoles of Wavelength/4 from its first edge.

Complete different substrate with higher distance one from another with the drawback that the separate connection to entity 90 is provided, with more losses and higher manufacturing complexity. Orientation of the printed dipoles is opposite, like in Fig. 5, where upper parts of the printed dipoles of TX antenna are printed on one side and upper parts of the Rx antenna are printed on other side of the substrate.

[0024] The proposed radiation elements are advantageously chosen to have:

Wideband radiation elements, (dipoles) having more than 10% bandwidth and input reflection coefficient S1 1 <-10 dB in operating frequency range

Planar feeding to the elements with symmetrical strip line

Printed one half of the dipoles on one side of the supporting substrate The second half of the dipoles on other side of the substrate Feeding transmission network being based on tapered strip line structure rather than typical quarter wave transformers

• Quarter wavelength reflector allowing high gain radiation diagram in from of the support structure, being narrow (less than 15 degrees) in the azimuth and wide (more than 60 degree) in elevation, is introduced.

[0025] The antenna has its receiving and transmitting part each with 8 printed dipoles, related matching structures and connections to the mm-wave front end 90.

[0026] In the Fig 5, lower part of the figure, an alternative arrangement of the 2 X 16 radiating elements example, is presented. For this arrangement, more space is required compared to 2 X 8 element arrangement, but the related beam in the azimuth is narrowed, meaning better spatial accuracy, and more gain, which is reflected in large communication range.

[0027] Typically substrate thickness of 0.127 mm may be used for 60 GHz

applications. The wideband dipoles upper and lower parts are

advantageously realized as planar ellipsoids to maintain wide band behaviour, they however may be realized by the plurality of the

approaches like printed n-togams, where N is larger than 4. If N is coming to be infinitive we have smooth ellipsoid like structures, which are advantageously proposed. For example, for N=5 we are speaking about "pentagram like" half printed dipole structure. The usage of the tapered feeding strips results in wideband impedance matching with smaller losses. This is important in order to maintain large manufacturing tolerances, which leads to improved yield and smaller system cost. The basic assembly idea proposed here is that the radio front-end chip is placed on the same substrate where the antennas are printed omitting the complicated and expensive mechanical interconnections. Considering the Fig. 5, High gain antenna functionality connection 50, it may turn out that before connecting the MIC chip to the antenna feeding network,

BALUNS may be required for Tx and Rx part due to the fact that the antenna has symmetrical feeding and MMIC 90 may have single ended or differential outputs and inputs but on the top of the assembled substrate. The reflector for the antenna dipoles, as well as cover to the assembly may be realized by the plurality of approaches. Advantageously the metalized plastics, is proposed as reflector by filling the distance to substrate with printed antenna by usage of foams for providing mechanical stability. The distance from the substrate to the metalized surface is between 0.2 and 0.25 wavelengths at central operating frequency.

The ideal, advantageous shape of the support structure 70 is proposed to be "spectacle frame like" shape. That means the support structure is looking as spectacles as in Fig. 1 having on left and right side two rods, by the plurality of the realisation, providing spectacles stability being placed between the user head and ears. Inside of these rods, the wire (part of the proposed apparatus wired connection functionality 60) is advantageously placed connecting on left side and on the right side signal processing unit 30 to the small loudspeakers, which may be put by the user in or on users ears, in the same manner as spectacles are today combined with the audio players. Those wires are carrying audio signals in audible spectrum to user ears. If the frequency range of 60 GHz is used the complete size of the integrated assembly of the antenna and front end would be in the approximate range of 20 mm to 90 mm with thickness of about 5mm, which may be integrated in the support structure 70. This would be enough to accommodate 16 dipoles in one row at 60 GHz. In the case of 8 dipoles the length would be approximately twice shorter, and very easy to integrate, however the related gain of the antenna would be smaller and radiation angle in azimuth would be wider, which is not ideal related to the spatial resolution need from the user point of view. Due to the spatial resolution option for the user, smaller resolution angle in the azimuth is advantageous, which requires more gain, and more dipoles in the row. By simple moving his head in slight direction, the user can after some training have quite a good feeling about the obstacles in front of him. Therefore the antenna operating at higher frequency may require less space for the same radiation diagram. On the other side higher frequencies are propagating with more loss, so it may turn out that more transmit power for the same range may be required. This would require more effort in the MMIC 90 design, also regarding current maturity of the semiconductor technologies for applications above 100 GHz. That is one of the reasons why advantageously deployment of the 60 GHz radar based system is proposed in the scope of this invention. The usage of the SiGe Technology for the deployment of the 60 GHz Integrated MMIC 90 is explicitly acknowledged as advantageous realisation of the Apparatus 100, main functional parts 10, 20, being described as basic functional blocks in the Fig. 6. This advantageously proposed topology of the 90, includes Tx output with PA (power amplifier) 91 functionality, RF input by LNA (Low Noise Amplifier) 92, VCO (voltage controlled oscillator) 93 with the Input Tuning signal, Prescaler 94, Quadrature Mixers 96, with the BBI

(baseband output signal I branch), and BBQ (baseband output signal Q branch), as well as signal conditioning sub-functionality 97. The functionality 90 must have power supply input, with the voltage level being dependent of the proposed semiconductor technology and preferable if SiGe is used with 3.3 V. Power distribution, ESD protections, and decoupling features, as well as matching circuitries, passive and active elements like buffer amplifiers and other sub-blocks are parts of the proposed functionality 90. Prescaler function may allow input for external PLL functions (out of functionality 90) and controlling. Functionality 96 comprises of related filtering structure for baseband signal provision and signal level adaptation required for the AD conversion functionality that is outside of the functionality 90. Splitters 98 are diving the power of the signals in the specific ratio. Splitting ratio on LNA 92 output is two.

[0029] The complete apparatus 100 may be also realized in a way that external MMIC Power Amplifier 99 is used by the plurality of the applied topologies and semiconductor technology. This approach may reduce the parasitic PA-LNA coupling compared to the case where both circuits are on the same chip. This approach may be also used to use to extend the range of the radar system by applying larger EIRP if the specific regulations for specific application has less severe constrains. The drawback is increased manufacturing complexity, followed by increased system cost on additional silicon and chip interconnections and potentially mechanical reliability.

[0030] The tentative operational ranges for providing functionality of the proposed system are presented in Table 1 , taking into account related ISM band frequencies, noise figures, and different EIRPs. These numbers are presented in rather indicative manner and do not present the final system parameters, so they are introducing non-optimal values. They however indicate that the operational ranges for utilizing advantageously FMCW concept may have sufficient range 5 and more meters for frequencies starting from 60 GHz and above. FMCW concept is proposed because it is possible to get both distance and Doppler information with the same integrated mm-wave front-end (90) by different art of input settings and signal processing. Table 1

[0031] This type of radar advantageously uses sawtooth (ramp-up and down) signal to modulate the VCO input and generate FMCW radar signal.

FMCW radar simultaneously sends and receives signals, in this case FM (linearly chirp-up and down). Parameters, such as modulation bandwidth and modulation frequency, directly affect radar's resolution - larger modulation bandwidth gives better range resolution, and lower modulation frequency better velocity resolution.

[0032] Considering all frequencies, it may be confirmed that almost at all systems ETSI regulative for the Short range devices with EIRP limits of 20dBm may be applied, which would simplify the world wide usage of the devices. Advantageously in the 60 GHz range EIRPs of the 25dBm for outdoors applications are allowed, which gives a sufficient margin. Considering advantageously proposed 60 GHz applications, the antenna gain of the 1 1-12 dBi for TX chain is theoretically envisaged for 8 ellipsoidal planar dipoles with linear feeding and reflector. By introducing 3 dB losses in substrate, matching and interconnections potentially 9dB effective gain may be combined with 16 dBm TX output giving EIRP of 24dBm. Also if the integrated functionality is giving 10 dBm operating gain, and if the losses are higher at Rx and Tx part, the degraded system performance would still be sufficiently good to have more than 3 meter range required by the application. FMCW bandwidth in 60 GHz range may be

advantageously 4 GHz and not 0.5 GHz like in table printed, providing respectively better range resolution. This corresponds that for the chirp time of 10ms and at range of 5m, reflects the beat frequency of less than 15KHz. Calculation of receiver sensitivity is performed with 100 kHz noise bandwidth, would require AD sampling frequency of at least 200 kHz, at the input of the signal processing unit 30.

[0033] The proposed apparatus hardware can also be used for providing

additional application features related to the communication in the same mm-wave frequency, by having additional method of operation, and specific signal processing. In this case, VCO functionality of 90 may be used in the way that VCO tuning is related to the transmission of data by acting as a communication modulator. This may be done in the way that different modulation schemes, by their plurality, may be applied and among them are following:

• On-Off keying

• N-PSK

• F

[0034] These classical communication modulation schemes can be combined in order to remain the low complexity of 90 and limited processing resources of 30.

[0035] This type of the simple communication means would allow point-to-point channel communication, especially for the following generic application purposes:

User identification, where the apparatus is obtaining the signals from other apparatus and resending the identification code.

• ^' Exchange of the data, related also to read out memory from Apparatus

100.

• Receiving the warnings and information from other systems

[0036] HMI Functionality 41 is related to the interface to the apparatus that may be provided by the plurality of the approaches like mechanical buttons, gestures or touches. The controlling function from the user side may be arbitrary complex, but following functionalities are obligatory:

• Turning off and on of the apparatus

Sound loudness regulation

[0037] Additional features like setting the centre frequency in audio spectrum related to non moving objects, the range of audio spectrum for different object speeds, as well as managing classic audio player controlling functions are feasible.

[0038] ACCU Power Supply 42 is necessary to provide the power to the active electronic circuitry of the Apparatus 100. The ACCU 42 is realised by the plurality of the technologies and shapes, advantageously proposed to use the polymer and Li Ion ACCU technologies used in the handheld devices. The capacity of the 42 is related to the size and technology, whereby the advantageously the 3.7 to 3.9V devices with 3-5Wh is recommended, which may be a realistic trade off between size, weight and one time charging time operation. There is a mechanical female jack for DC charger, being available as obligatory accessory.

[0039] Wired Connection functionality 60 is realised by the plurality of the

realisation compromising at least two wires as asset connecting signal processing functionality 30 to one loudspeaker. The wires are passing through supporting structure 70, advantageously inside rods of the spectacles-like shape of the 70, shown in the Fig 1.

[0040] System control functionality 40 , realised by the plurality of the controlling technologies and approaches provides:

Execution of the user commands coming from HMI functionality 41 , by setting parameters of the electronic circuitry within apparatus 100 Monitoring the charging and load status of ACCU functionality 42, and by the specific predefined event, by the plurality of the predefined events, issuing the initialisation of the warnings, by sending order to the signal processing unit 30, or by interfacing HMI functionality 41 .

[0041] Advantageously system control functionality is low power microprocessors by the plurality of the architecture and performance data. It may be integrated with signal processing functionality 30, and or be integrated together with MM-wave RF Front IC in System on Chip (SOC) 95 functionality.

[0042] Signal processing functionality 30 is realised by the plurality of the

realisation options, having means of analogue and digital processing or only analogue processing in the simplest realisation approach. In the simplest realisation options, analogue signals coming from radio wave receiving functionality 20, as a part of the functionality 90, are in the intensity and in frequency translated, filtered and amplified to the intensity and frequency in audio spectrum (20Hz -20KHz) without analogue digital conversion. Those signals are then provided to entity 60, and optionally to entity 80, for further provision to loudspeaker and to additional signal processing units. In the second realisation option, the input analogue signals from 20 (eg from 90) are AD converted, further processed by the digital means of processing, and after that by the means of DA (digital analogue) converters, analogue audio signals are provided to unit 60. Generally, for wireless connection functionality 80, digital signals are provided without DA conversion. Proposed and described digital processing means are comprising of: digital filtering, digital down conversion and other arbitrary type of the digital processing. The actual HW platform for the execution of the signal processing may be arbitrary CPU circuitry (like typical ARM processor platform with typical

environment), or additional θ unit with AD and DA converted and FPGA hardwired signal processing circuitry. Practically, if high performance CPU, like ARM is used, the complete functionality 40 may be realised by the same HW.

[0043] Wireless connection functionality 80 is an optional HW electronics being incorporated in the Apparatus 100. This HW feature is allowed by introducing additional enhancements to the basic Method of operations, new benefits for the user. Functionality 80 may incorporate short range wireless connectivity by its plurality in system and in the operating frequency bands that is in compliance with one or more following communication standards:

• Wireless LAN standards, typically standardised by IEEE, utilising 2.4 GHz, 5 GHz frequency band, 60 GHz frequency bands

• High Data Rate UWB Standards in 3-10 GHz Band

• Low Data Rate UWB pulsed based standards in 3-10 GHZ Band

• 463 MHz, 865 MHz and 915 MHz low data rate communication

standards including ZigBee protocol options

Bluetooth standards including low power Bluetooth by the plurality of the profiles

• New emerging 60 GHz communication standards

• DECT standards

Proprietary communication system characterised in using ISM band communication systems

[0044] This functionality is allowing wireless interfaces from 1-3 meter up to

several hundred meters. Three major reasons for the introduction of the functionality 80 are outlined:

Transmitting audio signal data (digitally modulated) to the micro receiver being positioned in the ear of the user, without using wired connection functionality like shown in Fig 4.

Two ways communication to the external processing unit, typically to the hand held device like shown in Fig. 4.

• Two way communication for exchanging the information obtained by the radar sensor to the external user owned database, which is placed on cloud, or on other memory entity. This may be done for example to profiles of the walking, with time stamps and with satellite location information. For example, if the user is taking his usual path, path is part of the specific moving profile and as such may be preload for the better navigation of the user.

[0045] Satellite position functionality 43 is an integrated receiver for global

position system by the plurality of the technologies and arbitrary

combination of the global system providers: GPS, Galileo, Glonass and others. The location information from functionality 43 may be combined with radar sensor information coming out of functionality 30, and information coming out from Accelerometer and Gyroscope sensor functionality 44.

[0046] Accelerometer and Gyroscope sensor functionality 44 is realised by the plurality of the technology. Advantageously 6 degrees of freedom is deployed for accelerometers and gyroscopes integrated in one package with MEMS technology. This optional functionality allows calculation of the precise position by combining the updated position coming from global satellite positioning system functionality 43. This calculation must be performed in the signal processing functionality 30 or in the remote Further signal processing unit 220, like indicated in Fig. 4, by transmitting signals over wireless connection functionality 80. In the case of the specific pre defined event, for example the detection of the specific object by the radar sensor, and it behaviour (speed for example), this event may be precisely geographically located. This information may be further used by updating the remote database, or for recalculating exact position or for other warning measures to be provided by audio means to the user.

Method of Operation

[0047] The basic Method operation complexity class 300 is firstly described, regarding Fig. 7. The user carries the apparatus 1 00, advantageously in spectacles like look, like in Fig. 1. The user turns on the apparatus and adjusts the loudness level via HMI functionality 41 . The signal processing unit is sending no audio alarm to the user meaning that the ACCU power supply functionality 42 has enough capacity for system operation. The user is moving his head left and right as a basic movement. In each azimuth direction where the head "is looking", the radar sensor being advantageously integrated in mm-wave front end IC 90 is active, by sending the radio signals through its sub-system radio wave transmission functionality 10 and receiving the radio signals by radio wave receiving functionality 20. These signals are analogy processed by signal processing functionality 30 in real time. The outputs of the signal processing functionality 30 are audio spectrum signals with specific frequency in audio spectrum and with specific signal strength. Due to the narrow antenna gain of the High antenna functionality 50, spatial differentiation in azimuth may be easy achieved. In this dedicated angle of observation the distance to the related object and its velocity may be qualitatively measured and transformed in audio signal. This

transformation is done by changing intensity and frequency of the audio spectrum, so the different distances and speeds of objects are mapped differently. Advantageously the mapping of the information in audio spectrum is done in the way, that if there is no movement of the obstacle, the specific frequency in audio spectrum will be used as a default frequency, and it is chosen to have decent, pleasant tonality for majority of the users, advantageously somewhere between 1 KHz and 4 KHz.

Considering the average expected speed of the user and his secure step forward, for the user is very important to get an information instantly if there are any object in the vicinity of 2 meters or below. The system may be set (limited on the system communication range) to produce sound intensity only if after some distance some object in specific spatial angle in azimuth is detected. For example, there is no signal sent to the ear if the distance to the object in some angle is larger than 3 meters. As the object gets closer than the reference detection distance (in this case 3 m), the intensity of the signal is increasing. By training the ear the user may very easily understand the distance to the object. The user may set this threshold for detection by using HMI functionality 41 . By moving the head in different angle position the user can easily scan all obstacles in the azimuth in a very short time. If there is a narrow obstacle in specific direction like a person standing or moving or like a traffic light or public light column, and the user is making fast movement from one side to another, the user will hear the sound with some frequency in very limited time corresponding to specific direction, and after that, the user may decide to "look" more in this direction and get the knowledge where the object is. The beauty of the proposed invention is that by passing the same path more times, the user will obtain the audio map of the surface, and remember the static obstacles, quite easily. That means without having complex machinery, this invention is providing affordable lightweight solution, where the user indirectly may obtain the information of the object in the similar feeling like today flight operators showing planes in different directions. If the frequency is changing by changing the intensity the user is getting the information that some object is moving with the specific velocity projection toward him. If the frequency and intensity decreases that may mean that the object is moving away of the user and contrary if the frequency increases as well as intensity that means that the object in specific spatial angle is moving toward the user. Practically if the user is for example in front of zebra crossing by looking on one side he may get the audio information if there is a moving object coming, before the decision to cross the street is done. It should be denoted that the antenna shape of the high gain antenna 50, which is generally wide band in elevation may be adopted in the way that the diagram may have maximum not necessarily in horizontal plane like shown in Fig. 2. That may give special freedom to the user to move also slightly the head up and down by scanning the environment. By introducing more rows of the radiating elements of the high gain antenna functionality 50 like in Fig. 5, the beam in elevation plane will be smaller, and the maximum gain will be increased. The disadvantage in this case is more required space for the antenna systems and therefore more "bulky" apparatus 100.

[0048] Advantageously the method of the operation 300 may be extended by Method of operation 310, if at least one of the following extended HW options are present:

• The signal processing unit 30 has more capability and more

processing power and the apparatus 100 has more memory If wireless connectivity functionality 80 exist, allowing connection to the computing and memory resources of the Further signal processing unit 220, where functionality 220 are realised by the plurality of the realisations options, and which may be part one of the different systems like: smart phone, tablet, remote PC, virtual processing resource on cloud, or dedicated processor functionality being declared as wearable or implanted processing functionality by the user of the Apparatus 100.

[0049] In this case the Method of operation 300 includes an option to perform the classification of the objects being detected by the specific person in 310:

The scattering reflection pattern, including also "scanning information" coming out of the user's head movement having apparatus 100, for the extending data base with reflection patters regarded to the specific user, group of the users or predefined reflection patterns. If the detected response is related to the specific group of the predefine objects, after performing dedicated pattern recognising by performing additional signal processing by functionality 30 or 220, the signal processing units 30, and 220, may issue an audio alert, being defined as audios signal with arbitrary content. Advantageously this may be object naming being sent to the user. In this case, signal processing functionalities 30 and 220 generate voice by artificial voice/speech synthesizers.

[0050] Advantageously the method of the operation 300 and Method of operation

310 may be extended by Method of operation 320, if at least 43 of 43 and 44 extended HW functionalities are present:

Satellite positioning functionality 43 is included in the Apparatus 100 Accelerometers and gyro sensor functionality 44 is included in the Apparatus 100

[0051] In this case the Method of operation 300, being extended by the method of operation 310 includes an option to perform additional navigation support to the user defined as method of operation 320:

[0052] The position of the user is determined by the global positioning

functionality 43, and this information is sent in parallel with radio scattering information coming from functionality 90 to the signal processing unit 30 or via functionality 80 to the further signal processing unit 220. Signal processing units 30 or signal processing unit 220 are then using localisation information, matching it with scattering information coming from functionality 90. The joint information is much more better than localisation information alone (precision of the satellite systems are limited typically more than 5m), because the non moving objects are being detected, and classified from the data base, as well as geographically compared with the map information of the area where the user is moving. This would be comparable to improving the accuracy of the satellite position functionality to the distances in the one meter or less accuracy. Results of applied method of operation 320 are offering additional information benefits to the user, containing:

Object identification

Improved localisation information

• Warning (Alerts) on potential obstacles in the moving part of the user.

[0053] Accounting signal information coming out of Method of operation 320 may be realised by plurality of the audio signals, whereby advantageously the speech signals are proposed by this invention. In this case signal processing functionalities 30 or 220 are generated the voice signals with voice/speech synthesizers.

[0054] By the inclusion of the functionality 44, the localisation information may be improved. Namely the localisation signals coming from the satellites are typically available in specific time period. Between two periods the user is moving and changing his position. By having inertial sensor, the position of the user between two satellite localisation refresh, the information or real time position is calculated by the functionality 30 or functionality 220 and used for the Method of operation 330 featuring: enhance the overall accuracy, object detection probability and quality of the warnings/alerts. The quality of the warnings/alerts is defined as increased event detection probability, or prediction of possible events in the future defined as predictability in larger future time step.

[0055] Furthermore the existing functionality 44, combined with processing

functionalities 30 and 220, are offering capabilities for Method of operation 330, where the features like

Trajectory recovery of the user may be used for behaviour analysis by the third parties

Additional body position and movement relevant event detection which may be used by the third parties

Precise mapping of the microenvironment, where the person is moving [0056] Typical application of this proposed Method of operation is described in the case of the Fireman using proposed Apparatus 1 00 and Method of Operation 300, 310, 320 and 330. The fireman could use the apparatus 100 in the shape of the spectacles, described in Fig. 1. After his entrance in the burning house that is filled with a dense smoke he could navigate in the house by using radar scattering, and audio information obtained from the remote processing unit in front of the burning building being monitored by his colleagues. He is connected by wireless means of 80, and his trajectory and local relative position (inertial sensor and gyroscopes) is transferred to outside of the building, combined with radar reflection patters coming out of the walls, doors, which are also known from the building plans and matched on outside processing unit 220. This will definitely improve his local micro location and help him by audio means to find direction, or in case of accident or injury, to send rescue team to pick him up. In that case, proposed invention may be usefully also for protecting human lives in giving more security and strong advantages.

Claims

1. Claim 1

Apparatus 100 comprising:

a) Radio wave transmitting functionality 1 0, being realized by the plurality of the semiconductor technologies, and by the plurality of the semiconductor sub- functionality integration level, with transmitting frequency being larger than 24 GHz

b) Radio wave receiving functionality 20, being realized by the plurality of the semiconductor technologies, and by the plurality of the semiconductor sub- functionality integration level, with transmitting frequency being larger than 24 GHz

c) Signal processing functionality 30, with the plurality of the analogue and digital signal processing means, with audio signal output in the audible spectrum and signal intensity being hearable by the human

d) Tx high gain antenna functionality 50, being realized by the plurality of the realization technologies with high gain radiation diagram in at least one dimension being connected to the transmitting functionality 10

e) Rx high gain antenna functionality 55, being realized by the plurality of the realization technologies with high gain radiation diagram in at least one dimension being connected to the receiving functionality 20

f) Wired connection functionality 60 being connected from one side to the signal processing functionality 30 and on the other side to the loudspeaker approaching human ear 210

g) And support structure functionality 70, being characterized as a mechanical structure by the plurality of the realization options, and in the realization option applied materials, mechanically connecting functionalities 10, 20, 30, 40, 50, 55 and 60 providing mechanical stability and mechanical connection to the head of the human 200

2. Claim 2

Like a Claim 1 , where the support structure 70 has a shape of the typical spectacle frame being attached on the human head 200 and human ears 210.

3. Claim 3

Where the Apparatus 100 described in the previous claims contains additional short range communication functionality 80 which is integrated in the support structure 90, with the plurality of the short range and long range wireless communication standard means.

4. Claim 4

Like a Claim 1 , where the high gain antenna functionalities 50 and 55 are realized as at least one row of the planar radiation elements being distributed along front part of supporting stricture 70, having in elevation direction wide angle radiation pattern 52 and in the azimuth narrow High gain radiation patern 51 , which is inherently achieved by having 4 or more radiation elements in respective row for entity 50, and entity 55, respectively.

5. Claim 5

as previous claims where analogue functionalities 10 and 20 are incorporated in mm-Wave integrated circuit 90, realized by the plurality of the

semiconductor technologies.

6. Claim 6

as previous claims where mm-Wave integrated circuit 90, signal processing functionality 30, and system control functionality 40, are integrated in System on Chip 95, realized by the plurality of the semiconductor technologies.

7. Claim 7

as previous claims where high gain antenna functionalities 50 and 55 have a capability of steering the antenna gain in the different directions, by the plurality of the realization options for providing steering.

8. Claim 8

as previous claims where the Apparatus 100 is containing Satellite position functionality 43, released by the plurality of the technologies, supporting the plurality of the existing satellite position systems.

9. Claim 9

as previous claims where the Apparatus 100 is containing Accelerometers and Gyroscope Sensor functionality 44, released by the MEMS technologies.

10. Claim 10

Method of Operation 300 where the functionality 10 is sending radio signals, through antenna functionality 50, where functionality 20 is receiving the radio signals, where the signal processing functionality 30 is generating audio signals, being hearable by the human in the specific hearing audio frequency and audio intensity signal level, where the distance to the obstacle in front of the supporting structure 70 is corresponding in the different audio signal intensity levels, which are after that, transmitted by the wired means of 60 to the human ear 210.

11. Claim 11

Method of Operation 300 where the functionality 10 is sending radio signals, through antenna functionality 50, where functionality 20 is receiving the radio signals, where the signal processing functionality 30 is generating audio signals, being hearable by the human in the specific hearing audio frequency and audio intensity signal level, where the velocity to the obstacle in front of the supporting structure 70 is reflected in the different audio frequencies within audible spectrum, which are after that, transmitted by the wired means of 60 to the human ear 210.

12. Claim 12

Method of Operation 300 like described in Claim 10 and Claim 11 , where the functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, depending on reaching one or more pre defined event states, where pre defined events states, are related to obstacle distance and obstacle velocity in front of the human.

13. Claim 13

Method of Operation 310 like described in Claim 10, Claim 11 , and Claim 12 where the functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, depending on additional information of detecting obstacle, where the processor unit 30 is performing object recognition by comparing the reflected radio signal pattern with classes of the memorized patterns, related to the predefined obstacles and moving status of those obstacles, where audio signals are audio information containing words.

14. Claim 14

Method of the operation 310 like in Claim 10, Claim 11 , and Claim 12, where the signal processing results of the entity 30 are transmitted by the wireless communication means entity 80 to the further signal processing unit 220, where this unit is performing object recognition by comparing the reflected radio signal pattern with classes of the memorized patterns, related to the predefined obstacles and moving status of those obstacles, sending then processed information back to the unit 30, which is that responding with sending audio information via wired connection functionality 60 to the human ear 210, where audio signals are audio information containing words.

15. Claim 15

Method of the operation 310 like in Claim 10, Claim 1 1 , and Claim 12, where the signal processing results of the entity 30 are transmitted by the wireless communication means entity 80 to the further signal processing unit 220, where this unit is performing object recognition by comparing the reflected radio signal pattern with classes of the memorized patterns, related to the predefined obstacles and moving status of those obstacles, and generating audio signals and sending them through own wireless communication means 230 remote functionality to the human ear 210 having loudspeakers with the enabled wireless interface 240, where audio signals are audio information containing words.

16. Claim 16

Method of the operation 320 like in Claim 13, Claim 14, and Claim 15, where apparatus 100 is containing satellite positioning functionality 43, providing to the signal processing unit 30 position information where this unit is performing object identification using location awareness, including warnings of other preselected objects in the vicinity of user, where functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, where audio signals are audio information containing words.

17. Claim 17

Method of the operation 320 like in Claim 13, Claim 14, and Claim 15, where apparatus 100 is containing satellite positioning functionality 43, providing to the signal processing unit 30, where the signal processing results of the entity 30 are transmitted by the wireless communication means entity 80 to the further signal processing unit 220, where unit 220 is performing object identification using location awareness, including definition of the warnings of other preselected objects in the vicinity of user, sending this information back via functionality 80 to functionality 30, where functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, where audio signals are audio information containing words.

18. Claim 18

Method of the operation 320 like in Claim 13, Claim 14, and Claim 15, where apparatus 100 is containing satellite positioning functionality 43, providing to the signal processing unit 30, where the signal processing results of the entity 30 are transmitted by the wireless communication means entity 80 to the further signal processing unit 220, where unit 220 is performing object identification using location awareness, including definition of the warnings of other preselected objects in the vicinity of user, and generating audio signals and sending them through own wireless communication means 230 remote functionality to the human ear 210 having loudspeakers with the enabled wireless interface 240, where audio signals are audio information containing words.

19. Claim 19

Method of the operation 330 like in Claim 16, Claim 17, and Claim 18, where apparatus 100 is containing accelerometer and gyroscope positioning functionality 44, providing to the signal processing unit 30 relative acceleration and body orientation position information, where this unit is performing real time accurate absolute position of the user, including precise mapping of the user in the micro environment, where functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, where audio signals are audio information containing words.

20. Claim 20

Method of the operation 330 like in Claim 16, Claim 17, and Claim 18, where apparatus 100 is containing accelerometer and gyroscope positioning functionality 44, providing to the signal processing unit 30 are transmitted by the wireless communication means entity 80 to the further signal processing unit 220, real time accurate absolute position of the user, including precise mapping of the user in the micro environment, sending this information back via functionality 80 to functionality 30, where functionality 30 is sending audio alerts with predefined content by the wired means of 60 to the human ear 210, where audio signals are audio information containing words.

21. Claim 21

Method of the operation 330 like in Claim 16, Claim 17, and Claim 18, where apparatus 100 is containing accelerometer and gyroscope positioning functionality 44, providing to the signal processing unit 30 are transmitted by

* the wireless communication means entity 80 to the further signal processing unit 220, real time accurate absolute position of the user, including precise mapping of the user in the micro environment, and generating audio signals and sending them through own wireless communication means 230 remote functionality to the human ear 210 having loudspeakers with the enabled wireless interface 240, where audio signals are audio information containing words.

22. Claim 22

Like all previous claims where the radio frequency related to the entities 10 and 20 and are realized as integrated mm-wave circuit comprising of the following integrated building blocks: PA (power amplifier functionality) 91 , LNA (low noise amplifier functionality) 92, Quadrature mixers 96, VCO (voltage control oscillator) 93, Prescaler functionality 94, Baseband conditioning functionality 97, Splitter functionality 98, realized by the plurality of the semiconductor technology.

23. Claim 23

like all previous claims where the radio frequency related to the entities 10 and 20, 50 and 55 are in 57-64 GHz Frequency Band.

24. Claim 24

Method of operation like in previous claims, where additionally the apparatus 100, and sub-system 90 is used for data communication with other communication system in the same frequency, where plurality of

communication system modulation and demodulation schemes, are performed in the signal processing unit 30.