On July 9, 2009, the US Patent & Trademark Office published a patent application from Apple that reveals various concepts behind a newly advanced facial detection and recognition system. Although the system described is primarily focused on use with portables such as the iPod and iPhone, it is evident that such a system will be implemented throughout Apple’s hardware lineup – and eventually spill over to future applications such as television, a vehicle navigation system, a video gaming system, video glasses and so forth, according to the patent. The system will be used to identify those in communications with users via various methods of messaging including email, instant messaging, video messaging, and/or user voice call. Yet the most advantageous aspect of this system will be found in its advanced security technologies that will go far beyond simple password protection. The user will be able to program the facial detection and recognition system so that only authorized faces could control access to specific applications, be it a spreadsheet or word processor app – or to even authorize the purchase of content at Apple’s iTunes Store.

Advanced Processing Environment

Apple makes the distinction between face detection and recognition very clear at the onset of this patent. Face detection and recognition are different processes. Face detection includes the process of detection and/or locating a face or faces within an image. Face recognition includes the process of recognizing that a detected face is associated with a particular person or user. Face recognition, however, is typically performed along with and/or after face detection.

Apple’s patent FIG. 4 shown below is a diagram of a computer processing environment 400. The processing environment may include a detection decision application 402, a face recognition decision application 404, and an input/output and/or application control application 406. The environment 400 may also include detection data 424 and recognition data 426, a face vector database 420 and/or an input/output interface configuration database 422. The detection data 424 may include, without limitation, data associated with knowledge-based detection techniques 428, feature-based detection techniques 430, template matching techniques 432, and/or appearance-based detection techniques 434.

The Input/Output Control Application

Apple’s patent FIG. 5 is a diagram of a face feature vector 500 including various facial features associated with a user or class of users according to an illustrative embodiment of the invention. The face feature vector 500 may include one or more elements such as, without limitation, eyes data 502, nose data 504, mouth data 506, chin data 508, face shape data 510, face areas data 512, face feature distance/angle/relation data 514, and/or skin color data 516.

In certain embodiments, the face feature vector may include other data associated with the detection data and/or recognition data. In one embodiment, with respect to face recognition, the vector 500 is derived from a detected face in an image, and used to identify a particular user’s face. In another embodiment, with respect to face detection, the vector is derived from a sensed image, and used to detect the presence of a face in the image.

In certain embodiments, the input/output control application determines an input interface feature and/or characteristic based on a decision signal from the decision application and/or decision application. In one embodiment, the input/output control application determines an alert output characteristic based on a decision signal from the decision application. For example, where the personal computing device is a cellular telephone, upon an incoming call, the device may sense whether the user is viewing its display. If the user’s presence is detected, the device may only provide a visual alert via the device’s display. If the user’s presence is not detected, the device may initiate an audible alert, e.g., ringtone, to alert the user about the incoming call. In this instance, the device may only apply face detection to determine whether any face is present and/or any person is viewing the device’s display.

Alternatively, if an incoming email is received by the iPhone, it may perform a face recognition step to identify the user. If the face of the user is recognized and/or authenticated, then the user is alerted about the email and the email may be made available to the user for viewing. If the face of the user is not recognized and/or authenticated, the iPhone may not initiate an email alert, and may hide, suppress, and/or block the content of the email from the unauthorized user.

Face Recognition Methodologies

The face pattern recognition application 416 may perform pattern recognition based on at least one of Bayes Decision Theory, Generative methods, discriminative methods, non-metric methods, algorithm-independent machine learning, unsupervised learning and clustering, and like techniques. The Bayes Decision techniques may include, without limitation, at least one of Bayes Decision Rule, minimum error rate classification, normal density and discriminant functions, error integrals and bounds, Bayesian networks, and compound decision theory. The Generative methods may include, without limitation, at least one of maximum likelihood and Bayesian parameter estimation, sufficient statistics, various common statistical distributions, dimensionality and computational complexity, principal components analysis, fisher linear discriminant, expectation maximization, sequential data, hidden Markov models, and non-parametric techniques including density estimation. The discriminative methods may include, without limitation, distance-based methods, nearest neighbor classification, metrics and tangent distance, fuzzy classification, linear discriminant functions (hyperplane geometry, gradient descent and perceptrons, minimum squared error procedures, and support vector machines), and artificial neural networks. The non-metric methods may include, without limitation, recognition with strings and string matching. The algorithm-independent machine learning techniques may include, without limitation, no-free lunch theorem, bias and variance, re-sampling for estimation, bagging and boosting, estimation of misclassification, and classifier combinations.

Apple credits Jeff Gonion (Campbell, CA) and Duncan Robert Kerr (San Francisco, CA) as the inventors of patent application 20090175509, originally filed in 2008.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. Feed patent number xx into this search engine for further details.

Jack Purcher, MacNN Senior Patent Editor

Apple’s redesigned 2009 iPod shuffle has been granted registered status in Europe under a series of eight design numbers starting with 001108526-001. On the same day, Apple’s iPod Shuffle is also shown to be a registered design in several other global databases including those in Singapore and China. The expiry dates range from 2014 in Europe and China to 2024 in Singapore. The design win for the iPod Shuffle has been credited to Apple’s Bartley K. Andre, Daniel J. Coster, Daniele De Iuliis, Evans Hankey, Richard P. Howarth, Jonathan P. Ive, Steve Jobs, Duncan Robert Kerr, Shin Nishibori, Matthew Dean Rohrbach, Peter Russell-Clarke, Douglas B. Satzger, Christopher J. Stringer, Eugene Antony Whang and Rico Zorkendorfer. Apple’s redesigned iPod Shuffle was introduced on March 11, 2009.

Jack Purcher, MacNN Senior Patent Editor

The US Patent and Trademark Office officially published a series of four newly granted patents for Apple Inc. yesterday and one of them was covered earlier this morning in our other report titled Apple Granted Patents for Smartbook & Find My iPhone Related Technologies. In fact, this is simply the second segment of that patent originally titled “Personal items network, and associated methods.” In this focused report, we explore Apple’s continuing march into sports monitoring devices and systems which we first covered in our March report. The difference in this report is that we see that the patent calls for a TV system to advance the concepts of movement and event monitoring systems in a very entertaining way. Lord knows that Apple TV needs a killer app of its very own to catapult this hobby device into being a contender for this all important in-home market segment. The patent also reveals an advanced heart rate monitor system and in many ways, Apple may be considering a Nike+iPod-like equivalent program for skiers, snowboarders and related sports.

Sophisticated Heart Rate Monitor

One of the key applications of this patent is that of a heart rate monitor strip. The wireless strip worn by the patient could notify medical staff that the patient is going into cardiac arrest.

In another embodiment, wireless data 17 is a signal indicative of actual heart rate, e.g., 100 beats per minute, such that monitor 10w may be used in fitness applications. Monitor 10w thus provides an alternative to “strap” heart rate monitors [such as a Polar] to monitor the heart in real time. Data 17 may be captured by a receiver such as a watch to display the data to the wearing user. The device could also be used for various other scenarios within a hospital environment such as monitoring a patient’s activity and to notify the appropriate personnel if a patient has fallen. It could also be set to monitor stress, pulse, respiratory rates etc.

Highly Advanced Sporting Event Sensor Monitoring

In Apple’s patent FIG. 8 shown below, we see two people engaged in karate training. Pluralities of Movement Monitoring Devices (MMDs) are attached to person “A” and person “B”. The data from these monitors are streamed to a remote reconstruction computer and receiver – which is noted below in FIG. 8A.

The Movement Monitoring Device strips are placed on persons A and B at appropriate locations, e.g., on each foot and hand, head, knee, and chest. In patent FIG. 8C we see a close up of boxer’s hand to illustrate that the strips could be placed on the hand under a boxing glove as well. So it’s not restricted to just karate.

In the case of the boxing gloves, we’re told that the Movement Monitoring Device includes an accelerometer oriented with a sensitivity axis aligned with a strike axis of the fist that transmits the strike data to the remote reconstruction computer and receiver through the gloves. Data transmissions from a monitor device of the invention, to a receiver, typically occur in one of three forms: continuous transmissions, “event” transmissions, timed sequence transmissions, and interrogated transmissions. In continuous transmissions, a monitor device transmits detector signals (or possibly processed detector signals) in substantially real time from the monitor device to the receiver to reconstruct the person’s activities and desired characteristics.

Possible Future Apple TV Features

Now this is where it could get very interesting in respect to monitoring sporting data for a future Apple TV application. Watching a fighting match is going to be taken to the next level and be a lot more interesting and engaging for the viewer. Accordingly the patent notes that data plots 154 and 156 (noted in context with FIG. 8A below) have further advantages for broadcast media in that data could be simultaneously relayed to the Internet or television (patent point 170) to display impact speed and intensity of blows given or received by persons A or B in real time, to enhance the pleasure and understanding of the viewing audience (i.e., viewers of television, and users of the Internet).

The fighter fan will be able to analyze such things as impacts per minute; average strike force per boxer; average punch power received to the head; average body blow power; and peak striking impact. Rotational information may also be derived with the appropriate detector, including typical wrist rotation at impact, a movement metric that may be determined with a spin sensor. The patent states that data 159 would be displayed in an easy to understand format, such as through bar graphs or other means. On this front however, I’m sure that Apple will amaze us with a sporting event monitoring user interface to boggle our minds.

The interesting part of this segment of the patent is that it’s just one sporting event example. You’ll have to use your imagination a bit here to realize that the monitoring systems detailed in this patent will apply to a plethora of sporting events well beyond fighting based contact sports. Think of football, downhill ski racing, extreme snowboarding, extreme dirt bike racing, baseball, soccer, hockey, wrestling, sports car racing, surfing contests and rodeo events like those at the annual Calgary Stampede – the greatest outdoor show on earth. I know this technology will extend to skiing and snowboarding as I noted in our opening summary that we covered this subject in March.

NASCAR Auto Racing

In 2006 NASCAR signed a video deal with Apple and in this patent we see that Apple could use this technology on both Apple TV and the iPhone. Apple’s patent describes the details of System 600 which is an application that includes an array of data capture devices 602 coupled to racecars to provide vehicle movement metrics.

In addition, the monitoring devices could also provide information to fans about any car’s speedometer, accelerometer, tachometer and/or gas gauges and could add spin sensors, temperature gauge, and driver heart rate sensors. An on-board computer can further provide position information about car 604′ position within the current race (e.g., 4.sup.th out of fifteen racecars). The system may also transmit data on events as they occur such as a car crashes, rollover or total rotations. The system may be available to customers as part of a certain event ticket package or available for a fee. If you accept it, then the race track information would be transmitted to your cell phone accordingly.

Resorts: Advanced Ski Lift Tickets

Apple’s technology will finally provide Ski Resort owners with a means to keep their ski trails safer in the future by being able to implement next generation Ski Lift Passes that will come with a built-in receiver and a multitude of advanced sensors including GPS. The monitors will allow resort owners to actually set ski hill speed zones that could be monitored for violators.

Ski Passes for specific runs will set speed limits of 35, 40 and higher mph and offenders could lose resort privileges, be fined or even removed from the premises for reckless behavior. The sensors will also be able to monitor if you’ve exceeded your jump limit and/or performed too many overhead flips. Some skiers may not approve of such restrictions, but for safety sake, it’s the best tool ever devised to assist ski resort patrollers. I think that this could eventually save lives over time – and you can’t beat that. On the positive flip side, the advanced ski lift passes will also be able to provide skiers with a Nike+iPod-like program for monitoring all manner of skill levels that you wish to challenge yourself with be it speed to a given time or an airtime value in jumps and so forth. I really enjoy the Nike+iPod program and could only imagine the fun it’ll bring to skiers.

Other Events Covered by the Patent

Apple credits Curtis A. Vock (Boulder, CO), Burl W. Amsbury (Boulder, CO), Paul Jonjak (Lafayette, CO), Adrian F. Larkin (Essex, GB) and Perry Youngs (Longmont, CO) as the inventors of granted patent 7,552,031, originally filed in December 2006. Technically, this patent may have an older history. It’s difficult to read the US Patent Document History associated with this patent.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or Issued Patent is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or Issued Patent should be read in its entirety for further details. For additional information on any granted patent noted above, simply feed the individual patent number(s) into this search engine.

Jack Purcher, MacNN Senior Patent Editor

The US Patent and Trademark Office officially published a series of four newly granted patents for Apple Inc. The notables within that group include one relating to a hybrid antenna that Apple prominently yet quietly assigns to that of smartbook. The second patent of interest is extraordinarily detailed and covers the widest range of future products imaginable. In fact this report will only cover one segment of the patent while the other information is presented in our supplementary report titled “Apple TV to Take Sporting Events to a Whole New Level.” This powerful patent was acquired by Apple as the key credit goes to an individual by the name of Curtis A. Vock of Boulder, CO. In fact, if truth be told, it would appear that Apple acquired this patent via the outcome of Civil Action Complaint 06-CV-02122 – PhatRat Technology LLC v. Apple Computer Inc. filed in Oct. 24, 2006. The patents known to be associated with PhatRat were “Personal Items Network, and Associated Methods” and “Activity monitoring systems and methods” covered in our March report. The technology found within this current patent covers subject matters ranging from the Apple iPhone 3G-S feature called “Find My iPhone,” to that of a heart monitoring system, an advanced Industrial Transport tracking system, a baby monitoring alarm system and other unique applications that could very well be worked into future iterations of Apple’s iPhone. Curtis A. Vock is credited with over 45 successful patents including those from PhatRat.

Granted Patent: Hybrid Antennas with Directly Fed Antenna Slots for Handheld Electronic Devices

Apple’s first patent generally relates to wireless communications, and more particularly, to wireless electronic devices and antennas for wireless electronic devices. Interestingly, the primary hardware that the patent focuses on initially is that of the ultraportable, which is technically defined by Wikipedia as a subnotebook/tablet-PC. The twist however, is that the patent is about a hybrid antenna consisting of one antenna for Wi-Fi/Bluetooth and the other for 3G data communications. When combined within the framework of a subnotebook that could run software such as an internet browser, voice-over-internet-protocol (VOIP) telephone call applications, email applications and media playback applications, we are then presented with the classic definition of what a smartbook is. The patent then shifts to defining “one suitable arrangement” which is described as “an example.” That example is that of the iPhone.

Apple’s key patent graphic example is that of FIG. 21. It’s a perspective view of an illustrative antenna having a directly fed slot antenna resonating element and an indirectly fed antenna resonating element in which an inductor is placed across the terminals of the antenna. FIG. 1 is a perspective view of an illustrative handheld electronic device with antenna structures in accordance with an embodiment of the present invention. Figure 1 is that of the iPhone.

The antenna structures and wireless communications devices support communications over any suitable wireless communications bands. For example, wireless communications devices may be used to cover communications frequency bands such as the cellular telephone bands at 850 MHz, 900 MHz, 1800 MHz, and 1900 MHz, data service bands such as the 3G data communications band at 2170 MHz (commonly referred to as the UMTS or Universal Mobile Telecommunications System band), the Wi-Fi (IEEE 802.11) bands at 2.4 GHz and 5.0 GHz (also sometimes referred to as wireless local area network or WLAN bands), the Bluetooth band at 2.4 GHz, and the global positioning system (GPS) band at 1575 MHz. The 850 MHz band is sometimes referred to as the Global System for Mobile (GSM) communications band. The 900 MHz communications band is sometimes referred to as the Extended GSM (EGSM) band. The 1800 MHz band is sometimes referred to as the Digital Cellular System (DCS) band. The 1900 MHz band is sometimes referred to as the Personal Communications Service (PCS) band.

Apple’s patent states in typical boilerplate fashion that “the foregoing is merely illustrative of the principles of this invention and various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention.” With that said, Apple covers every imaginable device on the planet including such things as a wrist-watch, pendant, earpiece, gaming device, GPS device, PDA, cell phone, media player etc.

Yet make no mistake about it: The initial product line listed for this patent under “detailed description” is that of the laptop and specifically “small portable computers of the type that are sometimes referred to as ultraportables.”

Apple credits Zhijun Zhang (Beijing, CN), Robert J. Hill (Salinas, CA), Robert W. Schlub (Campbell, CA), Juan Zavala (Watsonville, CA) and Ruben Caballero (San Jose, CA) as the inventors of granted patent 7,551,142, originally filed in December 2007.

Granted Patent: Personal Items Network, and Associated Methods

Apple’s patent generally relates to sensing systems monitoring applications in sports, shipping, training, medicine, fitness, wellness and industrial production. The invention specifically relates to sensing and reporting events associated with movement, environmental factors such as temperature, health functions, fitness effects, and changing conditions. The patent is divided into two distinct branches covering Movement Monitoring Devices and Event Monitoring Devices. In this particular patent, the basics of Apple’s background are important to read in order to grasp the scope of what is covered by this patent.

Apple’s Advances in technology have provided some quantification of movement. For example, GPS products now assist in determining the location of golf carts, vehicles and persons.

However, the detail of movement, minute to minute, second to second, is still not generally determinable in the prior art. For example, the movement of tangible objects typically involves (a) the shipment or carrying of goods and (b) electromechanical or motorized apparatus (e.g., planes, trains, automobiles, robots). The exact movements of such objects, and the conditions that they are subjected to, from point to point, are only qualitatively known. By way of example, a package is moved from location to location through delivery services like FEDERAL EXPRESS or UPS; however what occurred during transportation, and what transpired to the package, is anyone’s guess. Occasionally, an object within the package is broken, indicating that the package experienced excessive abuse; but whose fault it is, or how or when it happened – are not known. What environments the package experienced is also not readily known.

The movement of persons, on the other hand, typically involves human-powered transportation, e.g., facilitated by biking, a wheelchair, or a motorized vehicle, e.g., a car. Body movement involved in transportation is subjected to many forces, some of which are dangerous. But the prior art does not provide for this knowledge; there is no effective way, currently, to efficiently quantify human movement. In sports, physical fitness, and training, precise information about movement would assist in many ways. By way of example, how effective a hand strike is in karate or boxing is, today, only qualitatively known. Quantitative feedback would be beneficial.

It is, accordingly, one feature of the invention to provide systems and methods addressing the afore-mentioned difficulties. A further feature of the invention is to provide methods and devices to quantify movement in a number of applications. Another feature of the invention is to monitor and report meaningful environment information such as temperature and humidity.

Apple’s Find My iPhone

The patent first describes the technological make-up of a monitoring device which at one point describes port 16 as being that of a transponder that reflects event data to receiver 24. In one preferred embodiment, receiver 24 is part of the circuitry and packaging of a cell phone, which relays events (e.g., a movement event) to a remote storage facility. Think of that as being an essential component of Apple’s new “Find My iPhone” service.

In other embodiments, receiver 24 is part of the circuitry and packaging of a MP3 player, pager, watch, or electronic PDA. Receiver 24 may connect with headphones (not shown) to provide information to a user and corresponding to “event” data.

A monitor device 10 preferably includes an adhesive strip that provides for convenient attachment of the device to an object or person. As shown in FIG. 2, one such device 10” is shown coupled to adhesive strip 32 for just this purpose. Further, in one preferred embodiment, the detector 12 is also integrated within the ASIC as a solid-state accelerometer (e.g., using MEM technology). However, detector 12 can be a stand-alone element such as a piezoelectric strip, strain gauge, force-sensing resistor, weight sensor, temperature sensor, humidity sensor, chemical sensor, or heart rate detector.

Baby Monitor Alarm System

Apple’s patent covers an in-home baby monitoring alarm system that is applied to a child’s foot as if it were a band aid, as shown in FIG. 57 below. Once applied, the device synchronizes to baby body movement (such as repetitive movements associated with pulse or respiratory rate), breathing sounds or heart rate. The device then searches for “events” in the form of the absence of repetitive signals, indicating for example the danger of an absence of pulse, heart rate or respiration, in step 890. In step 892, the monitor device generates a wireless signal as a warning; that signal is received at a remote receiver at step 894. Once received, remote receiver generates an audible alarm (e.g., a buzzer sounds) or visible alarm (e.g., an LED is lit), in step 896.

As an alternative, the detector of the monitor device is a temperature (or alternatively a humidity) detector, and the alarm monitor merely tracks infant temperature for worried parents; such a device is useful for sick infants in particular. The temperature sensor can be coupled with other detectors (e.g., heart rate) to provide multiple functions, if desired.

The MMDs and EMDs of the invention thus have several other advantages. They may be used discretely and safely as medical diagnostic and monitoring detectors. With appropriate detectors, EMDs of the invention can for example provide for portable, wireless pulse oxymeters or blood glucose monitors. With the appropriate detectors in MMDs, rehabilitation clinicians would be able to quantitatively monitor metrics such as limb movement and balance. EMDs equipped with certain detectors may find use as real time, remote and inexpensive pH monitors and blood gas monitors.

Electronic Drink Coaster

Apple’s patent introduces us to the electronic drink coaster. Do I laugh now or later? Then again, once refined, it could actually have some benefit to both consumer and service establishment. The overview design is noted below as FIG. 70. The devices internal electronics sense the weight of a drink on the e-coaster (not the patent’s term) to automatically inform a restaurant or bar attendant wirelessly that the customer needs a new drink or refill. In one embodiment, a customer can also place an order from this electronic coaster. If it’s actually built into the table, then in might just work. Yet I don’t really think this falls into any feature or product line that Apple will invest time in.

Advanced Tracking System Senses Damages Apple’s patent provides us with patent FIG. 72 which covers a package management system and sensor. The sensors could be integrated directly into the shipping labels so as to communicate such data as impact, temperature, humidity etc.

By way of example, if the detector is an accelerometer and the MMD is designed to monitor “impact” (e.g., acceleration events that are less than about 1/2 second)–and yet impact data is not considered interesting unless the MMD experiences an impact exceeding 50 g’s–the preferred MMD used to accomplish this task would continuously monitor impact and tag only those impact events that exceed 50 g’s. The “event” in this example is thus a “50 g event.” Such a MMD is for example useful when attached to furniture, or a package, in monitoring shipments for rough treatment. The MMD might for example record a 50 g event associated with furniture shipped on Oct. 1, 2000, from a manufacturer in California, and delivered on Oct. 10, 2000 to a store in Massachusetts. If an event stored in MMD memory indicates that on Oct. 5, 2000, at 2:30 pm, the furniture was clearly dropped, responsibility for any damages can be assessed to the party responsible for the furniture at that time. Accuracy of the time tag information can be days, hours, minutes and even seconds, depending on desired resolution and other practicalities.

Apple credits Curtis A. Vock (Boulder, CO), Burl W. Amsbury (Boulder, CO), Paul Jonjak (Lafayette, CO), Adrian F. Larkin (Essex, GB) and Perry Youngs (Longmont, CO) as the inventors of granted patent 7,552,031, originally filed in December 2006. Technically, this patent may have an older history. It’s difficult to read the US Patent Document History associated with this patent.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or Issued Patent is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or Issued Patent should be read in its entirety for further details. For additional information on any granted patent noted above, simply feed the individual patent number(s) into this search engine.

Jack Purcher, MacNN Senior Patent Editor

On June 18, 2009, the US Patent & Trademark Office published a patent application from Apple that reveals one of the next chapters for Apple’s iPhone navigation systems and services. Implementations of the invention can realize one or more of the following advantages. The driver of a vehicle can be provided real-time information related to the operational state of the vehicle (e.g., the level of fuel in the vehicle). In some implementations, the driver is automatically provided advance warning of a refueling location and information about where the vehicle can conveniently be refueled. In other implementations, the driver is provided notification of a vehicle maintenance issue and given immediate information about conveniently located vehicle repair stations. These features can reduce driver distraction, reduce the risk of the vehicle running out of fuel and thereby improve road safety for the driver. The system also introduces the concept of filtering the information streamed to you. For instance, you may drive a Mercedes-Benz or other high end vehicle and so for repairs on the road, the system will only point out service stations for that model. The filter also allows users to choose their favorite gas stations, a required fuel type or even the types of restaurants you want to eat at while filling up. On the road, and only on the road, I look for Denny’s Restaurants – and so having them pointed out to me on this navigational system would help me plan a better trip. And lastly, the patent indirectly points to the navigational system being able to pick up television signals.

iPhone with Fuel Module

In Apple’s patent FIG. 1C we see the iPhone is configured to an in-vehicle computing system. This will be done by tethering, docking or simply by wireless means. It is here where we’re introduced to a new iPhone “fuel module” noted in the patent figure as feature 106. Apple notes that in some cases the iPhone will use the fuel module included with the vehicle’s computer system and even states broadly that the iPhone could actually be the both the navigation and vehicle computing system. At present it’s unknown whether or not the fuel module is already dormant in the current iPhone 3G-S.

Mapping Restaurants, Gas Stations & Repair Shops

Apple credits Joshua V. Graessley (Sunnyvale, CA) as the sole inventor of this patent application.

To see the latest 3G-S iPhone Navigational services just introduced at this month’s Apple developer conference – see Maps + Compass or just take a peek at a compressed list of the many services below.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. Feed patent number 20090157289 into this search engine for further details.

Jack Purcher, MacNN Senior Patent Editor

Fourteen months ago I wrote a very popular report called “Will Apple Open a Store in Second Life?” which was based on Apple’s patent application titled Enhancing Online Shopping Atmosphere . In it I presented evidence of Apple’s first look into creating an online retail-social environment for avatars. Today, we see Apple wildly advancing their technology on this front in respect to direct human-to-avatar expressions. When your head will turn right to left, so will your avatars in realtime. The technology takes advantage of Apple’s built-in iSight camera through a process described in the patent in which the camera sets your face up as a profile that replaces the mouse. As an avatar, your three-dimensional head is tracked by the video camera using motion vectors, which gives you the ability to move in six degrees of freedom, including up, down, left, right, backward, forward and free rotation etc. The avatar representation can be broadly applied in computer technology, whether in the form of a three-dimensional model used in computer games, a two-dimensional icon (picture) used on Internet forums or chatting websites.

One game that Apple singles out as an example is a volleyball game. The patent states that in some implementations, the two objects being visually tracked can take turns to move as well as manipulate a graphical icon shown on the display. In an example where the graphical icon is a volleyball, two people can take turns to move and spin the volleyball by simply moving their heads.

Enterprise Smart 3D Videoconferencing

In some implementations, the multiple displays are connected by networks. Two or more users can, on their respective displays, observe and control focus of a single desktop, or that of a presentation of a shared digital file. By way of illustration, when two or more users chat online and discuss a file presented on their respective displays, each of them can transition focus of the presentation of the file by moving his or her head.

In the same way that you could move your head as an avatar in an online environment, the enterprise worker could replace the cursor to being that of a pen or stylus to point at documents during a 3D desktop video conference. The worker could point to various points of a document or specimen as if he were in this shared workspace environment. Imagine the simplicity of the cursor being a representation of your hand. It would be freaky, but natural. Imagine the hand flipping through documents in Cover Flow and pulling out the file the worker wants to present to a colleague. The most important aspect of this invention is that it will work on OS X and Windows.

The patent goes on to state that example system objects in the workplace will include data files, applications, functions, windows, etc. In some implementations, the one or more windows (or applications user interfaces) shown on a display include an editor or a browser. A user can scroll a digital file presented by the editor or the browser in different directions such as up, down, left, and right simply by moving his head or the other object. For example, say there is a five-page file presented by a word processor on a display, and only the first page is shown within the workspace of the word processor. If the user would like to see the last page of the file, he could simply move his head down to scroll the digital file down to the last page. Optionally, he could flip the digital file page-by-page by slightly moving his head down over a relatively short distance.

One of the primary objectives of this patent is to allow a user to be hands free while navigating a user interface or interacting with an application on a computer system. The implications of this for the handicapped are immeasurable and will only add to Apple’s work in this important area.

Apple: Keep it Alive!

At the end of the day, it would appear with this second patent on avatar creation and development that Apple is taking this project very seriously. In typical Apple fashion, they’re applying incredible science to this project and have several consecutive real-world applications on the go that will service many important areas of the market. It will assist the handicapped, the enterprise worker and the consumer. We’ll see rich 3D interactive videoconferences and/or work sessions and experience next generation games and social networking applications. Apple’s previous patent demonstrated that they desire to bring a virtual Apple Store to the masses which would be a great tool to strengthen MobileMe.

All in all this is a great peek into Apple’s war room where next generation applications and devices are being developed for our continuing enjoyment. The only fly in the ointment is that the credited engineer behind this patent is Kevin Quennesson – The very same Kevin Quennesson that has recently left Apple for Cooliris. Let’s hope that Apple’s ongoing work on avatars doesn’t virtually disappear.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. Feed patent number 20090153474 into this search engine for further details.

Jack Purcher, MacNN Senior Patent Editor

On June 18, 2009, the US Patent & Trademark Office published a patent application of Apple’s which generally relates to a three-dimensional movie browser or editor. While the three-dimensional movie browser or editor described in the patent focuses primarily on a computer system using Final Cut Pro, it’s apparent that the technology in its more simplest of forms works itself down into other devices like the iPod touch, iPhone, Apple TV and/or other future devices be they televisions or a game console. The 3D browser could be used on any browser or application window such as QuickTime, iTunes, iMovie and sub systems such as DVD menus or DVD scene searching tools. Apple’s Cover Flow is a familiar and simple example of a 3D movie browser. As an interesting side note, the sole author of this Apple patent, Kevin Quennesson, left Apple in April of this year to join Cooliris , creators of the Cooliris 3D Wall. Cooliris is also available for the iPhone at the App Store. It makes you wonder how Apple ever let this creative engineer go and the direction that Cover Flow could have taken with him still on board.

Apple’s 3D Browser/Editor Image Analysis

Apple’s patent FIG. 3 (shown below) is a screen shot showing the three-dimensional content browser or editor with video frames displayed concurrently along a timeline of FIG. 1 with image analysis tools displayed along the timeline proximate to the video frames. The patent figure is included under the coverage of the software’s “Image Analysis” feature. This must be a part of Final Cut Pro as it provides image analysis tools that aren’t part of your every day simple iMovie tools – such as image histograms, image projections, image apertures, image profiles, image scattergrams, motion graphs (e.g., graphs that show the optical flow or amount of motion between frames), and image transforms (e.g., Hough Transforms and Fast Fourier Transforms).

Apple’s 3D Browser/Editor Software Stack

Apple’s patent FIG. 15 is a block diagram of an example software stack 1500. In some implementations, the software stack includes an operating system OS kernel 1510 (e.g., a UNIX kernel), a library system 1520, application frameworks 1530 and an applications layer 1540.

The library system 1520 provides various services applications running in the application layer. Such services can include audio services, video services, database services, image processing services, graphics services, etc. For example, the browser can make use of such services as QuickTime, Core Image, Core Video and OpenGL.

Simple 3D Movie Browser: Cover Flow

On Apple’s website under Apple TV Movies, you’ll find a video that shows you how Apple TV selects movies etc. On iTunes you could flip through movies for purchase or for rent using Apple’s Cover Flow feature as shown below.

Another Noteworthy Patent Applications Published Today: Another interesting set of Apple patent applications published today generally relate to Apple TV and more specifically the remote associated with it. This is really just a follow up to their more detailed patent 20090066648 titled GUI Applications for use with 3D Remote Controller published in March 2009. It was touted by some as Apple’s Magic Wand Patent. If you use Apple TV, then there’s not much to this patent for consumers. Apple’s four published patents offer but a handful of refinements. On technical grounds however, the patents go into more detail about the technologies that make the remote controller function – such as Z-direction input mechanisms or motion sensors. Yet in respect to the consumer refinements, the patent discusses such things as adding haptics to the remote; adding an on-screen virtual keyboard with predictive spelling; better zooming capabilities to get more details where applicable and an interactive screen saver which includes playing a series of photos from your iPhoto collection. The virtual keyboard for Apple TV is unlikely to be all that great but to Apple’s credit they added the new Remote iApp. Using the iPod touch or iPhone’s built-in keyboard capabilities for Apple TV will allow you to use its virtual keyboard and type in what you want a lot quicker than with a remote controller. See patents 20090158203, 20090153478, 20090153475, 20090158222 and 20090153389 for more details.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. Feed patent number 20090153648 into this search engine for further details.

Jack Purcher, MacNN Senior Patent Editor

The US Patent and Trademark Office officially published a series of 12 granted patents for Apple Inc. today. The notables within this group include one relating to Apple’s Final Cut Studio Soundtrack Pro 2 Surround Mixing wheel which covers surround sound mixing, another relating to Apple’s Finder, one that is interconnected with the iPhone’s GPS and finally one relating to iPhone theft detection.

Final Cut Studio: Surround Mixing

Apple’s patent generally relates to audio signal mixing techniques and, more specifically, to a technique for mixing multiple input channel signals, based on locations of indicators in a GUI, to generate multiple output channel signals. Specifically, the patent covers “Surround Mixing” which could be found in Final Cut Studio.

Apple’s patent states that “At some time after the multiple channel signals have been recorded, one might want to mix the multiple channel signals into an even greater number of output channel signals. For example, one might wish to take two input channel signals and mix them into four output channel signals to produce more of a “surround sound” effect.” The surround mixing tool can be found in Final Cut Studio as shown above – and in patent FIG. 1 below.

Apple credits William George Stewart (Cupertino, CA) and Michael Stephen Hopkins (Mountain View, CA) as the inventors of granted patent 7,549,123 for “Mixing input channel signals to generate output channel signals,” originally filed in June 2005.

Apple Finder Feature

Apple’s patent generally relates to computer systems with graphical user interfaces, such as window based systems; and more particularly to techniques for finding, moving, and copying objects in such systems. Specifically, the patent relates to Apple’s Finder’s spring loaded folders.

A new behavior of the graphical user interface has been provided which allows a user to open and close enclosures, such as folders in the Macintosh Finder environment, while dragging some other object. When the user pauses, gestures, or rolls over a hot spot on the object during the drag, a temporary window corresponding to that object is opened on top of the cursor. This allows the user to browse inside the enclosure and possibly open a hierarchy of enclosures contained within the newly opened window during the drag operation. The user thus has access to the entire storage system hierarchy during a drag operation. By using the spring loaded enclosure mechanism, the user is left free to browse while dragging, rather than being forced to set up source and destination windows before a drag begins. This greatly improves the basic copy and move functions provided by the graphical user interfaces based on windows and icons.

In FIGS. 14A-B, enclosure icon 1400 and a dragged icon 1401 are shown. The enclosure icon 1400 has a hot region, or temporary window open region, 1402 and a main box 1404. If the cursor is moved into the hot region 1402, as illustrated in FIG. 14B, then the sprung open enclosure will be open. If the cursor does not hit the hot region 1402, then no action occurs. If the mouse button is released over the main box 1404, then the dragged object goes inside the enclosure.

Apple credits Thomas J. Conrad (San Jose, CA) and Yin Yin Wong (Menlo Park, CA) as the inventors of patent 7,549,126 for “Computer system with graphical user interface including spring-loaded enclosures,” originally filed in July 2005.

Method of Controlling Movement of a Cursor on a Screen and a Computer Readable Medium Containing such a Method as a Program Code

Apple’s patent generally relates to a method of controlling movement of a cursor on a screen and a computer readable medium containing such a method as a program code. In particular, but not limited thereto, the present invention relates to a method implemented by a computer program for controlling movement of a cursor on a screen which is being used to display graphical data.

Other variations of the computer system can be envisaged. For example, the technology could be used in a specific application such as a GPS system in a car which may not include a keyboard or mouse but rather input buttons and menus on the screen. This patent could have played a part in Apple’s new Maps + Compass application demonstrated advanced GPS during last week’s WWDC.

Apple credits Gerhard Lengeling (Hamburg, DE) and Chris Adam (Norderstedt, DE) as the inventors of granted patent 7,549,131, originally filed in December 2002.

Other Granted Patents Issued Today:

Acceleration-based Theft Detection System for Portable Electronic Devices Apple’s patent generally relates to portable electronic devices. More specifically, this invention relates to theft detection systems for portable electronic devices – like the iPhone. Reference number: 7,548,161. Another security feature was introduced at WWDC 2009 which covers “Locate your iPhone on a map.”

Adaptive context sensitive analysis: Apple’s patent generally relates to the conversion of phonetic characters to written text and more particularly to converting words that are pronounced the same but written differently. Specifically, the patent relates to the Japanese written language of Kanji. Reference number: 7,548,863

Functional DMA performing operation on DMA data and writing result of operation: Apple’s patent generally relates to the field of integrated circuits and, more particularly, to direct memory access (DMA) in systems comprising one or more integrated circuits. Reference number: 7,548,997

Object selection using hit test tracks: Apple’s patent generally relates to methods for selecting objects from a moving image sequence of digitized or synthesized images, and more particularly, to a technique for storing auxiliary data in an item buffer, along with a video track, so as to precisely identify objects which can be selected from within each frame of the video track. Reference number: 7,549,122

Method and apparatus for image blending Apple’s patent generally relates to data processing systems using vector processing and Very Long Instruction Word (VLIW) architecture, more particularly to image blending. Reference number: 7,548,248

Generation and use of masks in MPEG video encoding to indicate non-zero entries in transformed macroblocks Apple’s patent generally relates to video encoding. More specifically, the invention relates to use of a non-traditional Zig-Zag Transform for Motion Picture Experts Group (MPEG) video encoding. Reference number: 7,548,583

Using order value for computing motion vector Apple’s patent generally relates to the field of multimedia compression systems. In particular the present invention discloses methods and systems for specifying variable accuracy inter-picture timing. Reference number: 7,548,584

Producing smooth motion compensated frames by combining multiple interpolation results Apple’s patent generally relates to video frame interpolation, and more specifically to combining results of multiple motion compensated interpolations in order to produce smooth motion in interpolated frames. Reference number: 7,548,664

And lastly, Apple was granted a patent for “Simulation of string vibration” (7,548,837).

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or Issued Patent is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or Issued Patent should be read in its entirety for further details. For additional information on any granted patent noted above, simply feed the individual patent number(s) into this search engine.

Jack Purcher, MacNN Senior Patent Editor

The attendees of this year’s WWDC were abuzz over some of the new iPhone 3G-S features that Apple’s Worldwide Product Marketing VP Phil Schiller introduced during his keynote. They were especially impressed with Apple’s new 3 megapixel autofocus functions along with on-iPhone video editing and Voice Controls. Brian Tong reporting for CNET TV captured the excitement over these particular features this way: “This will feature the touted 3 megapixel camera. It will be able to take photos a lot better. But it has this cool touch focus interface where you touch on an item – the camera and the software knows where to focus so they had house in the background and a flower in the foreground that they the used as example… Also you’re going to be able to get video capture – everyone’s been waiting for this. But the real slick-part about this is the fact that you can actually trim and edit your video clips and then upload them to YouTube, MobileMe or other services. Also some other features that were rumored – was voice control – so that you could actually talk by holding down the home button of your iPhone or iPod touch. It will actually enable you to “controool” your music collection or control it to call people. That was some really cool stuff” This WWDC 2009 Bonus report will show you that these cool features were borne from patents prior to the conference.

Groundbreaking Voice Control

Weeks prior to Apple’s developer conference a key patent report broke the news that Apple was Secretly Adding Voice Command Technology into Apple TV. The patent was credited to Apple’s leading Spoken Language Group engineer Jerome R. Bellegarda. In an effort to stay under the radar of the public eye, Apple’s name was omitted from the credits. But those of us in the know are well aware of Bellegarda’s groundbreaking research and development in this field on behalf of Apple. Apple’s release of this technology into its award-winning iPhone last week – has, without a doubt, poked the air out of Microsoft’s hot-air marketing balloon regarding voice control coming to Project Natal – sometime in 2010.

Additionally, a more detailed patent report titled “Apple Patent Reveals Voice Commands Coming to iPod, iPhone & Apple TV” covers voice commands eventually moving up into other iPhone functions regarding calendar meeting reminder notifications, incoming calls, voicemail, camera and yes, Apple TV. Apple is wise to focus and limit their introduction of voice command technology initially to iPhone-calling and iTunes music functions. It will allow iPhone 3G-S customers time to get familiar with these advanced functions while giving Apple engineers time to work out any of the preliminary foibles that might pop-up over the summer. Then they could begin to crank out a growing list of voice command functions as have been noted above.

Looking back to 2006, prior to Apple even unveiling their revolutionary mobile phone, one of our patent reports keenly pointed to a newly granted patent regarding voice commands and stated the following: “Being that the granted patent discusses the use of this system in relation to a user calling up a document, licence, memo and/or a producer’s agreement, it’s apparent that this patent could easily apply to a future iPhone.” Reporting and forecasting patent technologies accurately, allows our readership the ability to enjoy the roll out of new products even more. For patent sleuths wanting to review this particular patent in detail could do so by simply entering Apple’s patent number 7,127,394 into this search engine.

Groundbreaking Video Editing

The next coolest feature that is now available on the new iPhone 3G-S is their jaw dropping real-time video editor. This groundbreaking technology was first revealed to us in a 2008 Apple patent titled “Gestures for controlling, manipulating, and editing of media files using touch sensitive devices”

Right out of the gate in Apple’s patent abstract, they stated that “Specifically, gestural inputs of a human hand over a touch/proximity sensitive device can be used to control, edit, and manipulate files, such as media files including without limitation graphical files, photo files and video files.” Furthermore, in a report filed covering this patent, I pointed to a particular patent figure that covered Apple’s editing capabilities on a camera and noted that “While this may indicate a future Apple camera, it’s likely that the concept is meant to work with the camera on the iPhone.” Even Apple’s still camera autofocusing feature is generically covered in this patent. I say generically because Apple’s autofocus is a marketing trademark and therefore wouldn’t be discussed. Instead, the raw terminology used in the patent describes all-manner of zooming functions. The reference to zoom can be found listed 48 times in this patent. The zoom-in and zoom-out functions were covered in Phil Schiller’s keynote with the example of the “House and Flower.” A simple touch on the display on one of these objects would alter the view of the photo. It was a simple yet powerful visual.

What’s the Next Great Thing?

Just as we saw with the roll out of Apple’s 2006 patents being the precursor to the great 2007 iPhone – or even last week’s new iPhone 3G-S features being foretold in 2008 patents – Apple will continue to provide us with real insights, real facts and some really exciting concepts pointing us towards the next great thing. Every week MacNN’s patent blog is proud to present you with the very best patent news coverage to keep the excitement burning. And if the patent-pattern of the last two years is any indication – then we may be in store for Apple’s response to the emerging “smartbook.” Not because the rumor mill dreams of such – but because the facts bear that out. Stay tuned for a special report on this subject in the coming weeks – and stay connected year-round with our patent blog.

To Apple’s engineering teams and their greater developer community at large: Thanks for the great show and may I extend a hearty – Cheers!

Jack Purcher, MacNN Senior Patent Editor

On June 11, 2009, the US Patent & Trademark Office published an Apple patent generally relating to a method and system for prolonging emergency calls on the iPhone. The patent also details how an emergency phone call is made more difficult to disconnect by accident, how it preserves battery life by disabling non-essential hardware components – and how emergency mode enables emergency-phrase buttons on the iPhone that could easily communicate with an emergency operator your location (supported by GPS), a specific condition and/or the fact that you’re unable to speak due to your emergency condition.

The iPhone’s Emergency-Mode Processor

In Apple’s patent FIG.1 we see a schematic of the iPhone’s communication subsystem architecture (104) which includes an emergency-mode processor to process emergency calls. Once an emergency call is initiated, the iPhone’s emergency mode mechanisms kick in to prolong the length of the emergency call and avert premature termination.

Preserves Battery Life by Disabling Components

The iPhone’s emergency-mode processor may also preserve battery power by disabling non-essential hardware components, reducing power to the touch screen, disabling software applications, and/or reducing processor speeds. A non-essential hardware component may correspond to any hardware component that is not necessary for the emergency call to be conducted. For example, non-essential hardware components may include Bluetooth and Wi-Fi transceivers, secondary I/O devices, and/or camera sensors. However, hardware components that may be helpful in emergency situations, such as a GPS transceiver, may continue to be active in emergency mode.

Preventing Emergency Call Disconnect

To prevent an inadvertent end to the emergency call, the emergency-mode processor may make the emergency call harder to disconnect. For example, if the user presses a button to disconnect an emergency call, the emergency-mode processor may query the user for confirmation before disconnecting the call. The confirmation may be in the form of a button, a code or password, a verbal acknowledgement, and/or other input by the user. The emergency-mode processor may even disable the user’s ability to disconnect the call. As a result, the call may only be disconnected by someone (e.g., an emergency operator) on the other end of the emergency call. Further, the user may select settings to specify the level of difficulty and the methods of disconnecting emergency calls. The user may also select settings for each individual emergency number. For example, the user may disable the ability to disconnect a 911 call while activating a disconnect confirmation in other emergency calls.

Enabling Emergency Phrase Buttons

The iPhone’s emergency-mode processor may also enable emergency phrase buttons to shown on the display. In one example, the emergency phrase buttons are used by the user to communicate that they’re unable to speak. In one or more embodiments of the invention, the emergency phrase buttons allow preset and/or pre-recorded audio clips of phrases to be played in the emergency call. Certain phrases may be included with the iPhone and/or specified by the user. In addition, the phrases may be stored as audio files on the iPhone or generated in real-time using the iPhone’s speech synthesizer. For example, if the user is choking, the user may press an emergency phrase button that states his/her physical condition to a 911 operator. For example, a preset or prerecorded message could clearly communicate that the user has asthma and contain additional information about effective treatments specific to the user. See FIG. 2C below.

The user may also press buttons to communicate other information, such as his/her location (via GPS), and/or request the 911 operator to contact a friend and/or family member.

The numbers stored in the emergency number list may include, for example, police stations, fire stations, medical emergency lines, crisis hotlines, emergency family numbers, and/or work-related numbers.

Alternatively, the user may manually designate a call as an emergency by pressing a button, using a voice command, and/or providing other input upon connecting the call. If the call is an emergency, an emergency mode is activated (operation 308). If the call is a normal call, the emergency mode is not activated and the call is conducted normally.

Apple’s patent FIG. 4 is a flow diagram of the process of activating the emergency mode.

Apple lists Michael M Lee (San Jose, CA) as the sole inventor of patent application 20090149153.

Apple’s Emergency Call Warning: Apple’s iPhone Emergency Call webpage makes the following statement:

IMPORTANT: You should not rely on wireless devices for essential communications, such as medical emergencies. Use of any cellular phone to call emergency services may not work in all locations. Emergency numbers and services vary by region, and sometimes an emergency call cannot be placed due to network availability or environmental interference. Some cellular networks may not accept an emergency call from iPhone if it doesn’t have a SIM, if you locked your SIM, or if you haven’t activated your iPhone.

Other Patent Applications Published Today: Five additional patent applications from Apple were published today by the USPTO which include several on the subject of a Liquid-Metal Thermal Interface (20090145802, 20090149021, 20090146294), one on systems and methods for operating multi-level flash cells – multiple NAND flash memory cells specifically (20090147570) – and lastly, a continuation patent on Activity monitoring systems and methods (20090150114) which was covered in Apple’s granted patent as reported by us in March. The patent reveals sensors relating to extreme sports.

NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent applications and/or grants should be read in its entirety for further details. For additional information on any of today’s listed patent applications, simply feed the patent numbers into this search engine.

Jack Purcher, MacNN Senior Patent Editor