Apple reveals new Multi-Dimensional GUI
In November, the UPSTO Commissioner for Patents told the EE Times that his office was still falling behind and the backlog was growing. That understatement was clearly evidenced today when the UPSTO published between 65-70 Apple patents in a single day. While there were several technological themes that emerged within this allotment, such as back-up and programming, the number one theme of the day dealt with a new multi-dimensional GUI from Apple. “As the capabilities of processing devices progress” the patent reads, “so do the demands on the graphical user interface to convey information to the users in an intuitive manner.” Apple defines multi-dimensional as an interface capable of displaying 2.5D or 3D imagery. Apple introduces us to the Newtonian physics model that could be utilized in this new GUI and presents us with several patent figures that illustrate coming features and/or effects. One of these cool effects includes the ability to wrap open windows around multiple surfaces. There are approximately 12 patents relating to this proposed GUI and this report simply presents you with a few highlights.
Multi-Dimensional Desktop & Application Environment
Apple’s patent FIG. 2 is a block diagram of an example user interface architecture (200). The user interface architecture includes a user interface (UI) engine (202) that provides the user access to the various system objects (204) and conveys information about the system to the user.
Upon execution, the UI engine can cause the graphics device to generate a graphical user interface on an output device, such as a display device. In one implementation, the graphical user interface can include a multidimensional desktop (210) and a multidimensional application environment (212). In an implementation, the multidimensional desktop and the multidimensional application environment include x-, y- and z-axis aspects, e.g., a height, width and depth aspect. The x-, y- and z-axis aspects may define a three-dimensional environment, e.g., a “3D” or “2.5D” environment that includes a z-axis, e.g., depth, aspect. In an implementation, the multidimensional desktop can include use interface elements, such as visualization objects (220), a visualization object receptacle (222), and stack items (224). In some implementations, the visualization objects, the visualization object receptacle and the stack items can be presented in a pseudo-three dimensional (i.e., “2.5D”) or a three-dimensional environment as graphical objects having a depth aspect.
A visualization object can, for example, be a visual representation of a system object. In some implementations, the visualization objects are icons. Other visualization objects can also be used, e.g., alert notification windows, menu command bars, windows, or other visual representations of system objects.
In an implementation, the multidimensional application environment can include an application environment distributed along a depth aspect. For example, a content frame, e.g., an application window, can be presented on a first surface, and control elements, e.g., toolbar commands, can be presented on a second surface.
The User Interface Engine Architecture
Apple’s patent FIG. 5 is a block diagram of an example user interface engine architecture (500). The UI engine can, for example, include an interaction and visualization model engine (502), a physics engine (504), and a context engine (506). Other engines can also be included.
In one implementation, the physics engine can apply a physics aspect, such as Newtonian physics models based on mass, velocity, etc., to the visual representations of system objects, such as icons. In an implementation, the icons can be modeled as rigid bodies or non-rigid bodies. For example, placing an icon on a surface next to adjacent icons can cause the adjacent icons to shift positions in response to a simulated disturbance from the icon placement. In one implementation, icon magnetism can be selectively enabled or disabled by the user. In one implementation, icons return to their initial positions upon a disabling of the magnetism aspect. In another implementation, a magnet icon can have a magnetism aspect selected by the user, e.g., a magnetism with respect to a word processing application, or a magnetism with respect to two or more applications, or a magnetism with respect to the last time a document was accessed, e.g., within the last two days, etc.
Apple’s patent FIG. 6 noted above is block diagram of example system layers (600) that can be utilized to implement the systems and methods described herein. Other system layer implementations, however, can also be used.
In an implementation, a user interface engine, such as the UI engine or another UI engine capable of generating a three-dimensional user interface environment operates at an application level (602) and implements graphical functions and features available through an application program interface (API) layer (604). Example graphical functions and features include graphical processing, supported by a graphics API, image processing, support by an imaging API, and video processing, supported by a video API.
The API layer, in turn, interfaces with a graphics library layer (606). The graphics library layer can, for example, be implemented as a software interface to graphics hardware, such as an implementation of the OpenGL specification.
Example’s of Multi-Dimensional Desktop Environments
Apple’s patent FIG. 7, noted below, is a block diagram (700) of an example multidimensional desktop environment. In the example implementation, the multidimensional desktop environment includes a back surface (702) axially disposed, e.g., along the z-axis, from a viewing surface (704). In one implementation, the back surface can, for example, be a two-dimensional desktop environment, including one or more menus (701 and 703). In one implementation, the viewing surface can be defined by the entire image on a display device, e.g., a “front pane.” One or more side surfaces, such as side surfaces (706, 708, 710 and 712), are extended from the back surface to the viewing surface. A visualization object receptacle, e.g., an icon (714) is generated on one or more of the side surfaces, such as side surface (706). Although only one visualization object receptacle is shown, addition icon receptacles can also be displayed, e.g., along the side surface (708).
In one implementation, a reflection region (716) can be generated on the side surface, e.g., the “floor.” In an implementation, a reflection of the back surface and of graphical items placed on the reflection region can be generated, e.g., shapes (760 and 762) generate reflections (761 and 763) in the reflection region.
In one implementation, a motion model is dependent on a selected surface aspect. For example, an equable texture, such as an image of a hardwood floor or a polished metallic surface, can be associated with a rigid-body Newtonian physics model; conversely, a visible grid aspect, or a raised texture, such as an image of a carpet, pebbles, etc., can be associated with a grid snap. In another implementation, the motion mode and textures can be selected independently.
Apple’s patent FIG. 11, below, is a block diagram of another example multidimensional desktop environment which includes a back surface.
Apple’s patent FIG. 14, noted above, is a block diagram of another example multidimensional desktop environment which also facilitates a multidimensional application environment. For example, an application content frame (1410), e.g., a window displaying editable data, can be displayed on the back surface (1102), and one or more application control elements can be displayed on one or more side surfaces. For example, a three-dimensional function icon arrangement (1420) can be displayed on the surface 1108, and menu items 1430 can be displayed on the surface (1112).
The three-dimensional function icon arrangement can, for example, include three-dimensional function icons (1422, 1424, 1426 and 1428). In one implementation, each three-dimensional function icons include a function command on each surface, and each three-dimensional function icon. can be rotated, positioned, and manipulated through the use of an input device, such as a mouse.
Windows Could be Dragged or Displaced on one or more Surfaces
Apple’s patent FIG. 15 noted below, is a block diagram of another example multidimensional desktop environment which illustrates how windows could be dragged or displaced across one or more surfaces. That could be a very cool feature.
Multidimensional Desktop Environment Geometries
Apple’s patent FIG. 16A below is a block diagram of another example multidimensional desktop environment which illustrates an arcuate back surface 1602 that is axially disposed, e.g., along the z-axis, from a viewing surface 1604. In one implementation, a reflection region (1116) can be generated on the side surface (1606), e.g., the “floor.” In an implementation, the side surfaces (1606, 1608, 1610 and 1612) could be defined by arcuate regions having curvature intersections 1607, 1609, 1611 and 1613, respectively.
Other multidimensional desktop environment geometries can also be used. For example, in one implementation, the multidimensional desktop environment can conform to a tetrahedron-shaped environment in which a front surface of the tetrahedron defines a viewing surface, and the remaining three surfaces define a left surface, a bottom surface, and a side surface. In another implementation, the multidimensional desktop environment can conform to a triangular environment, in which one axis of the triangle defines the viewing surface and the remaining two sides of the triangle define a left surface and a right surface. Other geometries can also be used.
In one implementation, a configuration tool can be used to facilitate configuration of the multidimensional desktop environment by the user. For example, a configuration menu can present one or more multidimensional desktop environment geometries for selection by the user, such as a rectangular geometry, an arcuate geometry, a triangular geometry, etc. Selection of a geometry can cause the multidimensional desktop environment to be rendered according to the selected geometry.
The Mirror Effect: An Exemplary Physical Arrangement
In a secondary patent, Apple presents us with FIG. 5B (as shown below) which illustrates a block diagram of an exemplary physical arrangement which the reflection content surface 508 mimics. In an implementation, the reflection content surface 508 can appear to a viewer observing along viewing angle 522 as a mirror at right angle to the visualization objects 516 and 518, and containing a reflection of the visual content of the back surface 502 and the application interfaces 504 and 506. The reflection surface 508 is not limited to being placed at the bottom of a multidimensional desktop environment 500 (i.e., to reflect upward), and can be placed anywhere in the multidimensional desktop environment 500 to reflect content in any orientation. In an implementation, scaling of the reflected content can be applied so as to simulate a curved reflecting surface.
Apple lists the following inventors: Chaudhri; Imran A.; (San Francisco, CA) ; Louch; John O.; (San Luis Obispo, CA) ; Hynes; Christopher; (Santa Cruz, CA) ; Bumgarner; Timothy Wayne; (Sharpsburg, MD) ; Peyton; Eric Steven; (Lisle, IL). For full details of this patent, see Multi-Dimensional Desktop. Other patents covering this subject matter include, Visualization and Interaction Models, Reflections in a Multi-Dimensional User Interface Environment and User Interface for Application Interface Manipulation. There are at least a dozen patents in total published today covering this new multidimensional interface project.
NOTICE: MacNN presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent applications and/or grants should be read in its entirety for further details.
Written and researched by Jack Purcher. Contact Author: emailneo@shaw.ca .
Top