Most software applications and many Web sites are built using graphical user interface (GUI) development tools. Such tools provide a set of controls—also known as widgets—for building GUIs. The controls include text and number fields, checkboxes, radio buttons, sliders, menus, scrollbars, buttons, knobs, dials, meters, and various types of windows.
GUI tools are supposed to make it easier and faster for programmers to develop GUIs. However, the help they provide is limited by several flaws:
■
Too low level: GUI toolkits provide low-level building blocks to allow maximum flexibility in what can be built. But that is misguided: it makes developing applications tedious, time consuming, and error prone. It makes common, ordinary UIs just as hard to implement as highly unusual ones; the common case suffers to allow for the rare case. Some toolkits are low level because the toolkit developers lacked the time, motivation, or skill to design high-level building blocks covering the desired range of GUIs.
■
Too unguided: GUI toolkits allow programmers to build GUIs that violate design guidelines. They don't help guide programmers toward good designs. Most toolkits claim to support a GUI standard—such as Windows or MacOS—but make it just as easy to violate the standard as to adhere to it. They allow designers to make poor choices, like using the wrong control for a setting. A control may look nice, but that's a minor detail if it's the wrong control or behaves unexpectedly.
■
Too focused on appearance: Most GUI tools require developers to spend too much time fiddling with the appearance and layout of their GUIs. Are the labels for settings aligned properly? Should a number be presented as a digital readout or a position on a dial? Should a choice be presented as radio buttons or a menu? What font should be used in a text field? These are mere presentation issues—the "low-order" bits of GUI design. The important issues are the semantics of the UI, such as whether a setting is a date, a filename, a volume level, or a choice between fonts. Decisions about presentation will change from day to day or even hour to hour as the design evolves and so should not require recoding. Changing the presentation of a choice from radio buttons to a dropdown menu should require changing only an attribute, not ripping out radio button code and replacing it with dropdown menu code. Time spent fiddling with presentation would be better spent learning about users, tasks, and work flow and planning appropriate functionality.
The primitiveness of most GUI tools is a problem because many GUIs are not designed by UI professionals. Rather, they are quickly assembled on tight deadlines by programmers who lack the UI expertise necessary to compensate for the lack of guidance from the toolkits.
The result is that many GUIs are full of design errors. Some errors are semantic and can be detected only by people who understand the application's target users and tasks. However, many GUI design errors would be easily detected by most UI professionals, even those unfamiliar with the users and tasks. Such errors are GUI control bloopers. They fall into two categories:
1.
Using the wrong GUI control
2.
Using a control incorrectly
GUI control bloopers harm usability. They also give customers an impression of a shoddy, unprofessional product, especially when a GUI has many of them. Fortunately, they are fairly easy for usability experts to spot. They are also concrete and relatively easy to explain. Finally, they are usually easy to correct unless they are due to limitations of the GUI tools used to build the software. This chapter describes the most common GUI control bloopers, with design rules for avoiding them.
LIAM E. GUMLEY , in Practical IDL Programming, 2002
GUI Components (Widgets)
A GUI application typically contains a variety of components known as widgets. Each widget has a particular appearance, and the application responds in a certain way when you interact with the widget. Inspection of the mapping GUI reveals the following widgets:
•
A menu bar at the top allows configuration of application properties.
•
A scrolling list allows a map projection to be chosen.
•
Three horizontal sliders allow the projection center and rotation to be changed.
•
A text entry field allows a map scale factor to be entered.
•
A graphics window displays the name of the currently selected map projection.
•
Labels at the bottom of the GUI display text information, including the current cursor location.
•
A system menu button at top left and a Close button at top right (depending on platform) allow the GUI to be closed.
The application responds to interaction with each widget in different ways. For example, when a new projection is selected, the map is redrawn. When the cursor is moved over the map, the position display is updated. Thus the GUI application includes code not only to create widgets, but also to take action when the user interacts with each widget (i.e., an event handler). To create a GUI application, you need to arrange appropriate widgets in a sensible way, and then provide code that handles user interaction. These two aspects of a GUI—the appearance and layout of widgets, and their response to user interaction—reflect the essence of GUI programming.
John Sammons , in The Basics of Digital Forensics (Second Edition), 2015
Alert!
Dependence on the tools
Graphical User Interface (GUI)-based forensic tools can become a crutch. "Push-button" tools can make exams much more efficient, but they don't relieve the examiner of the responsibility to understand what's going on beneath the surface. Examiners need to understand not only what the tool is doing, but also how the artifact in question is created to begin with.
Some of the forensic tools that an examiner may use are listed in Table 3.2. Many of these companies offer video tutorials or demonstrations of their products. These can be a great source of additional information. They are typically available from the manufacturer's website or on YouTube. This is in no way meant as an endorsement of a specific tool. These are only a representative sampling of the many tools that are available.
Table 3.2. Some hardware and software tools that may be found in a digital forensics laboratory
Elizabeth Rosenzweig , ... David Juhlin , in Successful User Experience: Strategies and Roadmaps, 2015
User Interface and Interaction Design
The physical and graphical layer between a person and the technology for computing is the user interface, which includes modalities of interaction that make us the interaction design. It is the design of these elements that creates the overall UX.
Graphical user interface (GUI) is composed of elements:
▪
Graphics (see Figure 3.1):
▪
Windows: an element of the user interface that contains some of the information of the system. The Window is a metaphor for a view, and is part of a larger information system.
▪
Icons: a graphical representation of an idea. For example, an image of an arrow pointing down is an icon for the function—download.
▪
Labels: name or title of a software function or file.
▪
Dialog box: a window that is usually displayed over other graphical elements in a screen. The dialog box provides immediate information to the user and requires some form of interaction to close it, for example buttons such as "OK" or "Cancel" are often part of the dialog box.
▪
Menus: an element of the user interface that lists commands or functions for a computer application or system (Figure 3.2).
Figure 3.2. Menus on a computer.
Figure 3.1. Windows, icons, labels.
▪
Interactive graphical affordances:
▪
Buttons used for selecting items and initiating actions;
▪
Checkboxes used for selecting items;
▪
Sliders used for navigating through windows (Figures 3.3 and 3.4).
Figure 3.3. Buttons and checkbox.
Figure 3.4. Scroll bars.
▪
User interactions include modalities such as gesture, voice, audio, and other sensory interactions:
▪
Gestures include:
▪
Point and click**;
▪
Drag and drop;
▪
Swipe.
▪
Biometrics include:
▪
Eye-tracking and eye-gaze used for input;
▪
Voice and natural language processing;
▪
Galvanic skin response technologies for tracking and adjusting interfaces based on user's state of stimulation.
Another key element in a good UX is the information architecture (IA). IA can be defined as the organization of information within the system or product this includes the labels and menu structure as well. On a Web site the IA encompasses the menu structure and labels as well as the navigation from one page to another.
Daniel Wigdor , Dennis Wixon , in Brave NUI World, 2011
Lessons from the Past
The graphical user interface (GUI) is a good example of a promise that morphed over time. Doug Englebart is often called the inventor of the mouse, and he is, but the mouse was just a means to an end. The end, the goal of Engelbart's vision, was to augment the human intellect. Over time and in the hands of others, the promise of GUI morphed into making tasks easier and broadening the market for computing technology. In many ways this latter promise was realized. Fifty-odd years since Englebart's original paper, a vast domain of new capabilities has been extended to a wide range of people. Today, typical users compose typeset documents, create complex financial models, and access vast quantities of information (and misinformation) with relative ease. While cumbersome interfaces stubbornly persist and are constantly created anew, the ubiquity of technology is a testament to a lowering of barriers (i.e., the increased ease of use) for these new capabilities. The question of whether the individual's, the collective, or the average "intellect" is augmented by this growth in technology is highly debatable, and we have no desire to address that question in this book.
The de facto promise of the GUI, to make tasks using a computer easier, raises the bar for every product that adopts a GUI. The users expect more than functionality: They expect that the promised functionality will be easy to use. For example, introducing a new mark-up language with the promise of some new exciting features, then expecting that to create a new mass market product would be absurd to the point of delusional. The promise of easier access to capabilities is already thoroughly infused in users' expectations as they use a product. Parts of the product that fail to meet these expectations will go unused. It is not uncommon to discover users asking for functionality that already exists in a product. This perpetuating irony is driven in part by a lack of transparent ease of use in esoteric elements of the product. The tragedy of unused functionality is painfully acute because the development team has gone to the trouble and expense of building capabilities. These capabilities match users' needs, and "only" the interface got in the way. These are truly missed opportunities.
Implicit in the promise of each new paradigm are pitfalls for the development team. The first pitfall is that the development team may assume that simple adoption of the trappings of the interface ensures delivery on the promise. This was evident in many early GUI designs. They did include pull-down menus, dialog boxes, and a screen that showed what you were working on and would "get," yet they were unusable. The terms used in the menus were opaque to the intended users. The organization of the interface elements did not support expected task flow. The elements created on the screen seemed unrelated to the constructs that users employed in their work. The path to an intended result was totally unclear. In other words, while the GUI removed the syntactic demands and the memory requirements of a command system, it replaced those with a thicket of incomprehensible choices unrelated to the users' knowledge of the task domain. An even deeper and rare misunderstanding occasionally occurred. In these cases, the team failed to grasp some of the fundamental principles of the interface paradigm and adopted only the trappings. I have seen systems that ignored the object-operation syntax (select an object first, and choose the verb as a menu item) of the GUI and required the user to select an operation first and then the relevant objects would be highlighted. It was so contrary to expectations, it took some time to even grasp what was happening.
Cameron H. Malin , ... James M. Aquilina , in Malware Forensics Field Guide for Linux Systems, 2014
PDF Profiling Process: GUI Tools
▶ GUI-based tools can be used to parse and analyze suspect PDF files to gather additional data and context. There are three main tools in Linux used for this process: Origami Walker, PDFScope, and PDF Dissector. Although at the time of this writing PDF Dissector is no longer available for purchase (but is still supported by Zynamics), it is a powerful tool that many digital investigators added to their arsenal prior to its cessation and will be covered in this section.
Scanning for indicators of malice and examining file structure and contents
•
Building upon Didier Steven's PDF tools mentioned in the previous section, pdfid.py and pdf-parser.py, PDFScope is a GUI-based tool that provides the digital investigator with the functionality of these tools through a sparse and intuitive user interface, allowing for agile triage for indicators of malice.
•
Once a target specimen is loaded into the tool, existing file structures and action types can be explored through respective tabs at the top of the tool interface. As shown in Figure 5.89, a discovered JavaScript action type can be easily reviewed by clicking on the "/JS" tab.
FIGURE 5.89. PDFScope
•
Using the Object menu (Figure 5.90), the digital investigator can drill down further into the structure of the target file by navigating to or saving objects of interest.
FIGURE 5.90. Using the PDFScope Object menu to examine an object of interest
•
Origami is a framework of tools written in Ruby designed to parse and analyze malicious PDF documents as well as to generate malicious PDF documents for research purposes. The framework contains a series of Ruby parsers—or core scripts, scripts, and Walker (a GTK GUI interface) to examine suspect PDF files, depicted in Figure 5.91.
FIGURE 5.91. Origami Walker
•
Using Origami Walker, the digital investigator can quickly examine the structure and content overview of a target file specimen using the hierarchical expandable menu in the left-hand viewing pane while examining the respective PDF code, action items, and stream contents in the right-hand top and bottom viewing panes, respectively (Figure 5.92).
FIGURE 5.92. Extracting an encoded stream with Origami Walker
•
Upon selecting an object of interest, such as a stream, additional analysis options can be invoked by right clicking on the object and selecting the desired action, such as dumping a stream and searching for object references.
•
Specific key words/strings within an object name or body can be quickly located using the Walker search function in the Document menu, accessed from the toolbar.
Identifying and extracting malicious artifacts, scripts, and code
•
Zynamics' PDF Dissector88 provides an intuitive and feature-rich environment allowing the digital investigator to quickly identify elements in the PDF and navigate the file structure.
•
Anomalous strings can be queried through the tool's text search function, and suspect objects and streams can be identified through a multifaceted viewing pane, as shown in Figure 5.93.
FIGURE 5.93. Navigating the structure of a suspect PDF file with PDF Dissector
•
The contents of a suspicious object can be further examined by using the content tree feature of PDF Dissector.
□
Once a target object or stream is selected, the contents are displayed in a separate viewing pane.
□
Compressed Streams are automatically filtered through FlateDecode and decoded—the contents of which can be examined in the tool's built-in text or hexadecimal viewers.
□
The contents of a suspicious stream object (raw or decoded) can be saved to a new file for further analysis.
•
PDF Dissector offers a variety of tools to decode, execute and analyze JavaScript, as well as extract embedded shellcode.
•
Identified JavaScript can be executed within the tool's built-in JavaScript interpreter (Figure 5.94).
FIGURE 5.94. Executing JavaScript with the PDF Dissector JavaScript interpreter
•
Embedded shellcode that is invoked by the JavaScript can be identified in the Variables panel. Right clicking on the suspect shellcode allows the digital investigator to copy the shellcode to the clipboard, inspect it within a hexadecimal viewer, or save it to a file for further analysis.
•
Extracted shellcode can be examined in other GUI-based PDF analysis tools, such as PDF Stream Dumper,89 PDFubar,90 and Malzilla,91 which are described in further detail in the Tool Box section at the end of this chapter.
•
The Adobe Reader Emulator feature in PDF Dissector allows the digital investigator to examine the suspect file within the context of a document rendered by Adobe Reader, which may use certain API functions not available in a JavaScript interpreter.
•
Adobe Reader Emulator also parses the rendered structure and reports known exploits in a PDF file specimen by Common Vulnerabilities and Exposures (CVE) number and description, as shown in Figure 5.95.
FIGURE 5.95. Examining a suspect PDF file through the Adobe Reader Emulator
Online Resources
A number of online resources exist to scan suspicious PDF and MS Office document files, scan URLs hosting PDF files, or run suspicious document files in a sandboxed environment. Many of these Web portals also serve as great research aids, providing database search features to mine the results of previous submissions.
JSunpack-a JavaScript unpacker and analysis portal
http://jsunpack.jeek.org/dec/go
ViCheck.ca—Malicious code analysis portal; numerous tools and searchable database
https://www.vicheck.ca/
Document Analyzer—Malicious document analysis sandbox built upon the Joe Sandbox Desktop
http://www.document-analyzer.net/
WePawet—A service for detecting and analyzing Web-based malware (Flash, Javascript, and PDF files)
http://wepawet.iseclab.org/
XecScan—Sandbox that processes MS Office documents and PDF files
In Microsoft Vista for IT Security Professionals, 2007
Enabling Encryption on a File or Folder
We sometimes like to describe administrators as clickies and clackies—a clicky will want to find every possible way to use the mouse to accomplish an administrative goal; a clacky will generally use commands at the command prompt. Which type are you? Coming out of the world of software development, the author of this chapter is very much a clacky, although he does appreciate the power of a well-designed graphical user interface (GUI). We hope we can accommodate both kinds of administrators in the following instructions.
The GUI way to enable encryption on a file or folder in Windows Vista is as follows:
1.
Right-click on the file, select Properties, and on the General tab, click the Advanced… button, as in Figure 5.5.
Figure 5.5. Using Advanced Attributes to Encrypt Important Documents
2.
Check the box that reads Encrypt contents to secure data, click OK and then Apply or OK.
3.
You will be prompted to choose between encrypting just the file (or files) selected, or the file and its parent folder, as shown in Figure 5.6. If you selected a folder to encrypt, you will be asked if you want to encrypt just the folder, or the folder, its subfolders and files. Selecting to encrypt folders and subfolders, rather than files, is a good security measure, as it reduces the chance that a temporary copy of the file will be made and stored unencrypted in this folder.
Figure 5.6. Encrypting a File and Choosing to Encrypt Its Parent Folder
(You can, of course, enable encryption using the command line cipher /e < file > or cipher /e /s:<directory > .)
If you were paying attention on the Advanced Attributes dialog, you will have seen a grayed-out (disabled) Details button. This button is now enabled, because you now have some encryption details that you can view and modify. Clicking this takes you to the User Access dialog for this EFS file, as you can see in Figure 5.7.The User Access dialog displays the list of users who are given access to decrypt (read) and encrypt (modify) the file, their certificates, and the list of recovery certificates.
Figure 5.7. Listing the Users and Recovery Certificates for an EFS-Protected File
Recovery certificates are defined by the Group Policy (or local security policy if you are not on a domain) for EFS, and they represent the Data Recovery Agent (DRA).You need a DRA for those scary moments when the original owner of a file is no longer available to decrypt the file, and no one else has been given access through EFS.
Tools & Traps…
Jumping without a Backup Parachute
At one point, encryption in the hands of mere users was considered so dangerous and scary that Windows would not allow you to create an EFS-encrypted file that didn't have a DRA, but now those of you who like to live on the edge can change your EFS policy to remove the requirement for a DRA on encrypted files.
Encryption is already a double-edged sword: If you lose your password, you're not going to get your files back without brute force cracking your own password, or begging the owner of the DRA account. If you fly without a DRA, you're saying "if I forget my password, I'm comfortable forgetting all possibility of getting my data back."
If you don't use a DRA, at the very least you should back up your keys (you can do this from the User Access dialog) to a removable storage device. Because the keys are exported in a PFX file that has password protection, you should also write down the password for the PFX file, and store the password and file in a physically secure location.
If you can't back up your keys, back up your data.
Or, simply accept the risk that you will lose your files' contents completely.
Splat.
Depending on where you apply EFS policy in the Active Directory structure, you may wind up with several DRAs enacted on users who encrypt documents. This is not necessarily a bad thing, as different DRAs may be used for different purposes—an all-powerful, overarching DRA might be used to decrypt files for forensic investigation by a compliance team, whereas a DRA local to a particular team might be used to provide "oops" recovery for that team in the event of occasional slip-ups.
Jonathan Lazar , ... Harry Hochheiser , in Research Methods in Human Computer Interaction (Second Edition), 2017
12.3.2 Keystroke and Activity Loggers
Modern GUI windowing systems that support multitasking and concurrent use of multiple related tools present intriguing possibilities for HCI research. How often do users change applications? How many tasks do people work on at any given time? How long do users generally work within any given window before switching to another? What fraction of time is spent on overhead such as resizing windows, moving windows, or adjusting system controls? Answering these and other related questions requires data collected at the level of the operating environment.
Activity-logging software runs invisibly in the background, recording mouse movements, keyboard input, and windowing systems' interactions including window movement, resizing, opening, and closing. These tools act as proxies for user interaction events, recording events as they happen, before they are passed along to applications or the operating environment. Keyloggers are a special subclass of activity-logging software, focusing only on keyboard input. Activity-logging software has achieved a fair amount of notoriety in recent years, as these tools have been used as "spyware," to surreptitiously record user interactions in the hopes of stealing passwords, finding evidence of criminal behavior, or collecting evidence of spousal infidelity.
Commercial activity-logging products are often marketed as being tools for employers and parents to track inappropriate computer use by employees and children, respectively. Although some of these tools might be appropriate for data collection for research purposes, some antispyware programs may defeat or remove activity loggers. You may want to test the logging software on relevant computers and disable antispyware measures before trying to use these tools to collect data.
The disruption and recovery tracker (DART) (Iqbal and Horvitz, 2007) logged window positions and sizes, window actions, user activities, and alerts from various systems. DART's design presents an example of the responsible use of these tools for conducting legitimate research while remaining sensitive to privacy concerns. Keyboard logging was limited to a subset of possible choices, including menu shortcuts and some punctuation, and only a portion of each window title was collected. The resulting data therefore did not include file names, email addresses or subject lines, or web page titles. The analysis of more than 2200 hours of activity data collected from the main computers of 27 people over a two-week period generated numerous insights into time lost due to email or instant-messaging alerts, and how users respond to and recover from those interruptions (Iqbal and Horvitz, 2007). Logging studies can also be used to collect data on effective use of devices, as in a study that captured mouse movements of adults with disabilities as they completed everyday computer tasks, in the hopes of building tools that might adapt to better suit the needs of these users (Hurst et al., 2013).
Mobile devices present additional possibilities for activity tracking, providing not only the opportunity to record which keys and controls were activated and when, but also detailed information regarding users' geographic location. HCI researchers have used logs of user location to predict short-term motion of individuals in crowds in a city environment (Fan et al., 2015), to infer movement characteristics associated with depression (Canzian and Musolesi, 2015), and undoubtedly for countless other interesting questions. See Chapter 14 for further discussion of the possible uses of mobile and ubiquitous computing for tracking user activity.
Upside-Down ∀s and Algorithms—Computational Formalisms and Theory
Alan Dix , in HCI Models, Theories, and Frameworks, 2003
14.5.2 The Problem…
In a graphical user interface (GUI) or any single user interface, the succession of events in the program is straightforward:
1.
user event 1 arrives (e.g., mouse press)
2.
deal with event and update display
3.
user event 2 arrives (e.g., mouse release)
4.
deal with event and update display
5.
user event 3 arrives (e.g., key click)
6.
deal with event and update display
As we know, this can cause enough problems!
In a transaction-processing system, with one user, the application module may receive messages (with form data) in a similar fashion. However, the whole point of such systems is that they have many users. The module may therefore receive messages from different users interleaved:
1.
message 1 for user A received
2.
deal with message and generate new screen/Web page for user A
3.
message 1 for user B received
4.
deal with message and generate new screen/Web page for user B
5.
message 2 for user B received
6.
deal with message and generate new screen/Web page for user B
7.
message 2 for user A received
8.
deal with message and generate new screen/Web page for user A
The transaction-processing engine deals with passing the new screens back to the right user, but the application module has to do the right things with the form data in the messages. In the case of simple transactions, this may not be a problem; for example, if the application simply allows the user to enter an ISBN (International Standard Book Number) for a book and then returns data about that book, the system can simply deal with each message in isolation. However, a more complex dialogue will require some form of state to be preserved between transactions. For example, a request to delete a book may involve an initial screen where the user fills in the ISBN, followed by a confirmation screen showing the details of the book to be deleted. Only then, if confirmed, will the system actually do the deletion and generate a "book has been deleted" screen. Even a search request that delivers several pages of results needs to keep track of which result page is required and the original search criteria.
Getting back to my workplace in Cumbria in the mid‐1980s, the transaction systems in place at that stage dealt only with the simple stateless record-display transactions or multipage search transactions… and even the latter had problems. When several users tried to search the same database using the system, they were likely to get their results mixed up with one another! I was charged with producing the first update system. Whereas occasionally getting someone else's search results was just annoying, deleting the wrong record would be disastrous.
Aaron J. Quigley , Florin Bodea , in Human-Centric Interfaces for Ambient Intelligence, 2010
1.2 Background
The classical GUI has been successfully used since its inception, but the limitations of basic mouse and keyboard input are obvious, leading to the demand for new input and interaction modalities that will enhance the interface. Available since the early 70s in various forms, touch-sensitive surfaces are one solution. The PLATO IV,2 developed at the University of Illinois in 1972 and one of the first single-touch screens introduced, has a 16×16 array of touch-sensitive locations. One of the first multitouch systems was the flexible machine interface [12]. Here the touch-sensitive surface consisted of a frosted-glass sheet with special optical properties. When the sheet was touched, a black spot appeared that could be captured by the camera behind the sheet and recognized using simple computer vision. Importantly, this was only an input device; the user viewed the visual feedback on a different device.
Buxton3 underscored the difference between touch tablets and touch screens as follows:
▪
Touch tablets. The touch-sensitive surface is not used for feedback display.
▪
Touch screen. The touch-sensitive surface is overlaid by the display.
The soft machine concept [13] builds on the classical hard machine concept (e.g., an oven or washing machine). It uses graphics software to generate images of controls (e.g., keys, pushbutton switches, and slides on the screen) to give the appearance of a hard machine. To ensure the illusion, the soft machine display is touch-sensitive, enabling a user to operate its controls by direct touch, as if they were physical controls. This connection between display and action increases the sense of immediacy in the interface.
The XEROX 5700 Electronic Printing System [14], the first commercial soft machine, has obvious parallels with the current generation of multitouch display prototypes and systems. In the original, all controls of the soft machine were on a black-and-white touch screen display. The Lemur,4 an advanced example of a soft machine, released in 2003 is a multitouch music controller device with all controls (knobs and sliders) virtually created on the display. This interface is customizable to user preferences. Other advances such as haptic output can overcome the lack of tactile feedback on soft versions of hard machines. Sony's tactile touch screen feedback is based on an actuator, constructed as a multilayer sandwich of thin (0.28-μm) piezoceramic films, that "bends" the screen. The haptic effect can be coupled with visual or audio feedback to simulate the feel of buttons that click [15].
Rather than a dedicated sensing surface, alternative approaches often rely on computer vision for sensing. VideoPlace is a vision-based system capable of tracking hands and thus recognizing and interpreting a rich set of gestures [16]. Users are placed against a neutral background, making it possible to process their silhouette image. The system can detect when they "touch" the graphical objects projected on the wall in front of them and react to the contact. The concepts demonstrated in VideoPlace are used in current camera-based multitouch systems that are able to "see" in front of the display. Diffuse illumination or capacitance-based systems (described in detail in Section 1.4) can discriminate between touch and hover.
The Digital Desk [17] uses optical techniques to sense hands, fingers, and objects being moved on the desk's surface. It operates by projecting images on the desktop where documents can be placed. Using an overhead camera, it reacts to interaction with the objects on the desk and can scan the documents placed there. Although this is a front-projection system and occlusion can occur, the Digital Desk supports multitouch interaction through the use of both hands to rotate or scale documents or to manipulate objects on the desk.
Akin to the Digital Desk is Play Anywhere, a top-projection camera input system that allows multitouch user interaction [18]. It relies on a short-throw digital projector for top projection, an infrared illuminant, and an overhead camera to view the desk. These are all placed on the same stand, which makes this setup very portable because it can transform any flat surface into a multitouch sensitive display. The system can detect and track hands, fingers, and objects using a visual bar code scheme. The infrared illuminator illuminates the environment and causes a finger to cast an IR shadow on the display. If the shadow disappears, the system considers that the finger is touching the surface. By measuring the distance between the finger and its shadow, the system can also detect a hover.
More recent examples of touch-sensitive surface technologies include the Apple iTouch/iPhone product line and the SMART Table,5 (shown in Figure 1.4). The SMART Table uses digital vision touch (DViT), which relies on small cameras embedded in a device around the rim of the display [19]. When an object enters the field of view (FOV), the angle within the FOV of the camera is calculated. The SMART Table is multitouch and, multiuser, designed for primary education applications. Other examples include ThinSight [20], developed for small LCD panels only, which supports multitouch input using hands and objects (or a stylus in N-trig's6 case). ThinSight uses a 2D grid of retro-reflective optosensors (containing an IR light emitter and an optically isolated IR light detector, allowing it to emit light and to detect the intensity of incident light at the same time) placed behind an LCD panel. When a reflective object is placed in front of the panel, a part of the light is reflected back and detected by the optosensor. The data generated by the sensor grid is a low-resolution grayscale "image" that can be processed to extract information about the number of contact points and their position. N-trig uses a capacitive sensing system (i.e., signals are emitted that capacitively couple with users when they touch the surface) mounted in front of the LCD panel to detect touch, and a specialized sensor to detect and track the stylus.
0 Response to "Which Graphical User Interface Screen Element is Used to Select Data That Has Continuous Values"
Post a Comment