Table of Contents

Table of Contents. 1

Chapter 1. 3

Introduction. 3

1.1     What is Taprav?. 3

1.2     Why was it created?. 3

1.3     What do I need to run Taprav?. 3

Chapter 2. 5

Opening, Saving, Importing. 5

2.1     The Session  5

2.1.1              Creating a New Session  5

2.1.2              Opening a Saved Session  5

2.1.3              Saving a Session  5

2.2          Importing Data. 6

2.3     Pupil Data  6

2.3.1              Baseline Calculation  7

2.4     Task Model 7

2.4.1              Task Model Description File Format 7

2.5     Video. 8

2.5.1              Video Offset 8

2.6     Saving Sessions with Imported Data  9

Chapter 3. 10

The Visualization. 10

3.1     Pupil Response. 10

3.2     Task Model 10

3.2.1              Task Levels and Boundaries  11

3.2.2              Task Boundaries Overlay  11

3.3     Focus and Context Frames. 11

3.3.1              Local Frame  12

3.3.2              Global Frame  12

3.4     Views. 12

3.4.1              View Tabs  12

3.4.2              New   13

3.4.3              Rename  13

3.4.4              Close  13

Chapter 4. 14

Interacting with the Visualization. 14

4.1     Local Frame Interaction. 14

4.1.1              Zooming  14

4.1.2              Selection  14

4.2          Global Frame Interaction. 14

4.2.1              Resizing the Magnification Lens  15

4.2.2              Moving the Magnification Lens  15

4.2.3              Recreating the Magnification Lens  15

4.3     Menu Bar Actions. 15

4.3.1              View Menu  16

4.3.2              The Select Menu  17

Chapter 5. 18

Tools. 18

5.1          Filtering the Pupil Response. 18

5.2     Notes. 18

5.3          Statistics  19

5.4     Video. 19

5.4.1              Controlling the Video  19

5.4.2              View Play head  19


 

 Chapter 1                  

Introduction

This chapter describes the motivation for Taprav as well as the requirements for using Taprav.

 1.1        What is Taprav?

Taprav (Task Aligned Pupil Response Analysis and Visualization) is a visualization tool that enables the exploration of a hierarchical task model representation aligned with pupillary response data.

 1.2        Why was it created?

Software packages that ship with commercial eye trackers fall far short of what researchers need to analyze pupillary response data for interactive tasks.  Existing analysis software does not provide a tightly synchronized view of the user task model, video of on-screen interaction, and pupillary response.  We offer a pragmatic visualization solution that fulfills these needs along with a user interface that encourages exploration with focus and context perspectives.  In addition, although designed with pupil response analysis in mind, the architecture of Taprav has been designed to support any temporal coherent dataset measured during task execution.

 1.3        What do I need to run Taprav?

Taprav will run on any system where the Java Runtime Environment is installed.  The application itself is built on J2SE 1.4.2.  The most recent JRE can be downloaded Here.

Although not necessary for the various basic visualization features of Taprav, if you intend view video synchronized with the pupil and task model data you will also need to have QuickTime for Java installed on your machine. Currently, this means that only systems running Windows or MacOS will be able to play video through TAPRAV. Instructions for installing QuickTime for Java can be found here.

 Chapter 2                  

Opening, Saving, Importing

This chapter describes the process of opening and saving sessions, as well as importing data into Taprav.

 2.1        The Session

Any analysis work done on a data set is stored in Sessions.  A saved session stores imported datasets, time frames, selections, notes, and a variety of other settings necessary for keeping state.  Once a Session has been saved it can be reopened at a later time.

 2.1.1            Creating a New Session

When started, Taprav creates a new default session.  A new session can be created at any time by selecting File>New Session from the menu bar.  As only one session can be opened at a time, unless saved, the current session will be lost after creating a new session.

 2.1.2            Opening a Saved Session

To open a saved session, select File>Open Session from the menu bar.  Select the session file to open using the file chooser.  As only one session can be opened at a time, unless saved, the current session will be lost after opening the existing session.

 2.1.3            Saving a Session

The current session can be saved at any time by selecting File>Save from the menu bar.  If the session has not yet been saved, a prompt will be displayed requesting a file name for the Session. Alternatively, you can save the current session to a new location by selecting File>Save Session As from the menu bar.  Session files are automatically saved with the file extension *.ses.

 2.2        Importing Data

The current version of Taprav allows you to import pupil data, hierarchical task models, and video into the session.

 2.3        Pupil Data

Taprav supports the EyeLink ASC pupil data format (EDF data files can be converted to ASC format with the EyeLink EDFtoASC converter).

To import ASC pupil data into the current session, select File>Import>Pupil Data from the menu bar.

 2.3.1            Baseline Calculation

For analysis purposes, a pupil size baseline must be declared.  This value is obtained when importing pupil data by either giving the time period in the data collection where the baseline was measured, or by entering the baseline value directly.  In the former case, the baseline is calculated by averaging the pupil response values of the given eye during this period.

 2.4        Task Model

Hierarchical task models are imported into the current version of Taprav by reading in task model descriptions encoded in XML. The structure of the XML files is relatively simple, containing very few element types.

To import task model description files into the current session, select File>Import>Task Model from the menu bar and then select the data file from the file chooser.

 2.4.1            Task Model Description File Format

The XML format has three basic element types: task, start, and end.

The two attributes for task tags are name and offset.  The name attribute is the name of the task.  The name attribute is required.  The offset tag is an optional tag used to sync the task model to the corresponding pupil data.  The start time of the task block relative to the pupil data is given by adding the offset to the task start time of the task block frame.  Note that if the start time given in the start tag is correct relative to the pupil data, the offset tag is unnecessary.  Offset times are inherited from parent task blocks.

The start element describes the start time of a task block.  This tag has no attributes.  The start and end times are only necessary for the lowest level tasks.  Parent task times are determined by the child tasks.

The end element describes the end time of the task block.  Like the start element, this tag also has no attributes and is only necessary for the lowest level tasks.

The hierarchical structure of the task model is constructed by nesting task elements.  See the sample task model given in the samples directory for an example of a valid task model data file.

 2.5        Video

Importing video content into Taprav allows you to view video aligned with task model and pupil data.  Importing video is only available on those systems with QuickTime for Java installed (see section 1.3).

To import a video file into the current session, select File>Import>Video from the menu bar.  After selecting the video file from the file chooser, you will be prompted for the video offset.

 2.5.1            Video Offset

To properly align the video with other data sources, the video offset is used to set the zero time of the video relative to the pupil data.  This offset value should by the timestamp value where the video recording began.

 2.6        Saving Sessions with Imported Data

Imported task model and pupil data is internalized within sessions once saved.  Video data on the other hand is not.  A saved session retains file path information, but not the video content itself.  If a video file imported into a saved session is deleted or moved, Taprav will not be able to load the video content when the session is reopened.

 

 Chapter 3                  

The Visualization

This chapter describes the visualization paradigm used in Taprav.

 3.1        Pupil Response

Pupil response is plotted on the vertical axis over a horizontal timeline.  Here, pupil size is measured in percentage change in pupil size, or PCPS.  Subtracting the baseline value from the pupil size and dividing by the baseline yields the PCPS.  The redline running horizontally across the viewing rectangle represents the baseline value (0% PCPS).  Both the vertical and horizontal axes are of linear scale.

 3.2        Task Model

A rectangular block represents each task of the task model.  The width of a given task block corresponds to the time duration of the task.  The name of a given task block is drawn within the bounds of blocks (space permitting).  For more detailed information for a given task block, pass the mouse cursor over the task block for a tool tip task summary.

The task model itself is composed of a collection of task blocks.  The ordering of these blocks along the timeline gives the ordering relationships of the tasks in the model.  The task hierarchy is achieved by stacking these blocks into multiple levels.

 3.2.1            Task Levels and Boundaries

The vertical location of a task block indicates the level of the task model hierarchy the task occurs.  All task blocks of a given task level are filled with the same color.  The topmost row of task blocks belongs to the highest-level tasks, or level 0 tasks.  The task level decreases downwards, with the lowest level tasks on the bottom.    All child task blocks of a parent task are located below the parent block within the bounds of the parent block’s time span.  The child blocks do not overlap; instead, the block order is determined by time sequence.

Boundaries between task blocks are left unfilled. 

 3.2.2            Task Boundaries Overlay

To get a better sense of task model and pupil data alignment, Taprav includes a feature to render task boundaries over the local pupil visualization.  The widget to the right of the local task frame is used to control the task level boundaries that are drawn over the pupil data.  The widget itself is composed of a column of colored cells matching the task level colors.  A black bracket on the right of this column covers the range of task levels boundaries that will be drawn to the local pupil view.  Note that due to the boundary inheritance given by parent to child, the boundaries of level i include that of all levels 0 to i. The bracket illustrates this principle. To control the boundary widget, click a cell corresponding to the level you would like to see boundaries drawn.  Notice that the boundary lines drawn over the local pupil frame are the same as the range of levels selected by the boundaries widget.

 3.3        Focus and Context Frames

Given the sampling rate of eye tracking hardware and the duration of experiment trials, analysis of pupil response often carries the challenge of managing large datasets.  We have implemented a focus + context framework within Taprav to aide the user in navigating these large datasets.  This design allows the user to zoom to an area of interest while still being aware of the zoomed region’s time span relative to the entire data set.

 3.3.1            Local Frame

The top-most portion of the visualization panel is reserved for the local frame of the task model and pupil data.  The local frame is the “zoomed-in” area of the dataset.  This view is best used for detailed analysis.

 3.3.2            Global Frame

The bottom-most portion of the visualization panel holds the global frame of the task model and pupil data.  This frame shows the extent of the session data.  A blue rectangle is shown in the global frame corresponding to the time frame of the current local frame.  This is the magnification lens.  The global frame is not very useful for detailed analysis, however it is extremely useful for the analysis discovery process.  Within this frame of view the user is able to quickly and effortlessly navigate to any time region within the session data.

 3.4        Views

Through the use of views, Taprav allows the user to have multiple viewpoints of the same data source.  Each view can have a unique global, local, and selection frame.

 3.4.1            View Tabs

The user is able to quickly switch between views by way of a tabbed interface.  A row of view tabs is given at the bottom of the visualization panel.  The view tabs can be identified by unique names.  To look at a particular view, bring the desired view to the front by clicking the view tab you want to see. 

 3.4.2            New

Selecting Views>New from the menu bar creates a new view.  The new view takes on the properties of the currently active view.

 3.4.3            Rename

To rename the currently active view, selecting Views>Rename from the menu bar.  Enter the new name into the pop-up dialog box.

 3.4.4            Close

To close the currently active view, select View>Close from the menu bar.  Once closed, the view will be lost forever.

 

 

 

 

 

 

 

 Chapter 4                  

Interacting with the Visualization

This chapter describes the methods available to the user for interacting with the visualization.

 4.1        Local Frame Interaction

The user can interact with the local frame by zooming in and out of the current local frame.  In addition, finer statistical analysis within the local frame can be performed via selections.

 4.1.1            Zooming

To zoom in on an area of interest, hold down the left mouse button and drag across one of the local frame windows to create a zoom window over the area of interest.  Upon release, the local frame will zoom in to this window.

 4.1.2            Selection

In addition to zooming, the user can also perform selection within the local frame for further analysis.  To create a selection window, hold down the shift key while creating a time window as you would to zoom.  Upon release, the region within this window will be tinted gray to indicate selection.  For more details on the statistical uses of selection, refer to the discussion of the statistics window (section 5.3).

 4.2        Global Frame Interaction

The user interacts with the global frame by manipulating the magnification lens.  These actions have a direct effect on the local frame.

 4.2.1            Resizing the Magnification Lens

When on the vertical edges of the magnification lens, the cursor becomes a resize cursor.  The lens can then be resized by holding down the left mouse button while dragging one of the vertical edges of the lens.  When released, the local frame will adjust to adhere to the resized magnification lens.

 4.2.2            Moving the Magnification Lens

The cursor will appear as a hand when inside of the magnification lens.  The lens can be moved within the bounds of the global frame by holding down the left mouse button while dragging within the current magnification lens.  When released, the local frame will adjust to adhere to the repositioned lens.

 4.2.3            Recreating the Magnification Lens

The magnification lens can also be recreated ad hoc.  A new magnification lens is created by holding down the left mouse button while dragging a lens within the global frame. The initial click must be outside of the current magnification lens.  When released, the local frame will adjust to adhere to the newly created magnification lens.

 4.3        Menu Bar Actions

The menu bar provides additional actions to interact with the visualization.

 4.3.1            View Menu

The actions available within the View menu affect the temporal parameters of the visualization.  The View menu is split into Local Frame and Global Frame actions.

 4.3.1.1                  View>Local Frame

The Local Frame menu contains actions that affect information viewed within the local frame.

 4.3.1.2                  View>Global Frame

Global Frame submenu actions affect the content displayed within the Global Frame. Through these menu actions, the Global Frame can be set to a sub-range of the entire dataset.  Shrinking the Global Frame can be very useful when managing large datasets. 

The Global Frame can be adjusted in terms of the pupil data or the task model.

 4.3.2            The Select Menu

The selection menu actions affect the state of the Selection Frame.

 

 

 Chapter 5                  

Tools

This chapter describes the other tools provided by Taprav to help you explore your data.

 5.1        Filtering the Pupil Response

Eye Tracker hardware typically records pupil size events when the user looks away from the screen.  Events of this kind can be identified on the pupil response curve by abnormal valleys dipping below the baseline.  This data is usually not interesting to visualize, and is typically ignored during statistical analysis.  A simple filtering mechanism is built into Taprav to clean such unwanted data.  Filtering works by masking all pupil response events below a certain threshold value from visualization and statistical analysis.  To perform filtering on a session with imported pupil data, select Tools>Pupil Mask Settings from the menu bar.  In the dialog box that appears, set the threshold with the slider, or manually enter the threshold into the textbox.  Notice that the pupil response curve previews the resulting curve as you change the threshold.  Once the desired threshold is set, click OK to apply filtering.

 5.2        Notes

The notes panel allows you to make comments regarding a particular view.  This can be particularly useful when managing a large number of views or when doing collaborative work.  A view’s notes are available whenever the view is active.  Notes are saved automatically when text is typed into the notes textbox.

 5.3        Statistics

The statistics panel presents some simple statistics to the user based on the global, local, and selected frames.  Mean, min, and max pupil size, as well as standard deviation are presented in tabular form for each frame. 

 5.4        Video

The video panel allows the user to watch a video clip corresponding to the time span of the local frame.  This option is only available on those systems with QuickTime for Java installed (see section 1.3).

 5.4.1            Controlling the Video

Video content can be controlled in a way similar to many current media players.  The video player has play/pause, forward skip, and back skip buttons.  A time slider is also given in the video player window.  The video clip can be scrubbed by dragging the slider knob.

 5.4.2            View Play head

During video playback, a red vertical line representing the current video frame is displayed in visualization panel.  This play head is updated as the video time is changed.