Tutorial 01: How to generate GazeParser data file

GazeParser.Converter module includes conversion tools to generate GazeParser data file. Currently, SimpleGazeTracker CSV file (*.csv) and Eyelink EDF file (*.edf) are supported. GazeParser.Converter has TobiiToGazeParser() method, but it is under development and has only limited functions.

SimpleGazeTracker CSV to GazeParser

To generate GazePaser datafile from SimpleGazeTracker CSV file, use TrackerToGazeParser() function. In the following example, 'data.csv' is a SimpleGazeTracker data file to be converted (data.csv is in the current directory).

>>> import GazeParser.Converter
>>> GazeParser.Converter.TrackerToGazeParser('data.csv')

If this method run successfully, a file named 'data.db' is generated. This file can be read with GazeParser.load() method. TrackerToGazeParser accepts following optional parameters.

Name	Description
overwrite	By default, TrackerToGazeParser does not overwrite existing file. If overwrite=True is specified, TrackerToGazeParser overwrites existing file. Default value is False.
config	Specify a GazeParser.Configuration.Config object where conversion parameters are defined. If no configuration object is specified (i.e. config=None), TrackerToGazeParser uses default GazeParser configuration file. See Location of GazeParser Configuration file and Configure GazeParser for . details of default GazeParser configuration file. Default value is None
useFileParameters	If this parameter is True, conversion parameters are overwritten by parameters recorded in SimpleGazeTracker data file. Default value is True

Figure 1 shows a flow chart of setting 'config' and 'useFileParameters'.

Figure 1

GazeParser has a viewer of GazeParser data file. Executable file of the Viewer is app/Viewer.py in the GazeParser installation directory. If you are not sure of GazeParser installation directory, start pyhton interpreter and type as following.:

>>> import GazeParser
>>> print GazeParser.__file__
C:\Python27\lib\site-packages\gazeparser-0.6.0-py2.7.egg\GazeParser\__init__.pyc

In this case, C:\Python27\lib\site-packages\gazeparser-0.6.0-py2.7.egg\GazeParser is the GazeParser installation directory. Viewer.py is at C:\Python27\lib\site-packages\gazeparser-0.6.0-py2.7.egg\GazeParser\app\Viewer.py. If you installed GazeParser using win32 installer, you will find a shortcut for the Viewer in 'GazeParser' program group (Figure 2).

Figure 2

Start Viewer and choose [Open] from [File] menu.

Figure 3

Figure 4 shows an example of data. Raw gaze data is plotted at 'plot area'. Abscissa and ordinate represents time (in msec) and gaze posiiton (in screen coordinate), respectively. Vertical green lines represent inserted messages. Hatched pale bule areas represent detected saccades. To pane or zoom plot, click 'pane/zoom button' and then left-drag (pane) or right-drag (zoom) on the plot area.

'Event list' is a list of fixation, saccades and messages. By double-clicking an item on the event list, plot area is redrawn so that double-clicked event is at the center of the plot area.

Figure 4

In Figure 4, obviously too many saccades are detected. The range of the abscissa is 3000 msec. Usually, saccades occur 3-4 times per second at most. In addition, some saccades have very long duration for their amplitutde. This is a typical conversion result when velocity threshold for saccade detection is too low. In such a case, prepare a GazeParser configuration file and re-convert the data. Copy the configuration file that was inserted when the data is recorded (See also Configure GazeParser). Suppose that SACCADE_VELOCITY_THRESHOLD in the configuration file is modified from 22.0 to 40.0. Then the configuration file is saved as 'velthreshold40.cfg'.

To re-convert data.csv using velthreshold40.cfg, execute following command from python interpreter.:

>>> newconfig = GazeParser.Configuration.Config('velthreshold40.cfg')
>>> GazeParser.Converter.TrackerToGazeParser('data.csv', config=newconfig, overwrite=True, useFileParameters=False)

In GazeParser 0.6.1 or later, TrackerToGazeParser() reads configuration file when name of the configuration file is passed as 'config'.:

>>> GazeParser.Converter.TrackerToGazeParser('data.csv', config='velthreshold40.cfg', overwrite=True, useFileParameters=False)

注釈

You can directly edit inserted parameters in SimpleGazeTracker CSV file. In such a case, you don't have to create and load a new configuration file. If you want to re-convert a few files, this method is handy; however, if you have to re-convert many files, probably it is easy to create a new configuration file. See Format of the SimpleGazeTracker CSV data file (0.5.3 to 0.6.6) for format of SimpleGazeTracker CSV file.

Figure 5 shows the reuslt of re-conversion. Saccades with unnaturally long duration are disappeared.

Figure 5

A value of 40.0 may be somewhat high as velocity threshold for saccade detection. Appropriate threshould depends on measurement noise. For example, when recording data with a high-speed camera, small measurement noises are likely to misdetected as a saccade. The data used for this example was recorded at 400Hz and measurement noise constantly exceeded the threshold inserted to SimpleGazeParser CSV file (22.0 deg/s). It is recommended that record saccades and determine parameters suitable for your camera at first.

GazeParser.Converter has several parameters besides SACCADE_VELOCITY_THRESHOLD. For example, SACCADE_MINIMUM_DURATION prevents to detect too short saccades. Low-pass filers can be applied to the data by setting FILTER_* parameters. See Applying low-pass filter for detail of low-pass fileter.

Eyelink EDF to GazeParser

Use EyelinkToGazeParser() method to convert an Eyelink EDF file to GazeParser data file.

import GazeParser.Converter
GazeParser.Converter.EyelinkToGazeParser('data.edf')

EyelinkToGazeParser accepts the same parameters as those of TrackerToGazeParser except useFileParameters. Saccades, fixations, blinks and events are imported from EDF file. Therefore, parameters concerning saccade detection in GazeParser.Configuration object (e.g. SACCADE_VELOCITY_THRESHOLD) don't affect conversion result. Parameters concerning display setting (e.g. SCREEN_ORIGIN, SCREEN_HEIGHT and SCREEN_WIDTH) affect data visualization with GazeParser data viewer.

Batch conversion

os.listdir() and os.walk() functions are useful to convert multiple SimpleGazeTracker CSV files. For example, following script converts CSV files in 'data' directory.:

#!/usr/bin/env python
import os
import GazeParser.Converter

for file in os.listdir('data'):
    if os.path.splitext(file)[1].lower() == '.csv': # Is file extension 'csv'?
        GazeParser.Converter.TrackerToGazeParser(os.path.join('data',file))

To convert all CSV files in subdirectories, use os.walk().:

#!/usr/bin/env python
import os
import GazeParser.Converter

for root, dirs, files in os.walk('data'):
    for file in files:
        if os.path.splitext(file)[1].lower() == '.csv': # Is file extension 'csv'?
            GazeParser.Converter.TrackerToGazeParser(os.path.join(root,file))

GazeParser data viewer can also be used to convert multiple files (see below).

Converting SimpleGazeTracker CSV to GazeParser with Data Viewer

If you use inserted parameters to convert SimpleGazeTracker CSV file, you can directly select SimpleGazeTracker CSV file from [Open] in the [File] menu. If CSV file is not shown in the dialog, select 'SimpleGazeTracker CSV file (*.csv)' as file type (Figure 6).

Figure 6

If corresponding .db file exists, a confirmation dialog for overwriting is presented.

Figure 7

To specify conversion parameters, open data conversion dialog from [Convert SimpleGazeTracker CSV] in [Convert] menu.

注釈

In version 0.6.0, menu item is mistyped as [Convert GazeTracker CSV]. Data conversion dialog is a prototype and will be modified in future.

Figure 8

To customize conversion parameters, select 'Use following parameters' (box 1). This radiobutton corresponds to useFileParameters optin of TrackerToGazeParser(). You can directly edit parameters or load GazeParser configuration file by clicking 'Load configuration file' (box 2). To start conversion, clikc 'Convert files' (box 3). If you want to overwrite existing .db files, check 'Overwrite' (box 4).

Multiple files in a directory can be selected at once (Figure 9).

Figure 9

An error dialog is presented if any errors occur during conversion. Figure 10 shows a dialog which is presented when output file cannot be opened. In many cases, the cause for this error is that 'Overwrite' is not checked.

Figure 10

If other error is presented, please confirm that appropriate parameters are set. When all files have been processed, conversion results are reported (Figure 11).

Figure 11

Applying low-pass filter

As sampling frequency becomes higher, measurement noise will disturb saccade detection more severely. GazeParser.Converter has following built-in low-pass filters to reduce measurement noise.

filter	FILTER_TYPE	Description
Moving average	ma	A sequence of averages of N consecutive samples. N is determined by FILTER_SIZE.
Butterworth filter	butter	Applying an N-th order Butterworth low-pass filter. Order is determined by FILTER_ORDER. Cut off frequency is determined by FILTER_WN. FILTER_WN must be from 0.0 to 1.0. The value of 1.0 correspond to the Nyquist frequency. That is, if sampling frequency is 250Hz, the value of 1.0 corresponds to 125Hz.
Forwar-backward Butterworth filter	butter_filtfilt	Applying an N-th order zero-phase Butterworth low-pass filter. Zero-phase means that no phase shift occurs in the output. The order and the cut off frecuency of the filter is determined by FILTER_ORDER and FILTER_WN, respectively.
(No filter)	identity

Figure 12 and 13 show raw data and outputs of these filters. A curve labeled 'raw' represents raw data. Curves labeled 'ma', 'butter' and 'butter_filtfilt' represent the output of corresponding filters (FILTER_SIZE=5, FILTER_WN=0.2 and FILTER_ORDER=3).

Figure 12

Figure 13

Figure 14 shows the effect of FILTER_WN on the output of the Butterworth filter. Smaller FILTER_WIN yields in smoother output. Notably, the output is delayed from the raw data due to phase shift of Butterworth filter.

Figure 14

Figure 15 shows the effect of FILTER_ORDER on the output of the Butterworth filter. Similar to FILTER_WN, larger FITER_ORDER yields smoother and delayed output.

Figure 15

In contrast, output of the zero-phase Butterworth filter is not delayed from the raw data.

Figure 16

Resampling data

Ideally, interval between successive samples is constant throughout recording.; however, the interval fluctuates because of several reasons such as version of camera's driver, compatibility between camera and frame grabber and so on. Because GazeParser's low-pass filter function assumes that the interval is constant, low-pass filter cannot be used if fluctuation of the interval is not negligibly small.

RESAMPLING option is added to GazeParser 0.6.6 to "rectify" data with fructuated interval. If positivie value is set to RESAMPLING attribute of Configuration object, TrackerToGazeParser() resamples raw data at a frequency specified by RESAMPLING attribute. Following code makes GazeParser resample raw data at 200Hz before applying low-pass filter.:

newconfig = GazeParser.Configuration.Config()
newconfig.RESAMPLING = 200
GazeParser.Converter.TrackerToGazeParser('data.csv', config=newconfig, useFileParameters=False)

Resampling is performed by using SciPy's linear interpolation function. Resampling rate should not be set to a value far heigher than the original frequency.