Building SimpleGazeTracker for unsupported cameras

If you have a camera unsupported by SimpleGazeTracker and can write C++ programs for your camera, it would be easy to build SimpleGazeTracekr customized for your camera. Table 1 shows source files of SimpleGazeTracker. Only Camera_foo.cpp and GazeTracker.h are camera-specific. That is, only Camera_foo.cpp and GazeTracker.h must be written to build SimpleGazeTracker for unsupported camera.

Table 1

DetectEye.cpp	Common	Detecting pupil from camera image, estimating gaze position, rendering camera preview image and calibration results.
GazeTrackerMain.cpp	Common	Initializing application, event handling, saving data.
PlatformDependent.cpp	Common	Defining Plat-form dependent operatoins (such as timer and file-manipulation).
tcpipcom.cpp	Common	Receiving commands and sending data via TCP/IP.
GazeTrackerCommon.h	Common	Defining constants, extern declerations of functions and variables.
Camera_foo.cpp	Camera-specific	Initializing camera, getting camera image ("foo" indicates edition).
GazeTracker.h	Camera-specific	Camera-specific header file.

In the camera-specific source file, following fucntions must be defined (Table 2).

Table 2

getEditionString()	Returning edition name. This function is used to display edition name when SimpleGazeTracker is started.
initCamera()	Initializing camera. In this function, camera is initialized and starts capturing.
getCameraImage()	Copying captured image to memory.
cleanupCamera()	Releasing camera resource. This function is called when SimpleGazeTracker is terminated.
saveCameraParameters()	Save camera parameters to configuration file. This function is called when SimpleGazeTracker is terminated.
customCameraMenu()	Defining camera-specific menu items.
updateCustomMenuText()	Updating camera-specific menu items.
getCameraSpecificData()	Returning camera-specific data. New in 0.6.5

Camera-specific functions

const char *getEditionString(void)

This function simply returns edition name. This function is used to display edition name when SimpleGazeTracker is started.

const char* getEditionString(void)
{
    return "OpenCV Edition";
}

int initCamera(void)

This function initializes camera. Following tasks should be done in this function.

1. Load camera-specific configuration file if necessary.

2. Initialize camera.

3. Start capturing.

The default camera-specific configuration file should be placed in the SimpleGazeTracker installation directory. It is rcommended to use:cpp:func:checkAndCopyFile to create a copy of camera-specific configuration file in the SimpleGazeTracker user configuration directory. Global variable g_ParamPath holds SimpleGazeTracker configuration directory. g_AppDirPath holds SimpleGazeTracker installation directory. In addtion, In version 0.6.6, g_CameraConfigFileName is added to hold the value of -cameraconfig commandline option. Don't modify value of these global variables in this function.

std::string cfgfname;

//Set configuration directory.
cfgfname = g_ParamPath.c_str();
cfgfname.append(PATH_SEPARATOR);

//-cameraconfig option is given?
if(g_CameraConfigFileName==""){
    //Check default configuration file of your camera.
    checkAndCopyFile(g_ParamPath, "CONFIG_FOR_YOUR_CAMERA", g_AppDirPath);
    //Set name of default configuration file of your camera.
    g_CameraConfigFileName = "CONFIG_FOR_YOUR_CAMERA";
}

//Append name of configuration file.
cfgfname.append(g_CameraConfigFileName.c_str());

Camera initialization codes depend on camera's SDK. Plese read SDK's document. An important point is that camera's image width and height should be equal to g_CameraWidth and g_CameraHeight, respectively. If not, captured image must be resized everytime new image is captured. The value of g_CameraWidth and g_CameraHeight is defined by CAMERA_WIDTH and CAMERA_HEIGHT in the SimpleGazeTracker configuration file. Don't modify value of these global variables in this function.

If you want to add camera-specific menu items, you have to set number of menu items to g_CustomMenuNum. See Camera_OptiTrack.cpp for an example of defining camera-specific menu items.

戻り値

S_OK if initialization succeeded, E_FAIL if initializataion failed. If E_FAIL is returned, SimpleGazeTracker terminates with error message.

int getCameraImage(void)

This function transfars image from camera and copy the image to frame buffer. Global variable g_frameBuffer holds the first address of the frame buffer. Format of the image must be 8-bit monochrome. Width and height of the image must be equal to g_CameraWidth and g_CameraHeight. The value of g_CameraWidth and g_CameraHeight is defined by CAMERA_WIDTH and CAMERA_HEIGHT in the SimpleGazeTracker configuration file. Don't modify value of these global variables in this function.

戻り値

S_OK if new image is copied to frame buffer. E_FAIL if new image is not available. If S_OK is returned, SimpleGazeTracker starts image analysis. Otherwise, SimpleGazeTracker checks messages and events and then call getCameraImage() again.

An important point for writing this function is whether the image trasfering function locks program until new image is available. For example, GetFrame() method in OptiTrack 2D SDK immediately returns regardless of image availability. In such a case, call image trasfering function in getCameraImage() and branch processing depending on image availability.

int getCameraImage(void)
{
    g_camera->GetFrame(0, &g_frame); //This method transfars new image if available.

    if(g_frame!=0){ //g_frame is not 0 if new image is transfarred.

        // Copy transfarred image to frame buffer here.
        // Then, return S_OK.
        return S_OK;
    }

    // if g_frame==0, new image is not available.
    // In this case, return E_FAIL.
    return E_FAIL;
}

This algorithm doesn't work well if image transfering function locks program. In the following example, OpenCV VideoCapture class is used to get camera image. As far as my experience goes, retrieve() locks program until new image is available in many hardware setup. This means that SimpleGazeTracker cannot process messages sent from stimulus presentation PC while program is locked, resulting in severe performance decrement.

One solution for this problem is using thread. Following function is a "call-back" function, which runs in a separalte thread. In this function, global variable "g_NewFrameAvailable" is set to true when new image is successfully copied to frame buffer.

int captureCameraThread(void *unused)
{
    cv::Mat frame, monoFrame;

    while(g_runThread) // Repeat while global variable g_runThread is true.
    {
        if(g_VideoCapture.grab())
        {
            g_VideoCapture.retrieve(frame);  //This method transfars new image.

            // Copy transfarred image to frame buffer here.
            // Then, set global variable instead of returning S_OK.
            g_NewFrameAvailable = true;

            // Sleep to save CPU resource.
            // g_SleepDuration should be slightly shorter than 1000/(frames per second).
            if(g_SleepDuration>0.0)
            {
                sleepMilliseconds(g_SleepDuration);
            }
        }
    }

    return 0;
}

This call-back function should be started in initCamera() function. Using SDL_CreateThread() is one of platform-independent ways to create a thread. Set true to global variable g_runThread before starting thread. g_runThread will be used to stop thread when application is being closed.

int initCamera( const char* ParamPath )
{
    // Read configuration file.
    // Initialize camera.

    // Set global flag and start thread.
    g_runThread = true;
    g_pThread = SDL_CreateThread(captureCameraThread, NULL);

    return S_OK;
}

In getCameraImage(), g_NewFrameAvailable is checked instead of calling grab() and retrieve(). If g_NewFrameAvailable is true, getCameraImage returns S_OK because new image has been copied to frame buffer in captureCameraThread(). Note that g_NewFrameAvailable must be set to false to prepare next image acuisition. If g_NewFrameAvailable is false, getCameraImage() returns E_FAIL.

int getCameraImage( void )
{
    if(g_NewFrameAvailable)
    {
        g_NewFrameAvailable = false;
        return S_OK;
    }
    return E_FAIL;
}

It is important to stop thread when application is going to be closed. In cleanupCamera(), g_runThread is set to false. By this operation, while-loop in captureCameraThread() is terminated and thread is finished. Use SDL_WaitThread to wait until thread has been finished (if you started thread using SDL_CreateThread, of course).

void cleanupCamera()
{
    g_runThread = false;
    SDL_WaitThread(g_pThread, NULL);
}

void cleanupCamera(void)

This function stops camera and releases resources allocated in initCamera(). This function is called when SimpleGazeTracker is terminating.

void saveCameraParameters(void)

This function writes camera-specific parameters to camera-specific configuration file. This function is called when SimpleGazeTracker is terminating.

This function will be unnecessary if no camera-specific parameters are updated during SimpleGazeTracker is runnning. In such a case, this function should return without doing anything.

int customCameraMenu(SDL_Event *SDLevent, int currentMenuPosition)

This function defines behavior of camera specific menu item. Note that rendering menu string on the application window is done by updateCustomMenuText(). The arguments SDLevent and currentMenuPosition hold event and current menu position, respectively. If custom menu is unnecessary for your camera, this function should return without doing anything.

In the following example, two camera-specific menu items, "Intensity" and "Exposure" are defined. All menu items must have unique position, which indicates order of the item in the menu list. A constant MENU_GENERAL_NUM indicates number of default menu items defined in GazeTrackerMain.cpp. It is recommended to set camera-specific item's position immediately after the last default menu item's position, that is, MENU_GENERAL_NUM, (MENU_GENERAL_NUM+1), (MENU_GENERAL_NUM+2), ...

in addition, , SDL/SDL.h must be included to decode event.

#define CUSTOMMENU_INTENSITY  (MENU_GENERAL_NUM+0)
#define CUSTOMMENU_EXPOSURE   (MENU_GENERAL_NUM+1)

#include <SDL/SDL.h>

Number of custom menu items must be set to g_CustomMenuNum in initCamera().

g_CustomMenuNum = 2

In customCameraMenu(), define processes corresponding to menu items using conditional branch. Please read SDL's documents for detail of SDL_Evenet. Finally, return S_OK.

int customCameraMenu(SDL_Event* SDLevent, int currentMenuPosition)
{
    switch(SDLevent->type){
    case SDL_KEYDOWN:
        switch(SDLevent->key.keysym.sym)
        {
        case SDLK_LEFT:
            switch(currentMenuPosition)
            {
            case CUSTOMMENU_INTENSITY:
                g_Intensity--;
                if(g_Intensity<1)
                    g_Intensity = 1;
                g_camera->SetOption(NP_OPTION_INTENSITY,(CComVariant) g_Intensity);
                break;
            case CUSTOMMENU_EXPOSURE:
                g_Exposure--;
                if(g_Exposure<0)
                    g_Exposure = 0;
                g_camera->SetOption(NP_OPTION_EXPOSURE,(CComVariant) g_Exposure);
                break;
            default:
                break;
            }
            break;

        case SDLK_RIGHT:
            switch(currentMenuPosition)
            {
            case CUSTOMMENU_INTENSITY:
                g_Intensity++;
                if(g_Intensity>=12)
                    g_Intensity = 12;
                g_camera->SetOption(NP_OPTION_INTENSITY,(CComVariant) g_Intensity);
                break;
            case CUSTOMMENU_EXPOSURE:
                g_Exposure++;
                if(g_Exposure>=479)
                    g_Exposure = 479;
                g_camera->SetOption(NP_OPTION_EXPOSURE,(CComVariant) g_Exposure);
                break;
            default:
                break;
            }
            break;

        default:
            break;
        }
    default:
        break;
    }

    return S_OK;
}

void updateCustomMenuText(void)

This function sets camera-specific menu string to a global variable g_MenuString. Position of camera-specific menu items must be equal to those used in customCameraMenu(). If custom menu is unnecessary for your camera, this function should return without doing anything.

#define CUSTOMMENU_INTENSITY  (MENU_GENERAL_NUM+0)
#define CUSTOMMENU_EXPOSURE   (MENU_GENERAL_NUM+1)

void updateCustomMenuText( void )
{
    std::stringstream ss;
    ss << "LightIntensity(" << g_Intensity << ")";
    g_MenuString[CUSTOMMENU_INTENSITY] = ss.str();
    ss.str("");
    ss << "CameraExposure(" << g_Exposure << ")";
    g_MenuString[CUSTOMMENU_EXPOSURE] = ss.str();

    return;
}

getCameraSpecificData(void)

New in 0.6.5

If your camera has input port, you can insert its value to the SimpleGazeTracker data file using this function. If this feature is unnecessary, this function should return without doing anything. Note that the return value must be unsigned int.

DWORD g_DigitalInput = 0;

unsigned int getCameraSpecificData( void )
{
    //Value of g_DigitalInput is set other function in the camera-specific source file.
    return (unsigned int)g_DigitalInput;
}

Global variables, functions and constants

These variables, functions and constants are declared in GazeTrackerCommon.h. Please include GazeTrackerCommon.h to use them.

double getCurrentTime(void)

Platform-independent function to get current time in milliseconds.

void sleepMilliseconds(int duration)

Platform-independent function to sleep process. Unit of duration is milliseconds.

int getDataDirectoryPath(std::string *path)

Platform-independent function to get full path to SimpleGazeTracker data directory. Path string to the data directory is set to argument path.

Currently, this function always returns 0.

int getParameterDirectoryPath(std::string *path)

Platform-independent function to get full path to SimpleGazeTracker configuration file directory. Path string to the data directory is set to argument path.

Currently, this function always returns 0.

int checkAndCreateDirectory(std::string path)

Platform-independent function to check existence of a directory. If directory is not found, this function try to create directory.

S_OK is returned if directory exists or directory is successfully created. Otherwise, E_FAIL is returned.

int checkAndCopyFile(std::string path, const char *filename, std::string sourcePath)

Platform-independent function to check existence of a file named filename. If file is not found in directory sperified by argument path, this function try to copy file from directory specified by argument sourcePath.

S_OK is returned if file exists or file is successfully copied. Otherwise, E_FAIL is returned.

int g_CameraWidth

This variable holds width of the frame buffer. Value of this variable is determined by CAMERA_WIDTH in the SimpleGazeTracker configuration file.

int g_CameraHeight

This variable holds height of the frame buffer. Value of this variable is determined by CAMERA_HEIGHT in the SimpleGazeTracker configuration file.

unsigned char *g_frameBuffer

This variable holds the first address of the frame buffer. Size of the frame buffer is equal to g_CameraWidth * g_CameraHeight.

std::string g_ParamPath

This variable holds SimpleGazeTracker configuration directory.

std::string g_AppDirPath

This variable holds SimpleGazeTracker installation directory.

std::string g_CameraConfigFileName

This variable holds value of -cameraconfig option. If -cameraconfig is not given, this valiable holds an empty string (i.e. ""). New in 0.6.6.

std::string g_MenuString[MAX_MENU_ITEMS]

This array holds menu strings. General (=camera-independent) menu strings are stored in the elements former than MENU_GENERAL_NUM. Don't modify general menu strings.

int g_CustomMenuNum

This variable holds number of custom menu items. Default value is 0. Don't forget to set value in initCamera() if your camera needs custom menu items.

Constants

• MENU_GENERAL_NUM Number of general menu items. Index of camera-specific menu item must be equal or greater than this value. This value is 6 (0.6.4).

• MAX_MENU_ITEMS Sum of MENU_GENERAL_NUM and number of camera-specific menu items must be equal or smaller than this value. This value is 12 (0.6.4).

• MENU_STRING_MAX Text of each menu item must be shorter than this value. This value is 24 (0.6.4).

• PATH_SEPARATOR '' in windows , '/' in Linux and MacOS X.