= Mymory Evaluation =

This wiki will be a scratch pad and contain a TODO list for people that will be involved in Mymory's context evaluation.

== Software to be installed ==

 * '''FirstSteps''' - By following the steps described there you will:
   * install UserObservationHub (please '''{{{svn update}}}''' if you already have an older version)
   * install [http://dragontalk.opendfki.de/ Dragontalk]: Observation plugins for '''Thunderbird''' and '''Firefox''' (please install the '''newest version''')

 * '''WindowsLoggerListener''' (Windows users only) - By following the steps described there you will:
   * install [http://pas.kbs.uni-hannover.de/ User Activity Logger]
   * configure property file ({{{./etc/userobservationhub.properties}}}) to let UOH work with output of the User Activity Logger

 * '''ScreenshotRecorder''' (optional) - By following the steps described there you will:
   * configure property file ({{{./etc/userobservationhub.properties}}}) to set the screenshot interval time or to deactivate the screenshot recorder


== Part 1: Logging user actions --> create a log file with NOPs ==

The first important part of the evaluation is the logging part. We need a log of your actions for some period - at least for a day, better for one week or so. To create this log and (iteratively) accumulate observed user actions, you just start / kill / restart the UOH:

 * '''Start''' the UOH (via the batch file for example) and just "do what you gotta do"! ;-)
 * Disabling UOH is not yet implemented - just '''kill''' the UOH (with multiple CTRL+C for example) - this will not harm anything! feel free to kill that beast whenever you want or have to!
 * '''Restart''' UOH again any time.

Restarting the UOH is no problem. Observed user actions are always appended to the log file ({{{userobservationhub_nops.log}}}).



== Part 2: Import legacy data (gnowsis, context threads) ==

DFKI researchers can use the following team set to check out all projects needed for this concern: Save this file '''[http://usercontext.opendfki.de/raw-attachment/wiki/MymoryEval/MymoryEval_PimoEditorBranch_TeamSet.psf MymoryEval_PimoEditorBranch_TeamSet.psf]''' and then use eclipse/File/import/Team/Team Project Set.

TODO: Upload runtime configuration here

Start runtime by launching MyContextEval.launch (TODO: runtime not yet uploaded here)

The easiest way to import the PIMO from [http://gnowsis.opendfki.de/ gnowsis beta] and to transform it to be conform with [http://pimoeditor.opendfki.de/ PIMOEditor] is to launch the MyContext.launch (TODO: upload .launch) and use the button at the Context "Cidebar" called "Sync with gnowsis".

TODO: Write how to export and import a list of old context threads (JConsole -> ...)


== Part 3: Assign contexts to NOPs --> create ground truth ==

The second important part is the '''manual''' assignment of the logged NOPs to the user's contexts.

To do so, Mymory created an "evaluation GUI", that eases this task.

Click on the button "ContextualizeObservedNopsGui" to open the evaluation GUI:

[[Image(EvalGuiSmall.png)]]

Recorded NOPs are presented as rows in this GUI. Columns indicate the NOP's date and time, the hopefully best screenshot for this time, type of the user action, as well as, further extracted information stored in the NOP that shall allow you to remember what you actually did then.

On the left side is a list of your context threads. Each has a differently colored background. If you miss some context thread here you can add a new one using the text field at the bottom of this left panel.

To assign one or more NOPs to a specific context thread you select one or more rows, thus, selecting one of more NOPs.
Then double-click on some context-thread on the left.

You can also add some meaningless NOP to a blacklist by right-clicking on some selected NOP row(s) and then choosing "add to blacklist" on the context menu. NOPs with the same pattern are also removed from the table. Here, "NOPs with the same pattern" means NOPs, that have identical values in the columns disregarding the date/time and screenshot of course.[[BR]]
'''Please take care'''! Currently, there is no easy way to undo this "blacklisting" of NOPs!

The context menu also allows to permanently "delete" NOPs. This is only needed in extrem cases.

After having assigned all NOPs to context threads, you click on the "Save" button, which creates an edited userobs log file.
This file is the '''ground truth''' that will be used to run evaluation calculations on.



== Part 4: Run evaluations on ground truth ==

TODO