Mymory Evaluation

This wiki will be a scratch pad and contain a TODO list for people that will be involved in Mymory's context evaluation.

Software to be installed

• FirstSteps - By following the steps described there you will:
  • install UserObservationHub (please svn update if you already have an older version)
  • install ​Dragontalk: Observation plugins for Thunderbird and Firefox (please install the newest version)

• WindowsLoggerListener (Windows users only) - By following the steps described there you will:
  • install ​User Activity Logger --> download the .rar file on that web page and run the installer in the Logging_Framework_0_2_0 folder.
  • configure property file (./etc/userobservationhub.properties) to let UOH work with output of the User Activity Logger

• ScreenshotRecorder (optional) - By following the steps described there you will:
  • configure property file (./etc/userobservationhub.properties) to set the screenshot interval time or to deactivate the screenshot recorder

Part 1: Logging user actions --> create a log file with NOPs

The first important part of the evaluation is the logging part. We need a log of your actions for some period - at least for a day, better for one week or so. To create this log and (iteratively) accumulate observed user actions, you just start / kill / restart the UOH:

• Start the UOH (via the batch file for example) and just "do what you gotta do"! ;-)
• Disabling UOH is not yet implemented - just kill the UOH (with multiple CTRL+C for example) - this will not harm anything! feel free to kill that beast whenever you want or have to!
• Restart UOH again any time.

Restarting the UOH is no problem. Observed user actions are always appended to the log file (userobservationhub_nops.log).

Part 2: Import legacy data (gnowsis, context threads)

DFKI researchers can use the following team set to check out all projects needed for this concern.
Best, you let eclipse create and switch to a new workspace[[BR]] Save this file MymoryEval_PimoEditorBranch_TeamSet.psf and then use eclipse/File/import/Team/Team Project Set.

Subclipse users: use MymoryEval_PimoEditorBranch_TeamSet_subclipse.psf and rebuild your workspace after importing.

Then, download this file runtime_MyContextEval.zip and unzip it to the same location where your (new) eclipse workspace is.
You hopefully end up having a directory runtime_MyContextEval next to all the other projects you just checked out via svn.

Add this as a new project to your eclipse workspace via eclipse/File/import/General/Existing Projects into Workspace. When you open the Run Dialog in eclipse (Run/Open Run Dialog) you should find new launch configurations like "Launch MyContextEval" or "Launch MyContext+MyDesk" for example.

import PIMO from gnowsis beta

Please make sure, that gnowsis beta is running, before going further.
The easiest way to import the PIMO from ​gnowsis beta and to transform it to be conform with ​PIMOEditor is to launch the MyContextMyDesk.launch and wait a few seconds (10+ seconds) until you see the Context "Cidebar" open up at the left side of your screen. There is a button called "<<Sync PIMO with Gnowsis>>" at the bottom of that GUI. Press it and wait until the button toggles back to its normal state - this may take a while (1 minute).

When finished, shutdown the Mymory platform by rightclicking the trayicon (blue cube with "M" on it) and choosing "shutdown". Killing the application can harm the RDF store and lucence and all these nice, sophisticated things that help to make the world a better place... ;-)

TODO: Write how to export and import a list of old context threads (JConsole -> ...)

Import PIMO from NEPOMUK

• Make sure NEPOMUK is running.
• in eclipse, run the target: Launch MyContext+MyDesk
• press button Sync NEPOMUK with Gnowsis
• wait (long)

Part 3: Assign contexts to NOPs --> create ground truth

The second important part is the manual assignment of the logged NOPs to the user's contexts.

To do so, Mymory created an "evaluation GUI", that eases this task.

• Launch MyContextEval (in eclipse)
• In the system-tray, right-click the mymory icon and click "ContextualizeObservedNopsGui" to open the evaluation GUI:

Recorded NOPs are presented as rows in this GUI. Columns indicate the NOP's date and time, the hopefully best screenshot for this time, type of the user action, as well as, further extracted information stored in the NOP that shall allow you to remember what you actually did then.

On the left side is a list of your context threads. Each has a differently colored background. If you miss some context thread here you can add a new one using the text field at the bottom of this left panel.

To assign one or more NOPs to a specific context thread you select one or more rows, thus, selecting one of more NOPs. Then double-click on some context-thread on the left.

You can also add some meaningless NOP to a blacklist by right-clicking on some selected NOP row(s) and then choosing "add to blacklist" on the context menu. NOPs with the same pattern are also removed from the table. Here, "NOPs with the same pattern" means NOPs, that have identical values in the columns disregarding the date/time and screenshot of course.
Please take care! Currently, there is no easy way to undo this "blacklisting" of NOPs!

The context menu also allows to permanently "delete" NOPs. This is only needed in extrem cases.

After having assigned all NOPs to context threads, you click on the "Save" button, which creates an edited userobs log file. This file is the ground truth that will be used to run evaluation calculations on.

Part 4: Run evaluations on ground truth

TODO