So you can't install the Javax Mail Plug-in...

Instead, you can manually e-mail me the files at the end of the experiment.
  1. Download the plugin archive file: CONQEvalNoMailPlugin.zip on dropbox or locally (slower).
    1. In Eclipse, go to File > Import > General > Existing Projects Into Workspace
    2. Select archive file
    3. Browse to the zip file on your hard drive
    4. In the Project window, you should see a project called CONQ-Eval-NoMail

    5. After clicking "Finish", you should see the new CONQ-Eval project in your workspace.
    6. If you get an error after the build is complete, go to the Problems view, and "Quick Fix" the error to set API Baseline errors to warnings:

  2. Run the plug-in by right-clicking the CONQ-Eval-NoMail project and selecting Run As > Eclipse Application:

    This will open up a brand new runtime workspace. To verify your plug-in is installed correctly, go to Search > Search and verify you have all 3 search tabs: Conquer Search, Score Search, and Text Search with Logging.

  3. Troubleshooting:
    • Make sure there is an "out" folder in the CONQ-Eval project (latest downloads should have this)
    • The plug-in tends to run better using Java 1.7 and Eclipse Juno for RCP developers
    • If you get an OutOfMemoryError, try going to the Runtime Configuration for running the plug-in, go to the Arguments tab, and adding VM arguments for -XX:MaxPermSize=350m and -Xmx2048m:

  4. Now we need some source code in our new runtime workspace. Download the archive file of all the projects to be used in the experiment: CONQEval.zip on dropbox or locally (slower).
    1. In Eclipse, go to File > Import > General > Existing Projects Into Workspace
    2. Select archive file
    3. Browse to the zip file on your hard drive
    4. In the Project window, you should see 5 programs: iReport, jajuk, javaHMO, jBidWatcher, and Rhino. All 5 should be checked.
    5. After clicking "Finish", you should see all 5 projects in your workspace.
  5. Complete the rest of the experiment, beginning with the example below.
  6. After completing the experiment, the survey will prompt you to e-mail the search data. Please complete this step manually by exporting the "out" folder in the CONQ-Eval-NoMail project as an archive file, and e-mailing me the file as an attachment.

    The data includes the queries you entered and the results output by the search tool. It uses a generic e-mail address to send from and NO personally identifiable information is collected.

 

Example Search Task

To make sure the eclipse plug-in is working correctly, please try to use the "Textual Search with Logging" tool on the following example.

jBidWatcher_D

The example search task is for jBidWater, an auction bidding, sniping, and tracking tool for online auction sites such as eBay or Yahoo. For this search task, you are given a screen shot of the feature being executed. After looking at the image, try to create a query that returns the source code implementing this feature.

Based on the screen shot, first try the query "remove auction":

Which only gives 1 result. Next, try the query "delete auction" instead:

Which returns results that appear to be highly relevant. You would then enter the best query, "delete auction", into the Google survey.

Next Steps

Start the google survey, then go back to the e-mail to get the links for your first search tool and 4 search tasks.