How to run Robotium Test and Analyze the Failures?

In my previous post, i explained on how to create Test Project. Now, i explain more clearly on Running the Test, Analyzing the failures and processing the Results.

I will Explain this process by taking the Example Project step by step as follows,

1. Import the test project
2. Fix the errors that see on importing the project.
3. Run the test on the Device/Emulator
4. Analyze in case of Failures
5. Export the test results

Import Test project

• Download test project from the link
• from your Eclipse Import test project as an archive file
  • File > Import >Existing Project into workspace > Select archive file > choose the downloaded zip file "ExampleTestProject_v3.6.zip", select both 'Notepad' and 'Notepad Test'
  • click on "Finish"

Fix Project Errors :

• If 'Fix Project Properties' errors is seen after importing the project,
  • Right click on your test project > Android Tools > Fix Project Properties

Run Test on Device/Emulator :

• after creating test project and writing test code, run the test
  • Right click on your Test project > Run as Android Junit Test

• On selecting Run as 'Android Junit Test', it will first compile the source project and install it on to the device then it will compile the Test project and install as a test application in the device/emulator.
  • Junit tab will open and shows the Status of test run
    • it will display the list of tests to run
    • progress of test run, if all test pass, status will appear in Green color, on failures it displays Red color.
    • it contains various other options to Stop test run, Rerun test, Show failures etc..

Analyze Failures

• If any tests failed, errors can be seen in the Failure Trace window seen at the bottom. Robotium will display the log in Failure trace.
• Double click on the error, takes you to a line in the code that is causing the failure. 
• If you want to debug, right click on the test and click debug
  • you need place some breakpoints in the code (similar to debugging java code in eclipse)

Re-Run the Test :

• if you want to run the test again, click on "Re-Run test" button in the Junit Tab.
• if you want to Rerun with failure test as first, click on "Re-Run Test -Failures First"

  Save the Logs :

• you can save the test report as an XML file
  • click on Test history icon in Junit test tab > Export and save the Log file
  • Log file contains test history and also Failure trace in case of Test failures.

Sample Pass XML Report :

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="0" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="42.967">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="42.967">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="12.73"/>

   6:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.084"/>

   7:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="21.152"/>

   8:     </testsuite>

   9:   </testsuite>

  10: </testrun>

Sample Failure XML Report:

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="1" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="46.887">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="46.886">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="15.986">

   6:         <failure>junit.framework.AssertionFailedError: Note 1 and/or Note 2 are not found expected:&lt;true&gt; but was:&lt;false&gt;&#13;

   7: at com.jayway.test.NotePadTest.testAddNote(NotePadTest.java:62)


   8: at java.lang.reflect.Method.invokeNative(Native Method)


   9: at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)


  10: at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)


  11: at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)


  12: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)


  13: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)


  14: at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)


  15: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)


  16: 


  17: </failure>

  18:       </testcase>

  19:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.946"/>

  20:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="20.954"/>

  21:     </testsuite>

  22:   </testsuite>

  23: </testrun>