How to run Robotium Test and Analyze the Failures?

In my previous post, i explained on how to create Test Project. Now, i explain more clearly on Running the Test, Analyzing the failures and processing the Results.

I will Explain this process by taking the Example Project step by step as follows,

1. Import the test project
2. Fix the errors that see on importing the project.
3. Run the test on the Device/Emulator
4. Analyze in case of Failures
5. Export the test results

Import Test project

• Download test project from the link
• from your Eclipse Import test project as an archive file
  • File > Import >Existing Project into workspace > Select archive file > choose the downloaded zip file "ExampleTestProject_v3.6.zip", select both 'Notepad' and 'Notepad Test'
  • click on "Finish"

Fix Project Errors :

• If 'Fix Project Properties' errors is seen after importing the project,
  • Right click on your test project > Android Tools > Fix Project Properties

Run Test on Device/Emulator :

• after creating test project and writing test code, run the test
  • Right click on your Test project > Run as Android Junit Test

• On selecting Run as 'Android Junit Test', it will first compile the source project and install it on to the device then it will compile the Test project and install as a test application in the device/emulator.
  • Junit tab will open and shows the Status of test run
    • it will display the list of tests to run
    • progress of test run, if all test pass, status will appear in Green color, on failures it displays Red color.
    • it contains various other options to Stop test run, Rerun test, Show failures etc..

Analyze Failures

• If any tests failed, errors can be seen in the Failure Trace window seen at the bottom. Robotium will display the log in Failure trace.
• Double click on the error, takes you to a line in the code that is causing the failure. 
• If you want to debug, right click on the test and click debug
  • you need place some breakpoints in the code (similar to debugging java code in eclipse)

Re-Run the Test :

• if you want to run the test again, click on "Re-Run test" button in the Junit Tab.
• if you want to Rerun with failure test as first, click on "Re-Run Test -Failures First"

  Save the Logs :

• you can save the test report as an XML file
  • click on Test history icon in Junit test tab > Export and save the Log file
  • Log file contains test history and also Failure trace in case of Test failures.

Sample Pass XML Report :

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="0" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="42.967">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="42.967">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="12.73"/>

   6:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.084"/>

   7:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="21.152"/>

   8:     </testsuite>

   9:   </testsuite>

  10: </testrun>

Sample Failure XML Report:

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="1" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="46.887">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="46.886">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="15.986">

   6:         <failure>junit.framework.AssertionFailedError: Note 1 and/or Note 2 are not found expected:&lt;true&gt; but was:&lt;false&gt;&#13;

   7: at com.jayway.test.NotePadTest.testAddNote(NotePadTest.java:62)


   8: at java.lang.reflect.Method.invokeNative(Native Method)


   9: at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)


  10: at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)


  11: at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)


  12: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)


  13: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)


  14: at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)


  15: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)


  16: 


  17: </failure>

  18:       </testcase>

  19:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.946"/>

  20:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="20.954"/>

  21:     </testsuite>

  22:   </testsuite>

  23: </testrun>

Getting Started with Robotium

Just before getting started, let us discuss few things about Robotium.

Robotium can be used to test applications where source code is available and applications where only APK file is available. 

In this example, let us discuss about the case when source code is available. I will discuss the other scenario in a separate post. let us start with setting up test environment first,

Requirements :

• Eclipse for building the Test project
• ADT (Android Development Kit)
• Android SDK (Software Development Kit)
• JDK (Java Development Kit)
• Latest Robotium Jar file

Prerequisites for Creating Test project :

1. Install JDK, Eclipse, SDK and ADT to your system
2. After installation, set proper path to Environment variables JAVA_HOME and ANDROID_HOME and add the SDK and JDK installation path to environment variable PATH.
3. Download the Latest Robotium jar file from http://code.google.com/p/robotium/downloads/list 

Create a Test project in Eclipse:

• Create the Test project by "File --> New --> Project --> Android --> Android Test Project", the window will open
• Enter the Project Name, click on 'Next'
• if project source code is already available/imported in eclipse, in the "Test Target" field, choose 'An existing Android project' and select from the list of Projects, click on 'Next'
• in the "Build Target", select the application development platform, suppose,  if application is developed on Android 2.2, choose API level as 8 and so on..
• click on 'Finish', a new test project will be created in the Eclipse.

Add the Robotium jar file to the Project

Right click on your project, --> Build Path --> Configure Build Path --> select "Libraries" tab
Then select "Add External Jars" -> select the Robotium Jar file downloaded
go to "Order and Export" tab, select the check box next to the Robotium jar file.

Create a Test case :

• Create New class file:
• select Project folder  --> "src" directory --> select package, Right click on it, select "New --> Class", enter the class name and click on 'Finish'.
• extend your class to ActivityInstrumentationTestCase2
• when eclipse shows error, click on import statement, which will import 'android.test.ActivityInstrumentationTestCase2'
• copy the following code to your project and do modifications when required

   1: public class <your class> extends ActivityInstrumentationTestCase2{  //no need to copy this line    

   2:     private Solo solo;

   3:     // replace the "SampleTest()" with your class name ex: MyTest()

   4:     // replace "MainActivity.class" with your application project's starting activity class name 

   5:      public SampleTest() {           

   6:          super(MainActivity.class);

   7:       }

   8:     //setUp() is run before a test case is started. This is where the solo object is created.     

   9:    @Override

  10:     public void setUp() throws Exception {

  11:         solo = new Solo(getInstrumentation(), getActivity());

  12:     }

  13:    //tearDown() is run after a test case has finished. 

  14:    //finishOpenedActivities() will finish all the activities that have been opened during the test      execution.

  15:    @Override

  16:    public void tearDown() throws Exception {

  17:       solo.finishOpenedActivities();

  18:    }

  19:   // your test case starts here... 

  20:   // Test case name starts with keyword "test"followed by your Test case name in Capital Letter

  21:   //here, "MyFirstTestcase" is the test case name, change this with your desired name . 

  22:   public void testMyFirstTestcase() throws Exception {

  23:              // write your test code

  24:   } 

  25: } //no need to copy this line

Now you can write your Test case in the test block

Next : How to Run the Test? -- see the Post