How to run Robotium Test and Analyze the Failures?

In my previous post, i explained on how to create Test Project. Now, i explain more clearly on Running the Test, Analyzing the failures and processing the Results.

I will Explain this process by taking the Example Project step by step as follows,

1. Import the test project
2. Fix the errors that see on importing the project.
3. Run the test on the Device/Emulator
4. Analyze in case of Failures
5. Export the test results

Import Test project

• Download test project from the link
• from your Eclipse Import test project as an archive file
  • File > Import >Existing Project into workspace > Select archive file > choose the downloaded zip file "ExampleTestProject_v3.6.zip", select both 'Notepad' and 'Notepad Test'
  • click on "Finish"

Fix Project Errors :

• If 'Fix Project Properties' errors is seen after importing the project,
  • Right click on your test project > Android Tools > Fix Project Properties

Run Test on Device/Emulator :

• after creating test project and writing test code, run the test
  • Right click on your Test project > Run as Android Junit Test

• On selecting Run as 'Android Junit Test', it will first compile the source project and install it on to the device then it will compile the Test project and install as a test application in the device/emulator.
  • Junit tab will open and shows the Status of test run
    • it will display the list of tests to run
    • progress of test run, if all test pass, status will appear in Green color, on failures it displays Red color.
    • it contains various other options to Stop test run, Rerun test, Show failures etc..

Analyze Failures

• If any tests failed, errors can be seen in the Failure Trace window seen at the bottom. Robotium will display the log in Failure trace.
• Double click on the error, takes you to a line in the code that is causing the failure. 
• If you want to debug, right click on the test and click debug
  • you need place some breakpoints in the code (similar to debugging java code in eclipse)

Re-Run the Test :

• if you want to run the test again, click on "Re-Run test" button in the Junit Tab.
• if you want to Rerun with failure test as first, click on "Re-Run Test -Failures First"

  Save the Logs :

• you can save the test report as an XML file
  • click on Test history icon in Junit test tab > Export and save the Log file
  • Log file contains test history and also Failure trace in case of Test failures.

Sample Pass XML Report :

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="0" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="42.967">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="42.967">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="12.73"/>

   6:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.084"/>

   7:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="21.152"/>

   8:     </testsuite>

   9:   </testsuite>

  10: </testrun>

Sample Failure XML Report:

   1: <?xml version="1.0" encoding="UTF-8"?>

   2: <testrun name="NotePadTest" project="NotePadTest" tests="3" started="3" failures="1" errors="0" ignored="0">

   3:   <testsuite name="AVD23 [emulator-5554]" time="46.887">

   4:     <testsuite name="com.jayway.test.NotePadTest" time="46.886">

   5:       <testcase name="testAddNote " classname="com.jayway.test.NotePadTest" time="15.986">

   6:         <failure>junit.framework.AssertionFailedError: Note 1 and/or Note 2 are not found expected:&lt;true&gt; but was:&lt;false&gt;&#13;

   7: at com.jayway.test.NotePadTest.testAddNote(NotePadTest.java:62)


   8: at java.lang.reflect.Method.invokeNative(Native Method)


   9: at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)


  10: at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)


  11: at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)


  12: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)


  13: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)


  14: at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)


  15: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)


  16: 


  17: </failure>

  18:       </testcase>

  19:       <testcase name="testEditNote " classname="com.jayway.test.NotePadTest" time="9.946"/>

  20:       <testcase name="testRemoveNote " classname="com.jayway.test.NotePadTest" time="20.954"/>

  21:     </testsuite>

  22:   </testsuite>

  23: </testrun>

How to test Android Long click menus using Robotium?

in Android, Long click menus can be found in List items, texts, views, screen.  

Robotium supports various functions to handle Long click menus,

• clickLongInList(int line)
• clickLongOnText(String text)
• clickLongOnView(android.view.View view)
• clickLongOnScreen(float x, float y)

Let us see usage of these APIs with an example test case and test code for the project shown on the right side.

project has a list of items and each support a long press functionality. on long press, will open a menu with various options like "Open, Delete, Edit title"

Let us write a test case for this project,

Test case for Android Long click menus :

Steps:

1. Open notepad application
2. press the menu item 'Add note' and add note 'Note 1'
3. long click on the 'Note 1' in the list, which will open a menu, select 'Delete' option.
4. verify the note is delete or not?

Test Code :

public void testRemoveNote() throws Exception {

        solo.clickOnMenuItem("Add note");

        //Assert that NoteEditor activity is opened

        solo.assertCurrentActivity("Expected NoteEditor activity", "NoteEditor"); 

        //In text field 0, add Note 1

        solo.enterText(0, "Note 1");

        solo.clickOnMenuItem("Save");

       solo.clickLongOnText("Note 1");

        //Clicks on Delete in the context menu

        solo.clickOnText("Delete");          

        //Note 1 test should not be found

        boolean expected = false;   

        boolean actual = solo.searchText("Note 1");

        //Assert that Note 1 text is not found

        assertEquals("Note 1 Text is found", expected, actual);  

    }

Explanation:

1. Add Note : use 'solo.clickOnMenuItem()' to press on the menu item 'Add Note'. 
2. Long click : 'solo.clickLongOnText()' is used to perform long click operation on the specified text.
   
   
   
   • use 'solo.clickOnText()', to select the text from long click menu.

test Android Menu items using Robotium

Robotium supports various functions to test Android menu items. in Android applications Menu items appear on the pressing the menu button after launching the application.

• pressMenuItem(int index)
• pressMenuItem(int index, int itemsPerRow)
• clickOnMenuItem(String text)
• clickOnMenuItem(String text, boolean subMenu)

Right side shown is the sample Notepad Applications, which contains menu item called "Add note".

Let us write sample code to handle the menu items.

Test case for Android Menu items :

Steps:

1. Launch application, verify the current activity
2. open Menu item 'Add note'
3. add new note
4. go back to previous activity
5. take screenshot

Test Code:

public void testAddNote() throws Exception {

        solo.clickOnMenuItem("Add note");

        //Assert that NoteEditor activity is opened

        solo.assertCurrentActivity("Expected NoteEditor activity", "NoteEditor"); 

        //In text field 0, add Note 1

        solo.enterText(0, "Note 1");

        solo.clickOnMenuItem("Save");     

        //Clicks on menu item

        solo.clickOnMenuItem("Add note");

        //In text field 0, add Note 2

        solo.enterText(0, "Note 2");

        solo.clickOnMenuItem("Save");

        //Go back to first activity named "NotesList"

        solo.goBackToActivity("NotesList"); 

        //Takes a screenshot and saves it in "/sdcard/Robotium-Screenshots/".

        solo.takeScreenshot();

        boolean expected = true;

        boolean actual = solo.searchText("Note 1") && solo.searchText("Note 2");

        //Assert that Note 1 & Note 2 are found

        assertEquals("Note 1 and/or Note 2 are not found", expected, actual); 

}

Explanation:

1. Verify the current activity : ‘solo.assertCurrentActivity()’ is used to verify the current opened activity, which returns a boolean value based on the comparison.


2. Open Menu item : ‘solo.clickOnMenuItem()’ is used to Open the Manu and click on the menu item with a given name.


3. ‘solo.goBackToActivity()’ is used to go back to the activity with a given name.


4. take screenshot : ‘solo.takescreenshot()’ will capture the android screen and save to SD card under “/sdcard/Robotium-Screenshots/”. Function supports two arguments


• with “no arguments” – this will save the screenshot with current date/time
• with “name” as an argument – this will save the screenshot with the specified name (String)