Defect Severity and Defect Priority

Source: http://geekswithblogs.net/ppothuraju/archive/2005/01/22/20664.aspx

The severity framework for assigning defect criticality that has proven most useful in actual testing practice is a five level scale. The criticality associated with each level is based on the answers to several questions.

First, it must be determined if the defect resulted in a system failure. ANSI/IEEE Std 729-1983 defines a failure as,

“The termination of the ability of a functional unit to perform its required function.”

Second, the probability of failure recovery must be determined. ANSI/IEEE 729-1983 defines failure recovery as,

“The return of a system to a reliable operating state after failure.”

Third, it must be determined if the system can do this on its own or if remedial measures must be implemented in order to return the system to reliable operation.

Fourth, it must be determined if the system can operate reliably with the defect present if it is not manifested as a failure.

Fifth, it must be determined if the defect should or should not be repaired.

The following five level scale of defect criticality addresses the these questions.

 

The five Levels are:

1. Critical

2. Major

3. Average

4. Minor

5. Exception

1. Critical – The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system.

 

2. Major – The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system. There is no way to make the failed component(s), however, there are acceptable processing alternatives which will yield the desired result.

 

3. Average – The defect does not result in a failure, but causes the system to produce incorrect, incomplete, or inconsistent results, or the defect impairs the systems usability.

 

4. Minor – The defect does not cause a failure, does not impair usability, and the desired processing results are easily obtained by working around the defect.

 

5. Exception – The defect is the result of non-conformance to a standard, is related to the aesthetics of the system, or is a request for an enhancement. Defects at this level may be deferred or even ignored.

In addition to the defect severity level defined above, defect priority level can be used with severity categories to determine the immediacy of repair. A five repair priority scale has also be used in common testing practice. The levels are:

1. Resolve Immediately

2. Give High Attention

3. Normal Queue

4. Low Priority

5. Defer

1. Resolve Immediately – Further development and/or testing cannot occur until the defect has been repaired. The system cannot be used until the repair has been effected.

 

2. Give High Attention – The defect must be resolved as soon as possible because it is impairing development/and or testing activities. System use will be severely affected until the defect is fixed.

 

3. Normal Queue – The defect should be resolved in the normal course of development activities. It can wait until a new build or version is created.

 

4. Low Priority – The defect is an irritant which should be repaired but which can be repaired after more serious defect have been fixed.

 

5. Defer – The defect repair can be put of indefinitely. It can be resolved in a future major system revision or not resolved at all.

Advertisements
Posted in Uncategorized | Leave a comment

Approach while Testing Mobile App

Source: http://www.mobileappstesting.com/mobile-testing-approch/

While doing mobile testing, it is important to have a look on the approach with which you proceed forward to test mobile applications.

Requirements Analysis considering Mobile Factors:-

Like any application testing, it is important to go through the requirements in deep and rectify the requirement bugs at the right time. If you are not getting requirements properly, just communicate with your client or lead about which information you will be requiring in order to work on Test Scenarios and Test Cases around it. If there are many requirements which are decided on the go, like in the client calls or in the meetings ,then note it down and star working on the various scenarios around them. Make sure to analyze the requirements keeping mobile factors in consideration as well.For e.g. if the requirements is ” mobile app download any media” then you should make sure to ask:-

  • What kinds of media files are going to be supported
  • If download is going to be on SD Card or internal memory
  • What if SD card will not be available
  • What if user changes the SD Card

Mostly the requirements are on higher level and may not answers all the scenarios that the application may have later on.

UI Mock Ups:-

If required ask your client to provide the wire frames or UI mock ups which will help you understand how the application screens will look like.Wire frames can also help you to go through the flow of the application.

Study the Architecture and Technology:-

By the time developers are busy in developing the architecture or going through requirement analysis,it is very important that a tester should also look in to the technology perspective of the application. For example if your client is developing any streaming or media sharing applications,then apart from regular requirement analysis a QA should make sure what kind of streaming technology is going to be used. What are the different API will be used in the applications.As the application is related to media then what are the different kind of media file types this application will support etc.

Competitors application Analysis:-

It is one of the key trick which will really help you a lot. Most of the time when you develop any application, then we have some similar applications already available in the market. Just download these app and verify various features and functionality in this application. For example some twitter based application.You can take a look on Seesmic,tvider,twitpic.These all are media sharing app on twitter and many Features resembles as finally they are developed based on twitter APIS. By following this practice you can:-

  • Suggest some new features and improvements in the product your org is going to develop.
  • Get some hands-on on the kind of technology and features being used
  • Help you being familiar with the app
  • Help you to chalk out some important scenarios

Keep in Mind The Target devices:-

This will be decided by your client or by your manager that which device they will be targeting the app on. It is very important if you can provide your inputs on that as well. For a testing purpose, make sure you have devices of different screen sizes and resolution. Also some device may support orientation and some may not. So this should also be taken in consideration. We will discuss later on about how to choose a right set of devices. The point here what I am making is we should be in condition to provide our inputs in this front as well.

Design Scenarios and Test Cases:-

Once Test Planning is done, it is very important to design some test scenarios and Test Cases. You can design a complete Test Suite which will have various modules feature wise.You may add some tabs dedicated to Performance ,system and interruption test cases as well.Make sure you involve these kind of generic test cases as well. Apart from bulky test suite,you should have a quick checklist which will cover important features in the application. So in case you need a quick round of testing then you can follow this checklist.

Analysis on Need of Tools and Utilities:-

Make sure to put some efforts on the kind of told you can use later which will help you test the mobile app.I am not only talking about the Automation  Tools but also some small utilities like log capturing tools,screenshot capturing tools,memory consumption by app checking tools etc. Early planning is very important as you have sufficient time available before actual testing Starts.

Keep performance in Mind:-

Performance is very important aspect for mobile applications. So while testing your application,apart from regular functional testing scenarios, analyze the application for app size, behavior of the application in long run i.e. behavior of app if app being used excessively for long time, application launch time and time to load any screen, behavior of app when app do not have enough space, simultaneously running many apps  should be considered.

Application Submission Related Strategy:-

Be sure about the distribution channel for your mobile application.If you are submitting your app to app store then go through the guidelines provided by apple to make sure that you application is getting accepted .Similarly depending on the application store,go through the application and verify that whether your app satisfies the criteria for that store.

Testing Among The Beta Crowd:-

Every one is having their own approach of using the application.Yes the QA will not decide whether to go for beta testing, but the product team can just decide on this.They can submit the app to beta testing services and can see the inputs from the crowd. Also there is another cost effective solution for this as well.The app can be distributed among the trusted employees and after the Use of 1-2 weeks you can see inputs from them. I feel it as very important aspects as everyone is having there own way of using the app and having multiple people reviewing the app can actually help as some bugs can be found in this phase as well. Many times testers test the application in order to complete the testing within time as they have challenge to complete the testing within time. In this case sometimes they miss some scenarios that the normal user can encounter. For example i you are working on you app where user will watch the movie. If movie is of 1-2 hours then will the tester actually see the complete movie ? Mostly no …he will just fast forward it and will verify the rest of the scenarios. But the actual user will be interested more in watching movies and hence if he finds disturbance in this process then he will just throw this application away. Also the testers keeps on changing the build as he keeps on getting new releases from developer however such kind of crowd uses the same build for several days.The probability to see long term impacts of using the application can be dig out more in this kind of testing which is normally not possible.

Posted in Uncategorized | Leave a comment

Mobile Application Testing – A Quality Approach

Source: http://www.xoriant.com/blog/software-testing-and-qa/mobile-application-testing-a-quality-approach.html

The ever increasing demand of mobile devices has given a push to software developers in taking the traditional web applications to mobile environment. The challenge is to provide user experience as similar and seamless across various mobile devices as possible in spite of the limitations which the mobile environment poses, adopting an agile methodology to develop the mobile applications for a diversified device environment, hardware and networking considerations.

Mobile device markets that includes Smartphones, Tablets, PDAs etc. is growing dynamically making the mobile application developers strive to deliver most robust, scalable applications with quality assurance Every device platform creates a unique testing environment challenging the mobile application developers to follow different testing strategies. Here we shall see how different types of testing approaches can be taken up for a variety of mobile platforms.

Numerous different mobile platforms available for mobile applications to name a few:

  • Apple’s iOS
  • Google’s Android
  • Nokia’s Symbian, Maemo and MeeGo
  • Palm/HP’s WebOS
  • Samsung’s Bada
  • RIM’s BlackBerry OS., and many more.

As mentioned, every platform needs a different testing approach. A combination of manual and automation testing can be done for an effective outcome.

Following are different types of manual testing for mobile environment:

  • Functionality Testing:

Functional testing mainly includes finding device specific bugs and navigational issues of application. This type of testing should be done at the initial stage when the application is under development. In functional testing we can check database or network queries with response times, crashes and memory issues. Functionality testing of a mobile application is a black-box type of testing to assure that the application is functioning as per the business specifications.

  • Usability Testing:

Usability testing encompasses mobile interface testing, application navigation testing, and intuitiveness of the application, consistency, and soberness of color scheme. An ISO standard defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”.

In Usability testing ease and efficiency of the user and content verification will be tested as defined in common Usability guidelines.

  • Network Connection Testing:

Every mobile application which utilizes internet needs to be tested under different networks. These are a few networks for testing applications:

2G (GPRS, CDMA, EDGE), 3G and WiFi.

This is a necessity because most of the times the server responses are different for different type of networks. This testing helps us check application behavior under different networks.

  • Installation Testing:

While installing, the application should install required softwares automatically on the device. And while uninstalling the application, it should remove all the available content and databases from the device which are used by application.

  • Performance Testing:

Performance testing is required for finding memory related issues in application when data is excessive, due to devices having less memory as compared to simulator, this testing is required to be done on device only. By this testing approach, we can find the stability and performance of application under excessive data.

  • Stress Testing:

This testing is required for getting the application response time while clicking on different buttons regressively and adding more data on device. We can get different crashes and hangings of application while running the application for long time.

In absence of mobile devices, simulators have always played a vital role for testing mobile device applications, although device testing is always the ideal way as it represents more likely end user scenarios, the significance of simulators in testing cannot be ignored. To have an effective testing with the help of a simulator, it’s necessary to explore all the capabilities of simulator.

Simulator versus device testing:

Simulators are mainly used for functional testing of basic flows. Simulators are not used for performance and usability testing, but the final testing is conducted on actual devices so crashes and hangings can be identified. One cannot get quality application while relying only on simulator. Device testing is necessary for all the applications, as with device testing we can understand the application behavior on different networks.

Simulator:

Is a software application that can accurately imitate a mobile phone. Simulator is mainly used by developers to check the functionalities implemented under development phase.

Let’s discuss the pros and cons of using simulators:

  • Pros:

– Helps in isolating issues which are not volatile network connection dependent
– Provides a wide variety of testing over different types of device simulators for the same build
– Allows to test the same build in multiple device screens.

  • Cons:

– Simulators of older generation handsets don’t resemble the device as closely
– Some issues which are hit by the speed at which input was given cannot be reproduced easily
– Hardware/Firmware environment variations detectable in device testing only
– Device testing is always preferred as it represents more likely end user scenarios.

Device: Is the actual handset where application installed and runs.

There are some pros and cons while using real devices for testing as well.

  • Pros:

– Finds actual issues of application.
– Finds crashes, memory leak issues which can not found on simulators.
– Checks application over 2G and 3G and different networks
– Checks application behavior while incoming call, SMS, MMS and alarm.

  • Cons:

– Expensive for compatibility testing of application over wide range of devices
– Consumes more time for adding excessive test data for testing purpose.

Checklist to follow while testing a mobile application:

 

Following is a basic checklist which is required while testing mobile application for any platform:

1. Installation & Uninstallation: To verify whether the application can be installed & uninstalled successfully.

2. Network Connectivity:

  • The application can use simultaneous connections properly
  • The application follows the GSM Offline profile correctly when making connections.
  • When GSM Offline profile is selected, application cannot take network connection or send an SMS/MMS
  • The application can utilize WLAN, 2G and 3G networks correctly.
  • Performance of application during network connectivity problem.

3. Call/SMS/alarm handling: Verify that Application pauses and resumes for the same state when there is an incoming phone call/SMS/MMS/Alarm notification.

4. Check the look & feel of the application

5. Content: Check if enough information is displayed

6. The application must function as defined in the Help, user’s guide, or functional specification

7. Performance : Application and inner pages load time.

Automation Testing:

 

Automation testing is usually extended to perform tasks impossible with manual testing in large applications. An automated software testing tool is capable of playing pre-recorded and predefined actions. The results are mapped to the expected behavior and report the success or failure of these manual tests. Once automated tests are created they can easily be repeated. Having experienced the efficiency of automated software testing, it has become an important part of an application testing.

 

Automation tools available for testing:

Mainly mobile testing is done manually on actual devices. Some of the following tools are available in to test the functionality as well as usability of application.

  1. Robotium for Android
  2. Testquest, try, and digia for Symbian
  3. FoneMonkey for IPhones
  4. Memory sweep for IPhone
  5. Other tools: eggplant, VNC Robot, Hopper and TestQuest.

Advantages and Limitations of Automation Tools:

 

Advantages:
– Multiple tests can run in less time with fewer resources
– Automated Tools run tests faster than human users
– Can reuse tests on different versions of an application.

Limitations:
– Unable to test all the functionalities and Usability of application through automation tools
– Proficiency is required to write the automation test scripts for application different functionalities
– Debugging the test script if any error is present in the automation test script
– Storing and maintenance of test data is difficult.

 

I believe that this comprehensive information will be quite helpful to you if you are looking at some methods for an effective approach to test a mobile device application. You can leave your comments and/or questions; I would be looking forward to having an interactive conversation.

Posted in Uncategorized | Leave a comment

Mobile Apps Interview

  1. What is the difference between Mobile Testing and Mobile Application Testing ?
  2. What is your approach while Testing Mobile Applications?
  3. Have you ever written a Test Plan?What are the things specific to Mobile Application would you emphasis on while writing test plan for Mobile Applications?
  4. Do you know Facebook?Tell me what are the High level test cases for Facebook Web Application and for Facebook Mobile Application?
  5. Can you please let me know,the devices you have worked upon?
  6. Testing of Mobile Application on Emulators.Can you let me know your view?
  7. Have you ever worked on any automation tool for Testing Mobile Application?
  8. Please tell me about your project.What kind of Mobile Applications have you worked upon?
  9. Do you have Idea about Mobile Operating Systems?
  10. Blackberry Devices have which Operating system?
  11. What is current iOS (iphone OS) version?
  12. You have two cases. 1st you can not disconnect your call and 2nd you can not send SMS from your devices.Tell me Severity and Priority in both the cases?
  13. What are different Mobile Platforms/OS?
  14. What are the different way you can install a Mobile Application?
  15. Have you ever worked on Device Anywhere?Do you have experience of working on it?
  16. Do you have Idea about application certification program like True Brew Testing(TBT),Symbian Signed Test Criteria,Java Verified Program?
  17. See this application(Interviewer is given a Handset with a Mobile Application installed).Tell me what are the bugs in this Mobile application/Game.?
  18. Have you ever worked on LBS Application ?
  19. How will you test a Location Based Mobile Application?
  20. How will you perform Performance Testing for a Mobile Application?
Posted in Uncategorized | Leave a comment

Hello World

Hello world!!!!

Posted in Uncategorized | Leave a comment

Hello world!

Welcome to WordPress.com! This is your very first post. Click the Edit link to modify or delete it, or start a new post. If you like, use this post to tell readers why you started this blog and what you plan to do with it.

Happy blogging!

Posted in Uncategorized | 1 Comment