iPhone App Test Cases

No.	Module	Sub-Module	Test Case Description	Expected Result
1	Installation		Verify that application can be Installed Successfully.	Application should be able to install successfully.
2	Uninstall		Verify that application can be uninstalled successfully.	User should be able to uninstall the application successfully.
3(i)	Network Test Cases		Verify the behavior of application when there is Network problem and user is performing operations for data call.	User should get proper error message like “Network error. Please try after some time”
3(ii)			Verify that user is able to establish data call when Network is back in action.	User should be able to establish data call when Network is back in action.
4(i)	Voice Call Handling	Call Accept	Verify that user can accept Voice call at the time when application is running and can resume back in application from the same point.	User should be able to accept Voice call at the time when application is running and can resume back in application from the same point.
4(ii)		Call Rejection	Verify that user can reject the Voice call at the time when application is running and can resume back in application from the same point.	User should be able to reject the Voice call at the time when application is running and can resume back in application from the same point.
4(iii)		Call Establish	Verify that user can establish a Voice call in case when application data call is running in background.	User should be able to establish a Voice call in case when application data call is running in background.
5(i)	SMS Handling		Verify that user can get SMS alert when application is running.	User should be able to get SMS alert when application is running.
5(ii)			Verify that user can resume back from the same point after reading the SMS.	User should be able to resume back from the same point after reading the SMS.
6	Unmapped keys		Verify that unmapped keys are not working on any screen of application.	Unmapped keys should not work on any screen of application.
7	Application Logo		Verify that application logo with Application Name is present in application manager and user can select it.	Application logo with Application name should be present in application manager and user can select it.
8(i)	Splash		Verify that when user selects application logo in application manager splash is displayed.	When user selects application logo in application manager splash should be displayed.
8(ii)			Note that Splash do not remain for fore than 3 seconds.	Splash should not remain for fore than 3 seconds.
9	Low Memory		Verify that application displays proper error message when device memory is low and exits gracefully from the situation.	Application should display proper error message when device memory is low and exits gracefully from the situation.
10	Clear Key		Verify that clear key should navigate the user to previous screen.	Clear key should navigate the user to previous screen.
11	End Key		Verify that End Key should navigate the user to native OEM screen.	End Key should navigate the user to native OEM screen.
12	Visual Feedback		Verify that there is visual feedback when response to any action takes more than 3 seconds.	There should be visual feedback given when response time for any action is more than 3 second.
13	Continual Keypad Entry		Verify that continual key pad entry do not cause any problem.	Continual key pad entry should not cause any problem in application.
14	Exit Application		Verify that user is able to exit from application with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point.	User should be able to exit with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point.
15	Charger Effect		Verify that when application is running then inserting and removing charger do not cause any problem and proper message is displayed when charger is inserted in device.	When application is running then inserting and removing charger should not cause any problem and proper message should be displayed when charger is inserted in device.
16	Low Battery		Verify that when application is running and battery is low then proper message is displayed to the user.	When application is running and battery is low then proper message is displayed to the user telling user that battery is low.
17	Removal of Battery		Verify that removal of battery at the time of application data call is going on do not cause interruption and data call is completed after battery is inserted back in the device.	Removal of battery at the time of application data call is going on should not cause interruption and data call should be completed after battery is inserted back in the device.
18	Battery Consumption		Verify that application does not consume battery excessively.	The application should not consume battery excessively.
19	Application Start/ Restart		1. Find the application icon and select it 2. “Press a button” on the device to launch the app. 3.Observe the application launch In the timeline defined	Application must not take more than 25s to start.
20	Application Side Effects		Make sure that your application is not causing other applications of device to hamper.	Installed application should not cause other applications of device to hamper.
21	External incoming communication – infrared		Application should gracefully handle the condition when incoming communication is made via Infra Red [Send a file using Infrared (if applicable) to the device application presents the user]	When the incoming communication enters the device the application must at least respect one of the following: a) Go into pause state, after the user exits the communication, the application presents the user with a continue option or is continued automatically from the point it was suspended at b) Give a visual or audible notification The application must not crash or hung.

Software Testing @ glance

Steps to perform software testing: 

• Understand requirements and business logic
• Determine required standards and processes
• Set priorities, and determine scope and limitations of tests
• Determine test approaches and methods
• Determine test environment, test ware, test input data requirements
• Set milestones and prepare test plan document
• Write test cases
• Have needed reviews/inspections/approvals of test cases
• Set up test environment
• Execute test cases
• Evaluate and report results
• Bug Tracking and fixing
• Retesting or regression testing if needed
• Update test plans, test cases, test results, tractability matrix etc.

 

Components of a Bug Report: 

• Application name
• The function, module, name
• Bug ID
• Bug reporting date
• Status
• Test case ID
• Bug description
• Steps needed to reproduce the bug
• Names and/or descriptions of file/data/messages/etc. used in test
• Snapshot that would be helpful in finding the cause of the problem
• Severity estimate
• Was the bug reproducible?
• Name of tester
• Description of problem cause (filled by developers)
• Description of fix (filled by developers)
• Code section/file/module/class/method that was fixed (filled by developers)
• Date of fix (filled by developers)
• Date of retest or regression testing
• Any remarks or comments

Web based applications testing guidelines:

Apart from functionality consider the following:

• What are the expected loads on the server and what kind of performance is expected on the client side?
• Who is the target audience?
• Will down time for server and content maintenance / upgrades be allowed?
• What kinds of security will be required and what is it expected to do?
• How reliable are the site's Internet / intranet connections required to be?
• How do the internet / intranet affect backup system or redundant connection requirements and testing?
• What variations will be allowed for targeted browsers?
• Will there be any standards or requirements for page appearance and / or graphics throughout a site or parts of a site?
• How will internal and external links be validated and updated?
• How are browser caching and variations in browser option settings?
• How are flash, applets, java scripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?

From the usability point of view consider the following:

• Pages should be 3-5 screens longer.
• The page layouts and design elements should be consistent throughout the application / web site.
• Pages should be as browser-independent or generate based on the browser-type.
• There should be no dead-end pages. A link to a contact person or organization should be included on each page.

Security Testing

If you are thinking about Security Testing, here we go:

Security testing:

Security testing is a process to determine that an information system protects data and maintains functionality as intended. The security testing is performed to check whether there is any information leakage in the sense by encrypting the application or using wide range of software’s and hardware's and firewall etc.

The six basic security concepts that need to be covered by security testing are: confidentiality, integrity, authentication, availability, authorization and non-repudiation. Security testing as a term has a number of different meanings and can be completed in a number of different ways.

• Authentication - Testing the authentication schema means understanding how the authentication process works and using that information to circumvent the authentication mechanism. Basically, it allows a receiver to have confidence that information it receives originated from a specific known source.

• Authorization - Determining that a requester is allowed to receive a service or perform an operation.

• Availability- Assuring information and communications services will be ready for use when expected. Information must be kept available to authorized persons when they need it.

• Confidentiality - A security measure which protects the disclosure of data or information to parties other than the intended.

• Integrity – Whether the intended receiver receives the information or data which is not altered in transmission.

• Non-repudiation - Interchange of authentication information with some form of provable time stamp e.g. with session id etc.

Types:

There are two types of security testing that can be performed on Web applications: static analysis and dynamic analysis. In addition, there are two ways of performing security tests: automated and manual.

Dynamic analysis involves performing tests on a running instance of an application and is also known as black box testing. The security test will involve sending requests to the application and observing the responses to see if there was any indication that security vulnerability may be present. Dynamic analysis can be an effective way to test applications, but it is important to understand some limitations. First of all, because the testing is based on analyzing request and response patterns, the results obtained are really only a guess about the internal state of the application -- the tester typically has no knowledge of the actual application source code and what the actual internal state of the application is. In addition, because the tester is only looking at the observable behavior of the application and cannot know the entire attack surface, there is a chance that areas of the application and components of its functionality will be excluded from the test.

Dynamic analysis can either be performed in an automated manner or manually. Web application scanning tools like those from Watch fire and SPI Dynamics are good example of automated dynamic analysis tools. Automated tools are good for finding much common vulnerability such as SQL injection and cross-site scripting (XSS).

Manual testing of Web applications is typically performed using a Web browser and a Web proxy tool like Paros or OWASP's Web Scarab. The commercial scanning tools also typically come with proxies as well so that analysts using their scanners can augment the scanner results with manual tests. Proxies allow the security analyst to create and send arbitrary requests to the application and inspect the results to look for evidence of security issues. As mentioned above, these manual tests to look for data leakage, failures to authorize activities and so on are required for a credible application security assessment.

Where dynamic analysis is performed against an actually running installation of an application, static analysis involves reviewing application assets like source code, configuration files and so on when they are static -- or at rest. This is also known as source code analysis or white box testing. Static analysis opens up opportunities for a more thorough analysis because the analysis being performed has access to the "ground truth" of the source code. Analysts do not have to observe the behavior of an application and make guesses about the internal state of the system; instead the analyst has access to the actual instructions the software will follow when put into production.

Security testing approach:

In order to perform a useful security test of a web application, the security tester should have good knowledge of the HTTP protocol. It is important to have an understanding of how the client (browser) and the server communicate using HTTP. Additionally, the tester should at least know the basics of SQL injection and XSS.

1. Password cracking:

The security testing on a web application can be kicked off by “password cracking”. In order to log in to the private areas of the application, one can either guess a username/ password or use some password cracker tool for the same. Lists of common usernames and passwords are available along with open source password crackers. If the web application does not enforce a complex password (e.g. with alphabets, number and special characters, with at least a required number of characters), it may not take very long to crack the username and password.

If username or password is stored in cookies without encrypting, attacker can use different methods to steal the cookies and then information stored in the cookies like username and password

2. URL manipulation through HTTP GET methods:

The tester should check if the application passes important information in the query string. This happens when the application uses the HTTP GET method to pass information between the client and the server. The information is passed in parameters in the query string. The tester can modify a parameter value in the query string to check if the server accepts it.

Via HTTP GET request user information is passed to server for authentication or fetching data. Attacker can manipulate every input variable passed from this GET request to server in order to get the required information or to corrupt the data. In such conditions any unusual behavior by application or web server is the doorway for the attacker to get into the application.

3. SQL Injection:

The next thing that should be checked is SQL injection. Entering a single quote (‘) in any textbox should be rejected by the application. Instead, if the tester encounters a database error, it means that the user input is inserted in some query which is then executed by the application. In such a case, the application is vulnerable to SQL injection.

SQL injection attacks are very critical as attacker can get vital information from server database. To check SQL injection entry points into your web application, find out code from your code base where direct MySQL queries are executed on database by accepting some user inputs.

If user input data is crafted in SQL queries to query the database, attacker can inject SQL statements or part of SQL statements as user inputs to extract vital information from database. Even if attacker is successful to crash the application, from the SQL query error shown on browser, attacker can get the information they are looking for. Special characters from user inputs should be handled/ escaped properly in such cases.

4. Cross Site Scripting (XSS):

The tester should additionally check the web application for XSS (Cross site scripting). Any HTML e.g. <HTML> or any script e.g. <SCRIPT> should not be accepted by the application. If it is, the application can be prone to an attack by Cross Site Scripting.

Attacker can use this method to execute malicious script or URL on victim’s browser. Using cross-site scripting, attacker can use scripts like JavaScript to steal user cookies and information stored in the cookies.

Many web applications get some user information and pass this information in some variables from different pages.

E.g.: http://www.domain.com/index.php?userid=123&query=xyz

Attacker can easily pass some malicious input or <script> as a ‘&query’ parameter which can explore important user/server data on browser.

Important: During security testing, the tester should be very careful not to modify any of the following:

• Configuration of the application or the server

• Services running on the server

• Existing user or customer data hosted by the application

Additionally, a security test should be avoided on a production system.

The purpose of the security test is to discover the vulnerabilities of the web application so that the developers can then remove these vulnerabilities from the application and make the web application and data safe from unauthorized actions.

When do we use Security Testing?

Security testing is carried out when some important information and assets managed by the software application are of significant importance to the organization. Failures in the software security system can be serious especially when not detected, thereby resulting in a loss or compromise of information without the knowledge of that loss.

The security testing should be performed both prior to the system going into the operation and after the system is put into operation.

Rigorous security testing activities are performed to demonstrate that the system meets the specified security requirements & identify the left out security vulnerabilities, if any.

The extent of testing largely depends upon the security risks, and the test engineers assigned to conduct the security testing are selected according to the estimated sophistication that might be used to penetrate the security.

What are the objectives of Security Testing?

Security defects do not come to surface that easily as other types of defects. Thus security testing is carried out to identify defects that are quite difficult to identify. The security testing is carried out to ensure that the software under test is sufficiently robust and functions in an acceptable manner even in the event of a malicious attack.

The objectives of security testing can be:

1) To ensure that adequate attention is provided to identify the security risks

2) To ensure that a realistic mechanism to define & enforce access to the system is in place

3) To ensure that sufficient expertise exists to perform adequate security testing

4) To conduct reasonable tests to confirm the proper functioning of the implemented security measures

Who should do the Security Testing?

Majority of the security testing techniques are manual, requiring an individual to initiate and conduct the test. Automation tools can be helpful in executing simple tasks, whereas complicated tasks continue to depend largely on the intelligentsia of the test engineer.

Irrespective of the type of testing, the testing engineers that plan and conduct security testing should have significant security and networking related knowledge, including expertise of following areas:

1) Network security

2) Firewalls

3) Intrusion detection system

4) Operating systems

5) Programming and networking protocols like TCP/IP

Security Testing versus Conventional Software Testing

– A Quick Comparison:

Security testing has following attributes:

• It emphasizes what an application should not do rather than what it should do.
• It sometimes tests conformance to positive requirements for instance - "User accounts getting disabled after five unsuccessful login attempts" etc.
• It is aimed to test the negative requirements stating something that should never occur. For example "An external attacker should not be able to modify the contents of the Web page" and "Unauthorized users should not be able to access the data."

Conventional testing has following attributes:

• It is aimed to test a positive requirement i.e. to create the conditions in which the requirement is intended to hold true and verify that the requirement is satisfied by the software.
• To apply conventional testing approach to the negative requirements, we need to create every possible set of non-feasible conditions.

Methods of Security Testing:

To confirm if a particular software application meets the security requirements, usually following two methods of testing are adopted

1) Functional security testing: It is meant to ensure that the software behaves according to certain specified functional requirements and is expected to demonstrate that the specified requirements are totally satisfied at the acceptable level. Functional requirement generally have a form like - "When a certain thing takes place, then the software must respond in a particular way."

2) Risk-based security testing: The first step in risk-based testing is the identification of the security risks and the potential loss associated with those risks. It tries to confirm the immunity against specific risks that have been identified through risk analysis effort. Risk-based testing addresses negative requirements, which state what a software system should not do. Tests for negative requirements are derived from a risk analysis, and generally cover not only the high-level risks identified during the design process but also address low-level risks derived from the software itself.

What are the common techniques for Security Testing?

Few of the security testing techniques commonly used are:

• Network scanning
• Vulnerability scanning
• Password cracking
• Log review
• Integrity checkers
• Virus detection
• War dialing
• War driving (wireless LAN testing)
• Penetration testing

In actual practice combination of many such techniques may be used to have a more comprehensive assessment of the overall security aspect.

How to write Security test cases?

I shall deal with the approach and the style for writing security test cases.

Before you begin writing your test cases, note that

1) It is important to segregate based on Roles (something like Admin, Manager, Supervisor etc.)

2) You will need to delve into the negative scenario for a particular event initially before taking up the positive scenarios. This will ensure continuity of the test cases and will greatly help.

—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0

Given below is a very simple example for the approach to write security test cases. Once you are aware of the knack of writing simple ones like this, you can graduate to writing larger and complex ones. Yes, It is a great aid to novice and intermediate testers to follow these and test the security for any kind of application.

NO.	Action	Test Steps	Pass/ Fail	Comments
1	Invoke the application by typing the URL “http://abhi/myProj/Default.asp”	The browser should be invoked and the application login page should appear.
	Verify the login security for the Project Lead.
2	Login with loginname as “abhilash” and password as “password56″.	The user should be logged in and be directed to the Home page.
3	Verify the menu structure on the Home page.	The home page should contain the following menu structure. - Projects - Tasks - Dashboard - Reports - Skills
4	Verify the menu dropdown for the “Projects” menu.	The “Projects” menu should contain the following menu items. - Create Task - Create Build - Create Module
***	Like this you will need to cover the other menus too.
	End of verification for the Project Lead.
	Verify the login security for a Team Member
1	Login with loginname “anoop” and password as “anooppass”.	The user should be logged in and navigated to the Home page.
2	Verify the menu structure on the Home page.	The home page should contain the following menu structure. - Projects - Tasks - Dashboard - Reports - Skills
3	Verify that the user cannot select the menu item “Projects”.	The user should NOT be able to select the menu item “Projects”. The menu should be displayed as disabled.
4	Verify that the user can select the menu item “Tasks”.	The user should be able to select the menu item “Tasks”. The menu should be displayed as enabled.
5	Verify that the user cannot select the menu item “Dashboard”.	The user should NOT be able to select the menu item “Dashboard”. The menu should be displayed as disabled.
6	Verify that the user can select the menu item “Reports”.	The user should be able to select the menu item “Reports”. The menu should be displayed as enabled.
7	Verify that the user cannot select the menu item “Skills”.	The user should NOT be able to select the menu item “Skills”. The menu should be displayed as disabled.
	like this you will need to tackle each content on respective pages.
	End of verification for the team member.

—-0—x—0——0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0—0—x—0—0

Web Security Testing Checklist:

1. Try to directly access bookmarked web page without login to the system.

2. Do not sign-on system, directly try to download the file from the available download url, such as the input http://url/download?name=file and check if the systems restrict you to download the file.

3. sign out and then press the Back button to access the page accessed before.

4. ID / password authentication method: check with valid and invalid passwords, password rules say cannot be less than 6 characters, user id and password cannot be the same etc.

5. Important information (such as passwords, ID numbers, credit card numbers, etc.) should not get displayed in the input box when typing. They should be all encrypted and in asterix format.

6. Manually change the parameter value in the URL to check if you can access special pages. For example, suppose in a web system If ordinary users access the corresponding url in the parameters l = e and the corresponding url for advanced users in the parameters l = s. Now if a user manually changes the value from e to s it should not allow you to access the page.

7. In the url, enter the following address to check if it can be downloaded restricted files: http://url/download.jsp?file=C: \ windows \ system32 \ drivers \ etc \ hosts,

http://url/download.jsp?file = / etc / password

8. after session time out try to access restricted page.

9. Error messages whether they contain SQL statements, SQL error messages, as well as web server's absolute path, etc.

10. ID / password authentication, the same account on different machines cannot log on at the same time. So at a time only one user can login to the system with a user id.

11. ID / password authentication methods, entered the wrong password several times and check if the account gets locked.

12. Add or modify important information (passwords, ID numbers, credit card number, etc.). Check if it gets reflected immediately or caching the old values.