The OWASP MSTG: Towards a Standard Methodology for Mobile App Security Testing

The software security testing landscape has changed. A few years ago, we were testing web apps, web apps and... more web apps. Some of us already got a little bored with testing all those web apps. Then the smartphone revolution happened. And, perhaps unsurprisingly, smartphone apps turned out to have their own classes of security vulnerabilities as well! As of now, pre-release security testing has become standard for mobile security apps just like it is in the web application world.

Security testing methodologies - at least the publicly available ones - haven’t quite caught up to this development yet, even though there are some fantastic resources. Security books, such as the Mobile Application Hacker’s Handbook, and web resources such as the Android and iOS testing guides offered by the InfoSecInstitute serve as standard references for mobile pen-testers. OASAM is a notable attempt to define a security assessment methodology for Android, but as of now it lacks completeness and industry acceptance. What’s missing is something with the maturity OWASP Web Testing Guide, only for mobile.

So why do we even need this? Many skilled mobile pen-testers follow their own (sometimes unwritten) methodologies and come up with great results. Having a standard methodology however helps create consistency between different tests, and can be used to establish a baseline level of coverage. Another plus is that we'll hopefully arrive at something resembling an industry consensus on certain questions specific to mobile security, such as:

  • What is the minimum set of test cases that should be performed in every mobile app security test?
  • What security measures are required under which circumstances? For example, no one would argue that client-server communication should be encrypted, but do we always require apps to implement SSL pinning? Or only when highly sensitive data is concerned? Or maybe, is it a bad idea?
  • What is the role of obfuscation and anti-tampering? Is this something that is helpful in certain cases? How do we deal with those things in our tests, and how do we evaluate their effectiveness?

To solve these questions, the OWASP Mobile Security team has started developing a brand new revision of the OWASP Mobile Security Testing Guide (MTSG). The goal of this project is to create a comprehensive methodology that covers the processes, techniques, and tools used during a mobile app security test, and defines a complete set of test cases that enables testers to deliver consistent and complete results.

The first step in creating the guide was to come up with a way of structuring the document and categorizing the test cases. For now, we have come up with 59 test cases grouped in 7 categories. The list is already pretty comprehensive, however it is not set in stone and we expect it to expand and evolve during the next weeks. Please let us know if we forgot anything important!

Here are some considerations that went into the creation of the list:

1. In order not to duplicate existing guides, most server-side issues have been grouped into a single test case (OMTG-CLTSRC-004) that will refer to the e OWASP web guide and potentially other relevant methodologies.

Only test cases that can be performed on the mobile device or adjacent network have been included. For example, even though a mobile client could conceivably authenticate users against a remote LDAP server, we did not include a test case for server-side LDAP injection. Similarly there are not test cases for server-side SQL injection, captchas etc. The reasoning: Server-side tests are essentially out of scope for this guide, and if we were to include all of them we’d have to essentially copy the entire content of the OWASP web guide, plus others (as mobile app might connect to various kind of servers).

However, we have made a few exceptions for test cases that are highly relevant for most mobile apps even though those test cases usually require server-side testing, such as test cases related to user authentication.

2. Separation between vulnerability test cases and basic / advanced app protection test cases:

Anti-tampering and obfuscation have become a thing in mobile security. There’s a lot of ways one can make it harder to reverse engineer mobile apps, however we would argue that having most of those features is not a requirement (for example, while obfuscating the Java Bytecode of an Android app might be a useful additional layer of defense in some cases, not having any obfuscation does not cause a vulnerability). Nevertheless, we have to deal with obfuscated apps and it would also be useful to have a way of assessing their effectiveness.

For now, we have grouped these test cases in two separate categories:

  • Testing Basic App Protections:

    Test cases for protections that should be standard for every app. This includes setting the right compiler flags for stack protection and PIE, detecting jailbreaks and pinning SSL certificates - all things that can be done with relatively low effort to improve the security of the application.
  • Testing Advanced App Protections:

    Advanced features that may be built in to add additional layers of defense against reverse engineering and tampering. This includes bytecode and binary obfuscation, code encryption, debugger detection, virtualization, white box cryptography, and so on. Please comment if you feel that there's anything missing in the list.


Call for Authors and Reviewers

We're now at a stage where we would like to get community feedback on the proposed list of test cases. Please have a look at the test cases Google Doc and leave your comments.

If you would like to contribute to the guide, now is the best time to sign up! We're beginning to work on the actual content and are looking for experts to volunteer either as authors or reviewers. Whether you have too much time on your hands and want to own a whole category, or just contribute content for a single test case, all help is appreciated. Simply sign up for the OWASP Mobile Security Project mailing list, post your short bio and we'll add you as an editor.

Any other thoughts? Let us know in the comments!

About the Author

Bernhard Mueller is a full-stack hacker, security researcher, and winner of BlackHat's Pwnie Award.

Follow him on Twitter: @muellerberndt