Somesh Jha Assistant Professor, Computer Sciences Department, University of Wisconsin

Towards Resilient Malware Detectors

Abstract:

In today's interconnected world, malware, such as worms, viruses, and trojans, can cause havoc. A malware detector (commonly known as virus scanner) attempts to identify malware. In spite of the importance of malware detectors, their detection capabilities have gained little over the years, leading to today's reactive mode of operation where attackers have the upper hand.

In order to measure the strengths and limitations of existing malware detectors, we present a technique based on program obfuscation for generating tests for malware detectors. Our technique is geared towards evaluating the resilience of malware detectors to various obfuscation transformations commonly used by hackers to disguise malware. We also demonstrate that a hacker can leverage a malware detector's weakness in handling obfuscation transformations and can extract the signature used by a detector for a specific malware. We evaluate three widely-used commercial virus scanners using our techniques and discover that the resilience of these scanners to various obfuscations is very poor.

This lack of resilience is caused by the continued use of pattern-matching approaches in commercial virus scanners. The fundamental deficiency in such pattern-matching approaches to malware detection is that they are purely syntactic and ignore the semantics of instructions. We introduce a malware-detection algorithm that addresses this deficiency by incorporating instruction semantics to detect malicious program traits. Experimental evaluation demonstrates that our malware-detection algorithm can detect variants of malware with a relatively low run-time overhead. Moreover, our semantics-aware malware detection algorithm is resilient to common obfuscations used by hackers.

Short Bio

Somesh Jha received his B.Tech from Indian Institute of Technology, New Delhi in Electrical Engineering. He received his Ph.D. in Computer Science from Carnegie Mellon University in 1996. Currently, Somesh Jha is an Assistant Professor in the Computer Sciences Department at the University of Wisconsin (Madison), which he joined in 2000. His work focuses on analysis of security protocols, survivability analysis, intrusion detection, formal methods for security, and comnbating malicious code. Recently he has become interested in privacy-preserving protocols.