Quick Anti-Debugging Trick for GDB

Right now, I’m listening to Sepultura’s Bestial Devastation and thinking about an interesting post not related to any public bugs, so here is the most common anti-debugging method for the GNU Debugger (aka. GDB).

GDB as you most likely already know is just a nice front end to ptrace(2). When it debugs a binary it uses exec* family system call first, and when attaching to an already running process it uses ptrace() directly.

With this in mind you can just use a simple function that will try to ptrace() your code. If ptrace() fails with -1 then it’s really likely that you’re running into GDB since ptrace() failed because your code is already being debugged using ptrace() :P. Also, since ELF provides us with .ctors and .dtors we can perform this test even before main() at __do_global_ctors_aux() simply like this:

#include <sys/ptrace.h>
#include <unistd.h>
#include <stdio.h>

void check(void) __attribute__((constructor));

void
check(void)
{
   if (ptrace(PTRACE_TRACEME, 0, 0, 0) == -1) {
       printf("lulz!\n");
       _exit(-1);
   }
}
int
main(void)
{
  printf("do stuff outside GDB\n");
  return 0;
}

This simple check() function can be really helpful when you really want to know when you are in a debugger. Here is the result of this little example program:

sh-3.2$ gcc an.c -std=c99 -pedantic -Wall -o an -ggdb
sh-3.2$ ./an
do stuff outside GDB
sh-3.2$ gdb -q ./an
Using host libthread_db library “/lib/libthread_db.so.1″.
(gdb) run
Starting program: /home/xorl/an
lulz!

Program exited with code 0377.
(gdb)

I know there are more better ways to perform GDB anti-debugging but this is just to fill some space. This is known.. I know :p In the future I’ll probably post some more info on this subject.

~ by xorl on January 1, 2009.