xorl %eax, %eax

CVE-2009-1378: OpenSSL DTLS fragment handling memory DoS

leave a comment »

This bug was reported by Robin Seggelmann and it was discovered by Daniel Mentz. It affects OpenSSL prior to 1.0.0-beta2 release. Here is the vulnerable code from the latest OpenSSL release which is 0.9.8k:

3  * DTLS implementation written by Nagendra Modadugu
4  * (nagendra@cs.stanford.edu) for the OpenSSL project 2005.
5  */
6 /* ====================================================================
7  * Copyright (c) 1998-2005 The OpenSSL Project.  All rights reserved.
 ...
552 static int
553 dtls1_process_out_of_seq_message(SSL *s, struct hm_header_st* msg_hdr, int *ok)
554 {
555         int i=-1;
556         hm_fragment *frag = NULL;
557         pitem *item = NULL;
558         PQ_64BIT seq64;
559         unsigned long frag_len = msg_hdr->frag_len;
560
561         if ((msg_hdr->frag_off+frag_len) > msg_hdr->msg_len)
562                 goto err;
563
564         if (msg_hdr->seq <= s->d1->handshake_read_seq)
565                 {
566                 unsigned char devnull [256];
567
568                 while (frag_len)
569                         {
570                         i = s->method->ssl_read_bytes(s,SSL3_RT_HANDSHAKE,
571                                 devnull,
572                                 frag_len>sizeof(devnull)?sizeof(devnull):frag_len,0);
573                         if (i<=0) goto err;
574                         frag_len -= i;
575                         }
576                 }
 ...
578         frag = dtls1_hm_fragment_new(frag_len);
 ...
609         }


The bug is simple. An attacker can send numerous SEQ handshake packets to to trigger many allocations with size of frag_len which is calculated at lines 568-575. The allocation takes place at line 578 of dtls1_process_out_of_seq_message() function. To fix this, the following patch was used at ssl/d1_both.c which is where this function is located:

-	if (msg_hdr->seq <= s->d1->handshake_read_seq)
+	/* Try to find item in queue, to prevent duplicate entries */
+	memset(seq64be,0,sizeof(seq64be));
+	seq64be[6] = (unsigned char) (msg_hdr->seq>>8);
+	seq64be[7] = (unsigned char) msg_hdr->seq;
+	item = pqueue_find(s->d1->buffered_messages, seq64be);
+ 
+	/* Discard the message if sequence number was already there, is
+	 * too far in the future or the fragment is already in the queue */
+	if (msg_hdr->seq <= s->d1->handshake_read_seq ||
+		msg_hdr->seq > s->d1->handshake_read_seq + 10 || item != NULL)
 {
 unsigned char devnull [256];


Those new checks ensure that only messages with sequence numbers less than 10 in advance will be buffered by that routine. A PoC trigger code for this bug was published by Jon Oberheide and you can find it here.

Written by xorl

May 18, 2009 at 21:37

Posted in bugs

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s