FreeBSD wpa_supplicant(8) Base64 Integer Overflow

This bug was reported to the FreeBSD project by Jonathan Bokovza on 6 August 2009 and it affects 7.2-STABLE and probably other releases as well. wpa_supplicant is a WPA Supplicant component as you can read from its man page. Here is the buggy code as seen in src/contrib/wpa_supplicant/base64.c from FreeBSD’s CVS.

/**
 * base64_encode - Base64 encode
 * @src: Data to be encoded
 * @len: Length of the data to be encoded
 * @out_len: Pointer to output length variable, or %NULL if not used
 * Returns: Allocated buffer of out_len bytes of encoded data,
 * or %NULL on failure
 *
 * Caller is responsible for freeing the returned buffer. Returned buffer is
 * nul terminated to make it easier to use as a C string. The nul terminator is
 * not included in out_len.
 */
unsigned char * base64_encode(const unsigned char *src, size_t len,
			      size_t *out_len)
{
	unsigned char *out, *pos;
	const unsigned char *end, *in;
	size_t olen;
	int line_len;

	olen = len * 4 / 3 + 4; /* 3-byte blocks to 4-byte */
	olen += olen / 72; /* line feeds */
	olen++; /* nul termination */
	out = os_malloc(olen);
	if (out == NULL)
		return NULL;
    ...

	if (out_len)
		*out_len = pos - out;
	return out;
}

As you can see, during the first arithmetic operation, it multiplies ‘len’ with some constant values. If ‘len’ is large enough, it could result to an integer overflow and the subsequent call to os_malloc() would allocate incorrect number of bytes which will later result to heap memory corruption. The proposed patch is a simple check after the calculation.

olen++; /* nul termination */
+ if (olen < len) + return NULL; out = os_malloc(olen); if (out == NULL) [/sourcecode]