Parse Amazon S3 Logs with PHP

Amazon let’s you turn on logging for S3 buckets to capture (lots) of information on what’s happening there.

I wrote a PHP script to download those log files and parse the HTTP GET requests into a database for easier analysis. The data could also be sent to another source if you preferred.

S3logs-php on Github

Example Usage:

require 'vendor/Autoload.php';

$cfg = require 'config/config.php';

$logs = new \S3LP\Logs( $cfg['aws'], 'logs/', 'storage/logs/' );
try {
  $files = $logs->download();
} catch(Exception $e) {
  exit( $e->getMessage() );

//Parse logs to array
try {
  $data = $logs->parseLogs($files, 'DELETE');
  if(!$data) {
    exit('No GET requests found in logs');
} catch(Exception $e) {
  exit( $e->getMessage() );

//Save to DB
try {
  $db = new \S3LP\Db( $cfg['pdo'] );
  $inserts = $db->insert($data);

  echo "Inserted $inserts rows";
} catch(Exception $e) {
  exit( $e->getMessage() );

Get the code on Github.

WordPress Backups to AWS S3

Amazon Web Services S3 (Simple Storage Service) is a cheap and reliable way of storing data and is ideal for backups. Scheduling regular automatic backups of your WordPress website to S3 is pretty easy with a plugin, but it can be worth tweaking your AWS Credentials for better security.

This post will show you how to create a new user on your AWS account that has limited S3 permissions. It means if your site is ever compromised and the credentials stolen you’ll be in a far better position than having used your root AWS details! It’s also especially useful if you are managing backups of multiple client sites and do not want cross-access.

Step 1 – Create a new user with IAM in the AWS Console

  • Log into the AWS Console. Go to Services > Security & Identity > IAM
  • Create a new user (e.g. backup_myexample)
  • Copy and paste the Access Key and Secret somewhere; we’ll use those within WordPress shortly.

After creating your new user, go to their Policies and create a new inline policy. We’ll use inline, rather than group permissions so that each user you create (for backing up different websites) is isolated to their own S3 path.

Give the policy a name and paste and modify this Policy Document. Change my_awesome_bucket and my_directory to the bucket and path you’re using for these backups.

  "Version": "2012-10-17",
  "Statement": [
  "Sid": "Stmt1441240868000",
  "Effect": "Allow",
  "Action": "s3:*",
  "Resource": [

Your screen should look a bit like this
AWS IAM Policy

Step 2 – Install & Configure BackWPUp

  • Log into your WP Dashboard, go to Plugins > Add New Plugin and search for BackWPUp
  • Install it and create a new job. For testing you may want to do Database backup only, or list of plugins. This is much faster that a full site (Files) backup. Once you know it’s working setup a full site backup.
  • Set the backup to S3 Service.
  • On the S3 Service page select your Region and paste in the Access Key and Secret key from before.
  • Type in your bucket name and path to store the backups. It should match the IAM Policy Document


Save your settings and run the job.

The plugin logs will let you know if it worked.

A few notes

  • The IAM Policy allows all S3 actions on the given S3 path. I was not able to get this plugin to work with more restrictive permissions.
  • The new S3 Standard-IA class is good for these backups. The storage cost is cheaper than the Standard class without sacrificing redundancy as with Reduced Redundancy Storage. The downside is that downloads of these objects are more expensive.
  • Remember to check your backups periodically