Parse Amazon S3 Logs with PHP

Amazon let’s you turn on logging for S3 buckets to capture (lots) of information on what’s happening there.

I wrote a PHP script to download those log files and parse the HTTP GET requests into a database for easier analysis. The data could also be sent to another source if you preferred.

S3logs-php on Github

Example Usage:

<?php
require 'vendor/Autoload.php';

$cfg = require 'config/config.php';

$logs = new \S3LP\Logs( $cfg['aws'], 'logs/', 'storage/logs/' );
try {
  $files = $logs->download();
} catch(Exception $e) {
  exit( $e->getMessage() );
}

//Parse logs to array
try {
  $data = $logs->parseLogs($files, 'DELETE');
  if(!$data) {
    exit('No GET requests found in logs');
  }
} catch(Exception $e) {
  exit( $e->getMessage() );
}

//Save to DB
try {
  $db = new \S3LP\Db( $cfg['pdo'] );
  $inserts = $db->insert($data);

  echo "Inserted $inserts rows";
} catch(Exception $e) {
  exit( $e->getMessage() );
}

Get the code on Github.