Piwik Import Apache Server Access Logs Hourly No Duplicates

In order to regularly import your server’s access logs into Piwik, you will have to do three simple steps.

Step 1: Create a File to Store Commands

Create a new file called piwiklogimport.sh. Open it and type in the following:

eval $(awk '{ print "count="$1}' /home/username/public_html/piwik/numberoflines.txt) && python /home/username/public_html/piwik/misc/log-analytics/import_logs.py --token-auth=73952ab983b94872q2368q9ndu2ole6e --idsite=2 --skip=$count --url=http://domain.com/piwik /usr/local/apache/domlogs/username/domain.com && wc /usr/local/apache/domlogs/username/domain.com > /home/username/public_html/piwik/numberoflines.txt

Save it and upload it inside your Piwik directory. Change this file’s permissions to 400.

Cron File Import Piwik Apache Server Access Logs Permissions

Step 2: Create a File to Track Imports

Create a new file called numberoflines.txt. Open it and type in 0. Save it and upload it inside your Piwik directory. Change this file’s permissions to 600.

Create File Import Piwik Apache Server Access Logs Permissions

Step 3: Create a Cron Job for Importing Apache Access Logs

Create a new cron job. Set it to run on the 59 minute mark (for every hour) as such:

59 * * * *

For the cron command, use the following:

/bin/sh /home/username/public_html/piwik/domain.com/piwiklogimport.sh >/dev/null 2>&1

Go ahead and save this cron job.

Cron Job Import Piwik Apache Server Access Logs Automatically

Now the Apache access logs should be automatically imported at the 59-minute mark of every hour without importing previously imported lines of the log.

Import Piwik Apache Server Access Logs Automatically

Further reading: