What are You Looking for?

ngtop – Nginx Access Log Analyzer

ngtop is a powerful command-line tool designed to provide insightful analytics from Nginx access logs. This guide will cover the installation, configuration, and usage of ngtop to help you analyze web traffic effectively.

Installation

To install ngtop, you need to have Go installed on your system. Clone the repository and build the tool using the following commands:

git clone https://github.com/facundoolano/ngtop.git
cd ngtop
go build

Alternatively, you can download the pre-built binaries from the releases page.

$ go install github.com/facundoolano/ngtop@latest

 

Configuration

ngtop uses environment variables for configuration. The primary variables include:

  • NGTOP_LOG_PATH: Path to the Nginx access log file.
  • NGTOP_DB_PATH: Path to the SQLite database file.
  • NGTOP_PATTERN: Log pattern to parse the log file (defaults to the common Nginx log format).

Example configuration in a .env file:

NGTOP_LOG_PATH=/var/log/nginx/access.log
NGTOP_DB_PATH=/path/to/ngtop.db
NGTOP_PATTERN='$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"'

Usage

Initializing the Database

To initialize the SQLite database with your access log data, use the init command:

ngtop init

This command will parse the log file specified in NGTOP_LOG_PATH and store the data in the SQLite database.

Querying the Data

ngtop allows you to query the stored data using various filters. Here are some common commands:

  • Request Counts: Get the total number of requests. 
    ngtop count
  • Filter by URL: Get request counts for a specific URL. 
    ngtop count --url /path/to/resource
  • Filter by Status Code: Get request counts for a specific HTTP status code. 
    ngtop count --status 404
  • Filter by User Agent: Get request counts for a specific user agent. 
    ngtop count --user-agent "Mozilla/5.0"
  • Filter by Time Range: Get request counts within a specific time range. 
    ngtop count --from "2024-01-01T00:00:00Z" --to "2024-01-31T23:59:59Z"

Advanced Queries

You can also perform more complex queries using SQL directly on the SQLite database. For instance, to get the top 10 URLs by request count, you can use:

SELECT url, COUNT(*) as request_count 
FROM requests 
GROUP BY url 
ORDER BY request_count DESC 
LIMIT 10;

Updating the Database

If you have new log entries, you can update the database using:

ngtop update

This command will parse new entries in the log file and update the SQLite database accordingly.

Conclusion

ngtop is a versatile tool for analyzing Nginx access logs, providing insights into web traffic through simple and advanced queries. With its easy configuration and powerful querying capabilities, it is an essential tool for web administrators and developers looking to understand their server traffic better.

For more detailed information and advanced usage, refer to the ngtop GitHub repository.

ByRavishanker Kusuma
Updated on