Benchmarking Tool

A flexible HTTP benchmarking tool for load testing APIs with support for multiple endpoints, dynamic parameter generation, and comprehensive reporting.

Features

✅ Test multiple HTTP endpoints with various methods (GET, POST, PUT, DELETE, etc.)
✅ Advanced parameter generation system with multiple generator types
✅ Support for path parameters, query parameters, and request bodies
✅ Dynamic parameter value generation (random integers, formatted strings, choices, etc.)
✅ Flexible endpoint selection strategies (round-robin, weighted, random)
✅ Fixed and ramp RPS load generation with a bounded worker pool, optional queue depth, and token-bucket burst
✅ Configurable test duration and request timeouts
✅ YAML-based configuration for easy setup
✅ Streaming metrics with latency percentiles (p50/p90/p95/p99), achieved RPS, and per-endpoint breakdown
✅ Text and JSON report output (-format, -out flags)
✅ Failsafe auto-shutdown — abort the run when the failure ratio over a sliding window crosses a configured threshold
✅ Static string values and generator references with clear $ref syntax

Getting Started

Prerequisites

Go 1.24.2 or newer (see go.mod)

Quick Start

Clone and build:

git clone <your-repo-url>
cd benchmarking-tool
go mod tidy              # Install dependencies
go build -o benchmarking-tool

Run with the included example or the root config.yaml default:

./benchmarking-tool                          # uses config.yaml when present
./benchmarking-tool config-examples/simple-example.yml

Create your own configuration:

cp config-examples/simple-example.yml my-test.yml
# Edit my-test.yml to match your API
./benchmarking-tool my-test.yml

Command-line flags

The config file is a positional argument (defaults to config.yaml). Output is controlled by flags, which must precede the config path:

./benchmarking-tool [-format text|json] [-out FILE] [config.yaml]

Flag	Default	Description
`-format`	`text`	Report format: `text` (human-readable table) or `json` (stable machine-readable schema).
`-out`	(stdout)	Write the report to `FILE` instead of standard output.

./benchmarking-tool -format json -out report.json my-test.yml

Configuration

The tool uses YAML configuration files to define endpoints, parameter generation, and test execution settings. See config-examples/simple-example.yml for a working example.

Configuration Structure

# Base URLs for your API
baseUrls:
  - "https://api.example.com"
  - "https://backup-api.example.com"

# Test execution settings
execution:
  mode: "fixed"                # Only "fixed" mode is currently supported
  durationSeconds: 60          # Test duration in seconds
  requestTimeoutMs: 5000       # Individual request timeout in milliseconds
  requestsPerSecond: 50        # Target average requests per second
  # Optional — concurrency and rate shaping (see "Fixed RPS: workers and queue" below)
  # maxWorkers: 16             # Cap concurrent in-flight HTTP requests (default: auto)
  # maxQueueDepth: 32          # Buffered jobs between scheduler and workers (default: 2 × maxWorkers)
  # rateBurst: 1               # Token-bucket burst for the rate limiter (default: 1)
  # seed: 42                   # Fix the random seed for reproducible parameter generation (default: random per run)

# Named parameter generators (reusable across endpoints)
parameterGenerators:
  user_id:
    type: "formattedInt"       # Generate formatted string from integer
    min: 1000
    max: 9999
    format: "user_{}"          # Results in "user_1234", "user_5678", etc.
  
  age:
    type: "randomInt"          # Generate plain integer
    min: 18
    max: 65

# API endpoints to test
endpoints:
  get_user:
    path: "/api/v1/users/{user_id}"
    method: "GET"
    pathParameters:
      user_id:
        $ref: "user_id"        # Reference to named generator
    queryParameters:
      fields: "id,name,email"  # Static string value
      format: "json"           # Another static string

# How to select endpoints during testing
endpointSelection:
  strategy: "weighted"         # "roundRobin", "weighted", or "random"
  weights:
    get_user: 0.6
    create_user: 0.4

Fixed RPS: workers and queue

Fixed mode targets an average requestsPerSecond using a token bucket (golang.org/x/time/rate). A scheduler acquires tokens at that rate and pushes work to a bounded queue; worker goroutines (up to maxWorkers) dequeue work, build each request, and execute it with a shared http.Client. The HTTP transport’s idle connection limits scale with maxWorkers so many concurrent requests to the same host are not artificially serialized.

Field	Meaning
`maxWorkers`	Maximum number of requests executing at once. If omitted or `0`, it defaults to `min(256, max(1, requestsPerSecond))`. Allowed range after resolution: 1–8192.
`maxQueueDepth`	How many scheduled requests may wait for a free worker. If omitted or `0`, it defaults to 2 × maxWorkers. Maximum 1_000_000.
`rateBurst`	Burst size for the limiter (how many permits can accumulate). Default 1 if omitted or `0`. Allowed range: 1–10_000.
`seed`	Optional. Fixes the seed for all random parameter generation and endpoint selection, so repeated runs generate the same value streams per worker. Omitted means a fresh random seed each run.

When workers cannot keep up (slow server, low maxWorkers, or a small queue), the scheduler does not block indefinitely: if the queue is full, that scheduling slot is dropped and counted. The total dropped count appears in the final report as Dropped (backpressure) (and in the run log when any drops occurred). Completed requests are still recorded in metrics as today; dropped slots never become HTTP attempts.

For a small local example including these fields, see test_config.yaml.

Parameter Generators

Parameter generators create dynamic values for requests. They can be defined as named generators (reusable) or inline within endpoint definitions.

Parameter Value Types

The configuration supports three ways to specify parameter values:

Static Strings: Plain string values are treated as static

queryParameters:
  format: "json"            # Always returns "json"
  version: "v1"             # Always returns "v1"

Generator References: Use $ref to reference named generators

pathParameters:
  user_id:
    $ref: "user_id"         # References parameterGenerators.user_id

Inline Generator Definitions: Full generator objects

bodyParameters:
  age:
    type: "randomInt"
    min: 18
    max: 65

Generator Types

Authoritative examples (each file is validated by go test ./config/...):

config-examples/simple-example.yml — minimal GET/POST patterns
config-examples/config-clean-example.yaml — $ref, static strings, nested bodies
config-examples/parameter-showcase.yml — one section per generator type
config-examples/enhanced-config.yml — multi-endpoint, weighted selection
config-examples/advanced-example.yml — aliases (random_int, params / fields), nested arrays
config-examples/failsafe-example.yml — ramp + failsafe capacity search

Schema reminders

Use $ref: "name" to reuse a named generator; a bare string like field: "name" is a static value.
parameters / properties are canonical; params (templates) and fields (objects) are accepted aliases.
random is an alias for randomString.
Headers are sent as literal strings (no {{...}} substitution).

The following generator types are implemented:

`randomInt` ✅

Generates random integers within a specified range.

type: "randomInt"
min: 1              # Minimum value (inclusive)
max: 100            # Maximum value (inclusive)

Output: Plain integer (e.g., 42, 87)

`formattedInt` ✅

Generates formatted strings using random integers.

type: "formattedInt"
min: 1000
max: 9999
format: "user_{}"   # {} is replaced with the generated number

Output: Formatted string (e.g., "user_1234", "user_5678")

`static` ✅

Returns a fixed value.

type: "static"
value: "fixed-value"

Output: The exact value specified

`choice` ✅

Randomly selects from a list of values, optionally with weights.

type: "choice"
values: ["active", "inactive", "pending"]
weights: [0.5, 0.3, 0.2]  # Optional: probability weights

Output: One of the specified values

`randomString` ✅

Generates random strings with specified length and character set.

type: "randomString"
length: 10
charset: "alphanumeric"  # alpha, alpha_lower, alpha_upper, numeric, alphanumeric, hex, alphanumeric_space

Output: Random string (e.g., "a3B7xK9mP2")

`template` ✅

Generates values using templates with embedded parameters.

type: "template"
template: "Hello {{name}}, you are {{age}} years old"
parameters:
  name:
    type: "choice"
    values: ["Alice", "Bob", "Charlie"]
  age:
    type: "randomInt"
    min: 20
    max: 50

Output: Rendered template (e.g., "Hello Alice, you are 35 years old")

`object` ✅

Generates JSON objects with specified properties.

type: "object"
properties:
  name:
    type: "choice"
    values: ["John", "Jane", "Mike"]
  age:
    type: "randomInt"
    min: 18
    max: 65
  active:
    type: "choice"
    values: [true, false]

Output: JSON object (e.g., {"name": "John", "age": 42, "active": true})

`array` ✅

Generates arrays with random length and elements.

type: "array"
minLength: 1
maxLength: 5
elementGenerator:
  type: "randomInt"
  min: 1
  max: 100

Output: Array of values (e.g., [42, 17, 89])

`sequence` ✅

Monotonic counter (thread-safe), optional string format with {} or %d.

type: "sequence"
start: 1
increment: 1   # default 1
format: "ORD-{}"   # omit for plain integers

`uuid` ✅

RFC 4122 version-4 UUID string.

type: "uuid"

`timestamp` ✅

Current time in UTC. format: unix (int64 seconds), rfc3339, iso8601 (same as RFC3339Nano), or default RFC3339Nano.

type: "timestamp"
format: "rfc3339"

`randomFloat` ✅

Uniform float in [min, max]. Use min / max in inline maps; for named generators in the top-level parameterGenerators map, use minFloat / maxFloat if you need non-integer bounds. Optional precision (decimal places, -1 to skip rounding).

type: "randomFloat"
min: 0.5
max: 99.99
precision: 2

`randomBool` ✅

true with probability probability (0–1). Alias: trueProbability (same meaning).

type: "randomBool"
probability: 0.7

Endpoint Configuration

Each endpoint defines how to make requests to a specific API path.

endpoints:
  endpoint_name:
    path: "/api/path/{param}"    # URL path with parameter placeholders
    method: "GET"                # HTTP method
    headers:                     # Optional: custom headers
      Content-Type: "application/json"
      Authorization: "Bearer token"
    
    # Parameters for URL path (replace {param} placeholders)
    pathParameters:
      param:
        $ref: "generator_name"   # Reference to named generator
      # or inline definition:
      param:
        type: "randomInt"
        min: 1
        max: 100
    
    # URL query parameters (?key=value)
    queryParameters:
      limit:
        type: "randomInt"
        min: 10
        max: 50
      filter:
        $ref: "active_filter"    # Reference to named generator
      format: "json"             # Static string value
    
    # Request body (for POST, PUT, PATCH requests)
    bodyParameters:
      type: "object"
      properties:
        user:
          type: "object"
          properties:
            name:
              $ref: "user_name_generator"
            age:
              $ref: "user_age_generator"

Endpoint Selection Strategies

Controls how endpoints are chosen during testing.

`roundRobin` ✅ Implemented

Cycles through endpoints in order, ensuring equal distribution.

endpointSelection:
  strategy: "roundRobin"

`weighted` ✅ Implemented

Selects endpoints based on specified weights (probabilities).

endpointSelection:
  strategy: "weighted"
  weights:
    get_user: 0.6      # 60% of requests
    create_user: 0.3   # 30% of requests
    delete_user: 0.1   # 10% of requests

`random` ✅ Implemented

Randomly selects endpoints with equal probability.

endpointSelection:
  strategy: "random"

Execution Modes

Fixed Mode ✅ Implemented

Targets an average request rate over the test duration using a rate limiter, with bounded concurrency (maxWorkers) and an optional job queue (maxQueueDepth). See Fixed RPS: workers and queue for defaults and backpressure behavior.

execution:
  mode: "fixed"
  durationSeconds: 300         # Run for 5 minutes
  requestsPerSecond: 100       # Target average 100 RPS
  requestTimeoutMs: 2000       # 2 second timeout per request
  maxWorkers: 32               # Optional; omit for auto
  maxQueueDepth: 64            # Optional; omit for 2 × maxWorkers
  rateBurst: 1                 # Optional; default 1

Ramp Mode ✅ Implemented

Linearly interpolates the target rate from startRps to endRps across the whole run duration — useful for finding the load at which an API's latency or error rate starts to degrade. The rate is recomputed continuously (no stepping), and the same bounded worker pool, queue, and drop-on-full backpressure apply as in fixed mode. Both startRps and endRps must be between 1 and 1,000,000; ramp can also ramp down (startRps > endRps).

execution:
  mode: "ramp"
  durationSeconds: 30          # Ramp spans the full duration
  requestTimeoutMs: 2000
  ramp:
    startRps: 5                # Rate at t=0
    endRps: 200                # Rate at t=durationSeconds
  maxWorkers: 64               # Optional; auto-sizes to the ramp peak (max of start/end, capped at 256)

When maxWorkers is omitted it auto-sizes to the ramp's peak rate so the pool can serve endRps; set it explicitly to deliberately cap concurrency below the peak. See config-examples/ramp-example.yml for a complete example.

Failsafe (automatic shutdown on failures) ✅ Implemented

An optional execution.failsafe block aborts the run early when the target starts failing, instead of hammering a broken service for the remaining duration:

execution:
  failsafe:
    failureRatio: 0.5       # Abort when >= 50% of recent requests failed; (0, 1]
    windowRequests: 100     # Sliding window size; optional, default 100

A request counts as failed using the same rule as the report's Failed Requests: a transport error (timeout, connection refused, …) or an HTTP status >= 400.
The ratio is evaluated over a sliding window of the last windowRequests completed requests. The window must fill once before evaluation, so it doubles as the minimum sample count — a single early failure can never abort the run.
On trip the run stops gracefully (same path as reaching the duration): in-flight work drains, the full report is still produced with an abort marker, and the process exits non-zero so CI pipelines can detect it. Note that requests still in flight at the trip are cancelled and counted as failed, so the report's final totals can show a somewhat higher error rate than the target alone caused; the trip's own observedRatio is snapshotted before that shutdown noise arrives. The text report shows ⚠ Run aborted by failsafe at 12.4s: failure ratio 0.62 >= 0.50 over last 100 requests; the JSON report carries a summary.failsafe block (trippedAt, observedRatio, threshold, windowRequests).
Combined with ramp mode this becomes a capacity search: ramp the rate up until the target starts failing, and the failsafe ends the run at that point. See config-examples/failsafe-example.yml.

Complete Example

The repository includes several example configurations:

✅ config-examples/simple-example.yml: Basic configuration demonstrating GET and POST endpoints with parameter generation
✅ config.yaml: Default configuration file (copy of simple-example.yml)
✅ config-clean-example.yaml: Enhanced example showcasing the new $ref syntax and static strings

All files in config-examples/ are kept aligned with the current schema; go test ./config/... loads each of them in TestLoadConfig_AllConfigExamples.

Each file under config-examples/ is kept loadable by the tool. Prefer the files above over ad-hoc copies of older snippets.

Modes

fixed ✅ Implemented

Targets a configured average RPS across endpoints (round-robin, weighted, or random selection). Concurrency is capped by maxWorkers, with a bounded queue between the rate scheduler and workers; if the queue fills, excess schedule slots are dropped and summarized in the log. This mode is recommended for most load testing scenarios.

ramp ✅ Implemented

Linearly ramps the target RPS from startRps to endRps over the run duration, reusing the same worker/queue/backpressure topology as fixed mode. The report's Configured RPS row shows the span (e.g. 5 → 50 (ramp)) and Achieved RPS reflects the run's overall average. Good for capacity discovery and soak-to-peak profiles.

Output and Reporting

The tool provides comprehensive reporting including:

Request count and success/failure rates
Response time statistics (min, max, average, and p50/p90/p95/p99 percentiles)
Achieved RPS (requests completed ÷ elapsed wall-clock time) alongside the configured target
Error rate percentage
Status code distribution
Per-endpoint breakdown (request count, error rate, average, and p95 for each endpoint)
Detailed error message summary with occurrence counts (distinct messages capped at 100, with the remainder folded into an (other errors) bucket)
Execution duration and configured RPS (metrics reflect completed HTTP attempts only)

Metrics are aggregated as requests complete (streaming), so memory stays proportional to the number of endpoints, not the number of requests — the tool can sustain long, high-RPS runs without unbounded growth. Percentiles come from a compact log-linear latency histogram (worst-case ~3% bucket error, refined by interpolation; exact min/max are tracked separately).

During a fixed-RPS run, the runner also logs worker count, queue depth, and burst at start. If any scheduled requests were dropped because the job queue was full, the report's Dropped (backpressure) row shows how many (those slots are not counted in the request totals). Reported request times measure the HTTP exchange only (send → full response body received); parameter generation and request construction are excluded.

Example text output (-format text, the default):

--- Benchmark Report ---
Metric                  Value
------                  -----
Test Mode               fixed
Configured Duration     3s
Configured RPS          50
Total Requests          150
Successful Requests     150
Failed Requests         0
Dropped (backpressure)  0
Error Rate              0.00%
Elapsed                 2.982s
Achieved RPS            50.30
Min Request Time        270.334µs
Max Request Time        8.321875ms
Avg Request Time        970.439µs
P50 Request Time        469.674µs
P90 Request Time        2.29376ms
P95 Request Time        4.75136ms
P99 Request Time        7.405568ms

Status Code Distribution:
Status 200              150

Per-Endpoint Breakdown:
createUser              requests=40 errors=0.00% avg=1.006003ms p95=3.2768ms
getUser                 requests=110 errors=0.00% avg=957.506µs p95=4.816896ms

--- End of Report ---

With -format json, the same data is emitted as a stable schema ({config, summary}) where every duration carries both nanos and a human string, and per-status/per-endpoint maps are string-keyed — suitable for piping into jq or storing as an artifact.

TODO / Future Features

Contributing

Contributions are welcome! Please open issues or submit pull requests for new features, bug fixes, or improvements.

License

MIT License

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
config-examples		config-examples
config		config
generator		generator
metrics		metrics
reporter		reporter
runner		runner
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
config.yaml		config.yaml
go.mod		go.mod
go.sum		go.sum
main.go		main.go
main_test.go		main_test.go

Folders and files

Latest commit

History

Repository files navigation

Benchmarking Tool

Features

Getting Started

Prerequisites

Quick Start

Command-line flags

Configuration

Configuration Structure

Fixed RPS: workers and queue

Parameter Generators

Parameter Value Types

Generator Types

randomInt ✅

formattedInt ✅

static ✅

choice ✅

randomString ✅

template ✅

object ✅

array ✅

sequence ✅

uuid ✅

timestamp ✅

randomFloat ✅

randomBool ✅

Endpoint Configuration

Endpoint Selection Strategies

roundRobin ✅ Implemented

weighted ✅ Implemented

random ✅ Implemented

Execution Modes

Fixed Mode ✅ Implemented

Ramp Mode ✅ Implemented

Failsafe (automatic shutdown on failures) ✅ Implemented

Complete Example

Modes

fixed ✅ Implemented

ramp ✅ Implemented

Output and Reporting

TODO / Future Features

Contributing

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

`randomInt` ✅

`formattedInt` ✅

`static` ✅

`choice` ✅

`randomString` ✅

`template` ✅

`object` ✅

`array` ✅

`sequence` ✅

`uuid` ✅

`timestamp` ✅

`randomFloat` ✅

`randomBool` ✅

`roundRobin` ✅ Implemented

`weighted` ✅ Implemented

`random` ✅ Implemented

Packages