|
| 1 | +# ResultsAnalyzer — Public API Documentation |
| 2 | + |
| 3 | +Analyze and visualize the outcome of an AsyncFlow simulation. |
| 4 | +`ResultsAnalyzer` consumes raw runtime objects (client, servers, edges, settings), |
| 5 | +computes latency and throughput aggregates, exposes sampled series, and offers |
| 6 | +compact plotting helpers built on Matplotlib. |
| 7 | + |
| 8 | +--- |
| 9 | + |
| 10 | +## Quick start |
| 11 | + |
| 12 | +```python |
| 13 | +import simpy |
| 14 | +from matplotlib import pyplot as plt |
| 15 | +from asyncflow.runtime.simulation_runner import SimulationRunner |
| 16 | +from asyncflow.metrics.analyzer import ResultsAnalyzer, SampledMetricName |
| 17 | + |
| 18 | +# 1) Run a simulation and get an analyzer |
| 19 | +env = simpy.Environment() |
| 20 | +runner = SimulationRunner.from_yaml(env=env, yaml_path="data/single_server.yml") |
| 21 | +res: ResultsAnalyzer = runner.run() |
| 22 | + |
| 23 | +# 2) Text summary |
| 24 | +print(res.format_latency_stats()) |
| 25 | + |
| 26 | +# 3) Plot the dashboard (latency histogram + throughput) |
| 27 | +fig, (ax_lat, ax_rps) = plt.subplots(1, 2, figsize=(12, 4), dpi=160) |
| 28 | +res.plot_base_dashboard(ax_lat, ax_rps) |
| 29 | +fig.tight_layout() |
| 30 | +fig.savefig("dashboard.png") |
| 31 | + |
| 32 | +# 4) Single-server plots |
| 33 | +server_id = res.list_server_ids()[0] |
| 34 | +fig_rdy, ax_rdy = plt.subplots(figsize=(8, 4), dpi=160) |
| 35 | +res.plot_single_server_ready_queue(ax_rdy, server_id) |
| 36 | +fig_rdy.tight_layout() |
| 37 | +fig_rdy.savefig(f"ready_{server_id}.png") |
| 38 | +``` |
| 39 | + |
| 40 | +--- |
| 41 | + |
| 42 | +## Data model & units |
| 43 | + |
| 44 | +* **Latency**: seconds (s). |
| 45 | +* **Throughput**: requests per second (RPS). |
| 46 | +* **Sampled metrics** (per server/edge): series captured at a fixed sampling |
| 47 | + period `settings.sample_period_s` (e.g., queue length, RAM usage). |
| 48 | + Units depend on the metric (RAM is typically MB). |
| 49 | + |
| 50 | +--- |
| 51 | + |
| 52 | +## Computed metrics |
| 53 | + |
| 54 | +* **Latency statistics** (global): |
| 55 | + `TOTAL_REQUESTS, MEAN, MEDIAN, STD_DEV, P95, P99, MIN, MAX`. |
| 56 | +* **Throughput time series**: per-window RPS (default cached at 1 s buckets). |
| 57 | +* **Sampled metrics**: raw, per-entity series keyed by |
| 58 | + `SampledMetricName` (or its string value). |
| 59 | + |
| 60 | +--- |
| 61 | + |
| 62 | +## Class reference |
| 63 | + |
| 64 | +### Constructor |
| 65 | + |
| 66 | +```python |
| 67 | +ResultsAnalyzer( |
| 68 | + *, |
| 69 | + client: ClientRuntime, |
| 70 | + servers: list[ServerRuntime], |
| 71 | + edges: list[EdgeRuntime], |
| 72 | + settings: SimulationSettings, |
| 73 | +) |
| 74 | +``` |
| 75 | + |
| 76 | +The analyzer is **lazy**: metrics are computed on first access. |
| 77 | + |
| 78 | +### Core methods |
| 79 | + |
| 80 | +* `process_all_metrics() -> None` |
| 81 | + Forces computation of latency stats, throughput cache (1 s), and sampled metrics. |
| 82 | + |
| 83 | +* `get_latency_stats() -> dict[LatencyKey, float]` |
| 84 | + Returns the global latency stats. Computes them if needed. |
| 85 | + |
| 86 | +* `format_latency_stats() -> str` |
| 87 | + Returns a ready-to-print block with latency statistics. |
| 88 | + |
| 89 | +* `get_throughput_series(window_s: float | None = None) -> tuple[list[float], list[float]]` |
| 90 | + Returns `(timestamps, rps)`. If `window_s` is `None` or `1.0`, the cached |
| 91 | + 1-second series is returned; otherwise a fresh series is computed. |
| 92 | + |
| 93 | +* `get_sampled_metrics() -> dict[str, dict[str, list[float]]]` |
| 94 | + Returns sampled metrics as `{metric_key: {entity_id: [values...]}}`. |
| 95 | + |
| 96 | +* `get_metric_map(key: SampledMetricName | str) -> dict[str, list[float]]` |
| 97 | + Gets the per-entity series map for a metric. Accepts either the enum value or |
| 98 | + the raw string key. |
| 99 | + |
| 100 | +* `get_series(key: SampledMetricName | str, entity_id: str) -> tuple[list[float], list[float]]` |
| 101 | + Returns time/value series for a given metric and entity. |
| 102 | + Time coordinates are `i * settings.sample_period_s`. |
| 103 | + |
| 104 | +* `list_server_ids() -> list[str]` |
| 105 | + Returns server IDs in a stable, topology order. |
| 106 | + |
| 107 | +--- |
| 108 | + |
| 109 | +## Plotting helpers |
| 110 | + |
| 111 | +All plotting methods draw on a **Matplotlib `Axes`** provided by the caller and |
| 112 | +do **not** manage figure lifecycles. |
| 113 | + |
| 114 | +> When there is no data for the requested plot, the axis is annotated with the |
| 115 | +> corresponding `no_data` message from `plot_constants`. |
| 116 | +
|
| 117 | +### Dashboard |
| 118 | + |
| 119 | +* `plot_base_dashboard(ax_latency: Axes, ax_throughput: Axes) -> None` |
| 120 | + Convenience: calls the two methods below. |
| 121 | + |
| 122 | +* `plot_latency_distribution(ax: Axes) -> None` |
| 123 | + Latency histogram with **vertical overlays** (mean, P50, P95, P99) and a |
| 124 | + **single legend box** (top-right) that shows each statistic with its matching |
| 125 | + colored handle. |
| 126 | + |
| 127 | +* `plot_throughput(ax: Axes, *, window_s: float | None = None) -> None` |
| 128 | + Throughput line with **horizontal overlays** (mean, P95, max) and a |
| 129 | + **single legend box** (top-right) that shows values and colors for each line. |
| 130 | + |
| 131 | +### Single-server plots |
| 132 | + |
| 133 | +Each single-server plot: |
| 134 | + |
| 135 | +* draws the main series, |
| 136 | + |
| 137 | +* overlays **mean / min / max** as horizontal lines (distinct styles/colors), |
| 138 | + |
| 139 | +* shows a **single legend box** with values for mean/min/max, |
| 140 | + |
| 141 | +* **does not** include a legend entry for the main series (title suffices). |
| 142 | + |
| 143 | +* `plot_single_server_ready_queue(ax: Axes, server_id: str) -> None` |
| 144 | + Ready queue length over time (per server). |
| 145 | + |
| 146 | +* `plot_single_server_io_queue(ax: Axes, server_id: str) -> None` |
| 147 | + I/O queue/sleep metric over time (per server). |
| 148 | + |
| 149 | +* `plot_single_server_ram(ax: Axes, server_id: str) -> None` |
| 150 | + RAM usage over time (per server). |
| 151 | + |
| 152 | +## Behavior & design notes |
| 153 | + |
| 154 | +* **Laziness & caching** |
| 155 | + |
| 156 | + * Latency stats and the 1 s throughput series are cached on first use. |
| 157 | + * Calling `get_throughput_series(window_s=...)` with a custom window computes |
| 158 | + a fresh series (not cached). |
| 159 | + |
| 160 | +* **Stability** |
| 161 | + |
| 162 | + * `list_server_ids()` follows the topology order for readability across runs. |
| 163 | + |
| 164 | +* **Error handling** |
| 165 | + |
| 166 | + * Multi-server plotting methods validate the number of axes and raise |
| 167 | + `ValueError` with a descriptive message. |
| 168 | + |
| 169 | +* **Matplotlib integration** |
| 170 | + |
| 171 | + * The analyzer **does not** close figures or call `plt.show()`. |
| 172 | + * Titles, axes labels, and “no data” messages are taken from |
| 173 | + `asyncflow.config.plot_constants`. |
| 174 | + |
| 175 | +* **Thread-safety** |
| 176 | + |
| 177 | + * The analyzer is not designed for concurrent mutation. Use from a single |
| 178 | + thread after the simulation completes. |
| 179 | + |
| 180 | +--- |
| 181 | + |
| 182 | +## Examples |
| 183 | + |
| 184 | +### Custom throughput window |
| 185 | + |
| 186 | +```python |
| 187 | +fig, ax = plt.subplots(figsize=(8, 3), dpi=160) |
| 188 | +res.plot_throughput(ax, window_s=2.0) # 2-second buckets |
| 189 | +fig.tight_layout() |
| 190 | +fig.savefig("throughput_2s.png") |
| 191 | +``` |
| 192 | + |
| 193 | +### Access a sampled metric series |
| 194 | + |
| 195 | +```python |
| 196 | +from asyncflow.metrics.analyzer import SampledMetricName |
| 197 | + |
| 198 | +server_id = res.list_server_ids()[0] |
| 199 | +t, qlen = res.get_series(SampledMetricName.READY_QUEUE_LEN, server_id) |
| 200 | +# t: [0.0, 0.1, 0.2, ...] (scaled by sample_period_s) |
| 201 | +# qlen: [.. values ..] |
| 202 | +``` |
| 203 | + |
| 204 | +--- |
| 205 | + |
| 206 | +If you need additional KPIs (e.g., tail latency over time, backlog, or |
| 207 | +utilization), the current structure makes it straightforward to add new helpers |
| 208 | +alongside the existing plotting methods. |
0 commit comments