GenCos market MARL

Public API for the GenCos benchmark: a 5-agent rolling competitive market with exact PD-IPM clearing and ramp coupling. For physics, see Physics → Markets. For the benchmark task definition, see Benchmarks → GenCos.

Pure-functional core

from powerzoojax.envs import (
    MarketMARLState,
    MarketMARLParams,
    make_market_marl_params,
    market_marl_reset,
    market_marl_step,
)

These are JIT-compatible pure functions. market_marl_reset samples a random episode start index from the load-profile pool; market_marl_step enforces ramp bounds at the LP level via offer_sced.

powerzoojax.envs.market.market_marl_core.MarketMARLState

Bases: MarketState

Market MARL environment state.

Extends MarketState with per-unit dispatch profit from the last step, a fixed-length circular buffer of mean-LMP history for observations, and an episode start index for sampling diverse 48-step windows from a large profile pool.

episode_start_idx

Sampled uniformly from [0, T-1] at each reset (where T is the number of rows in load_profiles). The actual profile row used at step t is (episode_start_idx + t) % T. This gives diverse temporal coverage across training episodes when load_profiles is a large GB dataset pool.

powerzoojax.envs.market.market_marl_core.MarketMARLParams

Bases: MarketParams

Market MARL environment parameters.

Extends MarketParams with GenCos-specific fields.

powerzoojax.envs.market.market_marl_core.make_market_marl_params(case, load_profiles, *, n_segments=3, max_markup=2.0, max_steps=48, delta_t_hours=0.5, steps_per_day=48, lmp_scale=100.0, cost_thermal_weight=1.0, lmp_history_len=4, ramp_up_mw_per_step=None, ramp_down_mw_per_step=None, ramp_rate_fraction=0.5)

Build MarketMARLParams from a CaseData and load profile array.

Parameters:

Name	Type	Description	Default
case	CaseData	CaseData instance (e.g. from create_case5()).	required
load_profiles		(T, n_loads) array [MW] — absolute MW values.	required
n_segments	int	Number of offer-curve segments per generator.	3
max_markup	float	Maximum fractional markup (offer = base * (1 + m * max_markup)).	2.0
max_steps	int	Episode length in steps.	48
delta_t_hours	float	Time resolution [hours].	0.5
steps_per_day	int	Steps per day (for sin/cos obs features).	48
lmp_scale	float	LMP normalisation scale [$/MWh] for observations.	100.0
cost_thermal_weight	float	Weight on thermal overload cost signal.	1.0
lmp_history_len	int	Number of past mean-LMP values in the obs buffer.	4
ramp_up_mw_per_step	Optional[ndarray]	(n_units,) MW — per-step ramp-up limit. If None, defaults to ramp_rate_fraction × p_max.	None
ramp_down_mw_per_step	Optional[ndarray]	(n_units,) MW — per-step ramp-down limit. If None, defaults to ramp_rate_fraction × p_max.	None
ramp_rate_fraction	float	Fraction of p_max used as default ramp rate when ramp_up/down_mw_per_step are not provided. Default 0.5 (50% of p_max per step, i.e. 100% per hour at 30-min resolution).	0.5

Returns:

Type	Description
MarketMARLParams	MarketMARLParams.

powerzoojax.envs.market.market_marl_core.market_marl_reset(key, params)

Reset the market MARL environment.

Samples a random episode start index from [0, T-1] where T is the number of rows in load_profiles. Each episode then accesses profile rows (episode_start_idx + t) % T for t=0..max_steps-1, giving diverse temporal windows across training episodes when load_profiles is a large pool (e.g. a full year of GB half-hourly demand data).

Runs the exact SCED with truthful prices at the sampled start row to initialise the unit dispatch and LMP history.

Parameters:

Name	Type	Description	Default
key	PRNGKey	PRNG key used to sample episode_start_idx.	required
params	MarketMARLParams	MarketMARLParams.	required

Returns:

Type	Description
MarketMARLState	MarketMARLState at t=0 with sampled start index, truthful dispatch
MarketMARLState	and initialised LMP history.

powerzoojax.envs.market.market_marl_core.market_marl_step(key, state, flat_actions, params)

Single-step market MARL transition with ramp-bounded rolling market.

Action mapping

flat_actions ∈ [-1, 1]^{n_units * n_seg} m = (a + 1) / 2 ∈ [0, 1] sorted_m = sort(m, axis=-1) (enforce monotone offers) offer_prices = base_seg_prices * (1 + sorted_m * max_markup)

Ramp coupling

p_min_rt[i] = max(p_min[i], prev_dispatch[i] - ramp_down[i]) p_max_rt[i] = min(p_max[i], prev_dispatch[i] + ramp_up[i]) offer_sced is called with p_min_rt / p_max_rt to enforce these bounds in the LP itself (solver-layer enforcement, not post-hoc clipping).

Reward: dispatch_profit_i = LMP[node_i] * P_i * dt − TC(P_i) * dt where TC(P) = (a/3) P³ + (b/2) P² + c P.

Auto-reset: when done, state is replaced with reset state (stop_gradient).

Parameters:

Name	Type	Description	Default
key	PRNGKey	PRNG key (split for auto-reset).	required
state	MarketMARLState	Current MarketMARLState.	required
flat_actions	Array	(n_units * n_seg,) flattened actions in [-1, 1].	required
params	MarketMARLParams	MarketMARLParams.	required

Returns:

Type	Description
	(final_state, done, reward_vec, info)
	where reward_vec is (n_units,) dispatch profit per agent.

MARL adapter

from powerzoojax.rl import MarketMARLEnv

MarketMARLEnv wraps the pure core with the per-agent dict layout used by IPPO trainers. Agents are named genco_i, one per generator. The wrapper does not duplicate physics — it only handles action / observation splitting.

Private observation layout per agent is 8 + lmp_history_len:

• normalized first-segment base price
• normalized own p_max
• normalized own last dispatch
• normalized own last dispatch profit
• normalized own ramp-up headroom
• normalized one-step-ahead total-load forecast
• sin(t), cos(t)
• mean-LMP history

The wrapper returns per-agent reward and done dictionaries. The underlying pure core returns (final_state, done, reward_vec, info) and keeps market safety diagnostics such as cost_thermal_overload in info.

powerzoojax.rl.market_marl.MarketMARLEnv(params)

Bases: MultiAgentEnvironment

GenCos bid-based market — multi-agent RL environment.

Agents are generator companies (GenCos). Each step: 1. Each agent submits n_seg-dimensional bid (markup levels). 2. Exact SCED clears the market with ramp-bounded dispatch bounds. 3. Each agent receives dispatch profit as reward.

Parameters:

Name	Type	Description	Default
params	MarketMARLParams	MarketMARLParams (from make_market_marl_params).	required

Usage::

from powerzoojax.case import create_case5
from powerzoojax.envs.market.market_marl_core import make_market_marl_params
from powerzoojax.rl.market_marl import MarketMARLEnv
import jax.numpy as jnp

case = create_case5()
mid_load = (case.load_d_max + case.load_d_min) / 2.0
profiles = jnp.tile(mid_load[None, :], (48, 1))
params = make_market_marl_params(case, profiles, n_segments=3)
env = MarketMARLEnv(params)
key = jax.random.PRNGKey(0)
obs, state = env.reset(key)
obs2, state2, rew, dones, info = env.step(key, state, {n: jnp.zeros(3) for n in env.agent_names})

observation_space(agent=None)

Private observation space per agent: shape (self._obs_dim,).

Dimension = 8 scalar features + params.lmp_history_len LMP history values. For the benchmark default (lmp_history_len=4) this equals OBS_DIM=12.

[own_cost_b, own_p_max, own_dispatch, own_profit,

own_ramp_headroom, demand_forecast_t+1, sin(t), cos(t), lmp_hist[0..lmp_history_len-1]]

action_space(agent=None)

n_seg-dimensional action space in [-1, 1] (same for all agents).

compute_market_metrics(rollout_market_info)

Compute market-specific metrics from collected rollout info.

Called by make_ippo_train after each training update when this env is detected. rollout_market_info values have shapes:

lmp          : (n_steps, n_envs, n_nodes)
unit_power   : (n_steps, n_envs, n_units)
ramp_binding : (n_steps, n_envs)   — fraction of binding units

Implemented metrics (all keys prefixed "market/"): price_volatility — mean CV (std/mean) of per-env LMP time series HHI — mean Herfindahl-Hirschman Index across steps/envs market_share_std — std of time-averaged dispatch shares across agents (summary statistic accompanying dispatch_share_{ag}) ramp_binding_rate — mean fraction of ramp-constrained units per step dispatch_share_{ag} — per-agent time-averaged dispatch share (values ≈ 1/n for equal share; sum ≈ 1.0 across agents). Together these form the market_share_dynamics time-series signal: one value per agent per training update, suitable for plotting dispatch evolution. planner_cost_ratio — planner_cost / RL_gen_cost ∈ (0, 1]. Planner = ramp-free DC-OPF with true costs (offer_sced with base_seg_prices, no ramp bounds). Measures dispatch cost efficiency relative to the cost-minimising planner. A value close to 1.0 means RL dispatch is nearly as cheap as the planner; lower values indicate higher RL generation cost. NOTE: this is a COST RATIO, not a welfare ratio. True social_welfare_ratio = (V − RL_cost) / (V − planner_cost) where V is consumer value (VOLL × demand). V is not defined in this codebase, so a proper welfare ratio cannot be computed without an arbitrary consumer-side assumption. This metric is therefore not a social-welfare ratio. Requires rollout_market_info to contain load_mw (n_steps, n_envs, n_nodes) and gen_cost (n_steps, n_envs); omitted silently if absent (backward-compatible with old IPPO versions).

NOT implemented (explicit future work, G5 partial — not claimed as done): social_welfare_ratio — Requires consumer value V (VOLL × demand) for the ratio (V − RL_cost) / (V − planner_cost) to be well-defined and ∈ [0, 1]. V is absent from the current codebase; any choice would be arbitrary. exploitability_proxy — requires a best-response oracle or self-play; a price-taking approximation would conflate dispatch suboptimality with strategic effect. collusion_indicator — requires knowing the single-agent optimal markup, which is undefined without a Nash equilibrium solver. convergence_stability — implemented separately in the IPPO training loop (ippo.py), NOT computable from rollout data alone; exported as market/convergence_stability.

Usage example

import jax
import jax.numpy as jnp

from powerzoojax.case import load_case
from powerzoojax.envs import make_market_marl_params, market_marl_reset, market_marl_step
from powerzoojax.rl import MarketMARLEnv

case = load_case("5")
profiles = jnp.full((48, case.n_loads), 100.0, dtype=jnp.float32)
params = make_market_marl_params(case, profiles, n_segments=3, max_steps=48)

# 1. functional core (no per-agent layout)
state = market_marl_reset(jax.random.PRNGKey(0), params)
flat_actions = jnp.zeros(case.n_units * 3)
state, done, reward_vec, info = market_marl_step(
    jax.random.PRNGKey(1), state, flat_actions, params
)

# 2. MARL wrapper (per-agent dict)
env = MarketMARLEnv(params)
obs_dict, state = env.reset(jax.random.PRNGKey(2))
action_dict = {f"genco_{i}": jnp.zeros(3) for i in range(case.n_units)}
obs_dict, state, reward_dict, done_dict, info = env.step(
    jax.random.PRNGKey(3), state, action_dict
)