What Happens When a Genetic Algorithm and Deep Reinforcement Learning Share the Driving Seat: Plasma Flow Control on a Square Cylinder
We built a framework that couples a genetic algorithm with PPO inside an uncertainty-guided active-learning loop, wrapped around a GP-Transformer surrogate. Here is what we found, how it works, and what we would do differently.
The short version
Flow past a square cylinder at Re = 200 sheds a von Kármán vortex street: alternating vortices that cause oscillating lift, vibration, and drag. Two dielectric barrier discharge (DBD) plasma actuators, mounted at the trailing-edge corners, can suppress this — but only if you pulse them correctly. Pulsing is defined by three numbers: the duty-cycle frequency f, the duty-cycle ratio τ (how much of each cycle the plasma is on), and the phase difference φ between the upper and lower actuators.
We wanted the single best combination of those three numbers. A proper grid search would have needed on the order of 10,000 unsteady CFD simulations; each one took about 20 hours on 90 CPU cores. That was not happening. So we built a surrogate-assisted evolutionary reinforcement learning (SA-ERL) framework: a GP-Transformer surrogate stands in for the CFD solver, a genetic algorithm explores globally, a PPO agent refines locally, and the surrogate’s own uncertainty decides where to spend the next real simulation.
The answer it found — f = 20 Hz, τ = 0.9, φ = 0 — cut mean drag by 52.1%, suppressed lift fluctuations by 97%, and used about 10% less power than continuous forcing. Just as important, the comparison of optimizers gave a clear, data-backed answer to a question people argue about online: genetic algorithms and deep reinforcement learning each fail in a characteristic way, and combining them fixes both failure modes.
Results at a glance (Re = 200, verified by full CFD re-simulation)
| Quantity | Baseline | Optimized (ERL) | Change |
|---|---|---|---|
| Mean drag coefficient Cd,avg | 1.44 | 0.69 | −52.1% |
| Lift fluctuation Cl,rms | 0.44 | 0.014 | −97% |
| Normalized actuation power vs. steady forcing | — | — | about −10% |
Reference: Table VII of the paper. Continuous (steady) forcing reaches slightly better aerodynamics (≈55% drag reduction) but consumes more power; the duty-cycled optimum trades ~3 points of drag reduction for ~10% energy savings plus two extra control dimensions.
Why three AI methods, not one
Each ingredient in the framework exists because the others fail without it.
The GP-Transformer surrogate: a fast stand-in that knows when it is guessing
The surrogate maps the three actuation parameters to the aerodynamic responses (mean drag, lift RMS, and a normalized power metric). A Transformer encoder learns the nonlinear couplings between frequency, duty ratio, and phase; a variational Gaussian Process head outputs a predictive mean and a calibrated uncertainty. On held-out data the model reached R² = 0.9976 for mean drag and 0.9212 for lift fluctuations. The uncertainty number is what makes the rest of the loop honest: whenever the optimizer wanders into a region the model is unsure about, a real CFD run is scheduled there and the model is retrained.
Why a Transformer on top of a GP at all? Because optimizers spend most of their time querying the sparse, poorly sampled edges of the design space. In a boundary-extrapolation benchmark, the GP-Transformer kept lift-response R² above 0.9 where a standard GP dropped to 0.735 and a deep-kernel model to 0.45 (Appendix A of the paper). That robustness is the reason we trusted it inside the optimization loop.
The genetic algorithm: excellent explorer, poor finisher
The GA (population 100, tournament selection, blend crossover, Gaussian mutation, 5 elites) found a competitive solution within its first ten generations — and then essentially stopped improving. Over the remaining 1,990 generations of its budget it stayed locked at a duty-cycle frequency of 154 Hz, which is simply not where the optimum lives. Classic diversity collapse: once good solutions dominate the population, the GA stops visiting new regions.
Standalone DRL: excellent refiner, hesitant explorer
The PPO agent (actor-critic with two 128-neuron hidden layers, tanh-squashed Gaussian policy) did eventually find the low-frequency regime, but it took about 55 epochs, it converged to a noisier solution (f = 40 Hz, φ ≈ 0.5), and — the telling detail — it never settled the phase difference. The reward surface near φ = 0 is flat, and without external pressure the agent kept wandering around φ ≈ 0.5, which left it with residual lift fluctuations of 0.05, more than three times the ERL value.
ERL: the two cover for each other
In the ERL configuration, the GA runs five generations per evolutionary phase while PPO trains in parallel, and the two exchange information: elite GA individuals are injected into the PPO replay buffer, and the PPO policy generates mutation directions for part of the GA population. The outcome was consistent across every metric we tracked: ERL reached the best terminal reward (−0.51 vs −0.53), converged about 20 epochs earlier than standalone DRL, fluctuated the least, and — unlike DRL — locked onto φ = 0 smoothly by epoch 35. The GA branch supplied exactly the exploration pressure the flat reward region needed.
The physics of why 20 Hz, 0.9, and zero phase work
A square cylinder’s vortex street needs top-bottom asymmetry to exist. When both actuators fire in phase, they add momentum symmetrically, the separated shear layers elongate and reattach along the side walls, and the wake collapses into a steady, symmetric recirculation bubble instead of shedding vortices.
The spectral analysis makes this visible. In the baseline wake the natural shedding frequency (about 6 Hz) dominates at all three probe locations. Under the optimized actuation, that peak disappears; the strongest response is at the imposed 20 Hz forcing (−53 dB/Hz at the near-wake probe), and everything downstream is a broad, weak residual. The 20 Hz forcing dominates the natural instability — which is also why lock-in and resonance become unlikely. Mean streamlines and velocity profiles confirm the recirculation region is weakened and pushed downstream, and the near-wake turbulence intensity drops substantially.
Does it transfer to other conditions? A Re = 1000 check
We re-ran the entire pipeline at Re = 1000 with identical hyperparameters (Appendix B). The optimizer ranking repeated — GA stagnates first, DRL converges slowly with fluctuations, ERL finishes best — and the framework found a physically sensible new optimum: 400 Hz, τ = 0.91, φ ≈ 0.17 rad, giving 13.9% drag and 14.3% lift-fluctuation reduction against a much harder baseline (Cd = 2.01). The duty ratio stayed near 0.9, but the frequency scaled with the operating condition. We read that as a sign the framework is doing physics-aware search rather than memorizing one answer.
Honest limitations
Three things this paper does not do. First, everything is two-dimensional; three-dimensional wake instabilities at higher Reynolds numbers are out of scope. Second, the actuator is a Suzen–Huang electrohydrodynamic body-force model with fixed parameters (9 kV peak-to-peak, 1 kHz carrier, charge density 0.002 C/m³) — detailed plasma kinetics and thermal effects are neglected, and the power metric is a voltage-scaling estimate, not a measured one. Third, the duty-cycle frequency and phase conclusions are tied to this configuration; the framework is designed to re-optimize when the setup changes, which is a feature, but it means the specific numbers (20 Hz, 0.9, 0) should not be quoted as universal constants.
Cite the paper
Hamza, M., Nawaz, F., Ali, K., & Meng, X. “Optimization of duty-cycled plasma actuation parameters over a square cylinder using surrogate-assisted evolutionary reinforcement learning.” Physics of Fluids 38(11), 115138 (2026). pubs.aip.org · doi:10.1063/5.0352993
@article{Hamza2026SAERL,
author = {Hamza, Muhammad and Nawaz, Fahad and Ali, Kashif and Meng, Xuanshi},
title = {Optimization of duty-cycled plasma actuation parameters
over a square cylinder using surrogate-assisted
evolutionary reinforcement learning},
journal = {Physics of Fluids},
volume = {38},
number = {11},
pages = {115138},
year = {2026},
doi = {10.1063/5.0352993}
}Funding: National Natural Science Foundation of China (12472275, U2570219); National Key Laboratory of Aircraft Configuration Design (ZZKY-202501); 111 Project (B17037).
Frequently asked questions
What is evolutionary reinforcement learning (ERL)?
A hybrid optimizer that runs a genetic algorithm and a deep reinforcement learning agent (here, PPO) together, exchanging information in both directions. Elite GA solutions are injected into the PPO replay buffer, and the trained policy steers GA mutations. In this study ERL converged about 20 epochs faster than standalone DRL, reached a higher final reward, and settled the phase parameter cleanly while DRL kept oscillating.
Is deep reinforcement learning better than genetic algorithms for flow control?
Neither, on its own. On identical evaluation budgets, the GA plateaued after ~10 generations (stuck at 154 Hz), while standalone DRL converged slowly (~55 epochs) and never settled the phase difference. ERL beat both. If you are choosing between GA and DRL for an expensive CFD problem, the evidence here says: use both.
How much can DBD plasma actuators reduce drag on a square cylinder?
At Re = 200: 52.1% mean-drag reduction and 97% lift-fluctuation suppression, verified by full CFD re-simulation. At Re = 1000: 13.9% and 14.3% respectively, with a different optimal frequency (400 Hz vs 20 Hz). Steady continuous forcing achieved ~55% drag reduction but used ~10% more power.
What is a GP-Transformer surrogate model?
A surrogate that couples a Transformer encoder (which learns nonlinear interactions between input parameters) with a variational Gaussian Process head (which outputs predictions with calibrated uncertainty). The uncertainty drives the active-learning loop: expensive CFD runs are scheduled where the model is least confident. Achieved R² of 0.9976 (drag) and 0.9212 (lift fluctuations) on held-out data.
What is active flow control?
Modifying a flow in real time with external energy input. Here: two DBD plasma actuators at the trailing-edge corners of a square cylinder add momentum near the wall through an electrohydrodynamic body force. No moving parts, millisecond response.
Steady vs. duty-cycled plasma actuation — which is better?
For this configuration, duty-cycled. Steady forcing gives ~55% drag reduction; the optimized duty cycle gives ~52% while using ~10% less power, and it adds two controllable dimensions (frequency and phase). The paper recommends duty-cycled actuation for this setup.
Can deep reinforcement learning replace CFD?
No — CFD remains the ground truth. What the surrogate removes is the number of CFD evaluations: roughly 300 simulations instead of ~10,000. Every reported optimum was still re-verified with the full solver.
What is machine learning for fluid mechanics used for?
Broadly: active flow control, shape optimization, turbulence modeling, and reduced-order modeling. This paper belongs to active flow control, where learning algorithms search strategies that are hard to find by hand because the actuation parameters interact nonlinearly.
How many CFD simulations does surrogate-assisted optimization need?
Of order 10² here (300 initial LHS samples plus uncertainty-guided infills), versus ~10⁴ for a resolved grid search. At ~20 h per run on 90 cores, that difference is the entire reason the study was feasible.
What is AC-DBD plasma flow control using deep learning?
AC-DBD actuators generate a near-wall jet from a high voltage across a dielectric-covered electrode pair. “Using deep learning” means a reinforcement-learning agent picks the actuation parameters — duty-cycle frequency, duty ratio, phase difference — by interacting with a CFD model of the flow, instead of hand-tuning them.
What are the strengths and weaknesses of GAs and DRL?
GA: robust global search, gradient-free, but diversity collapse causes stagnation. DRL: directed local refinement of continuous parameters, sample-efficient near good solutions, but slow and oscillatory without exploration pressure. ERL pairs them so each covers the other’s weakness.
Is this paper peer reviewed? Where do I read it?
Yes — submitted 19 July 2026, accepted 30 October 2026, published 13 November 2026 in Physics of Fluids 38(11), 115138, AIP Publishing. DOI 10.1063/5.0352993, linked above.
What are the study’s limitations?
Two-dimensional simulations; a reduced-order body-force actuator model without plasma kinetics or thermal coupling; main results at Re = 200 (with a Re = 1000 supplementary validation). Optimized parameters are configuration-specific and should be re-optimized for other setups.
How should I cite this paper?
Hamza, M., Nawaz, F., Ali, K., & Meng, X. (2026). Optimization of duty-cycled plasma actuation parameters over a square cylinder using surrogate-assisted evolutionary reinforcement learning. Physics of Fluids, 38(11), 115138. https://doi.org/10.1063/5.0352993
Well written research article for combining evolutionary algorithms with reinforcement learning and developing a hybrid evolutionary reinforcement learning framework for flow control technology in the field of fluid mechanics. This article is really helpful and helping in literature study for computational fluid dynamics.
Thank you for your compliment on this the paper related to deep reinforcement learning , genetic algorithms and evolutionary reinforcement learning application in fluid mechanics.