Teaching AI to Tame Turbulence: How Evolutionary Reinforcement Learning Cut Drag by 52% on a Square Cylinder
Genetic algorithms, deep reinforcement learning, and a GP-Transformer surrogate joined forces to find the perfect plasma-actuator pulse settings — with 100× fewer CFD simulations than brute force ever could.

⚡ The Crux — 60-Second Summary
- The problem: Wind flowing past a square cylinder creates a chaotic, energy-wasting vortex street. Tiny plasma actuators can fix this — but no one knew the best settings for their on/off pulsing.
- The idea: We combined a genetic algorithm (broad exploration) with deep reinforcement learning (smart local refinement), guided by an AI “crystal ball” (a GP-Transformer surrogate) that predicts flow behavior without running expensive simulations.
- The result: 52% less drag, 97% calmer lift forces, and 10% less power than the best continuous strategy — using 100× fewer supercomputer simulations.
- Why it matters: A general recipe for optimizing any expensive physics problem — from aircraft design to energy systems — not just plasma flow control.
- Where: Physics of Fluids 38, 115138 (2026) — read the full paper
Imagine you want to reduce the drag on a bridge pier, a building, or a vehicle — not by reshaping it, but by blowing a tiny, invisible “wind” along its surface using nothing but ionized air. That technology — called a dielectric barrier discharge (DBD) plasma actuator — exists today. It has no moving parts, responds in milliseconds, and can suppress the chaotic vortex street that forms behind bluff bodies.
There’s just one problem: plasma actuators have too many knobs to turn.
In a new paper published in Physics of Fluids (38, 115138, 2026), our team at Northwestern Polytechnical University set out to answer a deceptively simple question: what is the single best way to pulse a pair of plasma actuators to minimize drag and kill lift oscillations behind a square cylinder? The answer required combining three of the most powerful ideas in modern optimization — genetic algorithms, deep reinforcement learning, and surrogate modeling — into one framework we call surrogate-assisted evolutionary reinforcement learning (SA-ERL).
This is the story of how it works, why each ingredient matters, and what it means for the future of AI-driven active flow control.
📋 Table of Contents
- 01The Problem: A 3D Needle in a 10,000-Simulation Haystack
- 02The Framework: Three AI Techniques, One Brain
- 03The Results: 52% Drag Cut, 97% Calmer Lift
- 04Does It Generalize? The Re = 1000 Test
- 05Explore Our Related Research
- 06Why This Matters Beyond Plasma Actuators
- 07The Bottom Line
- 08Meet the Research Team
- 09Frequently Asked Questions
- 10How to Implement GA, DRL, ERL & Surrogates for CFD
The Problem: A 3-Dimensional Needle in a 10,000-Simulation Haystack
When a fluid flows past a square cylinder at a Reynolds number of 200, it does something beautiful and annoying: it sheds alternating vortices — the famous von Kármán vortex street. These vortices create oscillating lift forces (which cause vibration and fatigue) and a large pressure drag (which wastes energy).
Our plasma actuators, mounted at the trailing-edge corners of the cylinder, don’t run continuously. Instead, they operate in duty-cycled mode, defined by three parameters:
- Frequency (f): how often the actuator switches on and off (we searched 1–1000 Hz)
- Duty-cycle ratio (τ): the fraction of each cycle the plasma is active (0.1–1.0)
- Phase difference (φ): the timing offset between the upper and lower actuators (0–2π)
These three parameters interact in nasty, nonlinear ways. A frequency that works brilliantly at τ = 0.9 may do nothing at τ = 0.5. A phase difference that suppresses vortices at one frequency can amplify them at another.
The brutal math: mapping this three-dimensional space with direct high-fidelity CFD simulations would need roughly 10,000 unsteady Navier–Stokes simulations. At 20 hours per simulation on 90 CPU cores, that’s a supercomputer-month. Nobody has that budget — not academia, not industry.
The Framework: Three AI Techniques, One Brain
Our SA-ERL framework (Fig. 6 in the paper) has four stages, each solving a specific failure mode of the others.
1. A GP-Transformer Surrogate: The “Crystal Ball”
The first breakthrough is a hybrid Gaussian Process–Transformer surrogate model — a neural network that speaks two languages.
A Transformer (the same architecture family that powers large language models like GPT) encodes the three actuation parameters through self-attention layers, learning the cross-parameter couplings that simpler models miss — for example, how frequency and phase difference jointly govern wake response. A variational Gaussian Process regression head then converts this latent representation into probabilistic predictions: not just “drag will be 0.69,” but “drag will be 0.69 ± 0.02, and I’m 95% confident.”
This uncertainty estimate is the secret weapon. It drives an active learning loop: whenever the model is uncertain about a region of parameter space, a new high-fidelity CFD simulation is run there to fill the gap. The result? The surrogate achieves R² > 0.99 for mean drag and R² > 0.92 for lift fluctuations on held-out data, with dramatic robustness near the boundaries of the design space — exactly where optimizers love to wander.
2. The Genetic Algorithm: The Wide-Eyed Explorer
We benchmarked three optimizers head-to-head on identical computational budgets (2×105 evaluations):
- Genetic Algorithm (GA): population-based global search. It explores broadly, but its population diversity decays over time — it gets “stuck” near its first good solution. In our runs, GA locked onto a suboptimal frequency of 154 Hz after just 10 generations and never escaped for the remaining 1,990.
- Standalone Deep Reinforcement Learning (PPO): gradient-informed local refinement. It learns efficiently from experience, but without exploration pressure it converges slowly and oscillates on flat reward landscapes.
- Evolutionary Reinforcement Learning (ERL): the best of both. A GA population maintains global exploration while a Proximal Policy Optimization (PPO) agent provides directed local refinement, with bidirectional information exchange — elite GA solutions are injected into the PPO replay buffer, and the PPO policy steers GA mutations.
The result was decisive. ERL reached a higher terminal reward (−0.51 vs −0.53), converged 20 epochs faster than DRL, and showed dramatically smaller fluctuations — because the GA branch prevented the policy from overfitting to narrow local reward ridges, while the PPO branch refined what GA found.
If you’ve ever searched for “genetic algorithm vs deep reinforcement learning,” this paper answers it with data: neither wins alone. Evolution wins when evolution and learning collaborate.
3. The Active Learning Loop: Staying Honest
A surrogate that leads the optimizer astray is worse than no surrogate. So our framework periodically re-verifies promising candidates with the real CFD solver and retrains the model from scratch, re-optimizing the GP inducing points. The loop terminates only when both the optimal reward and the maximum epistemic uncertainty stop improving — joint convergence of the policy and the model.
The payoff: the total cost dropped from O(104) to O(102) high-fidelity simulations — a 100× reduction — with physical fidelity preserved through direct CFD verification of every reported optimum.
The Result: Physics That Reads Like a Storybook
The ERL optimizer converged to a strikingly clean strategy:
🎯 The Optimal Strategy
f = 20 Hz, τ = 0.9, φ = 0
A slow, near-continuous, perfectly symmetric pulsing — the kind of answer an experienced flow-control engineer would nod at, except no engineer tried those exact numbers.
| Metric | Baseline | Optimized (ERL) | Improvement |
|---|---|---|---|
| Mean drag coefficient (Cd,avg) | 1.44 | 0.69 | −52.1% |
| Lift fluctuations (Cl,rms) | 0.44 | 0.014 | −97% |
| Normalized actuation power | continuous | duty-cycled | −10% vs. steady forcing |
And the physics tells you why it’s right. Power spectral density analysis of the wake shows the natural shedding frequency (6 Hz) completely disappears, replaced by the 20 Hz actuation frequency — the plasma forcing dominates the natural instability, making lock-in and resonance impossible. The separated shear layers elongate and reattach along the cylinder walls; the coherent vortex street collapses into a weak, symmetric recirculation bubble pushed downstream. Velocity deficit narrows, turbulence intensity in the near wake collapses, and the lift signal goes nearly flat.
Even better: compared to steady (continuous) plasma forcing, which achieved 55% drag reduction, the duty-cycled optimum achieves nearly the same aerodynamic performance (52%) while consuming 10% less power — and it adds two tunable control dimensions (f and φ) that steady forcing lacks. For engineers searching “plasma actuator duty cycle optimization,” this is a direct, quantified answer: pulse slowly, symmetrically, and nearly continuously.
Does it generalize? A test at Re = 1000
We re-ran the entire pipeline at Re = 1000 (a turbulent regime) using identical hyperparameters. The framework recovered the same optimizer ranking and discovered a new optimum (f = 400 Hz, τ = 0.91, φ ≈ 0.17 rad) with 13.9% drag reduction — automatically adjusting the frequency to the new operating condition rather than naively reusing the Re = 200 value. Same brain, new physics, no hand-tuning.
Why This Matters Beyond Plasma Actuators
- For the optimization community: this is among the first demonstrations of ERL on a genuine physics problem with expensive black-box evaluations, not a toy benchmark. The GA-explores / PPO-refines / surrogate-verifies recipe transfers directly to aerodynamic shape optimization, combustion control, and energy systems.
- For the ML community: the GP-Transformer’s boundary robustness (lift R² > 0.9 in sparsely sampled outer regions vs. 0.735 for standard GP and 0.45 for deep kernel learning) shows that attention-based surrogates with calibrated uncertainty are ready to replace hand-crafted response surfaces in Bayesian optimization pipelines.
- For engineers: active flow control has always been limited by the cost of exploring parameter space. This framework makes multi-parameter, multi-objective optimization of unsteady actuation computationally affordable — O(10²) simulations instead of O(104).
- For searchers of “LLMs for science”: this work sits at the intersection of large-model representation learning and physics — a template for how foundation-model architectures can be distilled into task-specific, uncertainty-aware surrogates for scientific discovery.
How to Implement GA, DRL, ERL & Surrogate Models for Your Own CFD / Active Flow Control Problem
Everything in this blog is reproducible. Whether you are a graduate student, a CFD engineer, or an AI researcher looking for a challenging testbed, here is the practical roadmap our paper lays out — the same pipeline you can adapt to airfoils, turbines, combustors, or any expensive black-box physics problem.
Step 1 — Define the design space and sample it (LHS + CFD)
Start with a Latin Hypercube Sampling (LHS) design of experiments: 150–300 points give a solid initial dataset for a 3–5 dimensional parameter space. Run each configuration with your CFD solver until statistically periodic forces are obtained, and record time-averaged drag, lift RMS, and any power metric. In our study, OpenFOAM with the pimplePlasmaFoam solver handled the plasma-body-force coupling, but any unsteady RANS/LES/DNS workflow works. Always complete mesh-independence and time-step-independence studies first — a surrogate is only as honest as the data beneath it.
Step 2 — Build a surrogate (Kriging → GP → GP-Transformer)
Begin simple, then scale up. A Kriging (Gaussian Process) surrogate is the classic baseline for aerodynamic optimization and needs only a few lines with scikit-learn or GPyTorch. If your response surface has sharp ridges and strong parameter coupling — as duty-cycled plasma actuation does — graduate to a deep-kernel / Transformer-GP hybrid that learns cross-parameter interactions while still returning calibrated uncertainty. That uncertainty number is not optional: it is what makes Step 4’s active learning loop possible.
# Surrogate baseline: Gaussian Process (Kriging-style) in a few lines
import torch, gpytorch
from gpytorch.models import ExactGP
from gpytorch.means import ConstantMean
from gpytorch.kernels import RBFKernel, ScaleKernel
from gpytorch.distributions import MultivariateNormal
class GPSurrogate(ExactGP):
def __init__(self, X, y, likelihood):
super().__init__(X, y, likelihood)
self.mean_module = ConstantMean()
self.cov_module = ScaleKernel(RBFKernel(ard_num_dims=X.shape[-1]))
def forward(self, x):
return MultivariateNormal(self.mean_module(x), self.cov_module(x))
likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = GPSurrogate(train_X, train_y, likelihood)
# predict mean AND epistemic uncertainty -> drives the infill criterionStep 3 — Add a Genetic Algorithm for global search
Use DEAP or PyGAD: tournament selection, blend crossover (pc ≈ 0.7), Gaussian mutation (pm ≈ 0.3), and elitism (5–10% of the population). Evaluate fitness on the surrogate, not the CFD solver — this is where the 100× speedup comes from. Expect GA to find a competitive region fast and then plateau; that plateau is exactly what the next step fixes.
Step 4 — Add PPO and close the ERL loop
With Stable-Baselines3, a PPO agent refines promising GA elites. The two-way coupling is the whole trick: inject GA elites into the PPO replay buffer, and use the PPO policy to generate mutation directions for part of the GA population. Reward shaping matters — combine normalized drag, weighted lift RMS, and a small uncertainty bonus (≈0.02σepi) to keep exploration alive.
# DRL local refinement with PPO (Stable-Baselines3)
from stable_baselines3 import PPO
# env.step(action) -> query GP-Transformer surrogate, return shaped reward
# R_t = -(C_d,avg + 0.4*C_l,rms) + 0.02*sigma_epi(x)
model = PPO("MlpPolicy", env, n_steps=2048, batch_size=64,
learning_rate=3e-4, verbose=1)
model.learn(total_timesteps=200_000)Step 5 — Active learning: let uncertainty buy your CFD budget
After each optimization phase, pick the top-performing candidates plus the highest-uncertainty points, re-run them in CFD, retrain the surrogate from scratch, and repeat. Stop when the best reward and the maximum epistemic uncertainty both plateau. This is what keeps the whole framework physically honest — and it is how ~300 simulations replaced ~10,000.
Want a head start? The full algorithmic details, hyperparameters (Tables III–VI), and the surrogate benchmark study (Appendix A) are in the open-access article — cite it, replicate it, and extend it.
The Bottom Line
We didn’t just find a good set of plasma actuator settings. We demonstrated a general-purpose recipe for optimizing expensive, nonlinear, multi-parameter physics problems: let a Transformer-GP surrogate stand in for the simulator, let evolution and reinforcement learning search together, and let uncertainty decide where to spend the next simulation. The wake behind a square cylinder is simply where we chose to prove it.
The von Kármán vortex street took a century to understand. It took AI about 300 simulations to tame it — and 10% less power than the best steady strategy known.
📰 Published in Physics of Fluids (AIP Publishing)
Physics of Fluids is a leading international journal dedicated to original theoretical, computational, and experimental contributions to fluid mechanics, published by AIP Publishing. This article appears in Volume 38, Issue 11 (November 2026), article 115138.
→ Read the full open article on pubs.aip.org · Explore the journal
📄 Cite This Work
M. Hamza, F. Nawaz, K. Ali, and X. Meng, “Optimization of duty-cycled plasma actuation parameters over a square cylinder using surrogate-assisted evolutionary reinforcement learning,” Physics of Fluids 38(11), 115138 (2026). pubs.aip.org
@article{10.1063/5.0352993,
author = {Hamza, Muhammad and Nawaz, Fahad and Ali, Kashif and Meng, Xuanshi},
title = {Optimization of duty-cycled plasma actuation parameters over a square cylinder using surrogate-assisted evolutionary reinforcement learning},
journal = {Physics of Fluids},
volume = {38},
number = {11},
pages = {115138},
year = {2026},
month = {11},
issn = {1070-6631},
doi = {10.1063/5.0352993},
url = {https://doi.org/10.1063/5.0352993}
}Meet the Research Team
This work comes from the Department of Aerodynamics, School of Aeronautics, Northwestern Polytechnical University (Xi’an, China), in collaboration with the National Key Laboratory of Aircraft Configuration Design.
Frequently Asked Questions
How can I implement GA, DRL, or ERL for my own flow control problem?
Follow the five-step roadmap in the implementation section above: (1) sample the parameter space with Latin hypercube sampling and run CFD, (2) train a Gaussian Process / Kriging surrogate, (3) attach a genetic algorithm, (4) couple it with PPO for local refinement, and (5) close an active-learning loop using the surrogate’s uncertainty. All hyperparameters are listed in the paper.
What is evolutionary reinforcement learning (ERL)?
A hybrid optimization framework that combines a genetic algorithm (global exploration) with deep reinforcement learning (local refinement), exchanging elite solutions and policy-guided mutations between the two.
How much can plasma actuation reduce drag on a square cylinder?
In this study, optimized duty-cycled DBD actuation at Re=200 reduced mean drag by 52% and lift oscillations by 97%, while using about 10% less power than continuous forcing.
What is a GP-Transformer surrogate model?
A hybrid AI model combining a Transformer encoder (for learning nonlinear parameter interactions) with a Gaussian Process regression head (for probabilistic predictions with calibrated uncertainty), used here to replace expensive CFD simulations.
Is deep reinforcement learning better than genetic algorithms for flow control?
Neither alone: standalone DRL converged slowly with persistent oscillations, while GA stagnated at a suboptimal solution. The combined ERL framework outperformed both, reaching a higher reward with 20 fewer epochs.
Can this optimization method be applied to other Reynolds numbers?
Yes — re-running the identical framework at Re=1000 yielded a new effective optimum (400 Hz) with 13.9% drag reduction, demonstrating operating-condition adaptability.
