Anything being done about people submitting PRs at very high volume?

[pragyaangaur]

Been noticing over the last couple of weeks that the repository is getting a pretty large volume of PRs, sometimes several from the same contributor in a single day.

Not trying to target anyone because I’m also new and learning, but I’ve mainly noticed this from @rois1995 and @ManasBagul23. Some of these PRs/commits also seem to be directly AI-generated or coauthored using LLMs.

I don’t think using AI itself is a problem. I’ve used it for the last couple of PRs I’ve put in too. My concern is mostly that it now makes it possible to submit a very large number of changes very quickly, while every PR still needs maintainer time to review.

Is there already some policy around this, or would it make sense to have some kind of limit/guideline on the number of PRs a contributor should have open at once?

[rois1995]

Sorry for the large volume of PRs. I have slowed down since talking with @bigfooted as I am aware of the burden of reviewing all of them. Most of the latest PRs resulted from splitting a larger one, so they are a cascade of bug fixes Claude found.

Unfortunately, I don't see a slowdown in the future either. I have already prepared a bunch of other PRs for bug fixes only related to the Adjoint, MUSCL_TURB, convergence acceleration, etc. I'll set aside the new-feature PR for now so the bug-fix PRs are well commented, tested, and easier to review. I will also help review other PRs to ease the overall workload.

I'll discuss the findings in the developer meeting so we can make a list of priority PRs to focus on and make everything smoother.

[pragyaangaur]

Thanks for the context! That makes sense, especially if you’re already coordinating with the maintainers and prioritising the PRs.

I think my concern is more general than these particular PRs though. As AI makes it possible to find and prepare fixes much faster, it might be useful to have some project-level guidelines around AI-assisted contributions and PR volume, so this doesn’t have to be figured out case by case.

I’m also not sure if SU2 currently has any AI disclosure policy. SunPy, for example, where I'm also a contributor, asks contributors to include a declaration explaining how AI was used in a contribution. Maybe something similar could be useful here too, even if it’s just a short disclosure in the PR template.

Otherwise, if it’s purely a matter of having AI find and prepare fixes, at this rate we could probably finish SU2 development in a month lol.

Would be interested to hear what the maintainers think, especially if this comes up at the developer meeting.

[bigfooted]

@pragyaangaur These 2 contributors that you mention show the difference between good and bad. ManasBagul23 is completely unknown, never participated in the community, suddenly jumps in and starts producing PR's that might technically solve an issue but they look like they were found with general 'find a bug' sweeps. It also looked like he was not a real user, just harvesting PRs. I have blocked him.

On the other end we have rois1995, who is well known in the community, participates in developer meetings and the su2 conference and is developing and improving significant parts of the SU2 codebase. We are also actively discussing how to deal with the recent increase in productivity because in the hands of an experienced researcher, claude and other AI tools can be very powerful and increase productivity.

We do not have an AI policy yet, I personally think it's completely fine to use AI as long as the end result is verifiably correct and is actually of benefit to the users. And I'd rather focus on helping a real user with real problems than a PR harvester who never actually compiled SU2 and is just solving non-issues that do not actually help users. And this is rather difficult to capture in an AI policy I fear because it's also about intention and not just about submitting a technically correct fix.

[pragyaangaur]

Thanks for the context, that distinction makes a lot of sense. I was mostly looking at it from the volume side and didn’t have the context about either contributor.

Agreed that an AI policy alone probably wouldn’t capture the actual problem here. Thanks for clarifying!