Meta used AI agents for 1,097 experiments, gaining 0.23% in search activity
A Meta research paper describes a month-long automated campaign that used about 360,000 GPU-hours and produced a small live-traffic gain, along with a slower search interface.
· 2 min read
- Meta's agents ran 1,097 experiments in 31 days and had 325 accepted, a 29.6% acceptance rate.
- The campaign used roughly 360,000 GPU-hours and lifted search-session volume by 0.23% in live testing.
- The final model slowed search interface rendering, and Meta did not disclose the size of the slowdown or the financial cost.
Meta researchers used AI agents to run 1,097 machine-learning experiments over 31 days, according to a report by CryptoSlate. The campaign consumed about 360,000 GPU-hours and raised search activity by 0.23%.
The work is described in a research paper dated Nov. 18. It shows how autonomous systems can build and test improvements to large recommendation models with limited human involvement. The researchers built a setup in which agents suggest changes to a model, edit its code, train new versions and evaluate the results.
The target was Meta's search-suggestion system, which decides what users see when they open the search bar before typing anything. Improving such a system normally means engineers must find possible changes, implement them and measure them through repeated training and testing. Meta automated much of that loop, while engineers kept control of research priorities and final deployment decisions.
The agents relied on Anthropic's Claude Opus 4.8 to explore architectural changes and produce candidates. Of the 1,097 candidate experiments, 325 were accepted, an acceptance rate of 29.6%. An earlier version of the system, which tested changes one after another instead of in parallel, had an acceptance rate of 15.6%. The researchers also said the agents found several architectural improvements that Meta's engineers had not explored before.
Meta put total compute use at roughly 360,000 GPU-hours. That figure covers completed experiments, extra tests that measured training variability, and failed or retried jobs. Because single training and evaluation runs took several hours, the researchers said parallel execution was needed to finish within a month.
After the accepted changes were combined into one model, Meta compared it with its existing search-ranking system on live traffic. Search-session volume rose by 0.23%, with a 95% confidence interval of plus or minus 0.15 percentage points. Other engagement measures, including participation and retention, also moved in a positive direction. The report notes that the platform handles more than a billion search-ranking requests a day, so a small gain could matter operationally.
There was a drawback. The new model made the search interface slower to render, though the researchers did not say by how much. The agents had optimized against offline measures of search engagement, while rendering speed was tracked separately. The researchers acknowledged that the system cannot identify every trade-off between its assigned targets and other parts of the user experience.
Meta did not disclose the financial cost of the campaign or compare its resource use with that of a conventional engineering team. The researchers still argued that the results show autonomous agents can do meaningful machine-learning engineering on production-scale systems. According to CryptoSlate, Meta now has to weigh the extra search activity and engineering productivity against the computing resources and slower interface before wider adoption.