Connect with us


AI smokes 5 poker champs at a time in no-limit Hold’em with ‘relentless consistency’ – TechCrunch



The machines have proven their superiority in one-on-one games like chess and go, and even poker — but in complex multiplayer versions of the card game humans have retained their edge… until now. An evolution of the last AI agent to flummox poker pros individually is now decisively beating them in championship-style 6-person game.

As documented in a paper published in the journal Science today, the CMU/Facebook collaboration they call Pluribus reliably beats five professional poker players in the same game, or one pro pitted against five independent copies of itself. It’s a major leap forward in capability for the machines, and amazingly is also far more efficient than previous agents as well.

One-on-one poker is a weird game, and not a simple one, but the zero-sum nature of it (whatever you lose, the other player gets) makes it susceptible to certain strategies in which computer able to calculate out far enough can put itself at an advantage. But add four more players into the mix and things get real complex, real fast.

With six players, the possibilities for hands, bets, and possible outcomes are so numerous that it is effectively impossible to account for all of them, especially in a minute or less. It’d be like trying to exhaustively document every grain of sand on a beach between waves.

Yet over 10,000 hands played with champions, Pluribus managed to win money at a steady rate, exposing no weaknesses or habits that its opponents could take advantage of. What’s the secret? Consistent randomness.

Even computers have regrets

Pluribus was trained, like many game-playing AI agents these days, not by studying how humans play but by playing against itself. At the beginning this is probably like watching kids, or for that matter me, play poker — constant mistakes, but at least the AI and the kids learn from them.

The training program used something called Monte Carlo counterfactual regret minimization. Sounds like when you have whiskey for breakfast after losing your shirt at the casino, and in a way it is — machine learning style.

Regret minimization just means that when the system would finish a hand (against itself, remember), it would then play that hand out again in different ways, exploring what might have happened had it checked here instead of raised, folded instead of called, and so on. (Since it didn’t really happen, it’s counterfactual.)

A Monte Carlo tree is a way of organizing and evaluating lots of possibilities, akin to climbing a tree of them branch by branch and noting the quality of each leaf you find, then picking the best one once you think you’ve climbed enough.

If you do it ahead of time (this is done in chess, for instance) you’re looking for the best move to choose from. But if you combine it with the regret function, you’re looking through a catalog of possible ways the game could have gone and observing which would have had the best outcome.

So Monte Carlo counterfactual regret minimization is just a way of systematically investigating what might have happened if the computer had acted differently, and adjusting its model of how to play accordingly.

The game originall played out as you see on the left, with a loss. But the engine explores other avenues where it might have done better.

Of course the number of games is nigh-infinite if you want to consider what would happen if you had bet $101 rather than $100, or you would have won that big hand if you’d had an eight kicker instead of a seven. Therein also lies nigh-infinite regret, the kind that keeps you in bed in your hotel room until past lunch.

The truth is these minor changes matter so seldom that the possibility can basically be ignored entirely. It will never really matter that you bet an extra buck — so any bet within, say, 70 and 130 can be considered exactly the same by the computer. Same with cards — whether the jack is a heart or a spade doesn’t matter except in very specific (and usually obvious) situations, so 99.999 percent of the time the hands can be considered equivalent.

This “abstraction” of gameplay sequences and “bucketing” of possibilities greatly reduces the possibilities Pluribus has to consider. It also helps keep the calculation load low; Pluribus was trained on a relatively ordinary 64-core server rack over about a week, while other models might take processor-years in high-power clusters. It even runs on a (admittedly beefy) rig with two CPUs and 128 gigs of RAM.

Random like a fox

The training produces what the team calls a “blueprint” for how to play that’s fundamentally strong and would probably beat plenty of players. But a weakness of AI models is that they develop tendencies that can be detected and exploited.

In Facebook’s writeup of Pluribus, it provides the example of two computers playing rock-paper-scissors. One picks randomly while the other always picks rock. Theoretically they’d both win the same amount of games. But if the computer tried the all-rock strategy on a human, it would start losing with a quickness and never stop.

As a simple example in poker, maybe a particular series of bets always makes the computer go all in regardless of its hand. If a player can spot that series, they can take the computer to town any time they like. Finding and preventing ruts like these is important to creating a game-playing agent that can beat resourceful and observant humans.

To do this Pluribus does a couple things. First, it has modified versions of its blueprint to put into play should the game lean towards folding, calling, or raising. Different strategies for different games mean it’s less predictable, and it can switch in a minute should the bet patterns change and the hand go from a calling to a bluffing one.

It also engages in a short but comprehensive introspective search looking at how it would play if it had every other hand, from a big nothing up to a straight flush, and how it would bet. It then picks its bet in the context of all those, careful to do so in such a way that it doesn’t point to any one in particular. Given the same hand and same play again, Pluribus wouldn’t choose the same bet, but rather vary it to remain unpredictable.

These strategies contribute to the “consistent randomness” I alluded to earlier, and which were a part of the model’s ability to slowly but reliably put some of the best players in the world.

The human’s lament

There are too many hands to point to a particular one or ten that indicate the power Pluribus was bringing to bear on the game. Poker is a game of skill, luck, and determination, and one where winners emerge after only dozens or hundreds of hands.

And here it must be said that the experimental setup is not entirely reflective of an ordinary 6-person poker game. Unlike a real game, chip counts are not maintained as an ongoing total — for every hand, each player was given 10,000 chips to use as they pleased, and win or lose they were given 10,000 in the next hand as well.


The interface used to play poker with Pluribus. Fancy!

Obviously this rather limits the long-term strategies possible, and indeed “the bot was not looking for weaknesses in its opponents that it could exploit,” said Facebook AI research scientist Noam Brown. Truly Pluribus was living in the moment the way few humans can.

But simply because it was not basing its play on long-term observations of opponents’ individual habits or styles does not mean that its strategy was shallow. On the contrary, it is arguably more impressive, and casts the game in a different light, that a winning strategy exists that does not rely on behavioral cues or exploitation of individual weaknesses.

The pros who had their lunch money taken by the implacable Pluribus were good sports, however. They praised the system’s high level play, its validation of existing techniques, and inventive use of new ones. Here’s a selection of laments from the fallen humans:

I was one of the earliest players to test the bot so I got to see its earlier versions. The bot went from being a beatable mediocre player to competing with the best players in the world in a few weeks. Its major strength is its ability to use mixed strategies. That’s the same thing that humans try to do. It’s a matter of execution for humans — to do this in a perfectly random way and to do so consistently. It was also satisfying to see that a lot of the strategies the bot employs are things that we do already in poker at the highest level. To have your strategies more or less confirmed as correct by a supercomputer is a good feeling. -Darren Elias

It was incredibly fascinating getting to play against the poker bot and seeing some of the strategies it chose. There were several plays that humans simply are not making at all, especially relating to its bet sizing. -Michael ‘Gags’ Gagliano

Whenever playing the bot, I feel like I pick up something new to incorporate into my game. As humans I think we tend to oversimplify the game for ourselves, making strategies easier to adopt and remember. The bot doesn’t take any of these short cuts and has an immensely complicated/balanced game tree for every decision. -Jimmy Chou

In a game that will, more often than not, reward you when you exhibit mental discipline, focus, and consistency, and certainly punish you when you lack any of the three, competing for hours on end against an AI bot that obviously doesn’t have to worry about these shortcomings is a grueling task. The technicalities and deep intricacies of the AI bot’s poker ability was remarkable, but what I underestimated was its most transparent strength – its relentless consistency. -Sean Ruane

Beating humans at poker is just the start. As good a player as it is, Pluribus is more importantly a demonstration that an AI agent can achieve superhuman performance at something as complicated as 6-player poker.

“Many real-world interactions, such as financial markets, auctions, and traffic navigation, can similarly be modeled as multi-agent interactions with limited communication and collusion among participants,” writes Facebook in its blog.

Yes, and war.

Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published.


Sega packs unreleased games, new arcade ports into Genesis Mini 2



A trailer detailing all the games on the Sega Genesis Mini 2, some of which are fresh releases or ports.

After revealing just 23 named titles back in July, Friday morning Sega announced the full lineup of 60 games that will be included on the limited supply of US Sega Genesis Mini 2 units starting on October 27. Beyond the usual retro suspects, though, that list includes a couple of games that have never been released in any form, as well as several fresh arcade ports and Genesis titles sporting brand-new features for their plug-and-play re-release.

Enlarge / The previously unreleased Devi & Pii looks like an interesting, competitive take on a Breakout-style game.

Those unreleased retro games include Devi & Pii, a title designed by Sonic 3 developer Takashi Iizuka. The “paddle-style game” looks like something of a cross between Arkanoid and Twinkle Star Sprites, with one or two players shifting back and forth to juggle angels and avoid bouncing devils.

The Genesis Mini 2 will also see the worldwide premier of Star Mobile, a game completed in 1992 by little-known journeyman developer Mindware but never actually released. The puzzle-heavy gameplay involves stacking stars on a carefully balanced mobile in a way that reminds us of the tabletop game Topple.

Besides those two never-before-seen titles, the Genesis Mini 2 features a few Sega arcade games that are being “ported” to Genesis-level hardware for the first time. These include:

  • Fantasy Zone: The cute-and-cuddly side-scrolling shooter gets ported to the Genesis by the same team that ported Darius on the first Genesis Mini, with a brand-new Easy Mode that wasn’t in the arcades.
  • Space Harrier and Space Harrier II: While the sequel was already technically native to the Genesis, these new ports use “modern technology” to provide a much smoother sprite scaling function than was previously possible on 16-bit hardware (it’s unclear if these new ROMs could run on a standard Genesis).
  • Spatter: A little-known 1984 maze game featuring a clown on a bouncing tricycle.
  • Super Locomotive: A 1982 train game focused on switching tracks to avoid collisions.
  • VS Puyo Puyo Sun: A competitive two-player-exclusive “demake” of the third game in the popular color-matching puzzle series, with “new rules not found in the original version.”

The version of <em>Outrun</em> on the Genesis Mini 2 will feature new music.
Enlarge / The version of Outrun on the Genesis Mini 2 will feature new music.

Other titles that were first released on the Genesis decades ago are getting brand-new features for their Genesis Mini 2 re-releases. These include:

  • Hellfire: Sega’s trailer talks up unspecified “new features and options for the Genesis.”
  • OutRun: The four in-game radio stations will now include “new [music] tracks not featured in the original.”
  • Phantasy Star II: The classic RPG now features “adjustable movement speed and a new easy mode.”
  • Rainbow Islands Extra: The “Extra mode” with brand-new enemies comes to North America for the first time.
  • Truxton: Comes with a new option to play background music “at its original arcade speed.”

None of these titles or additions may have the sheer star power of the shelved Star Fox 2 that finally appeared on the Super NES Classic Edition. Still, taken together, they show a certain level of care and attention to detail on Sega’s part that goes well beyond just throwing some ROMs on a cheap emulation box.

The complete list of titles available on the Sega Genesis Mini is included below (titles that were previously announced in July are marked with a *).

Genesis titles

  • After Burner II *
  • Alien Soldier *
  • Atomic Runner
  • Bonanza Bros. *
  • ClayFighter
  • Crusader of Centy
  • Desert Strike: Return to the Gulf
  • Earthworm Jim 2
  • Elemental Master
  • Fatal Fury 2
  • Gain Ground
  • Golden Axe II
  • Granada
  • Hellfire
  • Herzog Zwei
  • Lightening Force: Quest for the Darkstar *
  • Midnight Resistance
  • OutRun *
  • OutRunners *
  • Phantasy Star II
  • Populous
  • Rainbow Islands Extra
  • Ranger-X
  • Ristar
  • Rolling Thunder 2 *
  • Shadow Dancer: The Secret of Shinobi
  • Shining Force II
  • Shining in the Darkness *
  • Sonic 3D Blast *
  • Splatterhouse 2 *
  • Streets of Rage 3
  • Super Hang-On
  • Super Street Fighter II: The New Challengers
  • The Ooze *
  • The Revenge of Shinobi
  • ToeJam & Earl in Panic on Funkotron
  • Truxton
  • Vectorman 2 *
  • Viewpoint
  • Virtua Racing *
  • Warsong

Sega CD titles

  • Ecco the Dolphin (CD Ver.)
  • Ecco: The Tides of Time (CD Ver.)
  • Final Fight CD
  • Mansion of Hidden Souls *
  • Night Striker *
  • Night Trap
  • Robo Aleste
  • Sewer Shark
  • Shining Force CD *
  • Silpheed *
  • Sonic The Hedgehog CD *
  • The Ninjawarriors *

Bonus Games

New ports and/or previously unreleased titles

  • Devi & Pii
  • Fantasy Zone *
  • Space Harrier II (and Space Harrier)
  • Spatter
  • Star Mobile *
  • Super Locomotive
  • VS Puyo Puyo Sun

Ars Technica may earn compensation for sales from links on this post through affiliate programs.

Continue Reading


Hundreds of cosmetic MultiVersus mods stop working in apparent crackdown



Enlarge / Mods like this one, which replaces Lebron James with Jesus, no longer work in Multiversus.

The hit new free-to-play Warner Bros. arena fighter MultiVersus has enjoyed a robust modding scene since the game was still in its early access beta. This week, though, that community is on life support as players and developers report the game will no longer run with any modification installed.

Users on mod distribution site Game Banana started noticing the change around noon on Wednesday, with some discussing potential workarounds that might let their mods still work. By that afternoon, though, modder Ghost suggested on Twitter that “MultiVersus will no longer boot up if you have mods installed. They killed modding of any kind.”

The move was seemingly confirmed by MultiVersus Game Director Tony Huynh, who tweeted overnight that a user-reported game-crashing issue “might be because your client has been modded. If so you’ll need to remove the mods to play.”

In the few weeks since MultiVersus‘ public launch, modders have crafted hundreds of reskins that make existing characters look like other pop culture mainstays, from Lola Bunny and Luigi to Master Chief and Jesus Christ. Other mods could replace background art or music, add new visual effects to characters, or, uh, put duct tape over Velma’s mouth, if that’s what you’re into. These cosmetic mods only applied to a player’s local copy of the game and didn’t impact the integrity of online gameplay.

Buy some DLC instead?

This isn’t the first time Warner Bros. has made moves against MultiVersus modding. The publisher reportedly started issuing copyright strikes against Twitch streamers using modded characters in their videos earlier this month. At the time, Huynh confirmed in a tweet that “streaming with a modded client” was grounds for a DMCA copyright strike against at least one streamer.

While Warner Bros. hasn’t responded to a request for comment from Ars Technica, some players suspect the crackdown on mods could be an attempt to stifle competition for official MultiVersus DLC. The free-to-play game brings in revenue primarily by selling Season Pass content subscriptions that include new “variant” costumes and looks for many characters. Warner Bros. also sells separate DLC packs that include “unlock tickets” for upcoming characters like Rick & Morty and Black Adam.

Warner Bros. might worry that players will be less likely to buy this kind of content if similar (or near-identical) cosmetic content is available as free mods. But games like Crusader Kings II have managed to thrive with both paid DLC content and a wide range of free user-made mods, showing it’s not impossible for both types of content to coexist without ruining a free-to-play business model.

While modders haven’t given up on finding ways around MultiVersus‘ apparent new mod ban, Warner Bros. will likely keep squashing any new modification methods as well. When it comes to this version of the metaverse, apparently only officially recognized characters need apply.

Continue Reading


Embracer acquires rights to Tolkien-related IP, teases new LOTR films



Enlarge / A classic photo of famed author J.R.R. Tolkien, modified to reflect the new steward of his most famed intellectual property as of this week’s megaton acquisition announcement.

Getty Images / Sam Machkovech

Swedish game publisher Embracer Group has racked up headlines in the past few years thanks to megaton acquisitions of video game studios, and on Thursday morning, the company announced it has grown further still. Its latest acquisition spree includes one surprising company outside its usual business purview: Middle-earth Enterprises.

This company is better known as the exclusive handler of all Lord of the Rings and Hobbit novels, along with all intellectual property derived directly from those J.R.R. Tolkien fantasy masterworks. The financial terms of this deal with previous handler the Saul Zaentz Company were not disclosed, but Embracer is already eager to tease brand-new films featuring the aforementioned novels’ characters.

No production plans have been set for new films just yet; rather, Embracer says in a press release that it wants to “explore additional movies based on iconic characters such as Gandalf, Aragorn, Gollum, Galadriel, Eowyn, and other characters from the literary works of J.R.R. Tolkien.” The deal has a logical connection to Embracer’s existing business in the form of LOTR-themed board games already being produced by its subsidiary, Asmodee, while it’s hard to imagine Embracer not immediately assigning one of its many wholly owned video game studios to the franchise in one way or another. Embracer has yet to suggest plans to produce a Tolkien-themed video game, however. Instead, its Thursday announcement hinted to “new opportunities for fans to explore this fictive world through merchandising and other experiences.”

Importantly, this deal does not materially affect Amazon’s upcoming, highly budgeted Rings of Power TV series. That series’ pitch was led directly by the Tolkien estate and presented to various platforms before Amazon emerged with a winning bid, and the deal cleverly sneaked past Middle-earth Enterprises’ rights, because it involved texts and materials not covered by the long-standing SZC arrangement. Embracer suggests it “has financial interests” in the Amazon series, but it’s unclear whether Embracer will see any cut of that production’s profits, or if they merely benefit from more publicity and attention to all things Tolkien.

That’s a lot of Embracin’

Embracer named six other companies as acquisition targets, with four resembling traditional game development houses: Tripwire Interactive, the American studio responsible for the Killing Floor and Maneater series; Tuxedo Labs, a Swedish studio best known for the PC-exclusive destruction-simulation game Teardown; Tatsujin, a Japanese developer staffed by arcade-era developers with plans to work on series from the esteemed Toaplan games library (particularly the classic “shmup” Truxton and the beloved meme machine that is Zero Wing); and Bitwave Games, another Swedish studio with plans to both collaborate with Tatsujin and also create ports of NES-era classics like Sunsoft’s lost 8-bit gem Gimmick. The latter announcement appears to spoil at least one news item from a Sunsoft-related event scheduled for later on Thursday.

Outside of formal game studios, Embracer’s three other named acquisitions seem designed to diversify the company’s gaming business portfolio. The best-known of these, Limited Run Games, has risen to prominence among gaming fans in the past five years thanks to its focus on physical game releases. Typically, Limited Run opens up preorders for cartridges and discs of games that have previously launched as digital-download exclusives; once the preorder period completes, the publisher typically shuts down orders for the game in question, especially since the games in question are often cult classics or niche favorites (though it occasionally publishes games from larger game studios, particularly certain versions of games in the Doom series). Embracer did not suggest in its acquisition announcement that it plans to shift Limited Run’s business model.

Embracer is additionally acquiring Singtrix, a karaoke system manufacturer launched by patent holders previously involved in the Guitar Hero series. That acquisition’s announcement does little to clarify what consumers might expect from an Embracer-run Singtrix, other than a suggestion that the company is working on “the next pop culture musical experience,” which may or may not involve a future video game-like project. There’s a chance that Embracer’s third non-studio acquisition, the European peripheral manufacturer Gioteck, could help Embracer build some kind of physical musical peripheral to go with whatever Singtrix is whipping up, but that’s only speculation at this point.

In addition to all of these announcements, Embracer buried news of another game studio acquisition that it is not prepared to announce “due to commercial reasons.” Instead, Embracer suggested that this unnamed studio counts as “the third- or fourth-largest” game studio it entered plans to acquire today—which suggests that it’s not necessarily a massive studio. (If this were the third- or fourth-largest acquisition in the company’s entire history, on the other hand, that would be worth no less than $525 million, which is what it paid for the entirety of Saber Interactive in early 2020.)

Thursday’s news follows the shocking May announcement that Embracer had acquired the entire Western operations of Square-Enix, and those studios’ lucrative IP (including the Tomb Raider, Legacy of Kain, and Deus Ex series), for a mere $300 million. That announcement included confirmation that the studios were moving ahead with games in both the Tomb Raider and Deus Ex series, though neither Embracer nor its newly owned studios have revealed anything more about such games since the May news broke. That followed its acquisitions of well-known game studios such as Gearbox Entertainment (Borderlands), 4A Games (Metro Exodus), and Deep Silver Volition (Saint’s Row, Red Faction).

Continue Reading