Homepage / Technology / Google's AI subsidiary made a game-playing program that's entirely self-taught
Amazon says this Prime Day was its biggest shopping event ever Kudlow says President Trump is 'so dissatisfied' with China trade talks that he is keeping the pressure on As stocks regain their footing, an ominous warning looms Goldman Sachs downgrades Clorox to sell, says valuation is 'unsustainably high' How Satya Nadella has spurred a tripling of Microsoft's stock price in just over four years Kudlow says economic growth could top 4% for 'a quarter or two,' more tax cuts could be coming The one chart that explains Netflix’s stunning comeback US housing starts plunge 12% in June to a nine-month low Aerospace titans Boeing and Airbus top $110 billion in orders at Farnborough Target uses Prime Day to its advantage, logging its 'biggest online shopping day' so far this year Billionaire Marc Lasry sees bitcoin reaching up to $40,000 as it becomes more mainstream and easier to trade These are the 10 US airports where you're most likely to be hacked Amazon shares slightly higher as investors await Prime Day results Wreck of Russian warship found, believed to hold gold worth $130 billion A bullish ‘phenomenon’ in bond market is weeks away from fading, top credit strategist says Stocks making the biggest moves premarket: MS, GOOGL, TXN, UAL, NFLX & more Twitter shares up 50% since late April means most upside priced in, analyst says in downgrade EU fines Google $5 billion over Android antitrust abuse Mortgage applications fall 2.5% as buyers struggle to find affordable homes America may not have the tools to counter the next financial crisis, warn Bernanke, Geithner and Paulson Investors are getting spooked as the risk of a no-deal Brexit rises EU expected to fine Google $5 billion over Android antitrust abuse Ex-FBI chief James Comey urges Americans to vote for Democrats in midterm elections Elon Musk apologizes to British cave diver following baseless 'pedo guy' claim Disney, Comcast and Fox: All you need to know about one of the biggest media battles ever Xiaomi shares notch new high after Hong Kong, mainland China stock exchanges reach agreement The trade war is complicating China's efforts to fix its economy European markets set for a strong open amid earnings; Google in focus Hedge fund billionaire Einhorn places sixth in major poker tournament The biggest spender of political ads on Facebook? President Trump Asian stocks poised to gain after Fed's Powell gives upbeat comments; dollar firmer Stocks are setting up to break to new highs Not all FAANG stocks are created equal EU ruling may be too little, too late to stop Google's mobile dominance Cramer explains how Netflix's stock managed to taper its drop after disappointing on earnings Airbnb condemns New York City's 'bellhop politics,' threatens legal retaliation Amazon sellers say they were unfairly suspended right before Prime Day, and now have two bad choices Investor explains why 'duller' tech stocks can have better returns than 'high-flying' tech names Elon Musk is 'thin-skinned and short-tempered,' says tech VC Texas Instruments CEO Brian Crutcher resigns for violating code of conduct Google Cloud Platform fixes issues that took down Spotify, Snapchat and other popular sites Uber exec: We want to become the 'one stop' transportation app 'What a dumb hearing,' says Democrat as Congress grills tech companies on conservative bias Amazon shares rebound, report says Prime Day sales jumped 89 percent in first 12 hours of the event How to put your medical history on your iPhone in less than 5 minutes Investment chief: Watch these two big events in 2018 Even with Netflix slowing, the market rally is likely not over Cramer: Netflix subscriber weakness debunks the 'sky's the limit' theory on the stock Netflix is looking at watch time as a new area of growth, but the competition is stiff Why Nobel laureate Richard Thaler follows Warren Buffett's advice to avoid bitcoin Rolls-Royce is developing tiny 'cockroach' robots to crawl in and fix airplane engines After Netflix plunge, Wall Street analysts forecast just tame returns ahead for the once high-flying FANG group Roku shares rise after analyst raises streaming video company's price target due to customer growth China is investing 9 times more into Europe than into North America, report reveals Amazon says US Prime Day sales 'so far bigger than ever' as glitch is resolved Netflix is on pace for its worst day in two years US lumber producers see huge opportunity, rush to expand San Francisco to consider tax on companies to help homeless Homebuilder sentiment, still high, stalls as tariffs, labor and land drive up costs Powell backs more rate hikes as economy growing 'considerably stronger' Netflix history is filled with big stock declines – like today – followed by bigger rebounds Intel shares get downgraded by Evercore ISI due to rising competition from Nvidia, AMD Petco aims to reinvent the pet store with something you can't buy online Genetic testing is coming of age, but for consumers it's buyer beware Tech 'FAANG' was the most-crowded trade in the world heading into the Netflix implosion, survey shows Netflix weak subscriber growth may indicate a 'maturity wall' that could whack the stock even more: Analyst This chart may be predicting the bull market's demise Wall Street says Netflix's stock plunge is a ‘compelling’ buying opportunity because the streaming giant ‘never misses twice’ Tesla sinks after Musk tweets, again Boeing announces new division devoted to flying taxis Stocks making the biggest move premarket: NFLX, UNH, GS, AMZN, WMT & more Deutsche Bank downgrades Netflix, but says big subscriber miss is not 'thesis changing' IBM is experimenting with a cryptocurrency that’s pegged to the US dollar North Korea and Zimbabwe: A friendship explained Virgin Galactic spinoff Orbit to launch rockets from the UK with space deal Artificial intelligence will create more jobs than it destroys? That’s what PwC says ‘Treasonous’ Trump and ‘Putin’s poodle:' Scathing headlines follow the Trump-Putin summit China’s fintech companies offer ‘enormous’ opportunity, investment manager says Trump's performance at summit with Putin was 'unprecedented,' experts say Walmart and Microsoft link up on cloud technology as they both battle Amazon European stocks seen mixed amid earnings; Fed’s Powell to address Congress How I knew I should quit my day job and run my start-up full-time: Viral website founder China's stocks have been trounced, but the trade war may ultimately be good news for those shares Billionaire tech investor Peter Thiel bets on crypto start-up Block.one Asian shares subdued open after mixed close on Wall Street; energy stocks under pressure Amazon cloud hits snags after Amazon Prime Day downtime Netflix isn't doomed by one quarter unless people start questioning the long-term investor thesis Tech stocks set to sink on Tuesday after rough evening for ‘FANG’ Netflix plummets after missing big on subscriber growth This wristband lets humans control machines with their minds The U.S. has a rocky history convincing Russia to extradite computer criminals Amazon suffers glitches at the start of Prime Day Jeff Bezos is now the richest man in modern history 'The United States has been foolish': Read Trump and Putin's full exchange Goldman Sachs recommends these 5 highly profitable companies — including Nvidia — to combat rising inflation Goldman Sachs releases 'tactical' stock picks for this earnings season Three red flags for Netflix ahead of its earnings report The bond market may be raising recession fears, but don't expect one anytime soon Cramer: Banks are 'making fortunes' but are still as hated as they were during the financial crisis Putin told Trump at summit: Russia never meddled in US election


Google's AI subsidiary made a game-playing program that's entirely self-taught

Google‘s AI subsidiary DeepMind has unveiled the latest version of its Go-playing software, AlphaGo Zero. The new program is a significantly better player than the version that beat the game’s world champion earlier this year, but, more importantly, it’s also entirely self-taught. DeepMind says this means the company is one step closer to creating general purpose algorithms that can intelligently tackle some of the hardest problems in science, from designing new drugs to more accurately modeling the effects of climate change.

The original AlphaGo demonstrated superhuman Go-playing ability, but needed the expertise of human players to get there. Namely, it used a dataset of more than 100,000 Go games as a starting point for its own knowledge. AlphaGo Zero, by comparison, has only been programmed with the basic rules of Go. Everything else it learned from scratch. As described in a paper published in Nature today, Zero developed its Go skills by competing against itself. It started with random moves on the board, but every time it won, Zero updated its own system, and played itself again. And again. Millions of times over.

After three days of self-play, Zero was strong enough to defeat the version of itself that beat 18-time world champion Lee Se-dol, winning handily — 100 games to nil. After 40 days, it had a 90 percent win rate against the most advanced version of the original AlphaGo software. DeepMind says this makes it arguably the strongest Go player in history.

“By not using human data — by not using human expertise in any fashion — we’ve actually removed the constraints of human knowledge,” said AlphaGo Zero’s lead programmer, David Silver, at a press conference. “It’s therefore able to create knowledge itself from first principles; from a blank slate […] This enables it to be much more powerful than previous versions.”

Silver explained that as Zero played itself, it rediscovered Go strategies developed by humans over millennia. “It started off playing very naively like a human beginner, [but] over time it played games which were hard to differentiate from human professionals,” he said. The program hit upon a number of well-known patterns and variations during self-play, before developing never-before-seen stratagems. “It found these human moves, it tried them, then ultimately it found something it prefers,” he said. As with earlier versions of AlphaGo, DeepMind hopes Zero will act as an inspiration to professional human players, suggesting new moves and stratagems for them to incorporate into their game.

As well as being a better player, Zero has other important advantages compared to earlier versions. First, it needs much less computing power, running on just four TPUs (specialized AI processors built by Google), while earlier versions used 48. This, says Silver, allows for a more flexible system that can be improved with less hassle, “which, at the end of the day, is what really matters if we want to make progress.” And second, because Zero is self-taught, it shows that we can develop cutting-edge algorithms without depending on stacks of data.

More from The Verge:

Lego celebrates the women of NASA with new minifigs
Storm Ophelia was so unusual, it was literally off the charts
Bigelow Aerospace wants to put an inflatable space habitat in orbit around the Moon

For experts in the field, these developments are a big part of what makes this new research exciting. That’s is because they offer a rebuttal to a persistent criticism of contemporary AI: that much of its recent gains come mostly from cheap computing power and massive datasets. Skeptics in the field like pioneer Geoffrey Hinton suggest that machine learning is a bit of a one-trick pony. Piling on data and compute is helping deliver new functions, but the current pace of advances is unsustainable. DeepMind’s latest research offers something of a rebuttal by demonstrating that there are major improvements to be made simply by focusing on algorithms.

“This work shows that a combination of existing techniques can go somewhat further than most people in the field have thought, even though the techniques themselves are not fundamentally new,” Ilya Sutskever, a research director at the Elon Musk-backed OpenAI institute, told The Verge. “But ultimately, what matters is that researchers keep advancing the field, and it’s less important if this goal is achieved by developing radically new techniques, or by applying existing techniques in clever and unexpected ways.”

In the case of AlphaGo Zero, what is particularly clever is the removal of any need for human expertise in the system. Satinder Singh, a computer science professor who wrote an accompanying article on DeepMind’s research in Nature, praises the company’s work as “elegant,” and singles out these aspects.

Singh tells The Verge that it’s a significant win for the field of reinforcement learning — a branch of AI in which programs learn by obtaining rewards for reaching certain goals, but are offered no guidance on how to get there. This is a less mature field of work than supervised learning (where programs are fed labeled data and learn from that), but it has potentially greater rewards. After all, the more a machine can teach itself without human guidance, the better, says Singh.

“Over the past five, six years, reinforcement learning has emerged from academia to have much more broader impact in the wider world, and DeepMind can take some of the credit for that,” says Singh. “The fact that they were able to build a better Go player here with an order of magnitude less data, computation, and time, using just straight reinforcement learning — it’s a pretty big achievement. And because reinforcement learning is such a big slice of AI, it’s a big step forward in general.”

What are the applications for these sorts of algorithms? According to DeepMind co-founder Demis Hassabis, they can provide society with something akin to a thinking engine for scientific research. “A lot of the AlphaGo team are now moving onto other projects to try and apply this technology to other domains,” said Hassabis at a press conference.

Hassabis explains that you can think of AlphaGo as essentially a very good machine for searching through complicated data. In the case of Zero, that data is comprised of possible moves in a game of Go. But because Zero was not programmed to understand Go specifically, it could be reprogrammed to discover information in other fields: drug discovery, protein folding, quantum chemistry, particle physics, and material design.

Hassabis suggests that a descendant of AlphaGo Zero could be used to search for a room temperature superconductor — a hypothetical substance that allows electrical current to flow with zero lost energy, allowing for incredibly efficient power systems. (Superconductors exist, but they only currently work at extremely cold temperatures.) As it did with Go, the algorithm would start by combining different inputs (in this case, the atomic composition of various materials and their associated qualities) until it discovered something humans had missed.

“Maybe there is a room temperature superconductor out and about. I used to dream about that when I was a kid, looking through my physics books,” says Hassabais. “But there’s just so many combinations of materials, it’s hard to know whether [such a thing exists].”

Of course, this would be much more complicated than simply pointing AlphaGo Zero at the Wikipedia page for chemistry and physics and saying “have at it.” Despite its complexity, Go, like all board games, is relatively easy for computers to understand. The rules are finite, there’s no element of luck, no hidden information, and — most importantly — researchers have access to a perfect simulation of the game. This means an AI can run millions of tests and be sure it’s not missing anything. Finding other fields that meet these criteria limits the applicability of Zero’s intelligence. DeepMind hasn’t created a magical thinking machine.

These caveats aside, the research published today does get DeepMind just a little bit closer to solving the first half of its tongue-in-cheek, two-part mission statement. Part one: solve intelligence; part two: use it to make the world a better place. “We’re trying to build general purpose algorithms and this is just one step towards that, but it’s an exciting step,” says Hassabis.

Source: Tech CNBC
Google's AI subsidiary made a game-playing program that's entirely self-taught

Comments are closed.