The DeepReser League, developed by Amazon.com Inc.’s Amazon Web Services Cloud Business, is a branch of AI called Reinforcement Learning. In this type of Ai services, algorithms learn the right way to perform an action based on trial-and-error and observation. This technique differs from the type of AI commonly used in business called supervised learning, in which the algorithms are fed with trained data to learn how to detect images or make predictions. Here, cars supply their own data: images collected with cameras.
Anyone with an Amazon Web Services account can join the league. Teams or individuals can compete in online “virtual” races or individually around the world.
Teams build and train AI algorithms using Amazon Sage Maker software, fit them into 10-inch self-driving cars, and then bet them around a 17-foot by 26-foot track. The fastest car wins.
“It actually has practical applications,” said James Rhodes, chief technology officer of investment research firm Morningstar. Thanks to the training, the company expects to have dozens of projects based on reinforcement learning and other machine learning methods by the end of 2020.
In addition to training autonomous vehicles, experts say reinforcement learning can help robots to run faster or develop safety systems that automatically adapt to different environments. Mike Miller, general manager of AI devices at Amazon Web Services, said: “[It] is a very complex technology and has a very steep learning curve.
Algorithms are complex because they collect data on their own instead of feeding millions of images to learn, said Peter Stone, a professor of computer science at the University of Texas at Austin. Mr. Stone, president of AI software company KogitaI Inc., has nothing to do with Deepraser.
When programmers do the algorithms correctly, they write code to “reward” them for winning a race or avoiding an obstacle. In the case of algorithm-powered cars, this includes tasks such as staying close to the centerline of the track, reducing wide steering angles, and turning to avoid obstacles and crashes.
At Morningstar, more than 450 software developers, equity analysts, and quantitative researchers have formed nearly 100 racing teams in 10 countries since January, and Mr. Rhodes has begun employing technology.
To date, Morningstar has invested “north of tens of thousands of dollars” in microcars and training software, Rhodes said. “It’s bringing the virtual [world] into the physical space, especially for people who don’t need computer scientists,” he said.
Earlier this year, one of the Morningstar teams came up with an idea based on the reinforcement practice of looking for patterns in regulatory filings to more accurately identify different information. Another group has come up with a tool that uses a reinforcement practice to automatically find and fix broken links to financial institutions’ websites, Mr. Rhodes said. Two tools are under development.
Insurance Liberty Mutual, meanwhile, employs about 270 people, including software engineers and data scientists, and is involved in Depressor.
“It’s a fun way for people to get practical exposure to the most important algorithms in a secure environment, where they don’t mess up any major business applications,” said James McGlennon, the company’s chief information officer.
The company is already using other machine learning techniques to adjust prices for auto insurance based on risk factors and to look for anomalies in operations. The goal of the Depressor program is to think of ways the company can use reinforcement learning to help businesses, McGlennon said.
Ultimately, reinforcement learning is one of the many machine learning techniques used by companies, said Dario Gill, director of research for International Business Machines Corporation. He said the challenge was to rely solely on that technology when training a real-world autonomous vehicle. That said because a lot of methods depend on trial and error. “There is a reason why reinforcement learning is implicated in the world of games,” he added.