The TPU is in production use across Google's cloud, including powering the RankBrain search result sorting system and Google's voice recognition services.
Part of that has to do with the way application development is heading -- developers are building more and more applications in the cloud only, and don't want to worry about managing hardware configurations, maintenance and updates.
Another possible reason is that Google simply doesn't want to give its rivals access to the chips, which it likely spent a lot of time and money developing.
Analyst Patrick Moorhead said he expects the chip will be used for inferencing, a part of machine learning operations that doesn't require as much flexibility.
Right now, that's all Google is saying.
Holzle said that the company will reveal more about the chip in a paper to be released this fall.