Category Archives: AI

US$3/hour Burger-Flipping Robot

Source: LA Times, Feb 2020

Miso can offer Flippys to fast-food restaurant owners for an estimated $2,000 per month on a subscription basis, breaking down to about $3 per hour. (The actual cost will depend on customers’ specific needs). A human doing the same job costs $4,000 to $10,000 or more a month, depending on a restaurant’s hours and the local minimum wage. And robots never call in sick.

The Decline of Computers as a General Purpose Technology

Source: MIT, 2019

We conclude that the virtuous GPT cycle that has driven computing for decades is ending. This paper provides evidence that the GPT cycle is being replaced by a fragmented cycle where computing separates into specialized domains that are
largely distinct and provide few benefits to each other. This trend will have important implications for individual users and for the economy more broadly.

  • Technological and economic forces are making computer processors less general-purpose and more specialized. This process has already begun, driven by a slowing of Moore’s Law and the success of algorithms like deep learning.
  • Specialization threatens to fragment computing into “fast lane” applications that get powerful customized chips, and “slow lane” applications that get stuck using generalpurpose chips whose progress is fading.
  • The virtuous, general-purpose technology (GPT) cycle that has driven computing for decades is ending and is being replaced by a fragmented cycle where computing separates into specialized domains that are largely distinct and provide few benefits to each other.
  • In the long term, this fragmentation could slow the overall pace of computer improvement, jeopardizing an important source of economic prosperity.

For users who can profitably switch to specialized chips, there are likely to be significant gains, as we’ve seen with deep learning and cryptocurrency. For those who can’t switch, the picture will be bleaker as universal chip progress slows and with it, much of their computing performance improvements.

On a larger scale, we argue that the switch to specialization will worsen the economics of chip manufacturing, leading to slower improvements. Therefore, the move to specialized chips perpetuates itself, fragmenting the general -purpose model and splitting off more and more applications

Related Resource:  MIT Working Paper,  Nov 2018


Causality Eludes AI

Source: MIT Technology Review, Feb 2020

AI systems don’t understand causation. They see that some events are associated with other events, but they don’t ascertain which things directly make other things happen. It’s as if you knew that the presence of clouds made rain likelier, but you didn’t know clouds caused rain.

Understanding cause and effect is a big aspect of what we call common sense, and it’s an area in which AI systems today “are clueless,” says Elias Bareinboim. He should know: as the director of the new Causal Artificial Intelligence Lab at Columbia University, he’s at the forefront of efforts to fix this problem.

His idea is to infuse artificial-intelligence research with insights from the relatively new science of causality, a field shaped to a huge extent by Judea Pearl, a Turing Award–winning scholar who considers Bareinboim his protégé.

As Bareinboim and Pearl describe it, AI’s ability to spot correlations—e.g., that clouds make rain more likely—is merely the simplest level of causal reasoning.

It’s good enough to have driven the boom in the AI technique known as deep learning over the past decade. Given a great deal of data about familiar situations, this method can lead to very good predictions. A computer can calculate the probability that a patient with certain symptoms has a certain disease, because it has learned just how often thousands or even millions of other people with the same symptoms had that disease.

But there’s a growing consensus that progress in AI will stall if computers don’t get better at wrestling with causation. If machines could grasp that certain things lead to other things, they wouldn’t have to learn everything anew all the time—they could take what they had learned in one domain and apply it to another. And if machines could use common sense we’d be able to put more trust in them to take actions on their own, knowing that they aren’t likely to make dumb errors.

Today’s AI has only a limited ability to infer what will result from a given action. In reinforcement learning, a technique that has allowed machines to master games like chess and Go, a system uses extensive trial and error to discern which moves will essentially cause them to win. But this approach doesn’t work in messier settings in the real world. It doesn’t even leave a machine with a general understanding of how it might play other games.

An even higher level of causal thinking would be the ability to reason about why things happened and ask “what if” questions.

A patient dies while in a clinical trial; was it the fault of the experimental medicine or something else? School test scores are falling; what policy changes would most improve them? This kind of reasoning is far beyond the current capability of artificial intelligence.

Getting people to think more carefully about causation isn’t necessarily much easier than teaching it to machines, he says. Researchers in a wide range of disciplines, from molecular biology to public policy, are sometimes content to unearth correlations that are not actually rooted in causal relationships.

For instance, some studies suggest drinking alcohol will kill you early, while others indicate that moderate consumption is fine and even beneficial, and still other research has found that heavy drinkers outlive nondrinkers. This phenomenon, known as the “reproducibility crisis,” crops up not only in medicine and nutrition but also in psychology and economics. “You can see the fragility of all these inferences,” says Bareinboim. “We’re flipping results every couple of years.”

He argues that anyone asking “what if”—medical researchers setting up clinical trials, social scientists developing pilot programs, even web publishers preparing A/B tests—should start not merely by gathering data but by using Pearl’s causal logic and software like Bareinboim’s to determine whether the available data could possibly answer a causal hypothesis.

Eventually, he envisions this leading to “automated scientist” software: a human could dream up a causal question to go after, and the software would combine causal inference theory with machine-learning techniques to rule out experiments that wouldn’t answer the question. That might save scientists from a huge number of costly dead ends.

How do those social scientists, or any scientists anywhere, decide which experiments to pursue and which data points to gather? By following their intuition: “They are trying to see where things will lead, based on their current understanding.”

That’s an inherently limited approach, he said, because human scientists designing an experiment can consider only a handful of variables in their minds at once.

A computer, on the other hand, can see the interplay of hundreds or thousands of variables. Encoded with “the basic principles” of Pearl’s causal calculus and able to calculate what might happen with new sets of variables, an automated scientist could suggest exactly which experiments the human researchers should spend their time on. 

Bengio points out that fundamental knowledge about the world can be gleaned by analyzing the things that are similar or “invariant” across data sets.

Over time, with enough meta-learning about variables that are consistent across data sets, a computer could gain causal knowledge that would be reusable in many domains.

Pearl says AI can’t be truly intelligent until it has a rich understanding of cause and effect. Although causal reasoning wouldn’t be sufficient for an artificial general intelligence, it’s necessary, he says, because it would enable the introspection that is at the core of cognition. “What if” questions “are the building blocks of science, of moral attitudes, of free will, of consciousness,” Pearl told me.

You can’t draw Pearl into predicting how long it will take for computers to get powerful causal reasoning abilities. “I am not a futurist,” he says. But in any case, he thinks the first move should be to develop machine-learning tools that combine data with available scientific knowledge: “We have a lot of knowledge that resides in the human skull which is not utilized.”

Reproducibility of AI Results

Source: The Gradient, Feb 2020

A 2011 study found that only 6% of medical studies could be fully reproduced. In 2016, a survey of researchers from many disciplines found that most had failed to reproduce one of their previous papers. Now, we hear warnings that Artificial Intelligence (AI) and Machine Learning (ML) face their own reproducibility crises.

Ideally, full reproducibility means that simply reading a scientific paper should give you all the information you need to 1) set up the same experiments, 2) follow the same approach, and then 3) obtain similar results.

Chris Drummond has described the approach of using an author’s code as replicability, and made a very salient argument that replication is desirable, but not sufficient for good science. A paper is supposed to be the scientific distillation of the work, representing what we have learned and now understand to enable these new results. If we can’t reproduce the results of a paper without the authors code, it may suggest that the paper itself didn’t successfully capture the important scientific contributions.

even if we can replicate the results of a paper, slightly altering the experimental setup could have dramatically different results.

What Makes a ML Paper Reproducible?

Finding 1: Having fewer equations per page makes a paper more reproducible.

Finding 2: Empirical papers may be more reproducible than theory-oriented papers.

Finding 3: Sharing code is not a panacea

Finding 4: Having detailed pseudo code is just as reproducible as having no pseudo code.

Finding 5: Creating simplified example problems do not appear to help with reproducibility.


AI & Measuring Intelligence

Source: The Verge, Jan 2020

Measuring the intelligence of AI is one of the trickiest but most important questions in the field of computer science. If you can’t understand whether the machine you’ve built is cleverer today than it was yesterday, how do you know you’re making progress?

Beating humans at chess and Go is impressive, yes, but what does it matter if the smartest computer can be out-strategized in general problem-solving by a toddler or a rat?

This is a criticism put forward by AI researcher François Chollet, a software engineer at Google and a well-known figure in the machine learning community.

In a recent paper titled “On the Measure of Intelligence,” Chollet also laid out an argument that the AI world needs to refocus on what intelligence is and isn’t. If researchers want to make progress toward general artificial intelligence, says Chollet, they need to look past popular benchmarks like video games and board games, and start thinking about the skills that actually make humans clever, like our ability to generalize and adapt.

In your paper, you describe two different conceptions of intelligence that have shaped the field of AI. One presents intelligence as the ability to excel in a wide range of tasks, while the other prioritizes adaptability and generalization, which is the ability for AI to respond to novel challenges. Which framework is a bigger influence right now, and what are the consequences of that?

In the first 30 years of the history of the field, the most influential view was the former: intelligence as a set of static programs and explicit knowledge bases. Right now, the pendulum has swung very far in the opposite direction: the dominant way of conceptualizing intelligence in the AI community is the “blank slate” or, to use a more relevant metaphor, the “freshly-initialized deep neural network.”

General intelligence can generate task-specific skills, but there is no path in reverse, from task-specific skill to generality. At all. So in machines, skill is entirely orthogonal to intelligence. You can achieve arbitrary skills at arbitrary tasks as long as you can sample infinite data about the task (or spend an infinite amount of engineering resources). And that will still not get you one inch closer to general intelligence.

The key insight is that there is no task where achieving high skill is a sign of intelligence. Unless the task is actually a meta-task, that involves acquiring new skills over a broad [range] of previously unknown problems. And that’s exactly what I propose as a benchmark of intelligence.

If you want to one day become able to handle the complexity and uncertainty of the real world, you have to start asking questions like, what is generalization? How do we measure and maximize generalization in learning systems? And that’s entirely orthogonal to throwing 10x more data and compute at a big neural network so that it improves its skill by some small percentage.

… we need to stop evaluating skill at tasks that are known beforehand — like chess or Dota or StarCraft — and instead start evaluating skill-acquisition ability. This means only using new tasks that are not known to the system beforehand, measuring the prior knowledge about the task that the system starts with, and measuring the sample-efficiency of the system (which is how much data is needed to learn to do the task). The less information (prior knowledge and experience) you require in order to reach a given level of skill, the more intelligent you are. And today’s AI systems are really not very intelligent at all

Tech Wars

Source: ZeroHedge, Dec 2019

why has the trade war transformed into a tech war? 

The simple reason is that China could overtake the US as a major economic power by 2030. The US has been unconsciously fueling China’s ascension as a rising superpower by supplying high-tech semiconductor chips to Chinese companies. But recent actions by the Trump administration have limited the flow of chips to China, to slow their development in artificial intelligence (AI) and global domination.

A new report from the Global AI Index, first reported by South China Morning Post, indicates that China could overtake the US in AI by 2025 to 2030.

The index specifies that based on talent, infrastructure, operating environment, research, development, government strategy, and commercial ventures, China will likely dominate the US in the AI space in the next decade.

The tech war between both countries, to get more specific, has also blossomed into a global AI race, the report said.

By 2030, Washington is forecasted to have earmarked $35 billion for AI development, with the Chinese government allocating at least $22 billion over the same period.

China’s leadership, including President Xi Jinping, has specified that AI will be essential for its global military force and economic power competition against the US.

China’s State Council issued the New Generation Artificial Intelligence Development Plan (AIDP) back in 2017, stating that China’s AI strategy will allow it to become a global superpower.

In a recent speech, Xi said that China must “ensure that our country marches in the front ranks where it comes to theoretical research in this important area of AI, and occupies the high ground in critical and AI core technologies.”

Related Resource: SCMP, Dec 2019

The US is the undisputed leader in artificial intelligence (AI) development while China is the fastest-growing country set to overtake the US in five to 10 years on its current trajectory, according to The Global AI Index published this week by Tortoise Intelligence.

The index, which ranks 54 countries based on their AI capabilities, measured seven key indicators over 12 months: talent, infrastructure, operating environment, research, development, government strategy and commercial ventures.

The US was ahead on the majority of key metrics by a significant margin. It received a score of 100, almost twice as high as second-placed China with 58.3, due to the quality of its research, talent and private funding. The UK, Canada and Germany ranked 3rd, 4th and 5th respectively.


Evolution via multi-agent systems

Source: Quanta, Nov 2019

how AI agents could learn to use things around them as tools, according to the OpenAI team. That’s important not because AI needs to be better at hiding and seeking, but because it suggests a way to build AI that can solve open-ended, real-world problems.

“These systems figured out so quickly how to use tools. Imagine when they can use many tools, or create tools. Would they invent a ladder?”

The hide-and-seek experiment was different: Rewards were associated with hiding and finding, and tool use just happened — and evolved — along the way.

Because the game was open-ended, the AI agents even began using tools in ways the programmers hadn’t anticipated. They’d predicted that the agents would hide or chase, and that they’d create forts. But after enough games, the seekers learned, for example, that they could move boxes even after climbing on top of them. This allowed them to skate around the arena in a move the OpenAI team called “box surfing.”

The researchers never saw it coming, even though the algorithms didn’t explicitly prohibit climbing on boxes. The tactic conferred a double advantage, combining movement with the ability to peer nimbly over walls, and it showed a more innovative use of tools than the human programmers had imagined.