Category Archives: Visual Thinking

Seeing Faces

Source: Scientific American, Jun 2017

The ease of recognizing faces masks its underlying cognitive complexity. Faces have eyes, noses and mouths in the same relative place, yet we can accurately identify them from different angles, in dim lighting and even while moving. Brain-imaging studies have revealed we evolved several tiny regions the size of blueberries in the temporal lobe—the area under the temple—that specialize in responding to faces. Neuroscientists call these regions “face patches”. But neither data from brain scanners—functional magnetic resonance imaging—nor clinical studies of patients with implanted electrodes have explained exactly how the cells in these face patches work.

Now, using a combination of brain imaging and single-neuron recording in macaques, biologist Doris Tsao and her colleagues at Caltech have finally cracked the neural code for face recognition. The researchers found the firing rate of each face cell corresponds to separate facial features along an axis. Like a set of dials, the cells are fine-tuned to bits of information, which they can then channel together in different combinations to create an image of every possible face. “This was mind-blowing,” Tsao says. “The values of each dial are so predictable that we can re-create the face that a monkey sees, by simply tracking the electrical activity of its face cells.”

Tsao’s recent study suggests scientists may have been mistaken. “She has shown that neurons in face patches don’t encode particular people at all, they just encode certain features,” he says. “That completely changes our understanding of how we recognize faces.”

All that mattered for each neuron was a single-feature axis. Even when viewing different faces, a neuron that was sensitive to hairline width, for example, would respond to variations in that feature. But if the faces had the same hairline and different-size noses, the hairline neuron would stay silent, Chang says. The findings explained a long-disputed issue in the previously held theory of why individual neurons seemed to recognize completely different people.

Moreover, the neurons in different face patches processed complementary information. Cells in one face patch—the anterior medial patch—processed information about the appearance of faces such as distances between facial features like the eyes or hairline. Cells in other patches—the middle lateral and middle fundus areas—handled information about shapes such as the contours of the eyes or lips. Like workers in a factory, the various face patches did distinct jobs, cooperating, communicating and building on one another to provide a complete picture of facial identity.

… only needed readings from a small set of neurons for the algorithm to accurately re-create the faces monkeys were viewing, Tsao says. Recordings from just 205 cells—106 cells in one patch and 99 cells in another—were enough. “It really speaks to how compact and efficient this feature-based neural code is,” she says. It may also explain why primates are so good at facial recognition, and how we can potentially identify billions of different people without needing an equally large number of face cells.

Bret Victor – Humane Representation of Thought



New representations of thought — written language, mathematical notation, information graphics, etc — have been responsible for some of the most significant leaps in the progress of civilization, by expanding humanity’s collectively-thinkable territory.

But at debilitating cost. These representations, having been invented for static media such as paper, tap into a small subset of human capabilities and neglect the rest. Knowledge work means sitting at a desk, interpreting and manipulating symbols. The human body is reduced to an eye staring at tiny rectangles and fingers on a pen or keyboard.

Like any severely unbalanced way of living, this is crippling to mind and body. But it is also enormously wasteful of the vast human potential. Human beings naturally have many powerful modes of thinking and understanding. Most are incompatible with static media. In a culture that has contorted itself around the limitations of marks on paper, these modes are undeveloped, unrecognized, or scorned.

We are now seeing the start of a dynamic medium. To a large extent, people today are using this medium merely to emulate and extend static representations from the era of paper, and to further constrain the ways in which the human body can interact with external representations of thought.

But the dynamic medium offers the opportunity to deliberately invent a humane and empowering form of knowledge work. We can design dynamic representations which draw on the entire range of human capabilities — all senses, all forms of movement, all forms of understanding — instead of straining a few and atrophying the rest.

This talk suggests how each of the human activities in which thought is externalized (conversing, presenting, reading, writing, etc) can be redesigned around such representations.


Related Resources:

Wired, Jan 2014

MOST FORWARD-LOOKING DESIGNERS think about what might happen in the next five years. Bret Victor is more concerned about the next 500.

Print media, Victor says, has served us well. It helped us come up with new ways of representing knowledge that have been instrumental to human progress—things like charts, graphs, and mathematical notation. And yet print media has limitations. Namely, it engages a very narrow slice of our intellectual capabilities. Print is based on our eyeballs interpreting symbols. It doesn’t utilize our innate understanding of spatial relationships; it doesn’t take advantage of how deftly we learn by touching, holding, and manipulating objects with our hands.

Victor dubs it “the dynamic medium.” The basic idea is some sort of physical matter that has the ability to rearrange itself dynamically—think maybe some sort of computerized sand that could take any form at any time. Victor’s not concerned with the technological implementation of such a medium, though he’s utterly convinced that it will be feasible. He’s more interested in the intellectual breakthroughs it could yield.

What excites Victor about the dynamic medium is the possibility of new representational tools—leaps equivalent to the charts, graphs and notation of past centuries. Instead of reading about the global economy in all its intricacy, for example, imagine if you could hold a working model of it, or get inside it and have the model surround you.

as Victor sees it, human progress will eventually depend on such tools, ones that let us explore complex systems and concepts with our hands as well as our minds. (Victor’s big on the power of thinking with our hands.)

Medium, date indeterminate

He sees himself less as a designer/developer/engineer than as a researcher of computer-augmented creativity, much like his mentor Alan Kay (who pioneered graphical user interfaces and object-oriented programming) and his hero Douglas Engelbart (of “The Mother of All Demos” fame).

Victor declares that “the power to understand and predict the quantities of the world should not be restricted to those with a freakish knack for manipulating abstract symbols.”

 what would “post-paper” thoughts look like? Victor admits he has no idea. He just has a conviction about the medium that will enable them. “The important thing isn’t thinking about computers or programming as they are today, but thinking about moving from a static medium like marks on paper to a dynamic medium with computational responsiveness infused into it, that can actually participate in the thinking process,” he says.

how the media in which we choose to represent our ideas shape (and too often, limit) what ideas we can have.




Cyber-Physical World

Analyzing using Argument Maps

Source: Princeton Alumni Weekly, Jan 2015

Human Four-Dimensional Spatial Intuition in Virtual Reality

Source: Questia, Oct 2009

we show evidence that people with basic geometric knowledge can learn to make spatial judgments on the length of, and angle between, line segments embedded in four-dimensional space viewed in virtual reality with minimal exposure to the task and no feedback to their responses. Their judgments incorporated information from both the three-dimensional (3-D) projection and the fourth dimension, and the underlying representations were not algebraic in nature but based on visual imagery, although primitive and short lived. These results suggest that human spatial representations are not completely constrained by our evolution and development in a 3-D world.

Much effort has been made to challenge this cognitive limitation and to develop human four-dimensional (4-D) intuitions (Davis, Hersh, & Marchisotto, 1995; Gardner, 1969; Rucker, 1984; Seyranian, 2001; Weeks, 1985). Two basic techniques were proposed to help people obtain an intuition of 4-D space.

The first is by analogy to 3-D space. This technique has been widely used. For example, Berger (1965; Abbott, 1991) explained how a 4-D creature can enter a 3-D locked closet from the fourth dimension by describing how a 3-D creature enters a two-dimensional (2-D) enclosure from above without touching its walls.

The second technique is to lift an observer into the higher dimensional space, so that he or she can directly experience it perceptually (Abbott, 1991; Berger, 1965; Rucker, 1984; Seyranian, 2001). For example, Abbott suggested that a 2-D creature can obtain 3-D intuition when it is taken into the 3-D space and views its world from above. Although this approach is hypothesized to be the most powerful means of acquiring 4-D intuition, it was not possible to implement the technique until virtual reality was available (D’Zmura, Colantoni, & Seyranian, 2000; Francis, 2005).

Our Brains Process Visuals 60,000X Faster than Text

Understanding Visual Language

Source: Fast Company, Jun 2015

Symbolic icons tend to last.

A question designers may ask linguists has to do with how language changes over time. If language constantly changes because people and their culture evolve, why should the more visual aspects of language not follow suit, including the language of man-made objects?

Language is not just verbal or written. Speech as a means of communication cannot strictly be separated from the whole of human communicative activity, which also includes the visual. The word “imagination” definitely suggests that we can also think in images.

Visual language is defined as a system of communication using visual elements.The term visual language in relation to vision describes the perception, comprehension, and production of visible signs. Just as people can verbalize their thinking, they can visualize it.

A diagram, a map, and a painting are all examples of uses of visual language. Its structural units include line, shape, color, form, motion, texture, pattern, direction, orientation, scale, angle, space, and proportion. The elements in an image represent concepts in a spatial context, rather than the time-based linear progression used in talking and reading. Speech and visual communication are parallel and often interdependent means by which humans exchange information.