Computer Programs Become Art Appreciators

We humans might think we have the edge over machines when it comes to art appreciation, but computer scientists are perfecting programs that have a discerning eye.

To do this, Lawrence Technological University researchers Lior Shamir and Jane Tarakhovsky found a way to break down an individual painting into 4,027 “numerical image context descriptors.” They performed this operation on 1,000 paintings by 34 famous artists.

Then, Shamir and Tarakhovsky let the computer algorithm scrutinize all the paintings based on visual content alone, without input from a human.

The program was able to identify differences between modern art and classical realism and identify specific subgroups of painters. Based on similar visual content, the algorithm was able to place Michaelangelo, Raphael and Leonardo da Vinci, who are classified by art historians as High Renaissance painters, into the same subgroup. It was able to do the same thing for Baroque painters Johannes Vermeer, Peter Paul Rubens and Dutch master Rembrandt van Rijn.

In a similar manner, the computer was able to sort surrealist painters Salvador Dali, Giorgio de chirico and Max Ernst into the same category, and it grouped together post-impressionists like Paul Gauguin and Paul Cezanne.

Such pattern recognition can also be used in a narrower way.

In 2007, computer scientists at Israel's University of Haifa trained a computer to identify the work of a particular artist.

“As soon as the computer learns to recognize the clock drawings of Dali, it will recognize his other paintings, even without clocks,” University of Haifa researcher Daniel Karen said in 2007. “As soon as the computer learns to recognize the swirls of Van Gogh, it will recognize them in pictures it has never seen before.”

While still in its infancy, computer vision and analysis of art could one day prove useful in helping curators detect fakes.

Computers are also learning to recognize amateur art, as well as the works of the masters. In September, Brown University researcher James Hays and colleagues from the Technical University of Berlin produced a computer application that can recognize quick sketches of objects, even if the draftsmanship is lacking.

Hays and his colleagues collected sketches of everyday objects they obtained through Amazon's crowdsourcing Mechanical Turk program. The scientists fed the images into a computer, grouped by what object they were supposed to represent, and told the computer what the sketches depicted.

Now, the program can recognize drawings in real-time -- as a user draws on a tablet or computer, the application gradually zeroes in on what the drawing is supposed to represent.

Hays told the Brown Daily Herald that one day vision programs like his could help illiterate people use computers.

SOURCE: Shamir et al. “Computer analysis of art.” ACM Journal on Computing and Cultural Heritage 5: 7, July 2012.