Originally published on LinkedIn. The cover image was created using images generated by Shedevrum and DALL-E 2.
I’m Yulia Afletunova, and I’ve been working with data broadly for over five years. I started as a product manager, which was based on AI technology. I had time to be an ML engineer, and for almost four years, I have focused on the product side of data analysis.
In autumn, I came up with the idea of making a model that would use art images to determine a work’s authorship. Of course, I went to study existing solutions, and so I came across one very interesting story.
Let’s start with the fact that, in general, only a few groups of experts are engaged in identifying the originality of paintings. First of all, it is difficult to determine the authorship of a painting, if only because even an expert with many years of experience often can not say for sure whether the artwork is original or not. Many tests (examining the materials from which the canvas is made, taking X-rays of the painting, conducting laboratory tests, and more) and a detailed visual examination are necessary. Textures, depth of strokes, and what is not in our non-2-dimensional world help to determine the authorship of a work. On visuals alone, it is difficult to understand who painted this or that masterpiece. Moreover, it is only possible to say unequivocally about the authorship by studying other paintings by the same artist.
That is why there are agencies such as Art Recognition. Art Recognition is one of the few who have taken on the task of verifying the authenticity of works of art with the help of artificial intelligence. The word “help” is important here because, as far as I know, they do not limit themselves to artificial intelligence technologies alone. They do it very nicely — check out an example of their report on their website. There are some fascinating research papers there as well; I recommend looking at them.
And here’s the news from the world of artificial intelligence that has been on my mind for six months now, as if I were 16 again and reading a Dan Brown novel intermittently.
January 2023. An article appeared on the University of Nottingham website stating that a painting by an unknown author known as Tondo de Brécy, according to research by a team from the University of Nottingham and the University of Bradford, was probably created by the same artist as the Sistine Madonna — that is, the famous High Renaissance artist Raphael Santi (Sanzio). The discovery is, let’s face it, astonishing. And expensive.
Here are a couple of details about how this research was carried out. Tondo de Brécy has been studied in the art world for a long time and from all sides. To the naked eye, these are really two virtually identical works. However, experts have argued for about 40 years and could not agree on whether a great artist painted this work or whether this is a copy created roughly in the same period. And now, in January, the world has learnt that a neural network developed by Professor Hassan Ugail’s group has revealed that the Madonna matches the original Raphael Madonnas with 97% similarity, while the Child matches with 86%.
July 2023. The painting was exhibited at Cartwright Hall Art Gallery in Bradford, UK. This was quite a significant event.
October 2023. An article came out with the headline “Battle of the AIs: rival tech teams clash over who painted ‘Raphael’ in UK gallery”. As is common in news headlines, “battle” is an exaggeration in this situation, as is the competition between the teams. But the question is indeed an interesting and intriguing one. The fact is that at the same time, Art Recognition, represented by its CEO, Dr Carina Popovici, is conducting a separate AI-assisted study and reveals that the painting has an 85% probability of being painted by someone other than Raphael.
What is the difference between the two models? Carina Popovici tries to explain it in another publication. The fact is that the DNN used by the group from the University of Bradford is based on face recognition. But, as I mentioned, more than visual information is needed to determine the authorship of a painting. Art Recognition, therefore, uses datasets curated by historians and AI developers, supplemented with information about the work’s strokes and chromatics — i.e., the richness of the colours. Moreover, they populate their datasets with many copies, imitations and even fakes created by other AIs.
In the article, Carina Popovici calls attention to the fact that AI can be used as a valuable tool to enrich artistic expertise without contrasting its development with that of its peers. In addition, she adds that the team at the University of Bradford has yet to build a link between facial feature similarity metrics and the authenticity of a work and hopes that the paper by her colleague, Professor Hassan Ugail, will bring the necessary clarity to this issue.
December 2023. Prof Hassan Ugail and colleagues’ paper “Deep transfer learning for visual analysis and attribution of paintings by Raphael” has finally been published in Springer. They extracted features using a pre-trained ResNet50 model (omitting several layers) and used SVM for further classification (authentic or not). Beyond that, they used different edge detection methods to assess style and authorship. This sounds closer to what was discussed in Dr Karina Popovici’s article.
Early March 2024. The Financial Times releases a new chapter of this story. Here, both Ugail’s group and Popovici’s agency come to the common conclusion that, at this stage, AI is not a substitute for human expertise and needs some details and data that we can only observe as humans. At the same time, Ugail notes that the method proposed by Popovici is one of many correct ones, and other methods of authenticating works of art can be developed. But they still cannot replace humans, just as neither humans nor AI can unequivocally determine the authorship of an artwork. And all experts agree with this.
I want to say from myself that there will be more and more such stories, and that makes me happy. I am not frightened by how much artificial intelligence enters our lives, work, and learning. I see a window of opportunity to expand my field by exploring new technologies and methods, both for me and for the industry.
How often do you think about where else modern technology can be applied?