Turn your photos into algorithm paintings
If you would like to turn one of your photos into a unique algorithm painting, send me some photos and I’ll run them through the algorithm. Don’t forget to mention which of the algorithm variations you would like to use (Wavelets, Impressionist, Cactus or Animals). With your consultation I can then tweak the algorithms for a very personalised result and even highlight some details. If you like one or some of the final outcomes you can order them as a print.
The drawings and paintings of artists can teach us a lot about how we perceive our world and how the visual system works. In this project I explore how artists make use of the tricks of our visual system in their art and then write algorithms that use the same principles.
When people make a drawing of their surroundings, they often define objects by a line, which makes it easy to distinguish the object from the background. In the real world however, those lines are not there. It’s our brain that make us think that objects are clearly defined, even when there are a lot of shapes and colours in the background. The processes behind this phenomenon are known as edge detection. Mathematicians interested in computer vision are trying to mimic our visual system so that computers can make line-drawings from their surroundings as well (example). This can then, for example, be applied in robots to make it easier for them to distinguish objects from the background. At the moment however, we humans still do a better job.
Things begin to get even more interesting when we look at the variety of painting styles that artists have developed to express their impression of the world. Without explicitly knowing these things, often the styles used by artists show links with how our visual system processes images. I have tried to take some of these concepts from vision and used them to develop a computer program that turns photos into paintings with different styles to see how close they will resemble paintings that could have been produced by a human.
The first three photos were taken on a really nice hike with friends around Loch Venacher in October 2017 and the last photo is a view of Loch Lomond taken from Ben Lomond during another hike with the same friends the month before.
The first Painting algorithm I made is called Wavelets. Here, I have used slightly transparent rectangles as “brushstrokes” and made the direction of each brushstroke dependent on the contrast in the photo. In the example below you see how this results in the brushstrokes roughly following the shape of the hill.
The algorithm uses a rudimentary form of edge detection to highlight the most striking edges. If you want to know a bit more about the algorithm check out the extra information below:
At the strongest edges in the photo, where the contrast is highest, the brushstrokes consist of a black & white bar. This is a reference to “wavelets”, which are Gaussian Gabor functions with varying sizes, directions and spatial frequencies like below.
These wavelets are used by our visual system to build up an image.
The hues of the coloured brushstrokes are directly inherited from the photo. The brightness however, is changed to the same level for all strokes apart from very bright areas such as the sky (although this is optional). This makes the paintings look quite flat. Another consequence is that some of the hues when made brighter in this way are a bit unexpected, many strokes are blue where you may have expected a different colour. This is due to a trick from the visual system called colour constancy. When the sun is just above the horizon, the light is red, but when the sun is at its highest point the light is much more blue. Nonetheless, we always perceive oranges as orange and tomatoes as red, while in reality their colour changes depending on the time of day. Our visual system automatically corrects for these differences in colour by comparing the colour of an object with its surroundings. The algorithm doesn’t care about colour constancy and as a consequence you get surprising colours that make the painting quite abstract at times.
The input of the Wavelets Algorithm is a photo, click here for some more examples. The resolution of the photo does not have to be high for the algorithm to produce something aesthetically pleasing. Interestingly, the resulting images that I liked best were not the best photographs. If you would like to turn some of your photos into algorithm paintings, please send me an email with your request and a set of photos.
My partner Lawrie and I went on a long weekend to Inverness and one day we took a boat trip on Loch Ness. The first photo was taken from the spot where Lawrie’s brother had proposed to his wife just a month before (2017). The other two photos were used for the Wavelet algorithm as well (see above) and are taken around Loch Venacher.
The algorithm Impressionist is inspired by pointillism, a painting technique related to impressionism in which small dots are applied to the canvas that at a distance blend together. Instead of painted dots, my algorithm uses tilted blocks of pixels that are generated making use of the underlying brightness and hue of the original photo. The exact composition and location of these blocks have been randomised so no two renderings of the painting are the same – they are totally unique. To enhance the pointillist feel of the paintings the blocks making up the darker parts of the painting have lighter pixels mixed through and vice versa, the lighter blocks also contain darker pixels. A close-up of the blocks that make up the painting is shown below, revealing that the painting is digital. However, at a distance it looks as if it has been painted.
For more examples of the algorithm Impressionist click here.
The first photo was taken on Islay in December 2017. The second photo is a close-up of the Kelpies in Falkirk. The third photo is from a cactus at my parent’s home in Amsterdam.
This algorithm is based on the Wavelet algorithm, but shows different behaviour in areas with high contrast. It draws thin lines when it finds borders and shadows. The lines are straight and therefore give the paintings a bit of a prickly feel, which is why I gave it the name Cactus. Take a look here for some more examples.
These photos were taken on Arran with a group of friends. One of the group had started an B&B on Arran, so we were all invited to visit. This photo was taken in the garden where this golden-ringed dragonfly was taking a wee nap (2018).
This algorithm uses very short black brushstrokes to define edged, making them look soft. I’ve also set a brightness threshold to turn objects in the photo that brighter than this threshold into very light areas in the painting. This causes the abstract pattern in the background which I think is an interesting effect. Click here for more examples.