Data-driven vector graphics (2016)
Advances in machine learning has enabled researchers to create impressive data-driven tools and techniques for graphics.
However, we have not seen such advances in vector graphics. To this end, I have started looking into
vector graphical data and its applicability to machine learning, with the goal of producing useful data-driven tools for
Vector graphical data can be relatively noisy with vector layers representing a wide range of visual cues, like shape and
shading. As a first step in investigating this data, I decided on a more 'controlled' approach where the different components
in a training vector image are unambiguously labelled. This results in costly labelling of training data, but can give rise
to good or accurate results. In my Pacific Graphics
I demonstrate positive results for this approach.
2D colour gradients in vector graphics (2013-2015)
Vector graphics provides a powerful suite of artistic tools to define scalable 2D graphics. It is employed heavily in graphics design,
where crisp image quality at any resolution is essential. Vector graphics is also becoming increasingly popular
on the web because images are frequently scaled to fit any given viewing device. However, current vector graphics primitives, employed in
formats like SVG, do not handle complex variations in the colour function very well. From interviews with professional artists and designers,
pixel-based tools, such as Adobe Photoshop, are preferred over vector-based tools when the design requires more sophisticated variations in colour
than what the standard linear gradient tool can handle. This reluctance to employ vector graphics has spurred many graphics researchers, including myself,
to investigate whether more powerful, yet flexible, vector primitives can increase the attractiveness of vector graphics for graphic design.
The shading curve primitive, proposed in a 2015 CGF article,
represents a light-weight primitive for defining smooth 2D colour gradients from curves. This primitive was inspired by traditional drawing techniques
for achieving chiaroscuro. To achieve such effects in vector graphics, we argue that explicit control
over the colour gradient is needed. We believe we achieve such control via our so-called 'shading profiles', which are curves representing the colour
gradient perpendicular from a given geometric curve. For more details, have a look at our
project webpage and our Eurographics 2016 presentation (PDF or PPTX).
The gradient mesh tool is the current tool employed
by professional vector designers for defining 2D colour gradients. The mesh is topologically restricted to rectangular meshes, because it employs
Ferguson patches as its underlying interpolation method. We argue that this restriction heavily hinders the flexibility of the tool, an argument
not too different from supporters of subdivision surfaces in 3D design. Using subdivision surfaces, we were able to define an interpolation method
that supports arbitrary topology of the gradient mesh. See our CGF article for more details. A prototype software
of the tool, implemented by student John Kasper Svergja, is available on GitHub.
An alternative primitive that has received a lot of attention in the research community is the diffusion curve primitive.
In my PhD dissertation, supervised by Neil Dodgson and co-supervised by Jiri Kosinka, I argue for a superset primitive with support for both diffusion curves and gradient meshes. A brief overview of this
idea was presented at MMCS9 (an international conference on curves and surfaces: PDF and PPTX).
To achieve such a primitive, several problems must be dealt with. One of them is to provide fast random-access interpolation of
diffusion-curve constraints. A solution to this problem is to perform GPU 2D ray tracing. I have expanded the solution space of this approach to
support for diffusion points and local colour gradient constraints. This work was presented in a poster at SIGGRAPH 2016 (abstract
and poster pdf).
Artistically-inspired imaging (2011-2012)
Artistically-inspired imaging within my research field has been going on since the early 1990s. First, great techniques have been perfected over the
centuries to achieve perceptually-related effects (e.g. impressionism) and photorealism. A common theme in computer graphics is to automate these techniques
so that they can be employed in filters and tools. Second, looking at the development of art in the past centuries, we see a huge range of different types
of visualisations. To algorithmically produce these visualisations has turned out to be challenging - solutions to these visualisation problems
can be seen in applications like Photoshop, 3ds Max, and mobile apps for image and video editing.
Neo-impressionist paintings are filled with vivid colours and strong contrasts. The most prominent artist during this period,
Georges Seurat, used a beautiful blend of colours and an innovative contrast-enhancement
technique, based on the psychophysical 'mach band' illusion, to achieve
the impressive neo-impressionist look. It can be experienced, for instance, at the National Gallery in London where the masterpiece
'Bathers at Asnières' is on display. With collaborators Tania Pouli and Erik Reinhard, then at the Max Planck Institute for Informatics, we aimed to
emulate the contrast-enhancement technique of Seurat. Our method, which does not produce the visible halo-artefacts often created by previous methods, is
presented in a Computers and Graphics article.
The British painter Clive Head takes
an interesting approach to depicting a particular scene: he gathers visual information of the scene from different viewpoints and
sources, like photographs and sketches. The goal is never to faithfully reproduce the scene, but rather to depict an alternative, perhaps a more interesting,
scene or reality. Thus, he takes information from multiple viewpoints and includes this information into a single painting. Inspired by Head's approach, aided
by my collaborators Jan Kautz and James Tompkin, then at UCL, I made a framework for making multi-perspective images and videos. The framework, using
computer vision techniques, requires much less setup compared with previous methods. For more information, check out our
Abstract art (2012)
My supervisor at Cambridge, Neil Dodgson, is particularly interested in abstract art and has previously contributed to the field of computational
aesthetics. I was challenged to improve on recent work on
multi-class blue noise sampling, which can be used to produce coloured dot paintings,
similar to Damien Hirst's spot paintings. The previous work used a random-dart
sampling procedure, which is computationally expensive. In the discrete setting, where you only need to sample the colour dimension, and not the geometry
dimension, this seemed like an overkill. My method, presented at the computational aesthetics 2012 conference, outperforms this previous method in the discrete
setting by orders of magnitude. See our
project page for more details.