Designing tools for 3D content authoring based on 3D sketching

The increasing accessibility of real-time 3D rendering hardware has made 3D content creation a major means of expression and storytelling. But authoring 3D content requires interacting with the digital representations of shape and appearance that are compatible with rendering and animation algorithms. Triangular meshes, parametric material models and animation curves, while well suited to downstream computation, require artists to convey their ideas in terms of low-level commands that need to be learnt and remembered.
In this thesis, we explore the use of 3D strokes as a way for artists to express their ideas. Inspired by the way artists work with brush and canvas, we consider the artist's mark-making gesture as the main input to the authoring system. 3D strokes are flexible primitives that can be created in either 2D desktop user interfaces or in virtual reality (VR) interfaces, and they can encode a 3D shape or likewise the final appearance of a 3D painting. Designing tools that consider 3D strokes as a shape or appearance representation opens a large and exciting space to explore.
Designers can use 3D strokes as a partial representation of 3D shape. We investigate how to interpret a sparse 3D sketch into a 3D surface model. Since feature curves are a prominent part of the design and are finely depicted by the sketch, we recover a piece-wise smooth surface that preserves those sharp features. By obtaining a surface from 3D strokes, our algorithm allows to render the shape depicted by the sketch.
To better understand how 3D strokes can depict not only the shape but also the appearance of objects, we study the practice of VR painting among a community of artists that work with a commercial VR painting software. Based on this inquiry, we propose a design and implementation for 3D-Layers, a new interaction primitive for VR painting that embraces 3D strokes as the sole representation for both 3D shape and appearance, yet decouples edition of these two elements. Inspired by the usage of layer compositing in 2D digital painting, we support a non-destructive workflow to edit the appearance of a VR painting.
Hand-drawn animation is an expressive way to convey an animation with strokes. In "video doodles" animation, artists create an animated doodle that seems to live in the same 3D space as a captured video. Taking into account perspective effects and occlusions while drawing 2D strokes is not an easy task, so we leverage computer vision techniques to place strokes in 3D space and render them with respect to the video context. We design a 2D user interface that resembles traditional 2D motion design tools, to enable usersunfamiliar with 3D tools to create such animations.
Overall, we show that 3D strokes are an expressive representation for 3D content creation by proposing three systems that leverage 3D strokes or 3D sketches as interaction primitives for creative applications spanning shape, appearance and animation authoring. We approach system design from two complementary perspectives ; we develop novel algorithms to interpret strokes and low-level user input, and we design interactions to provide new ways for people to express their high-level intent.

@PhdThesis{Yu23,
  author       = "Yu, Emilie",
  title        = "Designing tools for 3D content authoring based on 3D sketching",
  school       = "Universit\'e C\^ote d Azur",
  year         = "2023",
  url          = "http://www-sop.inria.fr/reves/Basilic/2023/Yu23"
}