He was scrawny, malevolent, pathetic, terrifying—his desiccated body clad in a dirty loincloth, wisps of stringy hair dangling from his skull, bulbous eyes bloodshot, mouth a rictus of rotten fangs. When Gollum came into full view in the second Lord of the Rings film in 2002, he was hailed as a special effects marvel—the most realistic character created by computer graphics (CG) in the history of cinema.
Marschner in Cornell’s Gates Hall.Jeff Lower
Gollum’s startling verisimilitude, and that of the CG creations who’ve come after, owe their existence not only to the Hollywood digital effects artists who designed them, but to the computer scientists who enabled that work in the first place. That makes Steve Marschner, PhD ’98, one of Gollum’s godfathers—and he has the Academy Award to prove it.
A professor in Cornell’s computer science department since 2002, Marschner is a longtime CG researcher. He specializes in rendering natural materials—like skin, hair, wood, and fabric—that can be devilishly difficult to depict realistically. “If you look at the history of computer graphics, we got really good at plastic early on,” says Marschner, chatting with CAM in his Gates Hall office last fall. “It’s a very simple material. It’s homogeneous, it’s smooth on the surface, and it doesn’t have a lot of structure. Manufactured materials tend to have those properties, but things that are natural are much more complicated.”
Take skin, on which Marschner did some groundbreaking work on translucency that helped make Gollum seem so lifelike. As he notes, “Light doesn’t just reflect at the surface of the skin; it goes through it and interacts with the blood vessels, structures, and tissues below, and then it emerges. It’s much more complicated than, say, a ceramic cup”—he proffers a mug bearing the logo of Pixar, the famed animation studio where he spent last summer working with digital artists—“which is very opaque and homogeneous.”
If you are enjoying this article, please subscribe today. Cornell Alumni Magazine is better in print—and funded primarily through print subscriptions.
Part of Marschner’s research process is to tap expertise on the material in question, so he can study it at a more fundamental level than its outward appearance. Working on depicting wood, for instance, he has consulted with CALS forestry researchers; for fabrics, he’s talked to colleagues in Human Ecology’s Textiles and Apparel department. And in studying how to depict skin, he and his colleagues looked up medical physics literature regarding how to calculate dosages of laser therapy—since researchers in that field, he says, “have the same problem of how to compute how light bleeds through tissue.”
The citation for his technical Oscar, which hangs in his office.Jeff Lower
For the skin research, Marschner and colleagues won a 2004 technical Oscar. While that didn’t earn him a ticket to the star-studded red carpet ceremony that’s broadcast on TV, he did get to don a tux and accept the award—a plaque, featuring an image of the iconic statuette, that now graces his office wall—at a separate gala hosted by actor Jennifer Garner. “If you win the Oscar you get a statue,” he says with a laugh, “and if you win a technical Oscar you get a picture of the statue.”
Another major focus of Marschner’s research is hair, a substance that he says is much more challenging to depict than it may seem. The crux of the work—which has informed such animated films as Tangled, Zootopia, and Monsters University as well as live-action movies like the 2005 remake of King Kong—is understanding hair’s surprisingly complex structure, figuring out how that affects the way in which it reflects light (and therefore how we see it), and translating that knowledge to computer models.
While some people’s hair is made of round strands, he explains, in others they’re elliptical or flat. And the strands aren’t smooth; the surface is covered in scales, called cuticles, that are laid atop each other like roofing shingles. “Hair is interesting for a lot of reasons,” Marschner says, “but the most basic fact is that it’s made up of long, skinny objects. If you have perfectly combed hair, you’ll see a bright highlight that goes perpendicular to the fibers; in computer graphics, the geometry you need to figure out where that highlight happens has been known for a quite a while. But if you try to make things more accurate, you start to realize that hair is almost more like a glass cylinder than an opaque rod. Lots of light goes into the hair and refracts in and out, and the shape of the hair’s surface has a lot to do with that.”
Marschner’s work doesn’t just have applications in the virtual world. He’s the co-founder of a start-up company, called Weft, that aims to let people design their own fabrics and order them online. The project, a collaboration with colleagues at the Rhode Island School of Design, is based on a computer model allowing for highly realistic depictions of potential fabrics, generated from the level of each woven strand. Marschner’s research is also informing imaging that could help online shoppers—who can already view outfits modeled on rudimentary avatars of themselves—get a more realistic idea of what they’re buying. “There’s a difference between a polyester blouse that’s designed to look silky versus one that’s actually made from silk; it’s got a little bit different drape and sheen,” he says. “These things are subtle, but to make buying decisions you’d like to be able to see them. That requires a lot of accuracy.”
Asked if his insider knowledge of CG makes it harder to suspend his disbelief during a night out at the cinema, Marschner says that it depends—not so much on the technical elements, but on how good the film is. “The better the movie, the easier it is to not worry about the CG,” he says. “Sometimes I say that I have to watch a movie twice—once to pay attention to all the graphics, and again to ignore them and just watch the movie.” But, he adds, technology has advanced to the point where it can be hard even for experts to tell the real from the digital. “In the Lord of the Rings movies,” he says, “I remember thinking that they did a really nice job on the CG for some of the remote villages and sets—and then watching the DVD extras and realizing that they actually built all that stuff.”