Lucasfilm's deployment of AR will create the real world of "Star Wars"

Abervitz said: "We develop crazy clips, one-to-one size ships and characters, things that make you feel like you're in the scene. I could see certain people, a spacecraft taking off, robots running. It's like you walk into a scene, but it's very real." The ultimate goal is to turn the world of Star Wars into reality.

Wired magazine recently published an article saying that the robot C-3PO in Star Wars is impressive, but you may not imagine using this robot in your own living room. This week, Magic Leap and Lucasfilm's ILMxLAB announced a collaboration to bring such robots to ordinary users.

The following is the main content of the article:

Magic Leap founder Ronny Abovitz announced at the "Wired Business Conference" held in New York City last week that he has partnered with xLab, Lucasfilm's most forward-looking division.

It is reported that xLab was established last year with the goal of developing content experiences for immersive platforms such as virtual reality/augmented reality/mixed reality. In the future, the two parties will use Magic Leap's technology to jointly develop content related to Star Wars.

The two sides have not officially announced what kind of experience they will provide or whether Magic Leap's technology will be available to end users. However, the video Abervitz showed at the conference showed one possible application. This video features two of the most famous characters in the science fiction world and can talk to you. They are not in your world, but you are in their world.

You will encounter various problems. You will see the C-3PO asking you,"Can I talk to you? I regret to report that due to unforeseen circumstances, we were unable to reach the required agreement with Jabba the Hutt." At the same time, the R2-D2 robot came to the table in the room and projected holographic onto the table. He is not floating in the picture, the table top covers his lower body.

These are not characters played by actors, but computer images that look very real, but are different from anything you have seen before.

Lucasfilm and ILM have also previously worked with other companies, particularly technology companies. Similarly, Magic Leap has been working with New Zealand special effects production company Weta for many years. However, this time is not just a cooperation between the two companies, but a combination of two groups of crazy talents. They want to take Star Wars off the big screen and onto the streets.

To achieve this goal, the two sides have established what Abervitz calls a "semi-secret" joint laboratory at Presidio's ILM headquarters in San Francisco, which will include researchers and programmers from both companies and even Lucasfilm Story Group members. Vicki Dobbs, the xLab executive responsible for the matter, said: "I have been at ILM for 24 years, but I can't think of a similar experience. We have never seen anything that can tightly combine creativity and technology and bring innovation."

Adding "magic" to Star Wars

When xLab was founded a year ago, Lucasfilm and Magic Leap were discussing how to develop a longer-term relationship. xLab co-founder John John Gaeta joined ILM in 2013 as creative director focusing on emerging platforms. As he said, his job is to "listen to rumors from friends and colleagues and find talents who focus on new media." Under such circumstances, Gaeta began to hear about Magic Leap.

He contacted the company's founder, Abervitz, via social media, and the two sides began to communicate. Abervitz introduced Gaeta to veteran game developer Graeme Devine, who had just joined Magic Leap at the time. In the fall of 2014, the two invited Gaeta to Florida to see Magic Leap's technology in person.

What Gaeta first saw was not particularly attractive, when Magic Leap was developing a huge size prototype, The Beast. However, this is an unprecedented technology. Geeta said: "I've seen some crazy science laboratories with complex devices. This part didn't impress me. But after they started showing me, I experienced something I had never experienced in my life: I saw a computer-generated structure, stared at a point along the length, and the structure would adjust with the focus of my eyes."

This demonstration made Gaeta a supporter of Magic Leap. Gaeta was impressed, but he still needed to convince his colleagues. He said: "Even in the studio founded by Lucas, people are awakening slowly. Like other major events happening here, the arrival of a new era for Star Wars will take some time." By January 2015, a full ILM team had gone to Magic Leap. Subsequently, Lucasfilm President Kathleen Kennedy and General Manager Lynwen Brennan also visited the company.

There are certain reasons for the long exploratory contact between the two sides. Although Magic Leap has received more than $1 billion in financing, there is still no guarantee that the company will achieve long-term development. "They are one of the coolest companies ever, and we are just a small startup," Abervitz said. I think they had been observing us for some time before they came: "Will you survive six months later? Can you continue to deliver products?' It's important for them to continue to understand us."

But as the conversation progressed, the two companies began to see a shadow of themselves in each other. Rob Bredow, chief technology officer at xLab, said: "We were impressed by their approach from the beginning. What they say about 'add some magic' is very consistent with what John and I expect from xLab to put you in the magnificent world we've made."

Bredo calls the technology "experiential storytelling," like walking down the streets of Disneyland. He said: "The way you get content is different than you do in a movie theater. The experience they provide through immersive entertainment technology is somewhere in between."

Using ILM resources, Magic Leap's team began exploring such experiences. Abervitz said: "We develop crazy clips, one-to-one size ships and characters, things that make you feel like you're in the scene. I could see certain people, a spacecraft taking off, robots running. It's like you walk into a scene, but it's very real." The ultimate goal is to turn the world of Star Wars into reality. "We want to become a storytelling medium and eventually the standard medium."

Future development

This is a future-oriented approach. The two sides are still studying how to operate this joint laboratory and maintain daily communication. The focus of this project is on the one hand on what we know about the Star Wars experience and on projects that are still under wraps. Rachna Bhasin, Magic Leap's chief commercial officer, said: "We have seen a roadmap for the future, and if you think about it carefully, it will be very crazy. We are envisioning the experience of iconic characters and how we can expand the story and how the characters and clips can be presented in non-big-screen ways."

Lucasfilm President Kennedy said: "The partnership with Magic Leap is the relationship we hope to have: inspiration, cutting-edge technology, and innovation. With the help of world-class storytelling methods and technological talent in our joint laboratory, attractive mixed reality entertainment will take one step further."

So how will it develop after this? Gaeta has some ideas, which have to do with how you perceive Star Wars. Maybe X-wing fighter jets will fly randomly over your home, or maybe creatures from Cantina will climb onto your sofa. As artificial intelligence becomes more powerful, you can even talk to certain characters and share a glass of blue milk. For veteran science fiction fan Abervitz, such technology won't appear anytime soon. He said: "My life has been greatly influenced by Star Wars. I think it's because of these robots that I founded a robot company." C-3PO helped him regain his original intention, and based on current work by Magic Leap and ILMxLab, Abervitz may soon be rewarded.

Editor: Nancy