Using your Structure Sensor for the first time

The Structure App

The Structure app is the first app for the Structure Sensor available on the App Store. The Structure app includes a number of visualizations to show the raw data being captured and streamed by your Structure Sensor to your iPad.

The first is the infrared feed being captured by the Structure Sensor's infrared camera. To view this feed, connect your Structure Sensor to your iPad via the included Lightning cable, and then open the Structure app. Then, tap "IR" on the right hand menu.

In the lower left hand corner, you'll see a light bulb icon. Tap that icon to turn on the infrared LEDs to flood the scene with additional infrared light.

The next visualization you can explore is the raw depth stream being captured by your Structure Sensor. Tap the "Depth" link in the right hand menu to view the depth stream.

You'll notice that there is a circular target with a measurement next to it. This is the exact distance from the Structure Sensor to the object that the target is covering. You can move your iPad to move the target, or you can tap the screen to move the target without moving the iPad.

Now that you've seen the depth stream, you can see it registered with the color images coming from your iPad's megapixel camera. Tap the "Depth + Color" link in the right hand menu to see this.

Note: The alignment between the depth and color images will be imperfect when you first receive your Structure Sensor. An over-the-air update will be sent to your Structure Sensor in the future to correct this.

The Structure sample app also lets you update your firmware, as well as check the status of your Structure Sensor, including: serial number, firmware version and battery charge. To check the status of your Structure Sensor, tap the "gear" icon in the lower right hand corner of your screen.

Using the Fetch Sample App

Ready to use your Structure Sensor for augmented reality gaming? We've included a simple but fun sample app called Fetch (available on the App Store) to let you do exactly that! With Fetch, you control a cool cat on a hover board as he flies around your house, fetching power-up balls.

Finding a Suitable Scene

Fetch is more fun when you set up a scene with obstacles that your character can go on top of and behind. You might try putting down a few boxes, or a short table, or another obstacle that will become part of the game world

Fetch will let you use a space that's about two meters by two meters for gameplay. Make sure that you have room to walk around this space so you can scan in all sides of the obstacles within it.

Creating Your Game World

As soon as you've set up your scene and opened Fetch, use the pinch and zoom control on your iPad's touchscreen to adjust the area that the Structure Sensor will scan in for gameplay.

As soon as you've chosen your game area, hit the "Scan" button in the lower right. You will see the Structure Sensor capturing the obstacles in the area. Walk around the obstacles to scan in all of their sides.

If the Structure Sensor loses tracking while you scan in other sides of your obstacles, then try simply scanning obstacles from a single perspective.

As soon as you are done, hit the "Done" button. Now you're ready to play!

Fetch Game Controls & Gameplay

Playing Fetch is easy. Tap and hold your finger anywhere on your iPad's touchscreen, and the hover boarding cat will go to where your finger is. If you release your finger, the cat will slow down and stop.

To make the cat jump, use a double tap with two fingers. This will help you get your hover boarding cat on top of obstacles in the environment that you can't get around.

Hover over to one of the glowing balls for a power up that will let your hover boarding cat jump even higher.

While you play, notice how the hover boarding cat can interact with the obstacles you scanned in. He can hide behind them, jump on top of them, or go up them if they slope upwards. And, of course, he can't go through them.

Using The Viewer Sample App

The first sample app to try when you first attach your Structure Sensor to your iPad is the Viewer sample app. Viewer is available in the App Store.

Viewer provides a simple view of what the Structure Sensor sees alongside what your iPad's color camera sees. Here is what Viewer's output looks like:

Using the Scanner Sample App

In addition to using Uplink with Skanect, the Structure Sensor arrives with another option for those interested in 3D scanning. This is a simple 3D scanning app called Scanner, and is available as a sample app on the App Store. It's easy to start getting great looking scans quickly and consistently if you learn just a few simple things about using this app.

Finding Suitable Objects

A great first test for the Scanner sample app is to try scanning another person from the shoulders up to the top of the head.

The Scanner sample app works best for scanning objects the size of a medium-sized stuffed animal up to a human bust. Smaller items will lack detail and may challenge object tracking. Larger objects may be difficult to capture in their entirety.

The Scanner sample app may have difficulty capturing black surfaces and shiny surfaces. In addition, the Scanner sample app will have difficulty capturing intricate features, such as hairs, wires or other fine objects.

Setting Up Objects To Scan

For the best quality scan, please follow these general guidelines:

• Stand about 1 meter (or 3 feet) away from the object or the person
• Make sure that you have an unimpeded 360-degree path around the object or person so that you may capture an entire scan easily
• For objects, placing them at a standard tabletop height allows for the best combination of scanning quality and ease
• For objects, placing them on a smooth, flat surface ensures that the Scanner sample app will consistently capture great scans
• The Structure Sensor works best indoors or outside of direct sunlight

Positioning the Scan Area Cube

When you first open the Scanner sample app and connect your Structure Sensor to your iPad, you will see a large transparent cube appear on your screen. The bottom of this cube has an orange square. The orange square will automatically snap itself to flat surfaces such as floors or walls so that your scan is positioned on a flat plane.

Alternatively, if you point the Structure Sensor towards a person, the transparent cube will position itself around their upper torso and head for a bust scan.

To scan an object, start by standing about a meter (about 3') away from the object. Then, center the object within the transparent cube. Once the object is within the cube, you will see that the orange square will now adjust itself to fit over the object you plan to scan.

If parts of the object you plan to scan are not covered in orange, you are standing too close. Back up until those parts are covered.

Using the "Pinch" Gesture

Because the cube determines what will be captured by the Scanner demo, it is important to make sure the cube is the right size for the object to be scanned.

The size of the transparent cube can be easily adjusted using the typical iPad "pinch" gesture. Push your thumbs together on your iPad screen to make the cube smaller, and pull them apart to make it bigger.

If the cube is too small to contain the object you plan to scan, you can expand the cube by pulling your thumbs apart on your iPad screen. You may also try to move backwards so that you are further from the object you plan to scan.

If the cube is too large for the object you plan to scan, then your scan will be of poor quality and you may potentially capture objects in your scan that you do not want. You can shrink the cube by pushing your thumbs together on your iPad screen. You may also try to move closer so that you are nearer to the object you plan to scan.

For the highest quality scans, pinch to shrink the cube so it is just a bit bigger than the object within it that you plan to scan.

Scanning the Object

To begin your scan, press the blue "Scan" button.

Once you have started, move around the object to capture it. If you notice that there is a hole in the model, you can go back during the same scan to see if the Scanner sample app can fill it in.

If you have made a mistake during your scan, you can press the red reset button with the circular arrow on it to restart your scan.

Click the "Done" button when your scan is complete.

Maintaining Tracking

On your first few scans, you may notice that the Scanner sample app loses track of the object. A warning note will appear on your screen:

Try to rematch the scan to the object you are scanning to restart object tracking. If that doesn't work, try your scan again. If you consistently get object tracking errors during a scan of the same object, that object may not be suitable for scanning.

Viewing Your Completed Scan

To view your scan, click the "done" button. A review screen will appear. You can use your finger to drag the model around to see all sides of it.

You can also click "X-Ray View" to see the underlying mesh of the model that you have captured.

To share or export a scan, click the "email" link in the upper right hand corner. Click this to email an .OBJ file of your scan to yourself or a friend.

Click the "Back" link to reset the app so you can start your next scan.

More apps are coming soon!

At structure, we're planning on releasing more sample apps to expand the functionality of your Structure Sensor. In addition, developers worldwide are now using the Structure SDK to create more great apps, too.

We'll be sure to keep you posted whenever new sample apps are available for your Structure Sensor. And, if you're a developer, we're always here to help you build your own great app for the Structure Sensor!

Tutorial Videos

• Viewer and Scanner sample app tutorial
• An introduction to Uplink
• Skanect video tutorials

Need more help using your Structure Sensor?

If you have more questions, our Support Center has answers.