Getting Started with your Structure Sensor Pro

Congrats on your new Structure Sensor! We're sure you're eager to start exploring Structure Sensor's capabilities. To help you get the most out of your experience, we've put together this quick start guide to help familiarize you with this advanced accessory.

What's in the box?

Structrure Sensor Pros purchased directly from the Official Structure Sensor Store will arrive in a gift box, like so:

contents of Structure Sensor Pro box

Your Structure Sensor Pro should arrive with the following:

  • One Structure Sensor Pro
  • One USB charging cable
  • One magnetic screwdriver
  • Ten (10) M1.4 screws
  • One user manual
  • One Screw Information Card

Precision Bracket with Aluminum Latch

Precision Brackets with Aluminum Latch for most iPad models will arrive in a "bento box" style, which doubles as a carrying case for your assembled Structure Sensor, bracket, and cables:

Pictured: Newer, bento-style, Precision Bracket with Aluminum Latch packaging.

Structure Sensor Bracket For iPad Pro

Your Structure Sensor iPad Pro Bracket should arrive with the following (shown from top to bottom):

  • One Bracket Alignment Tool
  • One Structure Sensor Base Plate
  • One Bracket

Charging your Structure Sensor Pro

To preserve the battery life of your iPad, Structure Sensor Pro has its own internal battery. Charging your Structure Sensor's battery is simple: connect the charging cable to your sensor, and then plug the USB end into any strong USB 2.0-compatible device.

You can expect the following charge times:

  • 1.5A @ 5V = ~30 minutes (from a <5% charge state)
  • 0.9A/7.5W @ 5V = ~60 minutes (from <5% charge state)

0.9A can usually be expected from most computers. 1.5A can be expected from most USB wall chargers.

A blue LED indicator on the Structure Sensor's glass face will pulse when your Structure Sensor's battery is charging. As soon as your battery is completely charged, the LED indicator will stop pulsing and instead will stay on until you disconnect the charging cable.

In addition to connecting your Structure Sensor to any USB 2.0-compatible device, the USB Hacker Cable can also be used to charge your Structure Sensor and will charge the battery more quickly than the standard Power Adapter. USB Hacker Cables are available in the accessories section of the Structure Sensor Store.

Connecting Structure Sensor to your iPad

How to attach your sensor depends on what type of bracket you're using. For iPad compatibility information, see the support article titled "What iPads can I use Structure Sensor with?". Choose your bracket type below:

Your new Structure Sensor Pro is compatible with the original Structure Sensor type bracket. However, you need to use the new screws in the Structure Sensor Pro box, not the ones from the bracket box or the older sensor.

Attaching your Structure Sensor Pro

  • Structure Sensor Pro uses new, shorter screws to attach to iPad Brackets.
  • Do not use the longer screws which came in the iPad bracket box, or else you may damage your sensor.
  • In addition, please do not over-tighten the screws.

Additional Structure Sensor Pro Accessories

  • A Bracket Adapter comes pre-attached to your sensor. Please keep it attached.
  • Thanks to this adapter, all iPad brackets for the original Structure Sensor will be compatible with your Structure Sensor Pro.
  • Remember to use the USB cable in your Structure Sensor Pro box to charge your sensor before use. Any standard USB port will work.

Developer Portal

Next, if you're interested in developing your own 3D scanning software, you’ll want to sign up for the Developer Program to access the latest SDK.

Structure SDK is now available on iOS for the original Structure Sensor and Structure Sensor Pro. If you have already signed up for the Developer Portal and you need to retrieve your invitation, you can do so here.

Updating your Firmware

To ensure your best possible experience, we suggest updating your firmware upon receiving your sensor.

To make sure your firmware is up to date, please follow these steps:

  1. Download the Structure app from the App Store.
  2. Disconnect your sensor and launch Structure with the sensor disconnected.
  3. Connect your sensor when prompted.
  4. Wait. Sensor initialization can take 5 seconds or so.

    screenshot of app showing the sensor initializing

  5. Once initialized, tap the "i" icon in the upper right corner.screenshot of Structure app needing updated firmware
  6. If your sensor has a firmware update available, you will see this blue button (Note: if you are doing this on an iPhone, you may need to scroll down to see the button).
  7. Tap the blue banner to begin the firmware update process.

Please note, you will need to update your firmware quickly, as the longer the sensor is running, the more memory it will consume. If your sensor runs out of memory, you will need to unplug the sensor to clear the memory and try again.

Once your sensor is updated, it will reboot, and you will be on the latest available firmware.

Checking Depth Coverage

All of our sensors leave our manufacturing facility well-calibrated but a rough delivery, a drop, or other similar physical impacts may create a discrepancy between the sensor’s twin IR cameras. The result is similar to if you were to cross your eyes--the sensor will not be able to converge the two IR feeds to generate full-depth coverage. This phenomenon is called an IR offset.

Fortunately, this is not a physical defect of the sensor and in most cases, this can easily be fixed with a process called Stereo Image Refinement. Users in the past may have done this through CorePlayground; now this functionality can be done much more easily through Calibrator.

Do I have an IR offset? 

The following symptoms are associated with an IR offset:

  • No depth coverage
  • Bouncing bounding box
  • Instant tracking loss
  • Inability to capture a scan

To verify this, please follow these steps.

  1. Download the Structure app.
  2. Clean your sensor’s glass plate (as fingerprints and smudges can also lead to decreased depth coverage).
  3. Move to a room away from sunlight and face a flat, matte, non-black wall, around 1-1.5 meters (~3-5 feet) away.
  4. Launch Structure, wait for the sensor to initialize, and aim the sensor towards the wall.
  5. Good depth coverage will be shown with a solid colorization across the whole screen, like this:
  6. Poor depth coverage will be shown with anything less, such as this image:

If you are seeing anything less than the top image, your depth coverage can be improved. To do so, please follow the steps below.

Back to Top

Getting set up

  1. Launch the Structure app on your iPad and verify you are running at least firmware 1.2. Wait for the sensor to initialize. Once initialized, you can check your firmware status by tapping the “i” icon in the upper right. If you have a firmware update available, you can tap the blue button to update your firmware (iPhone users will need to scroll down). This process will take several minutes.
  2. Download Calibrator. Make certain the version number is at least 4.0. You can check this by either uninstalling the app and then reinstalling it from the App Store, or by going to Settings -> General -> iPad Storage -> Calibrator.
  3. Clean your sensor's glass plate. Make sure the plastic film that ships with your sensor is removed. If there are any smudges, dust, or fingerprints on the glass, clean them with rubbing alcohol and a microfiber cloth. Clean the glass plate in small circles, rather than from side to side, until all oils and particles have been removed.
  4. Set the scene. Stereo image refinement can only be performed indoors. Move away from any windows and find a blank wall. This wall should be neither black nor reflective. You will need to be at least 1-1.5 meters (~3-5 feet) away from the wall and ensure your sensor/iPad configuration is parallel to the wall. 
An example of a good scene An example of a poor scene

Back to Top

Running Stereo Image Refinement

  1. Launch Calibrator.
  2. When prompted, connect your sensor to your iPad.
  3. In the panel in the upper left, tap the "?" icon. This will pull up the Calibration Status pane with the option to run Stereo Image Refinement. Tap "Perform Refinement."
  4. You will be guided through a small explanation of Stereo Image Refinement. Swipe through the tutorial.
  5. Face a flat, featureless wall away from sunlight. Stand with the sensor perpendicular to the wall. When ready, tap "Start Refinement".
  6. Stay still. The refinement process will finish automatically. If you are satisfied with your refinement, tap "Save Refinement". Correct alignment should be 90% coverage within the inner 70% of the map.

Back to Top

Verifying your improved depth coverage

At this point, it is good to double-check and make sure everything has been saved correctly to the sensor. 

  1. Connect your sensor to your iPad again using the lightning or USB-C cable. 
  2. Launch the Structure app, as mentioned in the “Do I have an IR offset?” section.
  3. Face a flat wall, away from sunlight, and make sure your depth feed looks the same as you saw in Calibrator.
  4. Start scanning!

Back to Top

Calibrating your Sensor

You’ve installed your new sensor, you’ve made sure everything works, hardware-wise. The last thing you need to do before scanning is to calibrate. Using Calibrator to calibrate your sensor’s depth feed with the iPad’s color feed is far from intuitive (though we are working to improve this experience every day).

This guide is intended to be used alongside this tutorial video as a complete guide to successfully calibrating (which we promise, can be done!).

What is the importance of calibration?

Both the original Structure Sensor and Structure Sensor Mark II/ Pro work in tandem with your iPad’s color camera. The iPad’s camera applies the high-definition color and texture of the objects being scanned; Structure Sensor captures depth.

The problem is, the two cameras exist in two different places, and if run without calibration, would produce effects where the color from one object may shift over into the color of another. For example, say we were to scan a couch.

If we scanned the couch without calibrating the sensor properly, the scan might result in something like this:

See how the blanket bleeds into the pillow? That’s not right.

So we need calibration to inform the sensor and software of its place in space in relation to the iPad’s camera. When properly calibrated, the scan looks something like this:

As we can see here, there is a clear separation between the blanket and the pillow, and the color lays naturally where it should.

The sensor stores its calibration values locally, and it associates them with the iPad that was used to calibrate. Once you calibrate properly, you will not need to do so for quite a while longer, if ever.

The sensor holds two states of calibration: with and without the Wide Vision Lens.

Back to Top

Step 1: Wide Vision Lens Calibration

Things get even trickier with the addition of the Wide Vision Lens, as the fisheye balloons out the color stream. To help calibrate with the Wide Vision Lens, Calibrator introduces an extra step, utilizing this checkerboard. (You may skip this step if you do not have the wide vision lens).

  1. To use the checkerboard, print it out in landscape and make sure the squares are exactly 25mm in each direction.
  2. Post the checkerboard on a flat surface and make sure there are no bubbles.
  3. Attach your Wide Vision Lens, Launch Calibrator,  and tap “Calibrate with Wide Vision Lens” at the bottom.
  4. Read through the tutorial and tap “Start Calibration”.
  5. For the next part, you will need to position the iPad at various angles, matching the app’s AR overlay with the printed checkerboard. This can be a little confusing to start.

    The easiest way to think about how to position the iPad is to try and make sure the top right corner of the overlay matches up with the checkerboard, and then adjust the iPad’s angle to match the rest.

    The angles from which you will need to approach the checkerboard are as follows:

Straight on

Straight on, closer, and slightly from the bottom

From the right center, with the left edge of the iPad angled evenly away from the checkerboard

From the right top, with the left bottom edge of the iPad angled evenly away from the checkerboard

From the center top, with the bottom edge of the iPad angled evenly away from the checkerboard

From the left top, with the right bottom edge of the iPad angled evenly away from the checkerboard.

Back to Top

Step 2: The "Pollock Calibration"

The "Pollock Calibration" method has become the Structure standard for saving robust calibrations to the sensor. It is quick and reliable, regardless of light availability or environment. When possible, we recommend calibrating with this method.

  1. Download the Indoor Calibration Target.
  2. During the "How to calibrate" stage, bring the iPad close to the target, to the point where you can see the target in both IR and visible frames. Bring the iPad close enough so that the image is almost (but not quite) clipped. See the figure below for an example.
  3. Tap "Got it!" and hold still. It should take a few seconds.

  4. Proceed to the refinement stage (it should be pretty close to aligned, if not perfectly aligned), complete, and save the calibration.

  5. Alternate Step 2: Outdoor Calibration Mode

    Both calibration states (with and without the Wide Vision Lens) require bracket calibration. This is the software alignment between the iPad’s camera and Structure Sensor.

    Indoor and Outdoor Calibration modes are fundamentally the same; what changes is the gain and exposure settings of the sensor. Typically we suggest performing Outdoor Calibration, but if you live in a particularly cloudy area or are having trouble, we provide Indoor Calibration as an alternative.

    Calibration Instructions

    1. To begin bracket calibration, tap “Start Calibration”.
    2. Depending on your iOS device, you may be taken to a screen that discusses your bracket type. For all supported iPads, we include the XYZ extrinsic translations by default--these are the distance between the sensor and the iPad’s camera in terms of height, width, and depth in millimeters. If you have a custom case, you must make these measurements yourself (if you need to make these measurements, please check out this page). Assuming you have a bracket, tap your bracket type.
    3. You will be taken to a screen that says “Sunlight Required”.

      Fill up the sun meter by pointing the sensor outside or through an open window. If there is not enough IR available, you will see this toggle appear after five seconds:

      At this point, you can either tap the toggle to adjust the exposure and gain to begin outdoor calibration or continue pointing your sensor outside to try and fill the meter.

    4. Once full, you will see a split screen of color and IR. This is where bracket calibration occurs.

      Your goal here is to find a complex scene that is high enough in contrast that it appears well in both screens. In the picture below, I use the bare trees against the sky:

      As you can see, the trees show up fairly clearly in the IR feed as well as the color feed. This is an example of a scene with poor complexity.

    5. Once you find a complex enough scene, you don’t need to move a huge amount between frames--just move a little! This way you don’t have to continually find complex enough scenes.

      After two or three good frames, the app will switch to the final stage: refinement.

    Back to Top

    Alternate Step 2: Indoor Calibration Mode

    If you are calibrating at night, or in an area of the world that doesn’t see a lot of clear skies, you may not have enough ambient IR to complete Outdoor Calibration. Not to worry! If you are on this screen for five seconds, a toggle in the lower left corner will show up to enable Indoor Mode.

    What this does is adjust the exposure and gain settings of the sensor, making it more sensitive to IR light.

    The rules of calibration remain the same as Outdoor Calibration Mode; you need to ensure you are scanning a complex enough scene. What constitutes a complex indoor scene? A scene that has a lot of contrast on both the IR stream as well as the color stream. An easy example is a MacBook keyboard:

    This is because the black keys are clearly contrasting with the light gray chassis.

    You might imagine a plant would be a complex enough scene. Not so!

    You might also imagine a geometrically complex scene would be complex enough. Again, not necessarily!

    In this picture, we see a geometrically complex scene, but there are very little high contrast areas in the IR stream.

    While not perfect, the above scene was enough to complete bracket calibration successfully. The high contrast between dark and light created enough similarity between the scenes, and the software was able to do the rest.

    Back to Top

    Step 3: Refinement

    If you’ve made it this far, congratulations! You’ve made it through the hardest part.

    Refinement is the last part of calibration, where you manually tweak the calibration the software determined was correct. We do this by allowing you to drag the depth feed’s color overlay to match the physical scene.

    Please note! You do not need to perform refinement while looking outside! In fact, you might have better luck doing so indoors. We do our best to help you get the hang of it by working through this tutorial:

    Once you think you’ve got it, find a scene with some very clearly-defined edges, like this scene here:

    On iPads, you will drag the scene right and left. On iPhones, you will drag the scene up and down.

    Once the color (not the spaces where there is no color!) matches the physical scene, tap “Save Calibration”.

    The calibration state will save to the sensor.

    Remember, there are two calibration states that need to be saved: with and without Wide Vision Lens, so be sure to calibrate with both.

    Once you’re fully calibrated, you are good to go.

    Happy scanning!