Dive into the world of AR with ARKit

Augmented reality has been present in our mobile apps for quite some time. And today, we’re going to help you do the same! We’ve been offering our iOS development skills for years already and if you’re looking to start developing high-quality iOS apps with AR features too – this article may be a sound start!

So, without any further ado, let’s start!

What have we prepared for you

Introduction to AR and ARKit

When we first started working with AR, we thought that it would be great if we had a resource where we could find on one place just the basics about ARKit to help us to start with learning. So our iOS team has prepared a little introduction to ARKit for you to help start-up your AR journey.

AR stands for augmented reality and has been around for some time now, but more recently it has become available on personal devices like smartphones. VR immerses you in space where everything around you is virtual. AR places virtual objects in the real world around you.

The possibilities are endless. From furniture apps that let you try out how would some piece of furniture look in your home to augmented reality games that let you bring games inside the real world.

Apple has been keen to use the potential of AR and with the introduction of ARKit in 2017 as part of iOS 11, developers have been able to easily create all kinds of AR apps.

Apple has also been really active with developing ARKit and we got the second and third generation ARKit with iOS 12 and 13 respectively.

The latest iPad Pro has a LiDAR sensor that improves AR performance and its rumored that the next iPhone will have one too.

ARKit uses your device’s cameras and sensors to find out the information about the world around you and bring you a better AR experience.

ARKit takes advantage of the technology called Visual Inertial Odometry to track the world around you.

This enables your device to sense how it moves through the room. ARKit uses this technology to not only analyze rooms layout but also to detect horizontal planes like tables and floors.

And when you find planes around you, you can place virtual objects on them.

Throughout this blog, we will show you what are nodes, how to detect planes and place nodes on them, and how to spice it up with physics simulation.

Before continuing with reading good idea would be to check out Apple’s official documentation on ARKit.

Setting up the scene for AR

First, let’s clarify what is a node.

A node is a structural element of a scene graph, representing a position and transform in a 3D coordinate space, to which you can attach geometry, lights, cameras, or other displayable content.

• Official Apple documentation

ARSCNView (later we will refer to it as Scene View) is a view that enables you to display an AR experience with sceneKit. So before any other work on our AR app, we need to set it up and configure it properly.

Scene view is the base layer that blends virtual 3D objects and takes care of rendering our augmented reality experiences. It integrates AR sessions that manage lighting, camera drawing, and some other stuff like managing nodes for anchors.

Scene view has several other properties that can be useful for debugging purposes like showing feature points or displaying the physics body of your node.

ARWorldTrackingConfiguration is a configuration that is used for monitoring your device’s position and orientation while letting you augment the environment that’s in front of you.

Detecting planes

After we have configured our sceneView we need to set its delegate so we get notified when the plane is detected and the new ARAnchor has been added to the scene. To do so, we implement the method renderer(_:didAdd:for:).

First, we need to check that this anchor is ARPlaneAnchor (this is an anchor that represents plane surfaces).

Next, we create a new node and set its position using our anchor positions. After we have our node created, we just need to add it as a child node so it will be rendered in our scene.

After all this hard work we can finally see something appearing in our AR world. Next, let’s talk a little bit about these magic things that we refer to as nodes.

This picture is an example of how detecting planes works if configured right. 

Understanding nodes

Simply said nodes are objects that we see in our scene. But SCNNode object itself has no visible content, it represents only a coordinate space transform relative to its parent node.

We use the hierarchy of nodes to create its structure and then we add lights, cameras, and geometry to our nodes to create some visible content.

Here is a simple example of how to create a simple node and add it to your scene.

Nodes can be already designed 3D models or simple shapes that we create using basic geometry. And now when we know how to create nodes and place them inside our scene we should make things interesting with moving our node around.

The easiest way to move nodes is by using SCNAction move. First, we create move action by specifying the position where we want to move the node and duration so we can control the speed of node movement.

This video is an example of what kind of actions you could do to manipulate your node.

After we have an action object created we just call runAction on our node and our node will move and change its position. Of course, there are many more actions you can use for manipulating your nodes, some of them are rotating and fading out.

Using SceneKits physics engine

Let’s say we want to make our app stand out and not be just a boring app where we either have stationary objects or objects that move like robots and don’t interact with each other.

Luckily Apple has got our back with their SceneKit physics simulation.

It’s a realistic physics engine that manages all the calculations on physics bodies that we attach to our nodes. So, all we need to make our nodes interact with each other is attach a physics body to it.

So as you can see, when creating a physics body we need to specify the type of this body and its shape. The type can be kinematic, static, and dynamic.

Choosing a type of physics body

A static body is a body that is unaffected by forces or collisions and that cannot move, it’s a type that you would want to use for floors, walls, or any other immovable object.

The next type is kinematic. This is a body type that is unaffected by forces or collisions but that can cause collisions that affect other bodies when moved.

If you have some object that you want the user to move with his finger you would create an invisible kinematic body and use this body to collide with the object you need to move.

And the last type is dynamic, dynamic bodies are bodies that can be affected by forces and collisions. Dynamic bodies are used for objects for which we want to react to other bodies around them.

A good example would be a car, we want a car to interact with the floor for friction and to use engine force for realistic acceleration.

When you have chosen which type you need to use we need to define the shape of our body. Here we can create our custom shape our just set it to nil and let the sceneKit to calculate it for us.

For better performance, you should create some simple shapes like a box or cylinder.

One thing that we’ve learned the hard way is if you have scaled bodies and you use automatic shape creating by setting it to nil, it may not work as you expect and even debug option which shows you physics body can lie so the best method is to create a custom shape with the dimensions we need.

Also, a good thing to know is that all values in SceneKits physics simulation use the International System of Units (SI).

Spark the interaction between bodies

The last step to make bodies interact with each other is defining its categoryBitMask and collisionBitMask. These properties are used for defining body category and category of the body it interacts with.

When you have multiple physics bodies you need to set its categoryBitMask so you can identify this body when a collision happens and you use collisionBitMask to define with the body of which category it interacts with.

Optionally you can use contactTestBitMask property to define interaction where two bodies will generate contact message without being affected by the collision.

Some of the other useful things that we can find inside the physics engine are SCNPhysicsField and SCNPhysicWorld.

The physics field is used for creating forces that will affect all bodies in the area, forces like gravitation and electromagnetic force.

The video is an example of a body affected by the force of gravity.

The physics world is a property that holds SCNPhysicsWorld object and manages physics characteristics that affect the entire scene. One of those characteristics is SCNPhysicsBehaviour.

This is an abstract superclass for joints, vehicle simulations, and other high-level behaviors that we can incorporate in our physics world and apply them to our physics bodies so we can modify the results of the physics simulation.

Joints are physics behaviors that let you connect nodes just like you would do in real life with joints. Some of the joint types are SCNPhysicsHingeJoint, SCNPhysicsSliderJoint, and SCNPhysicsBallSocketJoint.

Also, physics simulation has vehicle behavior prepared for you which lets you make nodes that physically behave like real-life vehicles.

Now, it’s your move! 

These would be some basic terms and concepts that are used for creating AR apps and with a little effort, you can get a grasp of these basic building blocks that you can use for creating AR experiences. With every iteration of ARKit we get better performance and more features implemented so possibilities are endless.

Now when you understand how ARKit works and what you need to do to detect planes, add nodes and how to implement physics into your app to make it even more amazing you can take this knowledge and build upon it to make the AR world a better place.

Additionally, if you’ve been looking for more resources on mobile development, be sure to check out our article section. Also, feel free to take a look at how we took on the challenge of supporting different screen sizes while developing an iOS app! 

You have any questions about ARKit or have you worked with this technology and have tips for us? Say hello to us at hello@factory.dev