RealityKit meets SwiftUI
AR is in, and my understanding was — not. I had a hard time finding resources on how to add text to an ARView in an iOS application, not to mention manipulating that text with user interaction. So today, we’re taking it back to the basics. We will be building a simple counter app, except in a way we’ve probably never done before. We’re doing it in AR.
If you’d like to jump right into setting up the AR scene, skip ahead to “Setting up ARViewController”, or clone the repo here:
- iPhone 6s and up
- iPad 5th Generation and up
- iPod 7th Generation
ARView Counter App
Here is what will be the result of following today’s tutorial. A counter app being built in SwiftUI with the help of ARKit.
Both the visible button bar (right), and the
ARView and confined to their own views, so we will also learn how to pass object states between views, and update the
ARView whenever our count increases. Let’s get started.
Setting up Our Project
We will start off by selecting the Augmented Reality App when creating our new project. Hit next.
Then enter a name for the project, and be sure to keep SwiftUI selected. After hitting next, you’ll be prompted to find a location to save your project to. When done, select create, and Xcode will complete setup.
After selecting resume in the simulator pane, the starter code will be populated with presets for adding a box to our AR scene. We can see the box in the simulator above, but because there is no camera available to the simulator, it won’t have the AR experience.
At this point, you can go ahead and run the project (CMD + R) on your iOS device and see this box in the app.
Setting up our Views and Models
ARCounter directory selected, hit
cmd + option + n to create a new group (folder). Create 2 groups, 1 named
View and the other named
Model. Within the View group, add a new Swift file with
cmd + n .
Select Swift File, and name this file
Count . Repeat this process, creating 2 more Swift files named:
CounterButtonBarView . Move the files into their respective groups. Here’s an image for reference as to what your folder structure should look like.
Setting up Count
Count file, we will be creating the
Count class. This class contains a published variable named
Published is a property wrapper that will update all
ObservableObjects that have it as a property whenever it experiences a change in state. More on that later.
Setting up CounterButtonBarView
ContentBarButtonView we will want to add the code above. Here, we are setting up our interactive button bar so that way we will be able to update the count that will be displayed in our
line 5, we are declaring a variable for our instance of Count that is used to track the current count of our application.
line 8, we are embedding the following views in a horizontal stack.
line 10, we create our decrement button. Within this button, we set the action to decrement its views instance of Count by 1. It also prints to console which is helpful for debugging purposes. Within the label section for this button, we are setting an Image using an icon set provided by
line 24, similar button for resetting and incrementing the counter are created.
line 36, we are styling the ContentBarButtonView. Feel free to tweak these as you see fit.
Adding our Buttons to the ContentView
We are going to want to update our
ContentView to match the above code. The
ARViewContainer has been removed from the Content View (and is being moved to the file we created for it earlier.) It will also be removed from the body of the ContentView for the time being.
line 6, we are creating our instance of
line 9, we are embedding all further views in a
ZStack. This is because we are going to be overlaying the
CounterButtonBarViewover the AR’s camera view on the Z axis.
line 10, we are embedding the button bar in a
line 11, we are adding a
Spacer()to push down all remaining views to the bottom of the screen.
line 12, we are adding our
CounterBarButtonViewview and passing in the instance of counter we created for the application on
line 6. The bar button views handling of increment/reset/decrement will update this Count instance.
Now, with your iOS device connected, select it as the device target for build. After running
cmd + r, you will see an application with a black screen and button appear. On button tap, your console should be tracking the instance of Count as shown below.
Setting up ARViewController
Now, to setup our AR scene. The above code will handle the creation and updating of the displayed text every time one of the button we created earlier are tapped. We will simplify this by just redrawing the scene on every button tap.
line 6, we are setting up a reference that will take in an instance of the Count class we had created earlier. It is set to be an observed object, because it works with the Published property wrapper to update the view every time the published variable
line 8, we have the constructor for the ARView.
line 12, we have the function that will be called to update/redraw the AR view every time the
ObservedObjecthas a Published variable that changes. We call our custom
updateCounter()function from here, passing in the existing ARView.
line 17, we remove all anchors from the scene, clearing it.
line 24we create our
.generateTextand pass in a series of parameters. First, the text we want to display, which is just going to be the absolute value of the
count.num. Second, we pass in our desired extrusion depth to make the text 3d. Last, we set out font with a desired size & font weight.
line 35, we check the new value of
count.numand assign a color depending on whether it’s positive, negative or zero.
lines 37, we create a shader using
SimpleMaterial, passing in our color and some other desired presets (feel free to play with these).
line 38, we create our
Entityusing the mesh and shader we had created previously.
line 40, we are position out
textEntityto be in front of us by changing its position on the Z axis.
- And finally, on
lines 41 & 42, we are adding the entity to the anchor, and the anchor to the scene.
Once complete, the text will have been updated, an action triggered by the change in the published variable
num as a result of a user’s tap on button.
Final updates to Content View
Now, in our
ContentView we can add our
ARViewContainer, and pass in our instance of count.
And that should be it! Run the app and interact with the counter. Below, I have linked the repo again. If you face any troubles, and would like to see my full working code, feel free to clone it & work from it!
Link to Repo: https://github.com/martusheff/ar-counter
I find building off of tutorials to be a very good way to learn, so below are some challenges you may consider building upon on this app!
- Add a text field that updates the displayed text.
- Add a color picker to change the color of the text.
- Add an animation to the text every time the counter is updated.