ARKit

https://developer.apple.com/arkit/

by forgingahead • 9 years ago

295 183 9 years ago

developer.apple.com

tbrock• 9 years ago

Ooph, this is game changing.

There is a LOT of hardware out there that supports this.

The iPhone is the most popular game platform on the planet and, just like that, it will be the most popular platform for developing AR applications.

felipelemos• 9 years ago

Are you sure the iPhone is the most popular game platform? It is not even the most popular phone.

pat2man• 9 years ago

If you measure sales, it probably wins since iOS user spend a lot more than android users and mobile is huge: http://www.cnbc.com/2016/04/22/mobile-game-revenue-to-pass-c...

coda_• 9 years ago

I'm curious, which phone is more popular?

j2bax4• 9 years ago

According to this article, it took 1-4th place for most popular phone in 2016. https://www.fool.com/investing/2017/03/22/the-four-most-popu...

IBM2• 9 years ago

It's by far the most popular phone.

https://www.statista.com/statistics/276306/global-apple-ipho...

astrodust6• 9 years ago

You don't need the most market share to have the most gaming activity. People that use iPhones are much more active with gaming apps for whatever reason.

ClassyJacket4• 9 years ago

I'm almost certain it's the most popular phone by a very large margin. What other phone do you think sells better?

megablast2• 9 years ago

Of course it is, what do you imagine the most popular phone is? The Motorola v3?

foolrush• 9 years ago

Google Nintendo's AR Games if you haven't already seen them. Mind blowing fun, and years ahead of Apple's efforts. The archery and golf games were incredible.

I was hoping for years that it was a sign that Nintendo was going to release Super MARio...

https://youtu.be/45zzei39WMg

madeofpalk2• 9 years ago

Key thing with what Apple is doing is that it's completely marker less AR.

blazespin2• 9 years ago

This isn't game changing at all. AR without glasses is pretty weak.

What it is, is a great platform to develop on to get ready for AR glasses.

aylmao5• 9 years ago

I personally disagree. Give me some good AR maps app, or just something that will put wikipedia on top the real world and I'll have to buy battery packs to make for all the extra time I'll be on my phone.

marcusestes• 9 years ago

I'm currently working on a client project that uses the Vuforia library paired with an array of iPad Pros to pull off this sort of AR-light effect. But Vuforia relies on a tracking marker, and it's a huge pain in the ass for us. The idea of using lower-level camera and inertia sensor data to perform tracking without a marker is something I didn't expect to see without a Tango-like cluster of depth sensors.

Can't wait to try this out.

joejohnson• 9 years ago

demo of it already in action: https://twitter.com/heyadam/status/872278723700994048

krat0sprakhar• 9 years ago

The Wingnut AR demo that Apple showed during WWDC featuring the Unreal Engine was just mindblowing! Thankfully, it's available on Unreal's Youtube channel so you can still watch if you don't have Safari or the app - https://www.youtube.com/watch?v=S14AVwaBF-Y

tommoor• 9 years ago

At least Google uses web standards for their video

millstone4• 9 years ago

I wasn't aware of that, what format does Google use for its live streams?

I know that YouTube on Chrome uses VP9, which is not a web standard - or at least not more of a web standard than HTTP Live Streaming (which is what Apple uses for its keynote).

pjmlp4• 9 years ago

It is a Apple developer conference for software developers that care about making beautiful applications that take full advantage of native experiences on Apple platforms.

Any developer on this community can watch the video.

mikenew• 9 years ago

"can still watch if you don't have Safari or the app"

ugh

jeffhuys2• 9 years ago

You can also watch it using VLC.

smoser• 9 years ago

Or Edge.

paul9290• 9 years ago

It's just a 3D videogame and pardon I guess I'm missing the point of an AR video game vs. just sitting down and playing a 360 degree video game?

jayd164• 9 years ago

It's great because it does away with convenient touch controls and adds the fun of filming a table with an iPad.

aylmao5• 9 years ago

The content itself wasn't that big of a deal; it's the fact that the AR technology worked so well, and is available to anyone using the SDK is what was impressive and hype-worthy.

Doing AR from scratch is a huge endeavour but this should make it simple enough so that we start seeing more applications using it for more useful stuff.

proyb22• 9 years ago

Way more than 3D, I'm surprise it can remember position and flat plane accurately and butter smooth fps.

https://www.youtube.com/watch?v=Yphh1Ue3D6g

https://www.youtube.com/watch?v=-YzayB99piE

proyb22• 9 years ago

Using Metal 2 optimize, it's solid and I was even fooled by a red chair. IKEA would love this.

I realize on Google I/O 17 demoed Tango using Asus ZenFone with low frame rate is still not impressive. https://www.youtube.com/watch?v=Ey47-i33bLI

I thought Samsung S8 should have AR on a premium phones.

Compare to 60-120fps on iOS 11, yup, Apple Metal 2 could boost by reduce latency from 16.6ms to 8.xms and even more with direct display, it matched PlayStation VR refresh rate, real world tracking is what we need. https://www.youtube.com/watch?v=Yphh1Ue3D6g

blazespin2• 9 years ago

How was it mindblowing? I thought it was a pretty poor example of what AR can do. It was just a 3d game on a tabletop. Pokemon Go is a lot better, and it's pretty basic.

aedron• 9 years ago

It wasn't even a game, it was a scripted sequence with zero interactivity.

j2bax4• 9 years ago

I found the simple placement of the lamp and cup on the table to be more impressive and have greater real world use cases.

jensvdh• 9 years ago

Microsoft did this a couple of years ago with Hololens and Minecraft..

leggomylibro• 9 years ago

Shhh. And don't mention the Surface, either...

AceJohnny22• 9 years ago

You mean the $3000 headset that can only be used for AR?

https://www.microsoft.com/en-us/hololens/buy

threeseed8• 9 years ago

Microsoft didn't invent AR so what's your point ?

Also, Apple has almost never been the first to do anything. Tim Cook even admitted this yesterday. What Apple brings to the table is an actual polished solution. Which having used Hololens extensively it absolutely is not.

coldtea2• 9 years ago

Microsoft did this a couple of years ago with Hololens and Minecraft..

Yes, badly and expensive. And didn't follow up on it properly.

jayd164• 9 years ago

This is nice but what's unique about this? Is it significantly better or more performant? Is it just nice to have an integrated solution?

I'd love to see a breakdown of ARKit vs something like Vuforia. Here's a video with the equivalent functionality to the WWDC demo -- https://www.youtube.com/watch?v=JvE_7filGsY

coldtea2• 9 years ago

This is nice but what's unique about this? Is it significantly better or more performant? Is it just nice to have an integrated solution?

Ignoring all that other stuff, it's officially sanctioned and supported, part of the platform, requires no external libs, has Cocoa level documentation, and tons of developers will be using it very soon.

sambeau• 9 years ago

The building destruction physics were pretty impressive. Maybe I'm out of touch with modern games engines but I've not seen anything quite like that before.

amelius2• 9 years ago

I don't get the point of seeing anything in the background of the table, in this case the audience. Why not paint it with sky, then it's less distracting?

Oh, I get it, then it's not AR. Well, maybe AR is not such a good fit in this case to begin with.

ehsankia3• 9 years ago

The it's just watching a 360 youtube video on your phone.

This is still new and doesn't quite make full use, but there are many reason why you'd want to see the environment. For one, it gives a sense of scale. For example if you put a real life size dinosaur, seeing the surrounding allows you to understand the scale better. Next up, the AR application could very well interact with the world too, which brings new possibilities.

These are pretty new technology and no one knows the best way to use it yet. Maybe in the future we'll start putting a sky in the background. Maybe we'll find better ways to use AR. Both AR and VR are still in their infancy.

moduspol• 9 years ago

It demonstrates the power they have with AR. If they can show a town appear on a table with air strikes coming in in realtime, it implies that simpler more practical AR use cases will be much easier.

Prior to this, as a developer, you might think AR is limited to what Pokemon GO and the first example (like a cup of coffee on a table) show. After seeing this, if you were thinking of doing anything simpler than what they were showing, you'd likely see it as much more feasible than you previously did and perhaps look into including some of those features.

And this was at WWDC (their developer conference), so it certainly applies to a developer audience.

nihonde• 9 years ago

This kind of thing is the least interesting application of AR in my opinion. I want high-quality information overlays about the real world. I do understand that some people just want Starcraft on a coffee table, but it seems like a low ambition for Apple to showcase.

agmcleod• 9 years ago

would be exhausting to micro all those little units across a table by hand! :D

naravara• 9 years ago

This kind of thing is the least interesting application of AR in my opinion. I want high-quality information overlays about the real world. I do understand that some people just want Starcraft on a coffee table, but it seems like a low ambition for Apple to showcase.

It makes for a compelling proof of concept. Having a lot of individual things interacting with the environment independently is most of what you need, technically, for the high-quality overlays you're talking about. If you can do the former you can do the latter, it's just a sexier way of showing off the capabilities.

plg• 9 years ago

How do you play the game while holding the ipad with both hands?

j2bax4• 9 years ago

Having designed an AR experience for children and doing quite a bit of research into this problem a few years ago... You incorporate thumb controls on the sides of the device where you would naturally hold it. You use physical movement of the tablet to move things in the environment. Its limiting, but it works and can allow for fun interaction with the environment.

panic2• 9 years ago

Most apps these days run on Android, too, which makes using a single-platform framework like this a lot harder to justify. It's the same reason you don't see many games using SpriteKit -- it just doesn't make sense when you know you'll need to support Android in the future.

I wonder if it would make sense for Apple itself to support frameworks like ARKit and SpriteKit on Android. I think it would make people a lot more comfortable relying on them.

astrodust6• 9 years ago

Being iPhone only didn't stop Instagram from launching, getting huge, and branching out to other platforms. If this allows you to make a quicker prototype, it's a big deal.

daxfohl• 9 years ago

That was when Steve Jobs was still alive. Things have changed. iOS isn't the gold mine it used to be.

sidlls• 9 years ago

Mobile apps in general seem to be a race to the bottom from a business standpoint and Android is much more so that than Apple. I strongly dislike both platforms, and generally harbor a very negative opinion of Apple in general, but if I were to venture into mobile development I'd target Apple first and worry about Android (much) later.

weeks• 9 years ago

From what I've gathered this totally depends on your market. Creating a paid app or an app targeting US University students? iOS first. Creating an ad supported app or targeting an international market? Android first or use a framework that supports compiling to both platforms.

cageface3• 9 years ago

Apple keeps doubling down on platform specific services and APIs but as their market share dwindles this becomes increasingly untenable. Porting Apple Music to Android was a step in the right direction but they have a long way to go in the new world where services are more important than devices.

algesten• 9 years ago

How do you mean it dwindles? It seems to hold steady and increase in some countries.

This is a somewhat recent figure I found. https://9to5mac.com/2017/01/11/ios-market-share-kantar/

MBCook8• 9 years ago

I'm willing to bet the only reason they did that is because they already had beats customers on android.

The whole point of these APIs is to let developers make a better apps on iOS so people want an iOS device. If they port any of them to android all they're doing is making it easier for people to leave their platform.

threeseed8• 9 years ago

Why would Apple port core technologies to a competing platform ? Did Google port Play Services to Windows ? No. And no one would expect them to since it is a core differentiator for their platform.

ARKit isn't a backend service it's an OS library so doesn't make sense to be cross platform.

aylmao5• 9 years ago

Porting Apple Music to Android was a step in the right direction but they have a long way to go in the new world where services are more important than devices.

Are they though? There's still a huge amount of people buying phones and computers, which cost a lot of money, periodically. There's paid services, sure, like Apple Music, which might become self-sustaining businesses, but a lot of these other "services" by the big tech companies are operated at a loss, like email, maps, cloud-storage, messaging, vide-calling, AI assistants, etc.

The only reason to make free services like all those at a loss is to bring value to a platform so users will stick to it, and Apple keeping its services platform-specific makes its platform even more sticky. Apple benefits from this directly, because it makes money selling its platform. To use iOS you need a device from which Apple makes loads of cash.

Google on the other hand is still essentially just an ads company with hobbies, and its profit from the platform its building is much less murky, indirect, and IMHO flimsy. The only way their platform brings them substantial profits is by collecting data to better target ads, which is concerning too.

The lifetime of devices is growing, so in the long term we have to see if Apple can keep selling as many devices as it does; their main source of income is not entirely future-proof. If you ask me, however, Google might be in a more delicate position, especially considering the growing issues with internet ads (concerning privacy, tracking, obnoxiousness etc) and that so many players are in the ads business nowadays. Some, like Facebook, are moving in deep-- challenging Google in user data analytics and engagement time.

tl;dr: Only some services (like Music) make money. All the rest are more valuable if they serve your core business, and Apple is very much doing that.

cmsj2• 9 years ago

Mobile platforms have gotten large enough that market share isn't necessarily a useful metric. There are many hundreds of millions of iOS devices out there. That's a pretty decent market to be addressing. (It also happens to be almost all of the top end of the mobile market, so an iOS only offering has a TAM of hundreds of millions of people who are more likely to spend money. Not bad really :)

nxcho• 9 years ago

Unreal Engine and Unity is going to have ARKit support. If your ambition is to make an AR app for both iOS and Android I suspect that one of those engines are better.

I don't believe giving ARKit to away makes sense for Apple at all. It is, from what I can see, a pretty advanced technology and widespread adoption of it on other platforms will not help them selling more iPhones.

threeseed8• 9 years ago

Of course it makes sense and of course it will help sell phones.

This only works on iPhones/iPads with the A9 or A10 chip. Which means that if you want to use AR then you need to upgrade from your iPhone 6. It also will create a suite of AR apps which only run on iOS.

IBM2• 9 years ago

It absolutely does not make sense for Apple to support this on Android. AR is something that uniquely leverages assets that Apple has: its various AR related acquisitions (Metaio, FlyBy Media, Faceshift, Emotient, WiFiSLAM, PrimeSense) and most importantly their prowess in chip design.

It is trivial for the Android ecosystem to replicate an Instagram for their platform, it will be much more difficult to do this for AR and thus will be a meaningful source of competitive advantage.

There are real economic barriers to having the necessary hardware for AR. High-end Android phones don't ship anywhere near the numbers of iPhones to give it the scale required to make the necessary investments in hardware and software worth it (both from OEMs and Google). And that will ultimately limit the opportunity and market for developers, disincentivizing them to invest in it.

This is an area where Apple's scale, margins and ability to deploy new technologies and see meaningful adoption from users and developers (think about how fast Touch ID rolled down into the entire iPhone/iPad installed base, setting it up for Apple Pay years later) pulls up the ladder on would-be competitors.

panic2• 9 years ago

You're probably right, but I think it's important to note the difference between Touch ID (a user feature) and ARKit (a developer feature). Users won't know or care about ARKit if developers don't adopt it.

bpicolo• 9 years ago

Google is already doing similar, custom hardware and all: https://get.google.com/tango/

But it's definitely on fewer total devices, so Apple has a real edge here.

pier254• 9 years ago

Plus you have cross platform solutions that are as good if not better.

From game engines such as Unity which have plenty of AR plugins, to SkiaSharp (port of Google's Skia) using Xamarin.

warrenmiller• 9 years ago

Yes it does, eCPMs are about 5 times higher on iOS.

runesoerensen• 9 years ago

Saw this small test earlier today, pretty impressive stuff: https://twitter.com/kylebrussell/status/872123141346140160

Also, ARKit was discussed here yesterday https://news.ycombinator.com/item?id=14490239

HeyItsShuga• 9 years ago

The demo shown off in that video is available as a template in Xcode 9.

callumjones2• 9 years ago

Why has this comment thread devolved into comparing the number of Android OS installs versus the number of iPhones sold.

They are apples and oranges and the point is that the iPhone for a few years back is a guaranteed hardware specification that you can guarantee ARKit will work on. If you think about this for Android, you'd need to slice it by the devices that are compatible - it's not going to be a guarantee by OS.

ClassyJacket4• 9 years ago

I hope that ARKit can be a lesson in what works well and what doesn't, and spur the open source community to create something like it, but cross platform. As more phones get dual cameras, what we can do with software like this will only get better.

Look at OpenCV for example.

MBCook8• 9 years ago

ARKit doesn't seem to require dual cameras because it runs on iPads too. I do wonder if it uses the depth sensing second camera in the iPhone 7 Plus, or if that data isn't fast/precise enough to be useful.

altstar• 9 years ago

You need to that anyway with ARKit, it only runs on A9 and A10.

callumjones2• 9 years ago

Good catch, A9 isn't a few years back.

My argument probably isn't as valid but I still think the guarantee of 2 iPhone generations is safer than the fragmentation of Android devices that would be "released in the past 2 years".

jdauriemma• 9 years ago

Anyone know whether a JavaScript API is in the works?

MBCook8• 9 years ago

Apple does not provide JavaScript access to their APIs, so I doubt it. You'll only be able to use this in native apps. Something like PhoneGap or Cordova could try, but I wouldn't be surprised if they were serious performance issues.

pier254• 9 years ago

You should be able to make a native plugin for Cordova or React Native.

NativeScript gives you access to native API automatically.

oliyoung2• 9 years ago

There is https://github.com/jeromeetienne/AR.js/blob/master/README.md

lern_too_spel5• 9 years ago

What does this have over Qualcomm's AR SDK (now Vuforia) that has been available on iOS since 2011 and on Android a few years longer?

aey• 9 years ago

A larger price tag ;)

suyash• 9 years ago

ARKit is free and directly by the OS and device manufacturer.

threeseed8• 9 years ago

Vuforia is not owned by Qualcomm.

And so the advantage of ARKit is having Apple behind it.

qrbLPHiKpiux• 9 years ago

Purpose? Other then keeping consumer heads planted in the device? (The demo was neat, though)

aylmao5• 9 years ago

That's for the developers to figure out! Which is why it's an SDK and not some app with an API for extensions.

blt• 9 years ago

Anyone know what VIO method they use?

dcgoss• 9 years ago

AR goes beyond gaming - I thought the idea of using it for things like interior design was great. I remember Apple mentioning that IKEA used ARKit to project models of their furniture into rooms.

alasdair_• 9 years ago

It's probably worth noting that Google had a partnership with Lowes to do this with Project Tango about two years ago.

oliyoung2• 9 years ago

Because Safari is supporting WebRTC, ar.js is working on iOS now

https://twitter.com/fr3ino/status/872228441390813185

alexbilbie• 9 years ago

They should have called it TableView

Kyro38• 9 years ago

I've seen this pun multiple times. Could you explain it ?

jpttsn• 9 years ago

I'd guess it makes reference to UITableView, a piece of the UI framework most people use to code iOS apps.

random123456• 9 years ago

Demos of ARKit from twitter blows away Tango demos, even though Tango requires additional hardware. Will Google discontinue Tango?

tuyguntn2• 9 years ago

I am one of that unlucky customers who bought into the hype of Tango and bought Lenovo, probably I should sell it

jobigoud2• 9 years ago

Can you link some of those demos? I haven't seen anything very impressive so far. Also it doesn't do area learning as far as I can see.

tuyguntn2• 9 years ago

https://twitter.com/fr3ino/status/872228441390813185

https://twitter.com/heyadam/status/872278723700994048

https://twitter.com/kylebrussell/status/872123141346140160

marcusbrown• 9 years ago

The guy from "Pair" in the first episode of "Planet of the Apps" [https://www.planetoftheapps.com/en-us] tried to make a similar SDK his main business model but eventually failed to get the fundings from the VC.

I hope in the end he decided to stick his focus on the app (which is quite amazing), otherwise he's in big trouble now!

I really think that a lot of new apps will come out in this space, now that the technology is much more approachable.

WhiteNoiz3• 9 years ago

Not many people mention it, but ARKit also theoretically makes it possible to have 6DoF Google cardboard style VR as well.

Macsenour• 9 years ago

I said it for years... acceptance by mainstream will be AR then VR.

ARKit

Related Stories