Apple’s Shortcuts will flip the switch on Siri’s potential

https://techcrunch.com/2018/07/08/shortcuts-will-flip-the-switch-for-apple-on-siris-potential-just-not-how-you-think/

by evo_9 • 8 years ago

239 187 8 years ago

walterbell2• 8 years ago

This user-centric approach paired with the technical aspects of how Shortcuts works gives Apple’s assistant a leg up for any consumers who find privacy important. Essentially, Apple devices are only listening for “Hey Siri”, then the available Siri domains + your own custom trigger phrases. Without exposing your information to the world or teaching a robot to understand everything, Apple gave Siri a slew of capabilities that in many ways can’t be matched.

Is there a good doc or video that delineates what data stays on the device vs. being sent to Apple for processing? E.g. how much of this functionality will be available if you are not signed into iCloud?

mark2123• 8 years ago

from the keynote, no information is sent to Apple for processing. Whether it'll work if you're not signed into iCloud is a different issue but from the way it's described in various places in the WWDC Keynote, it ought to work just fine without any data connection, like airplane mode and no Wifi.

rueynshard2• 8 years ago

Wouldn't the user's speech data need to be sent to Apple to convert to text, or identify the intent first?

threeseed3• 8 years ago

This describes everything:

https://www.apple.com/business/docs/iOS_Security_Guide.pdf

marmshallow• 8 years ago

Is this a change for all Siri functionality? It currently requires internet on my iPhone.

ehsankia• 8 years ago

It's great to see Apple finally add some powerful customization, and for power users this is fantastic, but to average users this makes no difference. When it comes to normal queries, Siri is still far behind. Sure, with some extra work I can get almost any query to work, but most users don't want to manually code up all their queries.

jsjohnst• 8 years ago

None of the digital assistants (I have most of them) do anything truly useful out of the box for me, so the “needing to do work to customize” isn’t an issue for me. I’m actually happy I finally can with Siri, bit late, but better than never.

adam• 8 years ago

Perhaps it's already there, but I wonder if shortcuts will be able to be shared or there will be a library for them. So someone can pre-define all sorts of stuff, then I as a "basic" user can just add the shortcut vs. having to "program" it myself.

hug4• 8 years ago

Workflow (the mother of Siri Shortcuts, as I understand it) already has sharing functionality.

For example, this "get travel time to input destination" Workflow: https://workflow.is/workflows/ff987bcf0ad746d496415d7f4c75a8...

threeseed3• 8 years ago

but to average users this makes no difference

Ordinary users will just pick prepackaged workflows out of the gallery.

And some I can imagine being very popular e.g. "Post last photo to Instagram".

shiburizu• 8 years ago

Saw someone else comment on a different TechCrunch article here on HN about how these articles about Apple read like press releases. I really feel that now.

ndynan• 8 years ago

As a PM, this article is what I would write as sample press coverage I'd like for a product release.

Interesting to see that the author is a former PR member of http://my.workflow.is/ + has one recent article for TechCrunch.

Oh hey looks they were purchased by Apple: https://techcrunch.com/2017/03/22/apple-has-acquired-workflo...

Quick Edit: I'm not saying that this isn't good technology, rather I'm more concerned that TechCrunch is taking contributors who have ties to products being reported on and may have interests that are not being communicated to readers.

ianlevesque• 8 years ago

Yeah, this is native advertising and it's Everywhere.

codezero• 8 years ago

TechCrunch does this all the time, most of the "news" blogs do – I worked at a company that employed a PR firm that basically put our product release collateral directly onto a bunch of those blogs – it made me very uncomfortable.

dwaite2• 8 years ago

Workflow is the technology which is becoming the Siri Shortcuts editor.

staz• 8 years ago

The suite is back! http://www.paulgraham.com/submarine.html

minimaxir• 8 years ago

It should be noted that this is a Contributor article (common on weekends), not an article from the main TechCrunch staff.

It should also be noted that post-acquisition Workflow (where the author worked at and is noted in the author blurb) was the precursor team to Shortcuts.

drb91• 8 years ago

To be honest, I don’t feel like these sites even know what to ask. This may be the Gell-Mann Amnesia effect in action.

avip• 8 years ago

Where else did you think Tech content comes from?

erikpukinskis• 8 years ago

Much of it is independent enthusiasts dissecting public information.

simonh• 8 years ago

Company issues press release on new product. News companies relay information to readers. Readers complain the info just looks like a press release.

But as a reader, I do want to know this stuff.

fauigerzigerk• 8 years ago

You can get it from the source. No deceptive marketing tactics required.

https://www.apple.com/newsroom/

RankingMember• 8 years ago

It's fine if they make clear it's just a press release, but this is a press release masquerading as journalism.

ralfn• 8 years ago

Specifically because this is an announcement that is a copy of functionality shown for Android already.

It's Apple catching up.. which if this was a journalistic article would be mentioned.

"Apple announces same feature Google announced 6 months ago"

dwaite2• 8 years ago

Two of the issues with Siri is that Apple does not like to roll out features unless they work for everyone (across all languages), and discoverability of what it can do against disappointment in what it can't (really, you think defaulting to a Bing search is the best thing to do in this case?)

Siri Shortcuts lets you define your own local command and control phrases from presented actions, effectively solving both issues. It could (depending on per user investment) make Siri way more individually useful.

ronnier• 8 years ago

iOS 12 beta is amazing. It is extremely fast and feels like perfection. Hopefully apple will open the OS a little more for customization and apps to do more powerful things then it will truly be perfection.

Kpourdeilami• 8 years ago

How does it hold up for day to day use? Is it stable enough that could be installed on your main phone without many issues?

Razengan2• 8 years ago

I have installed it on my one and only iPhone (X) since it was made available. No issues so far except a couple random resprings (soft restarts.)

I can’t decide how much of an improvement it is over iOS 11 though (which was already performing good enough for me.)

I think this year’s big winner is macOS, not iOS.

kalleboo• 8 years ago

I've had my lock screen flip out a couple times, once requiring a manual reboot of my phone. The notifications list has graphical glitches. Some third-party apps are suspiciously crashy. It's probably the least problematic early iOS beta I've used, but like any beta I still wouldn't rely on it if missing a call would be a deal-breaker.

graeme4• 8 years ago

Fairly stable for me. Occasional freezes, sometimes requiring a reset.

Definitely feels faster, even on an iphone 8.

tzahola2• 8 years ago

Sounds like Automator for dummies.

saagarjha11• 8 years ago

A dumbed-down Automator, maybe. Please don’t insult the users of Workflow.

tzahola2• 8 years ago

I'm not insulting anyone. "X for dummies" is a book series.

mirceal2• 8 years ago

I personally turn off all notifications on my iPhone (except for a few select apps) and do not use Siri.

I can tell you that the Siri suggestions in Search really annoyed me (usually use search to find... apps, not to be fed all the crap from random apps).

I want a clean, non-intrusive experience. Apple should focus making the hardware better and to get the software out of the way (don’t make me think about it) instead of pulling all this crap on its users in the name of innovations.

mark2123• 8 years ago

a big part of the improved Siri integrations is helping group the notifications and give you the option to shut them off. So it really is trying to get the software out of the way if you want

jknz• 8 years ago

I can tell you that the Siri suggestions in Search really annoyed me (usually use search to find... apps, not to be fed all the crap from random apps).

In iOS 11 the delay for searhing for apps is unacceptable. instead of displaying the apps found right away (substring match over all apps should be instantaneous), it waits until all the crap from Siri and other random apps is fetched.

matt-attack• 8 years ago

Agreed. The only way I find & launch apps (besides the 4 on the dock) is by swipe-down from the middle of the homescreen, to reveal search, then to search for the app's name. This is very analogous to how I launch apps in OSX, namely command-SPACE, then start typing the application name.

It's extremely frustrating when iOS takes forever to find an app, or worse proioritizes all kinds of other garbage before the app's icon. If I type "waz" and I have the Waze app installed, I sure as hell expect to see the Waze icon instantly at the very top. Ideally, after typing just the "W".

cdmckay• 8 years ago

Which makes it even more annoying when you try click an icon you see and it’s replaced when Siri finishes, making you open a different app by accident. Terrible UX.

saagarjha11• 8 years ago

Apple should focus making the hardware better and to get the software out of the way

I…what? They clearly do, considering that they have the best smartphone hardware on the market and have what’s generally considered to be a good mobile OS? This feature is something that power users have been asking for years–if that’s not innovation in your eyes I’m not sure what is.

mark2123• 8 years ago

This is a great example of how owning the entire stack from CPU on up to user interface allows a company to do things that others just can't. Whether it's any good or not, well, time will tell. But it's not even possible for any other company (maybe Google with the Pixel, maybe).

rueynshard2• 8 years ago

I don't see why this could not be possible on Android. All Google needs to do is build an API for Assistant that other apps can use to 'donate' frequently used actions/intents. Assistant can then make suggestions to users based on its own analysis of users' activities in the same way Siri does.

Bahamut• 8 years ago

Apple could feasibly go further and do integration with macOS Siri as well down the line I'd imagine - Google can't do that nearly as deeply.

halflings3• 8 years ago

Not only is this possible in Android, but this already exists in Android:

https://developer.android.com/guide/actions/

flukus• 8 years ago

Cortana has had an API like this from the beginning, so I don't see why owning the entire sack is necessary.

jordansmithnz2• 8 years ago

I’m the developer of a popular timetable/schedule app for college. It seems like the perfect use case for Siri: “What class do I have next?”.

Up until now it hasn’t been possible due to the restrictive API. I’ve looked into the new shortcuts and Siri API docs and although the beta documentation is sparse, I’m confident I’ll be able to develop a natural, first class Siri integration.

Unfortunately, it won’t be trivial to implement (at least to do well). It probably won’t be done before the iOS 12 release, and I’m hesitant to start working it just yet due to the sparse beta docs.

Over the next year or two, I think there will be some great Siri integrations built. Hopefully, users discover and use them :)

Operyl2• 8 years ago

I think we’ll see more useful docs before release, they usually release more and more up until the iPhone launch events.

saagarjha11• 8 years ago

They're already up, though a bit sparse: https://developer.apple.com/documentation/sirikit/

aeleos2• 8 years ago

Do you mind sharing what the app is called?

Operyl2• 8 years ago

He created Class Timetable.

https://itunes.apple.com/us/app/class-timetable/id425121147?...

berti• 8 years ago

I always wanted something like this in my uni days, but was always too lazy to make it! More importantly than what, "Hey Siri, where's my next class?" ;)

-e- heh, quick LinkedIn stalk reveals you went to UC at the same time as me!

Nullabillity2• 8 years ago

Isn't that just.. a calendar?

conradev• 8 years ago

Have you checked out the two WWDC talk videos on developer.apple.com about Shortcuts? They should provide a lot more context and connect the dots between a lot of the new APIs.

jordansmithnz2• 8 years ago

Thanks, I think I’ve seen them but should definitely check them out again. I might have missed a thing or two :)

csomar• 8 years ago

Can't you do that by adding to the iPhone calendar?

princekolt• 8 years ago

I’ve been working on a Siri/Shortcuts-enabled App, and while I agree documentation is very sparse, the API is also very succinct. I only watched the WWDC session about the soup app and was able to get going from there.

threeseed3• 8 years ago

I also think this is going to revolutionise the TouchBar.

The ability to have contextual shortcuts could be pretty powerful.

MBCook4• 8 years ago

I think this is iOS only for now, but it seems like a very obvious thing to port to the Mac.

saagarjha11• 8 years ago

It’s already there in the form of Automator and AppleScript.

madrox2• 8 years ago

This is great for developers, but pointless for users. It doesn't change the user behavior, and the research shows people are using VA at home and nowhere else...a place better suited to Google Home and Echo.

There've been a lot of attempts by Apple to create richer hooks into apps like search integration, but they don't do much for engagement. There's some good ideas out there, along with anecdotal successes, but the interaction model isn't great and better app integration won't fix that.

MBCook4• 8 years ago

I use Siri on my phone all the time. Do you have a link to that research? I haven’t heard that before.

Given that iOS will be prompting people with possible Shortcuts based on what they do frequently I can easily see people starting to adopt this.

I don’t think you have to use Siri to trigger these, it’s just an obvious easy/fast way. Users could still use the shortcut app or widget to do it.

As I’ve been watching some of the Apple community on Twitter since this started to go into beta they’ve already produced some fun/surprising stuff. This seems like it’s going to be a BIG deal.

madrox2• 8 years ago

This is the best I can do right now. It isn't where I originally read it but google is failing me. https://creativestrategies.com/voice-assistant-anyone-yes-pl...

What it comes to is the people on HN don't represent most people using iPhones.

Angostura5• 8 years ago

Good news for HomePod and CarPlay users, and I suspect that the ability to have a short phrase, trigger a sequence of complex actions will make Siriuse out and about more common.

scarface744• 8 years ago

With more laws being passed about using your phone while driving and seeing that most cars have Bluetooth integration, it should become more popular. I use Siri all of the time while driving.

adityapurwa• 8 years ago

So basically they exposed an API to add to Siri a new skills. I wonder whether other platform already did simillar things? CMIIW, Cortana already exposed an API for us right?

MBCook4• 8 years ago

Really they’ve combined two things. They took an app called workflow, which they bought, would you can use to automate things on iOS. They extended it dramatically, and then added the magic ingredient of making it possible to have Siri call those actions that you define. Plus some AI stuff to suggest shortcuts for things you do frequently.

Alexa is quite popular because it has so many skills the people have created for it. That’s one of the reasons people claim Siri has been “behind“ and Amazon’s Echo devices have been doing so well.

I don’t know if Cortana or Bixby support anything like this.

halflings3• 8 years ago

Android has the same features:

https://developer.android.com/guide/actions/

https://techcrunch.com/2018/05/08/google-launches-slices/

halflings3• 8 years ago

Also, Shortcuts don’t require the web to work – the voice triggers might not work, but the suggestions and Shortcuts app give you a place to use your assistant voicelessly. And importantly, Shortcuts can use the full power of the web when they need to.
This user-centric approach paired with the technical aspects of how Shortcuts works gives Apple’s assistant a leg up for any consumers who find privacy important. Essentially, Apple devices are only listening for “Hey Siri”, then the available Siri domains + your own custom trigger phrases.

I don't get it. How is this different from Android? Android Actions [0] were announced before this. I think Assistant also works offline (with most voice commands + in voiceless mode).

[0] https://developer.android.com/guide/actions/

hug4• 8 years ago

SiriKit and Siri Shortcuts are very different things.

SiriKit allows you to build Siri support into your app a la Android Actions, but Siri Shortcuts is designed to allow drag-and-drop end-user "programming" of workflows that can be triggered by Siri.

"Hey Siri, I'm on my way home" could turn on you thermostat up, order you a pizza, remotely trigger your IoT enabled kettle and start playing your home-commute playlist.

For the more advanced of us, Workflow currently allows doing things like calling arbitrary REST APIs and parsing JSON. I've reverse engineered the API of a local coffee-ordering app so I can one-click order my morning coffee.

Next thing I'm planning is my "I need a coffee" button which will get the nearest cafe, order me a flat white, and pull up the directions.

icebraining• 8 years ago

Yeah, Siri Shortcuts seems more like using Tasker with the AutoVoice plugin.

sharcerer3• 8 years ago

Siri Shortcuts is same as Google Assistant routines. Only they didn't make an extra app for it. Say" I'm Home" and a lo of Assistant Actions will get triggered.

myrandomcomment2• 8 years ago

I switched to car play and use Siri a lot more then I used to. For most things it is fine. Others... this could help.

One thing I would like to know is why Siri cannot answer basic questions about what music is on my phone. For example, “what albums by New Order do I have on my iPhone?”. That does not work but “play album Substance 1987 by New Order” does.

robin_reala• 8 years ago

Siri on CarPlay is useless for me. Maybe it’s my situation (RP British English, driving in Sweden) but it usually takes 10 attempts to get it to recognise a track I want to be played, and getting routing information is easily double that for anything other than ‘drive home’. I one tried to get it to route me to ‘Malmö’ (third largest city in Sweden) and gave up after 30 attempts. I honestly have no idea what I’m doing wrong, but I suspect nothing and it’s just useless software.

Angostura5• 8 years ago

As another RP speaker using CarPlay, it works well. However you do have to heavily anglicise placenames when abroad. So I just tried ‘ get directions to Mal-moe’ and it worked first time.

thirdsun• 8 years ago

I'm not sure if this has changed yet, but until recently Siri regularly failed to recognize artists, albums or titles, which often happen to be english, if the device/siri was set to a different language. In my case that's german and I can't count the number of times I had to pronounce the artists I wanted Siri to play in a ridiculously german way.

English is so ubiquitous that, in my opinion, all Siri queries should be checked for both local language and english.

GeekyBear• 8 years ago

Think of Shortcuts as a visual scripting language that leverages app functionality and ties into Siri.

Either you can use drag and drop to write your own scripts, or you can run scripts written by others.

You can run your scripts in a variety of ways, including a trigger phrase you set with Siri.

Here's a short video showing the creation of a very simple script in an older version of the program.

https://www.youtube.com/watch?v=uK0sBtF5_1E

If you are a podcaster, you might create a more complicated script that converts a source audio file to MP3, adds MP3 tags and artwork to the resulting file and then uses FTP to upload the result to your podcasting network.

Another script idea would be to text someone a list of the blocks of free time you have open during a given workday based on your calendar data to help set up a meeting.

The possibilities are wide open. You can even tie directly into web API's.

LTL_FTC• 8 years ago

"hey Siri, destroy all evidence!" phone begins wipe

saagarjha11• 8 years ago

I’m sure you’re trying to be humorous, but just in case you’re not, this isn’t something that Shortcuts lets you do.

romangibson• 8 years ago

So, is it ready to do the complex task just like google and amazon echo?

TeMPOraL• 8 years ago

Does Google finally allows user to configure top-level actions, instead of the silly "Ok Google, ask $someapp to do something" (where what I want is simply "Ok Google, do something")?

singularity20012• 8 years ago

Or even better just "$act" when I'm alone in the room!

The "Ok Alexa ask $someapp to …" prefix is disgusting.

arthurofbabylon• 8 years ago

I agree with this idea for one reason in particular: as a user, truly leveraging Siri requires a manual engagement - the experience is not conversational, it is not like speaking with a person. It is like using a tool - one needs to conscientiously adjust oneself to follow its rules and get it to work as desired.

Users actively taking control over how they use Siri (as in iOS 12 Siri shortcuts) will almost certainly encourage them to more conscientiously adjust their usage behavior patterns.

There exists a general assumption that a voice has a human-intelligence behind it, but obviously now not all voices do. This poses a tough learning curve, as evinced by criticism of Siri as ineffective or plain bad. Yeah, Siri won’t respond the same way your boyfriend will when you ask him to find some good Chinese food, or express some feeling. But Siri will excell at setting alarms, or adding an event to the calendar, or starting a meditation with Timeless. It comes down to matching the language to the tool / following a protocol. It comes down to manually engaging with precision.

saagarjha11• 8 years ago

Siri won’t respond the same way your boyfriend will when you ask him to find some good Chinese food, or express some feeling

I think Siri did pretty well when I asked her these:

Q: Find me some good Chinese food

Siri: I found four Chinese restaurants a little ways from you:

Q: How are you?

Siri: Very well, thank you!

singularity20012• 8 years ago

That's about the only things Siri can handle without completely blowing it.

wool_gather• 8 years ago

Parent said:

respond the same way your boyfriend will

Note that she ignored the "good" part of the query, has no memory of the place you went to last time that you did/didn't like, can't filter by your preferred mode of travel, and so on.

whywhywhywhy• 8 years ago

There exists a general assumption that a voice has a human-intelligence behind it, but obviously now not all voices do.

This is one of the reasons I've always been bothered by the over-humanisation of Siri. I feel there is a belief on the Siri team that if they make it feel more like a human with jokes and overly verbose responses then people will be more forgiving when it fails.

I've always felt the opposite, that it gives it a "incompetent super-intelligence" sort of vibe instead and I start to wonder if when I ask it to set a timer for the 3rd time if I'd be less frustrated if it just responded with a confused beep rather than a quip.

mgiannopoulos• 8 years ago

That’s all good but I hope that Siri can be improved to understand voices like mine (a native Greek so my English accent is not perfect). Siri understands 40-50% of what I say. Google (on iOS, so same hardware) is more like 80-90%.

bitwize• 8 years ago

Google's voice recognition is nuts. One time I fired up Google's voice search and fed it some of the MST3K names for Dave Ryder from Space Mutiny (e.g., "Punch Rockgroin", "Blast Hardcheese", "Splint Chesthair", "Big McLargehuge", etc.) and was surprised at how many of the searches it got right -- capitalization, spacing, and all.

Shank3• 8 years ago

To me, the biggest thing here is that it takes a completely different approach than what has been the traditional path for voice assistants. In the past, it was always the game of waiting either for a custom skill or app, or it was hoping that Google or Amazon would program in some logic for handling a particular case.

Shortcuts enables basically any end user with enough devotion and dedication to short circuit this. It doesn’t require them to be an app developer and it doesn’t require them to learn code at all. The most basic shortcuts can be created without any if-then-else logic while enabling so much.

I’ve seen my mom ask her Google Assistant to do things with “and” a lot and they just don’t work because command chaining hasn’t been implemented. But with Shorcuts, she could conceivably make a chain and designate a phrase to be conversationally equivalent with an “and” in the middle.

Instead of having to explain to family why Siri can’t do X or Y, I can just make a Shortcut or show them how, and I’ve solved the problem rather than explaining why it can’t be done.

refulgentis• 8 years ago

Google has command chaining, as well as Routines - i.e. Shortcuts, but with intuitive UI.

Apple is moving Automator to iOS, this isnt voice UI, its imprecise programming: https://pbs.twimg.com/media/DhbmQJBX4AMEoho.jpg:large

saagarjha11• 8 years ago

The "voice UI" part isn't shown on that screen: you can assign a phrase to use to trigger these Shortcuts.

Shank3• 8 years ago

You can call Workflow/Shorcuts imprecise programming, but you're comparing apples to oranges. Routines don't let you arbitrarily group data and interact with it in a way that's friendly to non-programmers.

entropie2• 8 years ago

I’ve seen my mom ask her Google Assistant to do things with “and” a lot and they just don’t work because command chaining hasn’t been implemented

it is implemented. It just doesnt work often (or at all) for foreign languages.

I'am pretty sure this is the next big thing google is updating

Instead of having to explain to family why Siri can’t do X or Y, I can just make a Shortcut or show them how, and I’ve solved the problem rather than explaining why it can’t be done.

At least with GA and ifttt you can as of late make your own phrases (and responders). Nothing for the nontechies, but some progress there.

joshschreuder• 8 years ago

Do you know if it's possible to get the Home to respond to some external event? Doesn't seem to be possible to use as a Then with IFTTT (for example "When I arrive home Then say Welcome Home")

I found this:

https://www.npmjs.com/package/google-home-push

Which might let me write my own API to do something but it would be good if it were built in somehow.

practice9• 8 years ago

it is implemented.

AFAIK only for Google Home devices, not phones

Shank3• 8 years ago

Nothing for the nontechies, but some progress there.

This is the important thing. I'm a fairly competent programmer and if I'm really missing out on something, I really don't worry about it. The problem is that most users aren't programmers!

it is implemented. It just doesnt work often (or at all) for foreign languages.

Everyone has latched onto my comment because of command chaining, but this misses the point. The point was to give a concrete example of something that a user can now accomplish that they couldn't before. The Google Assistant cannot in fact build rudimentary constructions out of arbitrary system objects like Shortcuts can -- command chaining is just the example I picked.

sharcerer3• 8 years ago

Have your tried Assistant Routines? They're in the Assistant Settings. Multiple actions get triggered with single command

amelius• 8 years ago

Does anyone know of an extensive list of useful things that people typically ask their voice assistant?

As a non-user, and as someone who types faster than they speak, I find it hard to come up with compelling use-cases.

fouc• 8 years ago

Haven't looked for a list, that'd be a good idea.

So far all I do is:

"Remind me to wake up at 6am" or "Remind me to get my laundry in 30 minutes" (sets it in the todo list reminder app, since the alerts are better than the alarm app)

akvadrako• 8 years ago

I think you have a lot of company in the "only use Siri to set timers" camp. I also do alarms and appointments, but nothing else is reliable enough.

DennisAleynikov• 8 years ago

Stuff like estimates for driving time with traffic, or simple trivia questions is something I very frequently ask my google home. Otherwise when I'm using the phone I almost never use it unless I'm far from my phone and I need to do something simple.

scarface744• 8 years ago

I posted a list earlier, but hopefully you don’t type while you’re driving.

But things like Reminders are quicker with Siri. Especially things like Reminders based on external triggers - like locations and getting in and out of the car.

criddell5• 8 years ago

You can type faster than you speak on your phone?

I can't type that fast and when I'm driving I don't want to type at all. For me, being able to say "navigate to 123 main street" or "play the beatles" is very convenient.

subroutine6• 8 years ago

Cognitive science has a term called "Theory of Mind" (ToM), which refers a human cognitive ability that typically emerges in 4 year olds (In fact there is a famous experiment you should look up and try if you have any kids/nieces/nephews around that age), whereby humans are able to formulate the mind of another entity (the capacity it's working at; information it holds, and doesn't). This is what allows us to, for example, tell a lie. It also allows us to modify our approach in the way we communicate with dogs vs children vs adults, for better communication efficacy (or two different adults where one has high and one has low expertise in some domain, like programming).

In my opinion the biggest thing preventing me from using siri is not what it can and cannot do, but that it has been nearly impossible for me to develop a ToM for Siri. And since I simply dont know what siri is capable of, I only use this bot for a very narrow set of tasks. Furthermore one of my ToM priors for siri is that she has a mind that is incapable of learning. This is a big turnoff - bigger than i think we acknowledge; since we are used to interacting with entities that can, even if laboriously. For instance my Australian shepherd might not be able to bring me the newspaper; but if I really wanted her to do that task, I could slowly get her to approximate that behavior, and it would probably be satisfying to see progress in performance. With siri, I simply assume there are things she can do and things she cannot, and it'd be pointless to try and goad her into adding even a trivial task to her functional repertoire.

yonkshi• 8 years ago

The thought of Siri intentionally telling white lies is pretty funny and a bit creepy

- Hey Siri, am I fat?

- No Bruce, you are just big boned

- Thanks Siri.

rubatuga• 8 years ago

No need for down votes. Yeah it’s interesting how psychology plays a role in us interacting with a voice assistant too. I don’t think Siri will ever be able to judge you on something.

chrisweekly• 8 years ago

tangent: I'd ask anyone claiming to be "big boned" to show me a fat skeleton.

madeofpalk• 8 years ago

Is this unique to Siri, or do all voice assistants/computer programs exhibit this?

subroutine6• 8 years ago

I think it's all of them. Their abilities are just far more rigid than what we are accustomed to. That mixed with the fact that Siri is a black box that requires query to gain a piece of knowledge. This is not how we are used to doing it. A lot of the information we bake-in to the ToM we have for someone or something is based on observing how they behave and interact with others, or simply by themselves. I see my nephew about twice a year, and even with his rapidly changing capacity I can usually pin down what things he is/isnt capable of in a matter of days. Siri on the other hand is in my pocket almost every minute of every day, and i feel like i know nothing about her.

Long story short, I think direct vocal query to gain one piece of rote information is a terribly inefficiant way to develop a ToM of someone/something. If Apple wants to bring siri to the next level they need to figure out a way for us to learn about her abilities passively. The catch however is that I dont want siri bugging me all the time, but maybe sparsely she could give me a popup that says "hey I can do that for you". Particularly if she has 'seen' me do it 100 times and think she could save me a few minutes of effort. That imo would be more game-changing than adding additional siri-api for developers, which will add a bunch more eastereggs that i will never discover.

naeem• 8 years ago

This is an excellent comment.

graeme4• 8 years ago

Did you read the article? It's saying you'll be able to slowly "train" her by adding shortcuts.

Your comment seems to be a general gripe, and it doesn't address the fact that this new change may actually solve your problem.

(I agree with you, but I'm excited by this addition because it finally feels like a difference)

lostcolony3• 8 years ago

You're just provided with a way to automate a task and have Siri trigger it. It doesn't address the OP's key complaint that there's no clarity on what kinds of tasks Siri can do.

For instance, from 9to5mac, these comments - "While Siri and native integration with the operating system is fantastic, this is still very much the Workflow app, with a few extra bells and whistles. It works and function in a similar fashion by creating blocks that trigger one after another"

"One example of using Shortcuts with Siri is creating a Shortcut that grabs your current location, creates an ETA home, enables Do Not Disturb, and then sends a text to your roommate that you’re on your way home, and plays some music. All while you’re getting into your car, and preparing to leave. And all you have to say is “Hey Siri, I’m on my way home.” Alternatively, you could also just go into the Shortcuts app and start the request there."

All Shortcuts is adding is a way to trigger all of them in one new step. It's like a dumber IFTTT for Siri, basically, one you can't build new behaviors in, you can just script together existing ones. Useful, but it's still hard to tell what can Siri do.

acranox• 8 years ago

The lack of learning is what also bugs me about all the new tech. It's being used for targeted ads, and stuff like that, but not for trivial tasks that would actually improve my interaction with the interface. A few things that come to mind, are that I always listen to music as full albums, and yet the UIs still always try to get my to shuffle various playlists. Every day around the same time I text my partner nearly the same message, "I'm leaving work." and yet autocomplete still suggests other phrases. If I click the share button, I then go through 6 steps to share the article via a text message with the same person. Imagine if clicking share offered one click of "Would you like to send this as a text message to Jane?"

I'm not a fan of the current AI craze, and happily like my devices somewhat stupid, but if they're going to try and be smart, at least learn my predictable behaviors, and offer me shortcuts on those.

It's unclear to me what these new shortcuts really offer, but I'll be interested to try them. However I have a hunch what I want is actually even simpler than these will provide.

reaperducer• 8 years ago

FWIW, I switched to a new iPhone a few weeks ago, and it has learned at every workday at a particular time when I open a Messages conversation with a particular person and type "Good," it correctly auto-suggests the rest of the phrase that I sent at that time.

I have noticed, however, that it wont always make suggestions for similar repeated texting events. I think it has to do with limited space in the iOS auto-complete bar, because once I get past the first couple of words it fills in the rest.

If you're using iOS, is it possible that your usual phrase is too long to fit in auto-complete?

As for the other things you suggest, it appears this is the direction Apple is headed, based on the most recent WWDC keynote.

Infernal• 8 years ago

So I currently use a combination of Launch Center Pro and Workflow (the developer of which was purchased by Apple last year, and appears to be the source of what has now become Siri Shortcuts) to do what you're describing.

I have a Workflow that when run, uses Maps to estimate the drive time between wherever I am and my house, pads the estimate by 5 minutes, then adds that to the current time to get an ETA, formats it into a text message and sends the ETA to my wife.

In Launch Center Pro, I have a geo-fenced shortcut that presents a notification to run my workflow whenever it sees that I have left the area around my office building.

The UX is, halfway between my office building and my car, I get a notification from LCP - I tap the notification, Workflow thinks for a minute, and presents me with a text message to my wife and I tap "send".

As nice as this is already, I believe Siri Shortcuts may improve it in several ways:

1) Currently, apps are not allowed to send messages without explicit user interaction, so Workflow can't actually text my wife, it just presents me with an imessage screen and an already-written message that I must tap "send" on

2) Workflow does not support geofenced or time-based running of workflows, hence why I need LCP to launch my Workflow based on a geofence. If Siri Shortcuts supports this, then I won't have to rely on LCP anymore.

3) There is no way (that I'm aware of) to trigger my workflow automatically, I must tap a notification and unlock my phone, or run an app and select the workflow within it. An improvement would be a programmable Siri Shortcut ("Hey Siri, let my wife know my ETA"), and even better would be automatic running of workflows based on defined conditions, so my workflow would run and send a message automatically when crossing the geofence without me even needing to be aware of it (besides maybe a notification that lets me know the workflow has run).

epicide2• 8 years ago

For a few things ("I'm leaving work" messages, in particular), I've had good luck with Workflow[0][1]. Unfortunately, it's all manually programmed and no learning, but it is decently intuitive and powerful.

[0] https://www.workflow.is/ [1]: No affiliation -- their app just solves a need of mine.

spike021• 8 years ago

Yep, I've been thinking quite a bit about this lately as well.

I set the same alarms Monday through Friday to wake up for work (usually a few since I'm a deep sleeper). While I know I could schedule these, I usually prefer to manually set them, so I wind up doing that every night.

In the morning I can do, "Siri, turn off all alarms" and Siri will disable any that are set.

But at night time, I can't do "Siri, set my usual alarms for tomorrow morning."

Seems like a critical type of action that a predictive engine should be able to process. But perhaps I'm wrong.

otabdeveloper1• 8 years ago

Your post implies that kids under 4 years old can't lie.

This is, of course, patently untrue. (Mine told lies at 12 months.)

subroutine6• 8 years ago

I should have qualified that: "lie with high-rates of success". To clarify, at 1-years-old your infant could not formulate a high quality representation of your 'mind'. That is he/she couldnt fully appreciates that information held in your brain was different than the info available to themselves (the union/intersection/xor of info), nor did they have a strong intuition regarding how vastly different your two mental abilities were, without a theory of how minds work.

The classic experiment I mentioned above involves a child and two adults in a kitchen. One of the adults wants a piece of candy (or something - going from memory here), but the other adult says they cannot have it yet, and puts it in a cabinet. All 3 witness where the candy is placed. At which point the one who wants the candy leaves, but on their way out the door says "when i get back I want the candy". So then it is just one adult and the kid in the room. The adult asks the kid to move the candy to a different location in the kitchen. The kid does this and then sits back down. After a few minutes the adult says to the kid something like - "That other adult who wants the candy will be back in a few minutes. Where do you think they will look first when they get back?" - without fail 3 year olds will point to the cabinet where the candy is currently located, whereas 5 year olds will always point to the cabinet they adult who left saw it last. Something happens in brain development during age-4 that gives us the capacity to understand that different minds hold different information based on their own experiences. This task has been repeated dozens of different ways, including simply having an adult standing in a different location in a room as 3,4,5 year olds, where the adult's view of a wall (of pictures) is clearly obscured by some screen. Then they ask the kids stuff like... how many pictures of teddybears will that adult say there is on that wall? Again 3 year olds report what they themselves see, while 5 year olds perform like adults would on this task.

JacobJans• 8 years ago

Your comment makes me think of an interesting idea.

One of the problems with Siri/Google/Alexa is they try to be everything and do everything at once.

What if they split the expertise/domains they're capable of into different names?

For example, there could be a "mind" that is really good at playing music.

So you'd say, "Hey MusicBot, pause the song."

Or, you could say "Hey, InfoBot, what's the weather?"

Over time, you could get to know more "minds" that live in the cloud, and develop a theory of mind for each of them.

throw7• 8 years ago

I just saw advertised that google home/mini has at least one named bot for special domains... I believe it was Toby? who is a "chef".

fredley• 8 years ago

The famous experiment is the "Sally Anne task", which goes like this:

The child is in a room with Sally and Anne. Sally has a basket and Anne has a box. Sally puts the marble in her basket and leaves the room. While she's out, Anne steals the marble, and puts it in her box. Sally then comes back into the room.

You ask the child "Where will Sally look for her marble?". This tests whether or not they can fathom the world through Sally's eye, and see that her view of it will be different from their own.

The experiment was formulated by Simon Baron-Cohen.

LeifCarrotson• 8 years ago

For instance my Australian shepherd might not be able to bring me the newspaper; but if I really wanted her to do that task, I could slowly get her to approximate that behavior, and it would probably be satisfying to see progress in performance.

Given she's an Australian shepherd, it would probably be only a bit startling to subsequently come home and see her analyzing the finance section of the paper. :)

This is, I think, one of the failings of human ToM. We tend to underestimate entities which are different from us. The differences are not only of capacity and of information. They are differences of worldview, values, and biases that are very hard to capture, even between adult humans of different cultures. It's easy to imagine people as a less intelligent and less experienced versions of yourself, but that is not an accurate theory of their mind.

The same differences, only more so, are present between humans and dogs or humans and Siri. Given that the most empathetic and social people (with, I assume, a better theory of mind) are generally not found in technical roles, I wonder if they might find it easier to make that stretch and effectively use Siri or Alexa?

subroutine6• 8 years ago

Ha yes, im saying it'd be difficult to get her to bring the paper because she is already so busy day trading dogecoin.

I hear what you are saying but given we are the designers of the siri 'mind', we are in a unique position to make it more human-centric. It will be easier in the long run to adapt siri to humans than the other way around.

sirn• 8 years ago

To demonstrate what Shortcuts will be able to do, here's someone writing a C parser in Workflow (a predecessor to Shortcuts):

https://twitter.com/uroboro845/status/985169126871224321

(One of the replies is by Ari Weinstein, who is currently on the Siri Shortcuts team)

Tehnix• 8 years ago

A lot of people seem to mention Assistant Routines, but the major problem here is that it’s not available in a lot of countries outside the US (even not the UK..), meaning it’s not actually an alternative for the majority of the world.

The funny bit is though, that your Google Home will still recommend that you set up routines for things, except you just...can’t.

Since I don’t have it available myself, I cannot comment much on it, but while searching for why it wasn’t appearing in the Home app, I certainly didn’t get the impression that people are impressed by it :(

gkilmain• 8 years ago

To run an Alexa skill on iPhone you need to install the Alexa app and the Amazon app and use your Amazon app to handle the voice part. Then open the Alexa app to view the card. I hope I'm understanding the hype of the article correctly in that I can build a native app and use Siri to interact with it.

jtbayly2• 8 years ago

But does anybody know whether it will finally be possible for me to ask Siri to give me directions to my next appointment?

I’ve been waiting for this obvious functionality for years.

mergesort• 8 years ago

Yep, you can do that. This weekend I made a shortcut that goes through your calendar, finds the next appointment, and then picks your maps app of choice to load up directions into.

I took it another step further and am working on a version where it looks at the prices on Lyft and Uber, presents you with the respective options, and calls a ride to your next appointment. Shortcuts in iOS 12 are a real fun little thing to play with, the most fun I've had programming in a while.

Happy to send it to you. I looked for your contact info but couldn't find any way to contact you, but feel free to reach out in my profile if you want.

jtbayly2• 8 years ago

I'm not running the beta, so I guess don't bother. I'm sure I'll have fun creating it myself in a few months. :)

SN76477• 8 years ago

I just want macros. I would love to have a single button that does in OS functions, like having a button for EQ settings, or to launch a specific website in a browser (such as a different weather service)

mustacheemperor• 8 years ago

Looks like this update still won't let you ask Siri to play a specific song on any service except Apple Music, which is definitely my biggest gripe with Siri and the iPhone at large since switching from Android. I would switch to Apple Music, except then there'd be no way to wirelessly cast/play music through my component stereo. Semi related, if anyone has solutions to this problem I'd be very appreciative.

Apple’s Shortcuts will flip the switch on Siri’s potential

Related Stories