Shasta: AI-powered audio recording and editing

https://pages.adobe.com/shasta/

by nonoesp • 4 years ago

139 82 4 years ago

pages.adobe.com

nonoesp• 4 years ago

I discovered today that Adobe is introducing Project Shasta, a web-based AI-powered audio recording and editing tool. You can request access now [1].

It seems their intention is not only to provide AI-based audio editing and recording but also to cover the ground of what Riverside already offers.

Remote recording - Recording with others is as easy as sharing a link. Everyone’s audio is recorded in high quality locally, then Project Shasta syncs it back together in the cloud automatically.

"Soon to come features" list things we can already find in Descript, such as filler word removal or speech enhancement. And features already available in the beta include microphone checking for optimal quality and microphone distance, AI-powered audio, remote recording with guests, and project templates.

I was waiting to see what Adobe's take on AI-based audio editing and recording was, and it seems it's here. At the moment the project is referred to as Shasta; I imagine these capabilities either format a new product offering or are integrated with existing tools like Audition, Premiere, or After Effects.

What do you think?

[1] https://pages.adobe.com/shasta/request-invite

mrandish2• 4 years ago

As an old-school video editor who learned editing on analog VTRs (tape machines) and progressed to digital non-linear editing, I think the time saving potential of AI-assisted audio editing is substantial.

I still think it will be a while before AI is ready to create more than a first draft edit but if it handles 95% of the easy cases that's still a huge time-saving - especially on long-form projects. I spent so, so many hours of soul-sucking, rote drudgery doing things like manual audio ducking back in the day.

dilap• 4 years ago

Hopefully something like this catches on w/ smaller podcasters. Listening to shit-quality compressed internet audio in a podcast interviewing someone really interesting is just such a shame!

muttantt• 4 years ago

Well, there goes Andrew Mason's startup Descript...

preommr• 4 years ago

inb4 Adobe continues their streak of demonstrating they're incapable of building software and spends another couple of billion to buy out the competition.

luckydata• 4 years ago

it's not like it's a particularly out there idea, anything logical will have competition. I welcome that a large company with resources builds to market useful tools that I can expect to exist beyond the next round of funding.

adroitboss2• 4 years ago

Descript was the first thing I thought of. Since it does audio and video, I am not sure this can be called a competitor. Descript has such a head start with features I don't think Adobe will be able to catch up.

I wonder if they made Descript an offer before starting this project. Although, the base problem of editing audio or video with text isn't that difficult. They probably thought it could easily be cloned. The devil is in the details however, and the rabbit hole runs deep.

CharlesW2• 4 years ago

Descript has such a head start with features I don't think Adobe will be able to catch up.

What specifically is Descript doing that Adobe couldn't do relatively easily, or isn't doing already in its other NLE products (Premiere, Audition, Rush, After Effects)?

inasmuch8• 4 years ago

I know everyone hates Adobe (I certainly share some of the grievances about their pricing structure) but I have been using Photoshop for something like 23 years, working across half that many creative disciplines, and cannot overstate how much I value the consistency, reliability, and neutron-star density of its feature set. You call it bloated, I call it a Swiss army knife. I don't want every tool to be that way, but I love that some are. (The problem with this analogy is that Photoshop actually /is/ the best tool for many of the things it can do, where the Swiss army knife is solidly in jack-of-all-trades territory.)

Given this, I am very concerned about Adobe's continued march into web apps. For most of my UI and web design work, I use Figma, and quite like its feature set, but that I cannot use it offline and that I cannot work in a file that requires over 2GB of RAM due to its browser RAM cap is infuriating to me when Photoshop (the supposedly worse program) enables me to manually allocate as much of my system's 64GB of RAM as I please. It's difficult to imagine any web app stably and efficiently growing to Swiss army knife-levels of capability. Not just in terms of performance and resource management, but also GUI, keyboard shortcuts, etc.

I've long assumed my CC subscription would die with me, unlike many (most?) of my peers, but if the web-based Photoshop somehow became the primary Photoshop, I may have to cancel my subscription, and then kill myself.

quasarsunnix• 4 years ago

The problem is the bloat. I have CC access via work, but still choose to go with Affinity because of the performance issues. I counted 20 processes running the last time I had PS installed. The apps all take forever to load, drain battery if working remotely, and are far more unstable than in the past. I too grew up in PS in the 90s, but let's not pretend it's not become a dumpster fire - albeit a very feature rich dumpster fire as you stated.

I agree that Figma shares many of these same issues performance wise. Truthfully the only creative software I use that I truly enjoy these days is Blender. Hopefully one day the passion of Blender will make it to other areas in the creative software space.

inasmuch8• 4 years ago

I haven't used any of the Affinity apps. But are we talking about the same Photoshop?

I counted 20 processes running the last time I had PS installed

I don't check how many background processes are running when I use Photoshop because I don't notice them at all. Almost every feature I use in it feels responsive, especially compared to how it used to be. If you're talking about telemetry, yeah, I hate that too, but I've more or less resigned to it as a necessary evil in all my professional tooling.

The apps all take forever to load

Photoshop opens nearly instantly on my M1 Max Macbook Pro, as it did on my M1. It was slower on Intel, but what wasn't. I can open and close Photoshop three times before Discord even loads an empty window.

drain battery if working remotely

I've never tried a remote workflow with Photoshop, so can't speak to that at all. Definitely does not seem like what the program is optimized for, and I would think battery drain would be the fault of whatever tool you're using to remote in, no? Or do you mean just working on an unplugged laptop? In which case, I find Electron apps that do one task poorly tend to be less efficient than even heavier desktop apps that 100 things well, Adobe or otherwise.

and are far more unstable than in the past.

Sorry, what? I haven't had an Adobe app crash on me in probably eight years. I of course count that as good luck, but to say the current builds are less stable than older versions is just … insane to me. Thinking back to, say, the CS2/3 days, half my time in the program was spent Cmd+S-ing for fear of the inevitable hourly crash and loss of work. Photoshop crashes were a legitimate part of creative culture from like 1999 to 2012.

Sorry if I sound like an Adobe shill. We've just clearly had very different experiences with their products of late.

Where we are in agreement is that Blender is cool! I'm too much of a neophyte in 3D workflows to assess whether I think it's ultimately well designed (some bits seem to be … others not so much?), but it seems to run very well on the various machines I've tried it on and I have a great deal of respect for the nature of the program and the people behind it.

TheOtherHobbes• 4 years ago

Adobe keep adding pointless new features which I have to keep turning off in the preferences.

The new save dialog is insane. No, I don't want to save a copy just because I want to save a jpeg. Why is this even a question?

I have no idea why it isn't possible to copy/paste text with its blending options. Select All, Copy - Nope. All you get is the text, with none of the effects. (I know you can duplicate a layer - but not to a different document.)

Why does Creative Cloud install three versions? I suppose it's nice that I have v21, v22 and v23 but - why?

There still isn't a simple "paste clipboard to new document" command - something Paintshop Pro had back in the 90s.

Does anyone at Adobe actually use PS professionally? These are all simple usability issues that could be fixed with a bit of thought. But instead we get Neural Filters - useless - and the occasional core behaviour that changes for no good reason.

llanowarelves• 4 years ago

That's what's so frustrating about Adobe.

Given their position in the market, they could be cutting edge software performance wise and everything else. No one can say they don't have the resources.

Instead, they buy innovators (that solely exist in the first place because Adobe failed to keep up), do bare minimum to maintain, and charge rents.

dietrichepp• 4 years ago

I’ve been using Photoshop alternatives for a long time now, but when I was a kid, I had access to Photoshop (version 2.5 and later 5.0). I’m used to the alternatives. Mostly Gimp, Krita, Pixelmator, and Procreate.

Recently I had a chance to use Photoshop 7 on an old iMac G4 that I acquired. Surely this would be like living in the dark ages, banging some rocks together?

Well, no. It turns out that this old version of Photoshop is easy to use and powerful. I can quickly do simple graphic design tasks that would take me longer in other programs or leave me frustrated. I can also do a bunch of simple photo editing tasks that I’ve been missing out on. This experience really highlighted how I’ve gotten used to missing features or poorly-designed UIs in programs like Gimp. Simple things like messing with channels, changing the brush size, working with 16-bit color and alternative color spaces, layer styles, nice text layers—it’s all just there and working and ready to use, in this nearly 20-year-old software.

some-guy2• 4 years ago

I use the Windows version of Photoshop 7.0 still. It works out of the box with WINE as well on MacOS and Linux.

peterangular2• 4 years ago

Adobe CS5 user here - 12 year old software.

CS5 Master Suite w/discs and valid serial from a reputable seller on eBay still commands $200-300+ which is nuts to me. But, when I realize that I can do all of my modern graphics workflow in this 12 year old software I get it.

I would love to use some of the newer parts of the suite but the functionality of my CS5 still 100% covers my use-case. For photography I've ended up on Darktable, and if I need to manually retouch a photo there's nothing stopping me from a Darktable <> Photoshop CS5 workflow.

As much as I want to love Gimp - I can't. It is outright ineffective and lacks features compared to even my 12 year old Photoshop. The people who continue to tout it as a 1:1 replacement have never worked in a professional graphics workflow.

Also, Procreate is amazing for doing illustrations though - I really wish I had it when I was learning digital painting!

---

Regardless, just wandering sharing anecdotes because I relate to your experiences. Hard agree that legacy versions of Photoshop (and honestly the entire suite) are still insanely powerful. I also find them to be way more stable than their newer counterparts. They're just objectively a better financial decision - I bought this a decade ago and I don't have a monthly $50-60 Adobe draw on my bank account.

blagie• 4 years ago

Virtually all the free alternatives to Photoshop are similar to 1995-2000 era Photoshop. In that era, gimp and Photoshop were neck-and-neck, with gimp missing color models and deep color.

Photoshop from that era was pretty simple and easy-to-write. It worked well, and so did the alternatives. Adobe invested an incredible amount into research to build a more serious moat around 2022-era Photoshop.

tehlike• 4 years ago

Online/offline thing can be fixed with progressive web apps at least on Android. Figma has to implement it though.

inasmuch8• 4 years ago

I don't think I'll be designing on an Android device any time soon, but fingers crossed for implementation!

Keyframe3• 4 years ago

For all shit we give Adobe we have to give them credit. Over time their tools become incredibly good for productivity (even though they might suck hard in certain departments like performance). I've been around for long enough to see few of these. Freehand for example - there were tears and cries for help when they got ahold of it. Illustrator was shit at the time; Today it's awesome and an industry standard. Flash, same story. After Effects, same story. Premiere, same story. Indesign, same story. Etc.. Hard reason why most of those became industry standard was/is that Adobe understands how to cater to the user of those. Performance sucks, yes. Stability? Sometimes as well. User experience? The best. It's like with Microsoft and dev tools.

capableweb2• 4 years ago

Performance sucks, yes. Stability? Sometimes as well. User experience? The best

Not sure how this makes sense. If the performance is not good, and the stability is lacking, then the user experience cannot be the best. I rather use a video editor that makes me go through two menus instead of one, rather than having a video editor that crashes on me or takes ages to generate the proxy clips (or even need proxy clips for 4K and below footage).

krsw• 4 years ago

I think that cuts both ways in that while they do incrementally improve the products, they never would be industry standard if not for Adobe pushing them. Also worth noting that a lot of their portfolio was acquired once it had been threatening their turf and becoming established (Macromedia early on, Figma most recently). Their Suite approach lets them float tools for awhile before determining if they're worthwhile to continue developing or if they need to be axed, or merged into existing products.

TheRealPomax2• 4 years ago

Small fix: over time some of their tools become incredibly good for productivity.

Because they have an excellent track record of completely ignoring the glaring problems with some of their apps like become slower even though cpus and gpus are getting faster (looking at you, Lightroom team)

daniel-cussen• 4 years ago

OK I'm open to that side of things.

If Adobe pulls all the stops and produces value corresponding to what it charges, which is crazy--then, I might reconsider my views, maybe their business incentives have aligned very well now and they can become a virtuous monopoly.

You wouldn't know it from the comments on this site, though, my GOD, all negative yours is the first positive.

Further, Figma did a good thing for the end user by selling out to Adobe--at a very steep price. Their holding out for a huge price is virtuous, yes of course more money for them, but if you're selling out, charging a lot is virtuous, taking as big a bite as you can out of the giant.

zigzag3122• 4 years ago

For offline alternative to Figma check Lunacy. It's quite powerful, multiplatform and free. UI and workflow is well thought out.

It seems it's written in C# as it uses Avalonia UI.

https://icons8.com/lunacy

inasmuch8• 4 years ago

I will check it out—thanks for the tip!

rvz• 4 years ago

Enjoy it while it lasts as I bet that Microsoft will acquire the creators of Lunacy for their Surface products.

jjcm2• 4 years ago

Big disclaimer that I work for Figma, so HEAVY biases here.

In my eyes the 2GB/4GB limit for browser memory caps feels like an implementation issue rather than it being the wrong platform. At least for me, I value the ability to easily collaborate with people more than the need to have code be explicitly local or for things to be offline. The browser is excellent at that, and the explosion of online SASS products over locally installed apps in the last decade has shown it to be true for most use cases.

I don't think it's difficult at all to imagine a web app "stably and efficiently growing to Swiss army knife-levels of capability". Increasing that per-tab memory limit feels like an easier hill to climb than finding a way to provide that instant access for everyone and collaboration baked into locally installed apps.

Also don't kill yourself, even if it's a joke, life is more than software.

inasmuch8• 4 years ago

Appreciate your thoughts! The suicide bit was definitely a joke, so no worries there.

I hear what you're saying, and definitely get how the sharing bit that has helped Figma take off like it has is good for the browser. I'm a bit at odds with the industry, though, in that that's probably one of the things I like least about Figma. But a lot of that is because I'm not always the best team player :].

I won't pretend to know enough about what goes into building these programs to say whether the RAM cap or native app collaboration is the more difficult thing to overcome, but it's clear to me that I would prioritize these differently from others.

AuryGlenz• 4 years ago

On the other hand, I have absolutely no use for any collaboration (say, in Photoshop) with the work I do.

throwaway82388• 4 years ago

I work with the creative cloud/suite daily, and judging from what I read on twitter and in various forums, I get the feeling that I must be among the only happy Adobe customers out there.

I appreciate the continual new features and updates (exception: losing the Pantone solid book), but I feel the overall value proposition is better than it’s ever been. I remember spending hundreds annually on design suite upgrades. Now, a regular subscription gets me all of their software, a generous font selection, photoshop brushes, much more.

Taking Adobe for granted and kicking them around online has become a sport for some, but I suspect it’s a very loud but small segment of their customers.

inasmuch8• 4 years ago

Yeah, I hear you. A CC subscription is not cheap, but it's pretty incredible what you get for ~$600/yr, especially compared to what's offered by other subscriptions people pay for without blinking an eye.

I'm as eager—if not more—to take on massive greedy corporations as the next person, but my guillotine would be dull well before I got to Adobe's neck.

hamburglar• 4 years ago

This is a very old-man thing to say, but I think Photoshop peaked somewhere around version 6.0, which is what I’m still using. Amazing that it still runs, since it’s from the mid-90’s but it’s free of the cloud and really is very feature-dense. Thanks, windows appcompat team.

jannyfer• 4 years ago

This “Mic Check AI” - I actually just want to try this one feature.

I never know what gain I should set on my XLR mic + USB audio interface and I also don’t know how Teams/Webex will do to my signal. Is it compressing my signal? Noise gating it? No idea!

capableweb2• 4 years ago

You don't need AI, the one you have in your brain is enough, you just need to teach it what to look for :)

There is bunch of introduction material about "gain staging" online, here's just one to get you started: https://www.audiotechnology.com/tutorials/microphones-levels...

TheRealPomax2• 4 years ago

Or, if this is a solved problem with "AI" (a nonsense name for the most part, but if that's what they call it, that's what they call it) then don't and use the tools that get the job done. I have no reason to learn how a variable transmission engine works just to go shopping, and I have no reason to learn what to look for if tools do that for me. That's the whole point of good tooling: to turn conventional "you need to learn X before you can Y" into "this is archaic and no one needs to know this anymore". Or perhaps more in line with Adobe products: no one in their right mind is going to learn how to create uniform blurs with successive blends when they can just hit "filters -> blur -> gaussian blur". The tool made the traditional approach obsolete. Here's to many more things made obsolete.

samdotdesign5• 4 years ago

You can try it now at https://shasta.adobe.com/miccheck

steakknife5• 4 years ago

You can use it right now if you make a free account: http://shasta.adobe.com/miccheck

Not mobile friendly yet though.

adzm• 4 years ago

Really excited to see the transcription based editing. There is a ton of work that this can simplify. I really hope some of this gets integrated into Audition or AE etc as well.

DocJade• 4 years ago

The audio denoising / filtering they have seems quite impressive, if only I could use it in real-time on teams calls

samdotdesign5• 4 years ago

Hi! I’m Sam, lead designer of Shasta. You can check out the app at https://shasta.adobe.com (OP's post links to one of our launch announcement pages). Our enhance speech and mic check features are already available for everyone to try.

_HMCB_• 4 years ago

Thank you for stopping by and the link. I signed up for access using the OP’s link. Is that no longer required?

samdotdesign5• 4 years ago

Sorry for the confusion! Applying for access is still required to access Shasta, our recording + editing app. We’ve been slowly ramping up in getting people access to it… working hard on making sure it’s a super smooth experience.

Buuuut in the meantime, we launched two of our AI features as standalone mini apps that anyone can use for free, and those are called Mic Check and Enhance Speech. They’re fun, try them out and let me know what you think.

sullivanmatt• 4 years ago

You have a very neat website (https://sam.design). Learned something new today.

samdotdesign5• 4 years ago

Oh hey thanks!

_visgean• 4 years ago

Is the plan for this to be only an editor for podcasts or is there a plan to also serve the podcast through this platform? (i for example almost never listen to podcasts but would sometimes just enjoy reading transcripts...)

samdotdesign5• 4 years ago

We’ve been focusing on Shasta as a place to record and edit all kinds of spoken audio, including podcasts. As far as hosting and platform stuff, still figuring that out. But yeah, transcripts are great!

cercatrova• 4 years ago

Have you seen Whisper by OpenAI that was discussed recently [0]? The quality is extremely good for transcription and even translation, wondering if your team could use it as it's also open source.

[0] https://news.ycombinator.com/item?id=32927360

anigbrowl2• 4 years ago

Thanks for being available here. I used to beta test for Audition and am your target customer for this so I look forward to seeing this.

owlbynight• 4 years ago

Signed up for early access. We currently use Zencastr for recording our podcast, which has been great, but curious how Shasta will stack up and it'd be nice to not have to pay for two subscriptions.

thomasqbrady• 4 years ago

Is it still called "Sherlock-ing" if Apple isn't the one doing it?

Adobe XD : Sketch :: Shasta : Descript (https://www.descript.com), right down to the advertising style

CharlesW2• 4 years ago

This is Adobe's Descript killer for sure.

I personally don't think of making a copycat product as "Sherlocking". Wikipedia says Sherlocking is "releasing a feature that supplants or obviates third-party software". https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked...

jfengel• 4 years ago

Holy cow. I did not know about that. And the pricing seems extremely reasonable.

ohadron• 4 years ago

The Adobe XD :: Figma move didn’t work as well as they’ve hoped in the end…

relwin• 4 years ago

The denoising works very well. I found that mixing the enhanced audio with about -3 to -6 dB with the original voice audio works very well. (short clip at a brewery) https://youtu.be/M5pdHVoXQHE

I also tested with wind noise (outdoor speaker) and the audio "recovery" is amazing. It rescues bad field recordings that you normally can't use. I'm impressed!

steakknife5• 4 years ago

Shasta dev here. We're releasing an update soon to the enhance feature that does exactly this!

Krasnol• 4 years ago

While Project Shasta is in alpha, we only support Google Chrome on desktop.

I hate that this came up AFTER I logged in...

Tepix2• 4 years ago

Right now it shows this before i signed up. Also it only supports english.

steakknife5• 4 years ago

Officially it's Google Chrome, but for practical purposes any Chromium-based browser seems to work fine, e.g. Brave, Edge.

jjcm2• 4 years ago

As a Figma employee, there's obviously been a lot of shakeup recently for us. That said, this is one of the areas that actually makes me really excited to partner with Adobe. They've always been at the forefront of AI-esque assisted editing (content aware fill in photoshop was so far ahead of its time). Definitely excited to try this out and to see other AI projects by Adobe to help creatives.

osigurdson• 4 years ago

It is really cool but I think needs to sync with video. It seems that this would be relatively easy to do.

Shasta: AI-powered audio recording and editing

Related Stories