We have long had the technology to remove unwanted objects from images, but now Adobe has come up with a new feature that can remove objects from videos as well. To do this, Adobe has brought Photoshop’s Content-Aware Fill feature to After Effects.
Remove objects from videos is now easier
Adobe first introduced Content-Aware Fill in Photoshop with Creative Suite 6. The feature allows users to remove an unwanted object from an image by covering it with a copied portion of the image taken from the object’s surrounding and background. Now the same concept is being applied to remove objects from videos, but with added help from machine learning. Bringing Content-Aware Fill functionality to videos is part of Adobe’s Spring update to After Effects.
This new functionality uses Adobe’s Sensei AI platform and allows users to remove objects from videos without having to edit frame by frame. The use of AI helps remove objects from videos even if they are moving.
Content-Aware Fill for After Effects and Sensei work in tandem to give users a flawless editing experience. Content-Aware Fill for After Effects removes the object from one frame, and Sensei identifies and removes it from the rest of the frames.
Like Photoshop CS5’s Content-Aware Fill tool, After Effects’ version of the tool does all the processing on the local machine. However, Adobe says that as Sensei learns and improves, it should get better at removing objects from videos.
Removing an object from a video is not as simple as removing an object from an image. Users must create a mask to cover the erased region of the frames and define the region that has to be erased. Hand-animated or painted masks or even After Effects’ automated tracking tools can be used to do this.
One major challenge for the tool is when the mask used to cover the removed object is not in the frame seconds later. To overcome this, After Effects allows users to define a few areas of a frame that can be used as masks. Users can also create reference still frames in Photoshop to guide the tool while it does its own editing.
How will it be useful?
Adobe first showcased the technology at the its Sneak Peeks event in 2017 under the code-name Project Cloak. Now Content-Aware Fill for After Effects is available to After Effects CC 2019 users.
It is undoubtedly a very useful feature for those making movies, shows and even short YouTube clips. Adobe has suggested several use cases for the new feature, such as removing unwanted watermarks, rigging or boom mics from shots or even a person from a clip. Adobe adds that Content-Aware Fill could prove to be a very useful tool for 360-degree virtual reality projects in which it is difficult to hide everything around the camera.
In addition to adding this tool to remove objects from videos, Adobe has also added other updates to its video-editing software. Many of these updates are powered by Sensei, such as Auto-Ducking for Ambience, which now uses Sensei to offer a seamless experience. After the update, Auto-Ducking will automatically adjust the volume of background music tracks during dialogues in Audition.
Other new updates from Adobe
Other updates improve puppet rigging for Character Animator. With the update, users will see cleaner, automated lip syncs and workflow improvements. Users also get the new Freeform Project Panel, which enables editors to better organize and storyboard their clips in Premiere Pro. Now clips can be stacked, color-coded, and resized in the virtual workspace for a more intuitive workflow.
Adobe has also improved the performance of its applications. Export times for GPU rendering in Premiere are reduced, and mask tracking is 38 times faster for 8K videos and up to four times faster for HD scenes.
“Through optimized performance and intelligent new features powered by Adobe Sensei, video professionals can cut out more tedious production tasks to focus on their creative vision,” Adobe Vice President of Digital Video and Audio Steven Warner said.
Many other features are being rolled out across Creative Cloud’s video and audio apps. To see all the new features and updates, visit the Adobe Blog. The company will demonstrate the updates today at NAB 2019.