Posts in Linked

John Giannandrea on the Broad Reach of Machine Learning in Apple’s Products

Today Samuel Axon at ArsTechnica published a new interview with two Apple executives: SVP of Machine Learning and AI Strategy John Giannandrea and VP of Product Marketing Bob Borchers. The interview is lengthy yet well worth reading, especially since it’s the most we’ve heard from Apple’s head of ML and AI since he departed Google to join the company in 2018.

Based on some of the things Giannandrea says in the interview, it sounds like he’s had a very busy two years. For example, when asked to list ways Apple has used machine learning in its recent software and products, Giannandrea lists a variety of things before ultimately indicating that it’s harder to name things that don’t use machine learning than ones that do.

There’s a whole bunch of new experiences that are powered by machine learning. And these are things like language translation, or on-device dictation, or our new features around health, like sleep and hand washing, and stuff we’ve released in the past around heart health and things like this. I think there are increasingly fewer and fewer places in iOS where we’re not using machine learning. It’s hard to find a part of the experience where you’re not doing some predictive [work].

One interesting tidbit mentioned by both Giannandrea and Borchers is that Apple’s increased dependence on machine learning hasn’t led to the company talking about ML non-stop. I’ve noticed this too – whereas a few years ago the company might have thrown out ‘machine learning’ countless times during a keynote presentation, these days it’s intentionally more careful and calculated in naming the term, and I think for good reason. As Giannandrea puts it, “I think that this is the future of the computing devices that we have, is that they be smart, and that, that smart sort of disappear.” Borchers expounds on that idea:

This is clearly our approach, with everything that we do, which is, ‘Let’s focus on what the benefit is, not how you got there.’ And in the best cases, it becomes automagic. It disappears… and you just focus on what happened, as opposed to how it happened.

The full interview covers subjects like Apple’s Neural Engine, Apple Silicon for Macs, the benefits of handling ML tasks on-device, and much more, including a fun story from Giannandrea’s early days at Apple. You can read it here.

Permalink


Apple Commits to 2030 Carbon Neutrality Across Full Business

Every year Apple releases a new environmental report showing the company’s progress in environmental efforts, and alongside the release of this year’s report, the company has announced a new commitment for the decade ahead:

Apple today unveiled its plan to become carbon neutral across its entire business, manufacturing supply chain, and product life cycle by 2030. The company is already carbon neutral today for its global corporate operations, and this new commitment means that by 2030, every Apple device sold will have net zero climate impact.

“Businesses have a profound opportunity to help build a more sustainable future, one born of our common concern for the planet we share,” said Tim Cook, Apple’s CEO. “The innovations powering our environmental journey are not only good for the planet — they’ve helped us make our products more energy efficient and bring new sources of clean energy online around the world. Climate action can be the foundation for a new era of innovative potential, job creation, and durable economic growth. With our commitment to carbon neutrality, we hope to be a ripple in the pond that creates a much larger change.”

Achieving carbon neutrality for its corporate operations was a nice milestone for the company, but this new commitment appears far more challenging. Apple works with third-party suppliers and manufacturers all around the world to build its devices, and fulfilling this new goal depends a lot on those third parties. It will be interesting to see over the next decade all of the different actions that will be taken to find success in carbon neutrality, but the report of Apple including fewer accessories in the box with new iPhone purchases certainly seems like it would help.

Permalink

Apple Partners with Historically Black Colleges and Universities to Create New Coding Centers

Today Apple announced an expansion of its initiative of partnering with Historically Black Colleges and Universities (HBCUs) to create hubs for training the next generation of coders. 10 new HBCU coding centers are being added throughout the US, from which nearly 500 teachers and community leaders will soon participate in “a virtual Community Education Initiative Coding Academy that Apple is hosting for all initiative partners.” During this training:

Educators will learn the building blocks of coding with Swift, Apple’s easy-to-learn coding language. Participants will work in teams to design app prototypes to address real community challenges. After completing the coding academy, educators will begin to integrate the coding and creativity curricula into their communities by launching coding clubs and courses at their schools, hosting community coding events, and creating workforce development opportunities for adult learners.

This announcement comes as Apple just last week shared updates to its lineup of coding resources for students, educators, and families alike, demonstrating the company’s investment in developing coding initiatives across all age groups. The move also follows Tim Cook’s open letter in June addressing racism in America and subsequent creation of a new $100 million Racial Equity and Justice Initiative by the company. The executive leading this initiative, Lisa Jackson, commented on today’s HBCU news saying:

“Apple is committed to working alongside communities of color to advance educational equity,” said Lisa Jackson, Apple’s vice president of Environment, Policy and Social Initiatives. “We see this expansion of our Community Education Initiative and partnership with HBCUs as another step toward helping Black students realize their dreams and solve the problems of tomorrow.”

These last couple months have seen many companies express a desire to work toward pursuing racial equality and justice, but true change takes more than just words, so I’m glad to start seeing the early fruits of Apple’s new commitments.

Permalink


Apple Highlights Hardware and Apps in ‘The Whole Work-From-Home Thing’

In April 2019, Apple published a video called The Underdogs that followed the story of a team of co-workers designing a round pizza box. Today, the quartet is back in a sequel of sorts called The Whole Work-From-Home Thing.

The new video follows the same group of colleagues as they attempt to design an all-new box while working from home. The story follows the quartet as they work around the clock on a tight deadline while juggling personal obligations and coping with working remotely.

The pace is frenetic. Over the course of the multi-day ordeal, the group turns to their Macs, iPads, and iPhones to come up with ideas and design the box. They also rely on a wide array of apps, including third-party apps like MindNode and Adobe InDesign.

Like its predecessor, ‘The Whole Work-From-Home Thing’ is funny but succeeds at demonstrating ways that Apple hardware and apps can solve some of the problems facing many people these days. This video may hit a little too close to home and stress some people out a bit, but I enjoyed the lighthearted fun poked at working from home and think it’s well worth watching.

Permalink

Epic Games Releases iPhone App That Captures Facial Expressions to Unreal Engine

Source: Epic Games.

Source: Epic Games.

Epic Games has released a new iPhone app for videogame developers that captures facial expressions, piping them into the company’s Unreal Engine in real-time. As explained on the Unreal Engine blog:

Live Link Face streams high-quality facial animation in real-time from your iPhone directly onto characters in Unreal Engine. The app’s tracking leverages Apple’s ARKit and the iPhone’s TrueDepth front-facing camera to interactively track a performer’s face, transmitting this data directly to Unreal Engine via Live Link over a network.

What I find most interesting about Live Link Face is that Epic says it scales from solo developers working at home to sophisticated stage productions involving actors in motion capture suits and multiple iPhones. If so, that will make the app a terrific example of the sort of democratization of complex tools that technologies like ARKit and hardware like the iPhone’s TrueDepth camera make possible when integrated into existing workflows.

Permalink

Omni’s Revised 2020 Roadmap

In January, Ken Case of The Omni Group shared the company’s plans for 2020, which included the release of OmniPlan 4, the expansion features for OmniFocus for the Web, simplified app license management on the web, along with OmniFocus collaboration and improved in-app workflows. Omni Automation has shipped as part of all of the company’s products and OmniPlan and simplified licensing will launch soon. However, the combination of the global pandemic and announcements of WWDC has caused Omni to adjust its remaining plans, though its goals remain the same.

As Case describes it:

Our roadmap itself isn’t changing dramatically. We’re going to continue working on OmniFocus collaboration, and we’re going to continue improving the flow of using our apps. But the latest news from Apple has inspired us to take this work even further.

Omni has historically been at the forefront of adopting new Apple technologies. The company was an early adopter of the Cocoa frameworks and was among the first to develop a pro-level app for the iPad. With that in mind, Case announced that Omni would reevaluate its apps, considering how they can take advantage of Apple’s new frameworks:

as we redesign our apps, we’re going to leverage the latest technologies. We’re not going to completely restart our development from scratch—but we are taking a step back to think about how we would design and build our apps if we were starting again now, building on the latest technologies and taking into account everything we’ve learned from our customers – you! – about how you use our apps.

As Case notes, this is a very big undertaking. Users may need to wait a little longer for the next big update to some of their favorite apps, but taking the time to make the transition now will hopefully mean Omni’s apps will remain relevant for years to come.

Permalink

2020 Apple Design Award Winners: The AppStories Interviews

Federico and I had the pleasure of interviewing three of the 2020 Apple Design Award winners for AppStories. The awards, which were announced by Apple last Monday, recognize “outstanding app design, innovation, ingenuity, and technical achievement.”

For today’s special episode, we spoke with Majd Taby of Bergen Co., the creator of photo and video editing app Darkroom, Sam Rosenthal of The Game Band, the studio behind Where Cards Fall, which was a launch title on Apple Arcade, and Jenova Chen, of thatgamecompany and the creative director of Sky: Children of the Light, an innovative social adventure game. All three interviews are terrific conversations that reveal common threads of thoughtful design, innovative approaches that feature the latest Apple technology, and a deep understanding of their users.

Thank you to Majd Taby, Sam Rosenthal, and Jenova Chen for taking the time for the interviews, Apple for helping arrange them, and as always, thank you for listening to AppStories. We hope you enjoy the episode.


Permalink